Metricon 4 — The Importance of Context

September 27, 2009

Metricon 4 was held Tuesday, August 11, 2009, in Montreal, Quebec, co-located with the USENIX Security Symposium. This page contains the details of the meeting, including the original CFP, the final agenda, and the meeting’s Digest.

Agenda

• Baseline Scoring Methods
  • John Nye, Reproducible Measurement as a Foundation for Security Assessment Metrics
  • Ed Bellis, Orbitz, Orbitz SCAP Metrics
• Measuring Impact
  • Lloyd Ellam, SigmaRisks – The Ugly, The Bad, and The Good
  • Shivaraj Tenginakai – Metrics for Detecting Compromised Systems. Accompanying paper.
• Enterprise Security Management
  • Li Liu, PhD candidate – Security Metrics in Governance, Risk and Compliance
  • Jim Cowie, Renesys – Using Security Metrics to Motivate a Response to A Critical Vulnerability
  • Gene Kim, Tripwire and Kurt Milne, IT Process Institute – Foundational Practices that Optimize Security and Operations
• Lunch over discussion of handouts, including:
  • Measuring the future basis of competition among AV products
  • Performance Testing the Vulnerability Response Decision Assistance (VRDA) Framework
  • PCI DSS Statistics and Metrics
  • Techniques for Enterprise Network Security Metrics
  • CIS Consensus Project
  • SOX Material Weakness and CIO/CEO turnover
• Software Security
  • Gary McGraw, Cigital and Brian Chess, Fortify – The Building Security In Maturity Model
  • Sandy Clark and Matt Blaze, University of Pennsylvania – Does Software Quality Matter?
• Trends and Stats
  • Betsy Nichols, Plexlogic – Crunching Metrics from Public Data
  • David Shettler, DataLoss DB
• Security Manager Panel
  • Moderator: Jennifer Bayuk – Introduction
  • Panelists: Ed Bellis, Orbitz; Chris Walsh, SurePayroll; and Robert Masse, Reitmans Ltd.
• Further discussion over dinner

Dan Geer wrote up the meeting Digest.

Program Committee #

Chair: Jennifer Bayuk, Independent Consultant

Members:

• Warren Axelrod, Financial Services Technology Consortium (FSTC)
• Fred Cohen, Fred Cohen & Associates & California Sciences Institute
• Lloyd Ellam, Iceberg Networks
• Dan Geer, In-Q-Tel
• Andrew Jaquith, Forrester Research
• Wayne Jansen, National Institute of Standards and Technology (NIST) Gene Kim, Tripwire
• Gunnar Peterson, Arctec Group
• Chris Walsh, SurePayroll

Original Call for Participation #

Metricon 4.0 is intended as a forum for lively, practical discussion in the area of security metrics. It is a forum for quantifiable approaches and results to problems afflicting information security today, with a bias towards practical, specific approaches that demonstrate the value of security metrics with respect to a security-related goal. Topics and presentations will be selected for their potential to stimulate discussion in the workshop.

Metricon 4.0 will be a one-day event, Tuesday, August 11, 2009, co-located with the 18th USENIX Security Symposium in Montreal, Quebec. Beginning first thing in the morning, with meals taken in the meeting room, and extending into the evening. Attendance will be by invitation and limited to 60 participants.

All participants will be expected to “come with findings” and be willing to address the group in some fashion, formally or not. In keeping with the theme of The Importance of Context, preference will be given to the authors of position papers/presentations who have actual work in progress that demonstrates the value of security metrics with respect to a security-related goal.

Topics that demonstrate the importance of context include:

• Data and analyses emerging from ongoing metrics efforts
• Studies in specific subject matter areas
• Time and situation-dependent aspects of security metrics
• Long-term trend analysis and forecasts
• Measures of the depth and breadth of security defenses
• Metrics definitions that can be operationalized
• Incorporating unknown vulnerabilities into security metrics
• Security and risk modeling calibrations
• Security measures in system design
• Software assurance initiatives
• Security metrics relationship to security assessments

The program committee will also consider any innovative security metrics related work.

How to Participate

Submit a short position paper or description of work done or ongoing. Your submission must be brief – no longer than two pages including both text and graphical displays of quantitative information. Author names and affiliations should appear first in the submission. Submissions may be in PDF, PowerPoint, HTML, or plaintext email and must be submitted to metricon4@securitymetrics.org. These requests to participate are due no later than noon GMT, Monday, May 25, 2009 (a hard deadline). You should receive an email acknowledgment of your submission within a day or two of posting; take action if you do not.

The Program Committee will invite both attendees and presenters. Participants of either sort will be notified of acceptance quickly – by June 15, 2009. Presenters who want hardcopy materials to be distributed at the Workshop must provide originals of those materials to the Program Committee by July 27, 2009. All slides, position papers, and what-not will be made available to all participants at the Workshop.

No formal academic proceedings are intended, but a digest of the meeting will be prepared and distributed to participants and the general public. (Digests for previous Metricon meetings are on the past event pages mentioned above.) Plagiarism is dishonest, and the organizers of this Workshop will take appropriate action if dishonesty of this sort is found. Submission of recent, previously published work as well as simultaneous submissions to multiple venues is entirely acceptable, but only if you disclose this in your proposal.