Welcome
Welcome to securitymetrics.org, a community website for security practitioners. Securitymetrics.org offers a community blogging service (this page) and a members-only mailing list. See the Mailing List page for more details.

Announcing Metricon 5

Metricon 5 is the fifth annual conference dedicated to security metrics. It is a forum for presenting new approaches for measuring information security effectiveness, with a bias towards practical, specific approaches. Topics and presentations will be selected for their novelty and merit, and their potential to stimulate discussion.

With five years of organized conferences in the history books, this year's theme, appropriately, is Older But Wiser. Four years ago, presenters at the first Metricon discussed software security, benchmarking, identity management, enterprise case studies and many other topics. Since then, researchers and enterprises have continued to investigate new techniques. What have we learned? Given that we are trying to measure, measuring the security metrics field (and the success or failures of our own efforts) is also our responsibility.

The program is organized along three temporal perspectives:

  • Metrics Past. Which metrics techniques from 2006 worked, and which did not? And how can knowledge of the past inform the present and future?
  • Metrics Present. What is the state of the art as practiced today' by leading corporations, consultants and researchers?
  • Metrics Future. What new strategies for measuring security will emerge in the future?

Metricon 5 will be a one-day event, Tuesday, August 10th, 2010, co-located with the 19th USENIX Security Symposium in Washington, DC (http://www.usenix.org/events/sec10/). Metricon will begin bright and early in the morning, continue through a catered lunch in meeting room, and extend into the evening with informal discussion. Attendance will be by invitation. Capacity is limited to 60 participants.

Program

TimeTrack
0800–0900hBreakfast
0900–0930hAndrew Jaquith, Forrester Research, Welcome address and ''Five Years of Security Metrics: A Look Back''(info)
0930–1000hRichard Seiersen, Kaiser Permanente, ''Practical Security Metrics in the 4th Dimension''(info)
1000–1030hRH Powell, Akamai, ''Weathering Storms in the Cloud: Analyzing Massive Distributed Denial of Service Attacks to Better Prepare for the Future''(info)
1030–1100hMorning break
1100–1130hJohn S Quarterman, Quarterman Creations/CREC at the UT Austin School of Business, ''Spam Reputation as Output Measure of Infosec''(info)
1130–1200hGina Fisk, Los Alamos National Laboratories, ''Optimizing Performance Management using Adaptive Metrics, Fitness Functions, and the Balanced Score Card''(info)
1200–1230hFabio Massacci, Universita' di Trento, ''Which is the Right Source for Vulnerability Studies? An Empirical Analysis on Mozilla Firefox''(info)
1230–1345hLunch
1345–1415hElizabeth Nichols, PlexLogic: Security Metrics, ''Security Metrics: What’s Hot and What’s Not''(info)
1415–1445hLaura Glowick, Federal Home Loan Bank of Boston, ''Enterprise Security Dashboard''(info) also: FHLB's metrics catalog(info)
1445–1515hAfternoon break
1515–1545hAlex Hutton, Verizon Security Intelligence, ''Bridging Risk Modeling, Threat Modeling, and Operational Metrics With the VERIS Framework''(info)
1545–1615hMichael Smith, Fish Catchers Heavy Industries, ''Meta-Metrics: Building a Scorecard for the Evaluation of Security Management and Control Frameworks''(info)
1615–1730Rump session: open-mic discussion of current research and topics of shared interest
1730–Beer! Sponsored by Blue Canopy

Venue

Metricon 5 will be held at the Marriott Woodman Park Hotel, 2660 Woodley Road Northwest, Washington, DC, on August 10th, 2010. It is co-located with the USENIX Security 2010 Symposium.

Event Sponsors

BlueCanopy_Logo_04032010.png

Attendance

Attendance is by invitation only. If you would like to attend, send an e-mail to metricon5 at securitymetrics dot org.

All participants will be expected to "come with findings" and be willing to contribute to group discussions. Politeness will be praised; questions, encouraged; lurkers, flushed out.

The proceedings of all past meetings are available here:

For speakers

  • Deadline for final presentation: July 30th, 2010

Conference chairs

  • Andrew Jaquith, Forrester Research
  • Khalid Kark, Forrester Research

Program committee members

  • Jennifer Bayuk, Stevens Institute of Technology
  • Dan Geer, In-Q-Tel
  • Chris Walsh, SurePayroll
  • Wade Baker, Verizon Risk Intelligence
  • Ray Kaplan, Ray Kaplan & Associates
  • Michael Smith, Akamai Technologies
  • Daniel Arista, Syracuse Research Corporation

Mini MetriCon 4.5

Mini MetriCon 4.5 was held Monday, March 1, 2010, in SanFrancisco, California, adjacent to the USA RSA 2010 Conference. The presentations are posted as embedded links in the agenda; the original CFP remains available as well.

MetriCon 4.0

MetriCon 4.0 was held Tuesday, August 11, 2009, in Montreal, Quebec, co-located with the USENIX Security Symposium. See the MetriCon 4.0 page for the details of the meeting, including its CFP, the final agenda, and the meeting's Digest.

Mini MetriCon 3.5

Mini MetriCon 3.5 was held Monday, April 20, 2009, in SanFrancisco, California, adjacent to the USA RSA 2009 Conference. The presentations are posted as embedded links in the agenda; the original CFP remains available as well. Sadly, no Digest was ever completed.

MetriCon 3.0

The MetriCon 3.0 presentations and digest are available as attachments to the final agenda

Mini MetriCon 2.5 Presentations

The MiniMetriCon 2.5 presentations are available as attachments to the final agenda.


Metrics Catalog Project:

The Metrics Catalog Project was officially launched in June 2008. A major revision has been made available as of April 2009. To see the catalog on-line you can visit:

General information about the Metrics Catalog can be found in the following documents:

BEWARE: You will need a Javascript and Java enabled browser to optimally experience the content on these sites. Due to circumstances beyond our control, we cannot support any browser on Vista.

--Elizabeth Nichols, 3-July-2009

Logged in? Add a New entry to this blog!

April 15, 2007 10:58 PM
More Praise for Security Metrics
The bloggiste at Layer 8 just declared Security Metrics to be "That Good". I have no idea who shrdlu actually is. But whomever she is, she deserves a hearty thank-you and an offer of a beer should we ever meet in person. Here is a snippet of what she said:
I have found the Metrics Prophet for our times, 
and his name is Andrew Jaquith.

I stumbled home yesterday from work, 
sleep-deprived, jittery, and feverish from 
an oncoming cold.  I tucked myself into 
bed, hoping to sleep—but I could not sleep
until I had read Security Metrics cover to
cover.  It was That Good.

Now, either that makes me the biggest
saddo anorak west of the Pond, or it
means Jaquith is an extraordinary writer
about what would otherwise be an
extremely dull subject.  I would of course
prefer to think it’s the latter, and I’m
sure he would too.

First off, his writing is chock full of
playfulness and amusing literacy, from the
literary nods ("Call me Analyst.") to the
rimshots ("… the top and bottom 50% are
divided by—wait for it—the median!"). 

Secondly, his metrics are for the most part
accessible, meaning that as soon as I see
them, I think, “Yeah, I could get those!”
And a whole lot of them are ones I’d
already thought of, but there are a few
gems in there that were like little Altoids
in my mouth, that made me sit up and
go, “Whoa.”

You can see the rest of her review on her website. If you are thinking of buying the book, her comments should give you an idea of what is inside. She has some excellent and perceptive constructive criticisms also, which are all on target.

Ms. Shrdlu, thanks very much for the kind words. I especially appreciate that she caught my nod to Herman Melville in the first line of the book ("Call me Analyst.").

By Andrew Jaquith  Permalink
April 9, 2007 12:39 PM
Second Workshop on Security Metrics (MetriCon 2.0) — Call for Papers

MetriCon 2.0 CFP

August 7, 2007 Boston, MA

Overview

Do you cringe at the subjectivity applied to security in every manner? If so, MetriCon 2.0 may be your antidote to change security from an artistic "matter of opinion" into an objective, quantifiable science. The time for adjectives and adverbs has gone; the time for hard facts and data has come.

MetriCon 2.0 is intended as a forum for lively, practical discussion in the area of security metrics. It is a forum for quantifiable approaches and results to problems afflicting information security today, with a bias towards practical, specific implementations. Topics and presentations will be selected for their potential to stimulate discussion in the Workshop.

MetriCon 2.0 will be a one-day event, Tuesday, August 7, 2007, co-located with the 16th USENIX Security Symposium in Boston, MA, USA (http://www.usenix.org/events/sec07/). Beginning first thing in the morning, with meals taken in the meeting room, and extending into the evening. Attendance will be by invitation and limited to 60 participants. All participants will be expected to "come with findings" and be willing to address the group in some fashion, formally or not. Preference given to the authors of position papers/presentations who have actual work in progress.

Each presenter will have 10-15 minutes to present his or her idea, followed by 15-20 minutes of discussion with the workshop participants. Panels and groups of related presentations may be proposed to present different approaches to selected topics, and will be steered by what sorts of proposals come in response to this Call.

Goals and Topics

The goal of the workshop is to stimulate discussion of and thinking about security metrics and to do so in ways that lead to realistic, early results of lasting value. Potential attendees are invited to submit position papers to be shared with all. Such position papers are expected to address security metrics in one of the following categories:

  • Benchmarking
  • Empirical Studies
  • Metrics Definitions
  • Financial Planning
  • Security/Risk Modeling
  • Tools, Technologies, Tips, and Tricks
  • Visualization

Practical implementations, real world case studies, and detailed models will be preferred over broader models or general ideas.

How to Participate

Submit a short position paper or description of work done/ongoing. Your submission must be no longer than five(5) paragraphs or presentation slides. Author names and affiliations should appear first in/on the submission. Submissions may be in PDF, PowerPoint, HTML, or plaintext email and must be submitted to MetriCon AT securitymetrics.org.

Presenters will be notified of acceptance by June 22, 2007 and expected to provide materials for distribution by July 22, 2007. All slides and position papers will be made available to participants at the workshop. No formal proceedings are intended. Plagiarism constitutes dishonesty. The organizers of this Workshop as well as USENIX prohibit these practices and will take appropriate action if dishonesty of this sort is found. Submission of recent, previously published work as well as simultaneous submissions to multiple venues is acceptable but please so indicate in your proposal.

Location

MetriCon 2.0 will be co-located with the 16th USENIX Security Symposium (Security ’07). (http://www.usenix.org/events/sec07/)

Cost

$200 all-inclusive of meeting space, materials preparation, and meals for the day.

Important Dates

Requests to participate: by May 11, 2007
Notification of acceptance: by June 22, 2007
Materials for distribution: by July 22, 2007

Workshop Organizers

Fred Cohen, Fred Cohen & Associates
Jeremy Epstein, webMethods
Dan Geer, Geer Risk Services
Andrew Jaquith, Yankee Group
Elizabeth Nichols, ClearPoint Metrics, Co-Chair
Gunnar Peterson, Arctec Group, Co-Chair
Russell Cameron Thomas, Meritology

By gunnar peterson  Permalink
April 3, 2007 11:26 AM
Alex Hutton Likes Security Metrics
Alex Hutton was one of the editorial reviewers for several chapters of Security Metrics, and offered some excellent feedback during the writing stages. Now that the book has shipped, as a way of saying "thank-you" my publisher Addison-Wesley hooked him up with a copy. Alex seems to like the rest of the book, too:
A Book You Should Buy

Finally, I’d like to point you to this. It’s a book
every analyst should own, written by a very smart person
(Andrew Jaquith), and filled with - mostly  - very
good material... Metrics are great, but there are so
many, many ways to get them wrong... Do go out
and obtain a copy for yourself and/or your analysts.
You won’t be disappointed. It’s one of those books
you’ll actually use.

Thanks so much, Alex! When I see you at MetriCon 2.0 later this year, you've got a beer coming to you.

By Andrew Jaquith  Permalink
April 1, 2007 10:39 PM
Introducing Security Metrics, the Cartoon
Mark Curphey's cynical vehicle for ripping the security industry gains another blunt instrument: the Hamster Wheel of Pain, featured in Chapter One in Security Metrics: Replacing Fear, Uncertainty and Doubt. Mark was kind enough commission a cartoon based a quick e-mail from me. I think the cartoon shows that I am at least as cynical as he.
By Andrew Jaquith  Permalink


Weblog archives:
This site is not affillated with any organization, and the opinions expressed on this website are strictly those of the authors themselves.

To log in to the Securitymetrics.org website, create a profile first.

Attachments