Welcome
Welcome to securitymetrics.org, a community website for security practitioners. Securitymetrics.org offers a community blogging service (this page) and a members-only mailing list. See the Mailing List page for more details.

Announcing Metricon 5

Metricon 5 is the fifth annual conference dedicated to security metrics. It is a forum for presenting new approaches for measuring information security effectiveness, with a bias towards practical, specific approaches. Topics and presentations will be selected for their novelty and merit, and their potential to stimulate discussion.

With five years of organized conferences in the history books, this year's theme, appropriately, is Older But Wiser. Four years ago, presenters at the first Metricon discussed software security, benchmarking, identity management, enterprise case studies and many other topics. Since then, researchers and enterprises have continued to investigate new techniques. What have we learned? Given that we are trying to measure, measuring the security metrics field (and the success or failures of our own efforts) is also our responsibility.

The program is organized along three temporal perspectives:

  • Metrics Past. Which metrics techniques from 2006 worked, and which did not? And how can knowledge of the past inform the present and future?
  • Metrics Present. What is the state of the art as practiced today' by leading corporations, consultants and researchers?
  • Metrics Future. What new strategies for measuring security will emerge in the future?

Metricon 5 will be a one-day event, Tuesday, August 10th, 2010, co-located with the 19th USENIX Security Symposium in Washington, DC (http://www.usenix.org/events/sec10/). Metricon will begin bright and early in the morning, continue through a catered lunch in meeting room, and extend into the evening with informal discussion. Attendance will be by invitation. Capacity is limited to 60 participants.

Program

TimeTrack
0800–0900hBreakfast
0900–0930hAndrew Jaquith, Forrester Research, Welcome address and ''Five Years of Security Metrics: A Look Back''(info)
0930–1000hRichard Seiersen, Kaiser Permanente, ''Practical Security Metrics in the 4th Dimension''(info)
1000–1030hRH Powell, Akamai, ''Weathering Storms in the Cloud: Analyzing Massive Distributed Denial of Service Attacks to Better Prepare for the Future''(info)
1030–1100hMorning break
1100–1130hJohn S Quarterman, Quarterman Creations/CREC at the UT Austin School of Business, ''Spam Reputation as Output Measure of Infosec''(info)
1130–1200hGina Fisk, Los Alamos National Laboratories, ''Optimizing Performance Management using Adaptive Metrics, Fitness Functions, and the Balanced Score Card''(info)
1200–1230hFabio Massacci, Universita' di Trento, ''Which is the Right Source for Vulnerability Studies? An Empirical Analysis on Mozilla Firefox''(info)
1230–1345hLunch
1345–1415hElizabeth Nichols, PlexLogic: Security Metrics, ''Security Metrics: What’s Hot and What’s Not''(info)
1415–1445hLaura Glowick, Federal Home Loan Bank of Boston, ''Enterprise Security Dashboard''(info) also: FHLB's metrics catalog(info)
1445–1515hAfternoon break
1515–1545hAlex Hutton, Verizon Security Intelligence, ''Bridging Risk Modeling, Threat Modeling, and Operational Metrics With the VERIS Framework''(info)
1545–1615hMichael Smith, Fish Catchers Heavy Industries, ''Meta-Metrics: Building a Scorecard for the Evaluation of Security Management and Control Frameworks''(info)
1615–1730Rump session: open-mic discussion of current research and topics of shared interest
1730–Beer! Sponsored by Blue Canopy

Venue

Metricon 5 will be held at the Marriott Woodman Park Hotel, 2660 Woodley Road Northwest, Washington, DC, on August 10th, 2010. It is co-located with the USENIX Security 2010 Symposium.

Event Sponsors

BlueCanopy_Logo_04032010.png

Attendance

Attendance is by invitation only. If you would like to attend, send an e-mail to metricon5 at securitymetrics dot org.

All participants will be expected to "come with findings" and be willing to contribute to group discussions. Politeness will be praised; questions, encouraged; lurkers, flushed out.

The proceedings of all past meetings are available here:

For speakers

  • Deadline for final presentation: July 30th, 2010

Conference chairs

  • Andrew Jaquith, Forrester Research
  • Khalid Kark, Forrester Research

Program committee members

  • Jennifer Bayuk, Stevens Institute of Technology
  • Dan Geer, In-Q-Tel
  • Chris Walsh, SurePayroll
  • Wade Baker, Verizon Risk Intelligence
  • Ray Kaplan, Ray Kaplan & Associates
  • Michael Smith, Akamai Technologies
  • Daniel Arista, Syracuse Research Corporation

Mini MetriCon 4.5

Mini MetriCon 4.5 was held Monday, March 1, 2010, in SanFrancisco, California, adjacent to the USA RSA 2010 Conference. The presentations are posted as embedded links in the agenda; the original CFP remains available as well.

MetriCon 4.0

MetriCon 4.0 was held Tuesday, August 11, 2009, in Montreal, Quebec, co-located with the USENIX Security Symposium. See the MetriCon 4.0 page for the details of the meeting, including its CFP, the final agenda, and the meeting's Digest.

Mini MetriCon 3.5

Mini MetriCon 3.5 was held Monday, April 20, 2009, in SanFrancisco, California, adjacent to the USA RSA 2009 Conference. The presentations are posted as embedded links in the agenda; the original CFP remains available as well. Sadly, no Digest was ever completed.

MetriCon 3.0

The MetriCon 3.0 presentations and digest are available as attachments to the final agenda

Mini MetriCon 2.5 Presentations

The MiniMetriCon 2.5 presentations are available as attachments to the final agenda.


Metrics Catalog Project:

The Metrics Catalog Project was officially launched in June 2008. A major revision has been made available as of April 2009. To see the catalog on-line you can visit:

General information about the Metrics Catalog can be found in the following documents:

BEWARE: You will need a Javascript and Java enabled browser to optimally experience the content on these sites. Due to circumstances beyond our control, we cannot support any browser on Vista.

--Elizabeth Nichols, 3-July-2009

Logged in? Add a New entry to this blog!

January 10, 2007 3:27 PM
And So It Begins, With Small Saturated Spots
My publisher, Addison-Wesley, has recently updated the information on my book, Security Metrics: Replacing Fear, Uncertainty and Doubt on Amazon. Although I am particularly fond of the inside contents, I am also very pleased with the way the cover came out. AW's very talented Alan Clement put this great picture on the cover:

arj-frog.png

Isn't it nice? I like it not just because it's a great picture, but because it ties into the book's overall theme. Allow me to explain.

I am highly visual person. I have always believed that when you want to make a point, particularly to executives, you need to deliver it in a way that drives it home forcefully and clearly. Accompanying numbers with clear, crisp, honest graphics is an important way of doing this.

The great Edward Tufte, arguably the most influential information designer of the last 25 years, contends that the strategic application of small, saturated points of color in exhibits can draw attention to key points. His tastes run exactly opposite to the mainstream, as evidenced by many exhibits I see from reputable vendors. Most people's idea of a "slick exhibit" means three-dimensional bar charts clad in radioactive colors. But instead of drenching entire exhibits with leftover Sherwin Williams inventory, Tufte argues that in many cases exhibits should be relatively austere, save for those few key spots where you want to make your point. That is where you bust out the saturated color. In short, Tufte argues that restraint should rule. As a raging contrarian and visually driven person, I agree wholeheartedly.

Which brings me back to my book's cover. In the book itself, I have exercised fairly tight control over the presentation format of the exhibits, of which there are about 75. But the cover is something else again. My publisher wanted, for obvious reasons, to apply a common style to the cover that aligns it in with its other professional books. And of course I was happy to work with them on this. So when it came time to consider designs, AW described the overall aesthetic, and asked me for some general guidelines about cover art.

AW's professional books generally have a wildlife motif. I suggested that we look for an animal who camouflages itself or blends in with the natural surroundings, but also has some kind of small, dense, saturated marking or plumage. Alan sent me a dozen samples, which included tropical forests with wildly colored birds, an Arctic wolf in a snowstorm with shining yellow eyes, a tree owl with red eyes, a snake with unique markings, and several others. But the frog that you see above was far and away the most appealing. It was Alan's favorite, too.

Colors, security, metrics, outlier analysis, and book design — who knew they were all so related?

By Andrew Jaquith  Permalink
January 3, 2007 2:16 PM
SSL is a Concrete Sewer Pipe
My buddy Gunnar Peterson has recently been raging about the inadequacies of REST security, pointing out that RESTful folks who equate transport-level confidentiality (such as SSL provides) with "security" are only partly right. Gunnar makes some fairly involved references (Neal Stephenson) to make the point.

Of course, Gunnar is right.

When I speak with people about application security, I try to use a few snappy analogies to drive the point home. And with respect to the difference between transport-level security and message-level security, the analogy I use is to compare SSL to a concrete sewer pipe. You may not be able to break into it, but you sure as hell have no idea what's flowing through it.

By Andrew Jaquith  Permalink
January 1, 2007 11:12 PM
Coding in Anger

Last week's shutoff of this website's self-registration system was something I did with deep misgivings. I've always been a fan of keeping the Web as open as possible. I cannot stand soul-sucking, personally invasive registration processes like the New York Times website. However, my experience with a particularly persistent Italian vandal was instructive, and it got me thinking about the relationship between accountability and identity.

Some background. When you self-register on securitymetrics.org you supply a desired "wiki name", a full name, a desired login id, and optionally a e-mail address for password resets. We require the identifying information to associate specific activities (page edits, attachment uploads, login/log out events) with particular members. We do not verify the information, and we trust that the user is telling truth. Our Italian vandal decided to abuse our trust in him by attaching pr0n links to the front page. Cute.

The software we use here on securitymetrics.org has decent audit logs. It was a simple matter of identifying the offending user account. I know which user put the porn spam on the website, and I know when he did it. I also know when he logged in, what IP address he came from, and what he claimed his "real" name was. But although I've got a decent amount of forensic information available to me, what I don't have any idea of whether the person who did it supplied real information when he registered.

And therein lies the paradox. I don't want to keep personal information about members — but at the same time, I want to have some degree of assurance that people who choose to become members are real people who have serious intent. But there's no way to get any level of assurance about intent. After alll, as the New Yorker cartoon aptly put it, on the Internet no-one knows if you're a dog. Or just a jackass.

During the holiday break, I did a bit of thinking and exploration about how to address (note I do not say "solve") issues of identity and accountability, in the context of website registration. Because I am a co-author of the software we use to run this website (JSPWiki), I have a fair amount of freedom in coming up with possible enhancements.

One obvious way to address self-registration identity issue is to introduce a vetting system into the registration process. That is, when someone registers, it triggers a little workflow that requires me to do some investigation on the person. I already do this for the mailing list, so it would be a logical extension to do it for the wiki, too. This would solve the identity issue — successfully vetting someone would enable the administrator to have much higher confidence in their identity claims, albeit with some sacrifice

There's just one problem with this — I hate vetting people. It takes time to do, and I am always far, far behind.

A second approach is to not do anything special for registration, but moderate page changes. This, too, requires workflow. On the JSPWiki developer mailing lists, we've been discussing this option quite a bit, in combination with blacklists and anti-spam heuristics. This would help solve the accountability problem.

A third approach would be to accept third-party identities that you have a reasonable level of assurance in. Classic PKI (digital certificates) are a good example of third-party identities that you can inspect and choose to trust or not. But client-side digital certificates have deployment shortcomings. Very few people use them.

A promising alternative to client-side certificates is the new breed of digital identity architectures, many of which do not require a huge, monolithic corporate infrastructure to issue. I'm thinking mostly of OpenID and Microsoft's CardSpace specs. I really like what Kim Cameron has done with CardSpace; it takes a lot of the things that I like about Apple's Keychain (self-management, portability, simple user metaphors, ease-of-use) and applies it specifically to the issue of identity. CardSpace's InfoCards (I have always felt they should be called IdentityCards) are kind of like credit cards in your wallet. When you want to express a claim about your identity, you pick a card (any card!) and present it to the person who's asking.

What's nice about InfoCards is that, in theory, these are things you can create for yourself at a registrar (identity provider) of your choice. InfoCards also have good privacy controls — if you don't want a relying party (e.g., securitymetrics.org) to see your e-mail identity attribute, you don't have to release that information.

So, InfoCards have promise. But they use the WS-* XML standards for communication (think: big, hairy, complicated), and they require a client-side supplicant that allows users to navigate their InfoCards and present them when asked. It's nice to see that there's a Firefox InfoCard client, but there isn't one for Safari, and older versions of Windows are still left out in the cold. CardSpace will make its mark in time, but it is still early, methinks.

OpenID holds more promise for me. There are loads more implementations available (and several choices for Java libraries), and the mechanism that identity providers use to communicate with relying parties is simple and comprehensible by humans. It doesn't require special software because it relies on HTTP redirects to work. And best of all, the thing the identity is based on is something "my kind of people" all have: a website URL. Identity, essentially, boils down to an assertion of ownership over a URL. I like this because it's something I can verify easily. And by visiting your website, I can usually tell whether the person who owns that URL is my kind of people.

OpenID is cool. I got far enough into the evaluation process to do some reasonably serious interoperability testing with the SXIP and JanRain libraries. I mocked up a web server and got it to sucessfully accept identities from the Technorati and VeriSign OpenID services. But I hit a few snags.

Recall that the point of all of this fooling around is to figure out a way to balance privacy and authenticity. By "privacy", I mean that I do not want to ask users to disgorge too much personal information to me when they register. And correspondingly, I do not want the custodial obligation of having to store and safeguard any information they give me. The ideal implementation, therefore, would accept an OpenID identity when presented, dyamically collect the attributes we want (really, just the full name and websute URL) and pull them into our in-memory session, and flush them at the end of the session. In other words, the integrity of the attributes presented, combined with transience yields privacy. It's kind of like the front-desk guard I used to see when I consulted to the Massachussetts Department of Mental Health. He was a rehabilitated patient, but his years of illness and heavy treatment left him with no memory for faces at all. Despite the fact I'd visited DMH on dozens of occasions, every time I signed in he would ask "Have you been here before? Do you know where you are going?" Put another way, integrity of identity + dynamic attribute exchange protocols + enforced amnesia = privacy.

By "authenticity" I mean having reasonable assurance that the person on my website is not just who they say they are, but that I can also get some idea about their intentions (or what they might have been). OpenID meets both of these criteria... if I want to know something more about the person registering or posting on my website, I can just go and check 'em out by visiting their URL.

But, in my experiments I found that the attribute-exchange process needs work... I could not get VeriSign's or Technorati's identity provider to release to my relying website the attributes I wanted, namely my identity's full name and e-mail addresses. I determined that this was because neither of these identity providers support what the OpenID people call the "Simple Registration" profile aka SREG.

More on this later. Needless to say, I am encouraged by my progress so far. And regardless of the outcome of my investigations into InfoCard and OpenID, my JSPWiki workflow library development continues at a torrid pace.

Bottom line: once we have a decent workflow system in place, I'll open registrations back up. And longer term, we will have more some open identity system choices.

By Andrew Jaquith  Permalink


Weblog archives:
This site is not affillated with any organization, and the opinions expressed on this website are strictly those of the authors themselves.

To log in to the Securitymetrics.org website, create a profile first.

Attachments