Welcome
Welcome to securitymetrics.org, a community website for security practitioners. Securitymetrics.org offers a community blogging service (this page) and a members-only mailing list. See the Mailing List page for more details.

Announcing Metricon 5

Metricon 5 is the fifth annual conference dedicated to security metrics. It is a forum for presenting new approaches for measuring information security effectiveness, with a bias towards practical, specific approaches. Topics and presentations will be selected for their novelty and merit, and their potential to stimulate discussion.

With five years of organized conferences in the history books, this year's theme, appropriately, is Older But Wiser. Four years ago, presenters at the first Metricon discussed software security, benchmarking, identity management, enterprise case studies and many other topics. Since then, researchers and enterprises have continued to investigate new techniques. What have we learned? Given that we are trying to measure, measuring the security metrics field (and the success or failures of our own efforts) is also our responsibility.

The program is organized along three temporal perspectives:

  • Metrics Past. Which metrics techniques from 2006 worked, and which did not? And how can knowledge of the past inform the present and future?
  • Metrics Present. What is the state of the art as practiced today' by leading corporations, consultants and researchers?
  • Metrics Future. What new strategies for measuring security will emerge in the future?

Metricon 5 will be a one-day event, Tuesday, August 10th, 2010, co-located with the 19th USENIX Security Symposium in Washington, DC (http://www.usenix.org/events/sec10/). Metricon will begin bright and early in the morning, continue through a catered lunch in meeting room, and extend into the evening with informal discussion. Attendance will be by invitation. Capacity is limited to 60 participants.

Program

TimeTrack
0800–0900hBreakfast
0900–0930hAndrew Jaquith, Forrester Research, Welcome address and ''Five Years of Security Metrics: A Look Back''(info)
0930–1000hRichard Seiersen, Kaiser Permanente, ''Practical Security Metrics in the 4th Dimension''(info)
1000–1030hRH Powell, Akamai, ''Weathering Storms in the Cloud: Analyzing Massive Distributed Denial of Service Attacks to Better Prepare for the Future''(info)
1030–1100hMorning break
1100–1130hJohn S Quarterman, Quarterman Creations/CREC at the UT Austin School of Business, ''Spam Reputation as Output Measure of Infosec''(info)
1130–1200hGina Fisk, Los Alamos National Laboratories, ''Optimizing Performance Management using Adaptive Metrics, Fitness Functions, and the Balanced Score Card''(info)
1200–1230hFabio Massacci, Universita' di Trento, ''Which is the Right Source for Vulnerability Studies? An Empirical Analysis on Mozilla Firefox''(info)
1230–1345hLunch
1345–1415hElizabeth Nichols, PlexLogic: Security Metrics, ''Security Metrics: What’s Hot and What’s Not''(info)
1415–1445hLaura Glowick, Federal Home Loan Bank of Boston, ''Enterprise Security Dashboard''(info) also: FHLB's metrics catalog(info)
1445–1515hAfternoon break
1515–1545hAlex Hutton, Verizon Security Intelligence, ''Bridging Risk Modeling, Threat Modeling, and Operational Metrics With the VERIS Framework''(info)
1545–1615hMichael Smith, Fish Catchers Heavy Industries, ''Meta-Metrics: Building a Scorecard for the Evaluation of Security Management and Control Frameworks''(info)
1615–1730Rump session: open-mic discussion of current research and topics of shared interest
1730–Beer! Sponsored by Blue Canopy

Venue

Metricon 5 will be held at the Marriott Woodman Park Hotel, 2660 Woodley Road Northwest, Washington, DC, on August 10th, 2010. It is co-located with the USENIX Security 2010 Symposium.

Event Sponsors

BlueCanopy_Logo_04032010.png

Attendance

Attendance is by invitation only. If you would like to attend, send an e-mail to metricon5 at securitymetrics dot org.

All participants will be expected to "come with findings" and be willing to contribute to group discussions. Politeness will be praised; questions, encouraged; lurkers, flushed out.

The proceedings of all past meetings are available here:

For speakers

  • Deadline for final presentation: July 30th, 2010

Conference chairs

  • Andrew Jaquith, Forrester Research
  • Khalid Kark, Forrester Research

Program committee members

  • Jennifer Bayuk, Stevens Institute of Technology
  • Dan Geer, In-Q-Tel
  • Chris Walsh, SurePayroll
  • Wade Baker, Verizon Risk Intelligence
  • Ray Kaplan, Ray Kaplan & Associates
  • Michael Smith, Akamai Technologies
  • Daniel Arista, Syracuse Research Corporation

Mini MetriCon 4.5

Mini MetriCon 4.5 was held Monday, March 1, 2010, in SanFrancisco, California, adjacent to the USA RSA 2010 Conference. The presentations are posted as embedded links in the agenda; the original CFP remains available as well.

MetriCon 4.0

MetriCon 4.0 was held Tuesday, August 11, 2009, in Montreal, Quebec, co-located with the USENIX Security Symposium. See the MetriCon 4.0 page for the details of the meeting, including its CFP, the final agenda, and the meeting's Digest.

Mini MetriCon 3.5

Mini MetriCon 3.5 was held Monday, April 20, 2009, in SanFrancisco, California, adjacent to the USA RSA 2009 Conference. The presentations are posted as embedded links in the agenda; the original CFP remains available as well. Sadly, no Digest was ever completed.

MetriCon 3.0

The MetriCon 3.0 presentations and digest are available as attachments to the final agenda

Mini MetriCon 2.5 Presentations

The MiniMetriCon 2.5 presentations are available as attachments to the final agenda.


Metrics Catalog Project:

The Metrics Catalog Project was officially launched in June 2008. A major revision has been made available as of April 2009. To see the catalog on-line you can visit:

General information about the Metrics Catalog can be found in the following documents:

BEWARE: You will need a Javascript and Java enabled browser to optimally experience the content on these sites. Due to circumstances beyond our control, we cannot support any browser on Vista.

--Elizabeth Nichols, 3-July-2009

Logged in? Add a New entry to this blog!

November 29, 2005 3:18 PM
The Natives are Restless

Many readers know that my day job is as a security technology analyst for Yankee Group. Well, it's about that time of year where we start to wind down our research calendar. One of the things we're getting out the door is the 2005 Yankee Group Security Leaders and Laggards Survey, in which we ask a statistically relevant number of enterprises (500+) about their spending habits, preferred security suppliers, and future plans.

This year, we added a special set of questions designed to force choices between several competing alternatives. For example, we asked whether enterprises preferred to work with resellers, or direct with security vendors. Probably the most interesting (read: mischievous) question we asked was this one:

Tell us what outcome is desirable: 1) backporting future Vista security features to older Windows versions (XP, 2000) versus. 2) Enterprise migration to Vista

Now, you might think that companies would be jazzed up about the security improvements Microsoft has promised for Vista, and that upgrading would be something companies would prefer to do. Our data shows exactly the opposite:

  • 5% of customers found upgrading "extremely desirable": 12% said it was "desirable"
  • 30% were "neutral"
  • An even 26% said that backporting was either "desirable"; ditto with "extremely desirable"

But wait, there's more!

When we look only at what we call security "leaders" — those companies that spend the highest percentage of their IT budgets on security — the differences are even more pronounced. Fully 40% of Leaders felt that backporting was extremely desirable; after adding in the "desirable" percentage, the total favoring backporting is a sky-high 65%. That is a stunning number.

Now consider the additional fact that Vista won't run acceptably on hardware older than about a year. Consider also Joe Wilcox' observation that Microsoft has missed as hardware upgrade cycle. When you put all of those things together, it tells me that customers don't want a forklift upgrade to a more secure operating system forced on them.

Does this just reflect common sense? Probably. I just didn't expect the numbers to work out quite this unequivocally.

By AnonymousCoward  Permalink
November 14, 2005 11:14 AM
The Devil's Information Security Dictionary

Just saw the very funny Devil's InfoSec Dictionary on the CSO site. Of course, I had to add a few definitions of my own:

Blended threat
a hemlock smoothie
Process, Security Is A
a throw-away line that explains why security measurement is impossible
Risk management
a repeated process around the Hamster Wheel of Pain that vendors use to enumerate vulnerabilities you didn't know you had, followed by serial remediation of same. See "remediation"
Remediation
furious arm-flapping and showy activity designed to convince bosses that something is actually being done about vulnerabilities identified by third parties
Spear phishing
a sport undertaken by illiterate anglers
By AnonymousCoward  Permalink
November 9, 2005 11:24 PM
Making the wrong development choices

I hate to be a curmudgeon about this, but this fellow needs a beat-down:

Fixing AJAX: XmlHttpRequest Considered Harmful"

I offer this as exhibit A (as in AJAX) about why application security may well be intractable, in part because we've got mainstream technical outlets teaching techniques to evade well-founded security principles.

AJAX applications wouldn't be possible (or, at least, wouldn't be nearly as cool) without the XMLHttpRequest object that lets your JavaScript application make GET, POST, and other types of HTTP requests from within the confines of a web browser.... But the kind of AJAX examples that you don't see very often (are there any?) are ones that access third-party web services, such as those from Amazon, Yahoo, Google, and eBay. That's because all the newest web browsers impose a significant security restriction on the use of XMLHttpRequest. That restriction is that you aren't allowed to make XMLHttpRequests to any server except the server where your web page came from... Too bad -- your application is on www.yourserver.com, but their web service is on webservices.amazon.com (for Amazon). The XMLHttpRequest will either fail or pop up warnings, depending on the browser you're using.

(quick cut to Andy spitting up his coffee in disbelief at what he thinks he's about to read)

On Microsoft's IE 5 and 6, such requests are possible provided your browser security settings are low enough (though most users will still see a security warning that they have to accept before the request will proceed). On Firefox, Netscape, Safari, and the latest versions of Opera, the requests are denied...

(sounds like good security engineering to me... what's the problem?)

There is hope, or rather, there are gruesome hacks, that can bring the splendor of seamless cross-browser XMLHttpRequests to your developer palette. The three methods currently in vogue are:

(Danger, Will Robinson! Glad I didn't refill the coffee cup...)

Application proxies. Write an application in your favorite programming language that sits on your server, responds to XMLHttpRequests from users, makes the web service call, and sends the data back to users.
Apache proxy. Adjust your Apache web server configuration so that XMLHttpRequests can be invisibly re-routed from your server to the target web service domain.
Script tag hack with application proxy (doesn't use XMLHttpRequest at all). Use the HTML script tag to make a request to an application proxy (see #1 above) that returns your data wrapped in JavaScript. This approach is also known as On-Demand JavaScript.
The basic idea of all three hacks is the same: fool your user's web browser into thinking that the data is coming from the same domain as the web page.

(Excellent. A good summation of potential threat vectors. But I can't believe we're about to read a serious discussion, signed-off by a serious publisher, about how to evade security protections.)

A word of caution here: there is a good reason why XMLHttpRequests are restricted. Allowing them to freely access any domain from within a web page opens up users to potential security exploitation. Not surprisingly, these three hacks, which offload the request to your web server, potentially threaten to disparage your web server's identity, if not its contents. Caution is advised before deploying them.

(The obligatory "Kids, don't try this at home" message.... followed by the Snake River motorcycle jump.)

By AnonymousCoward  Permalink
November 7, 2005 10:38 AM
Graphical Integrity, Part I

The folks at the NY Times have put together a nifty interactive graphic that diagrams the various data breach cases that have been disclosed since January. It breaks down when each incident occurred, and categorizes the incidents by industrial sector and geography of the disclosed parties:

Stolen Identites: data breaches since January 2005

The graphic is interesting for two reasons. First, there's the obvious one: it's gives us a sense of the problem, quantitatively speaking.

For example — not surprisingly, financial services appears to have suffered the most: 46m, or 12% of the total account disclosures. With regard to type of account, most of the problems were related to credit cards, phone records and the like. E-mail/online payment accounts only comprised about 14% of the issues.

The second reason this graphic is interesting — and this is, in fact, the real point of this post — is the third graphic ("Case By Case"). The incidents are arranged horizontally, left-to-right, with shaded bubbles placed on the timelines at their respective dates. Cases with larger numbers of affected records have bigger bubbles. This is a pretty good time-series technique, in my opinion. Nice, but do you want to know what's even better than that?

Here's where our friends at the Times get all sorts of bonus points. As I mentioned, the bubbles are sized in proportion to the number of affected account records. But the bubble's size is NOT a simple linear mapping of records to diameter — but to AREA. Thus, for a given number of records, the radius of the circle is sqrt( records / ( n x Pi ) ), where n is a scaling factor.

Per Edward Tufte, scaling in proportion to area is a much more honest and appropriate technique to use than by diameter, due to the way the eye perceives such things. When @stake introduced its infamous 2x2 bubble chart back in early 2000, we scaled the "business impact" ratings in exactly the same way, using area scaling. We wanted issues scored a "5" to be perceived as 5x bigger than the "1" issues, not 25x.

On a side note, Tufte has been known to consult for the NY Times on occasion. This graphic would seem to have his fingerprints on it; I wonder if he had a hand in it. Anybody know?

By AnonymousCoward  Permalink
November 1, 2005 11:13 PM
Fun with Spam

Collecting Hamster Wheels of Pain is certainly a fun hobby. So is collecting the rather amusing e-mail addresses chosen by spammers to evade e-mail filters. Here are some good'uns from the 305 spam-grams from the past week:

  • Amyroh (x2; aka the name of a talented Sun Java engineer; clearly mined from apache.org dev lists)
  • brian@domain.com (x2)
  • C. Michael Patterson, M.D. ("Dear valued patient: get the safest prescription at the lowest cost here.")
  • Deandre Pryor ("Curious? Come explore")
  • Donny Cervantes (""Inexpensive tablets here"... yes, but are the windmills cheap too?)
  • drew.varner@oracle.com (subject: "Status", plus viral payload. It's not from Mary Ann...)
  • Earthlink Customer Support (x100, self-congratulatory e-mails about blocked viruses; legitimate sender)
  • eBay ("fraud alert", with link to a not-very-disguised Australian server)
  • Elmo Glass (subject: "you travel as horsemeat", with inline image advertising diet/sex/memory pills)
  • java2d-interest@sun.com (with Lovegate virus stripped out by Earthlink)
  • jimmy@allaire.com (subject: "owmfj")
  • Lyle Kramer (subject: "Hello Football Raunchy Blond Mom Gets Hardcore C**tbang")
  • Micah Becker (subject: "Re: on begin to tether")
  • Parker Hardy (admittedly this is very funny, at least for those of you who have a memory for late 70's primetime TV)
  • Santos McGee (Wavmokqlq@optonline.com; advertising cheap mortgages; as if anyone with a brain would entrust financial matters with someone willing to disguise their identity. Assume it was a real name: interesting wedding that must have been...)
  • Sherry Comer (subject: "That run an fondu")
  • trebor.a.rude@lmco.com (Interesting. With my A&D background, I interpret this as "Trebor A. Rude at Lockheed Martin Corporation)

Yeeeeargh.

There aren't any obvious patterns here, other than to note that we've got plenty of evasion methods on display: random dictionary constructions, website impersonation, old-fashioned come-ons and white-page-concatenation.

From a metrics standpoint, over the one-week span I've received 305 spams in my e-mail inbox, plus another 512 blocked by my ISP (since October 28; which equates to about 1000 weekly), for a total of about 1300 spams per week. I have perhaps received (max) 100 legitimate e-mails in the same period, and probably closer to 50. That means my e-mail inbox is over 96% spam. Swell.

By AnonymousCoward  Permalink


Weblog archives:
This site is not affillated with any organization, and the opinions expressed on this website are strictly those of the authors themselves.

To log in to the Securitymetrics.org website, create a profile first.

Attachments