Welcome to securitymetrics.org, a community website for security practitioners. Securitymetrics.org offers a community blogging service (this page) and a members-only
mailing list
. See the
Mailing List page for more details.
Announcing Metricon 5
Metricon 5 is the fifth annual conference dedicated to security metrics. It is a forum for presenting new approaches for measuring information security effectiveness, with a bias towards practical, specific approaches. Topics and presentations will be selected for their novelty and merit, and their potential to stimulate discussion.
With five years of organized conferences in the history books, this year's theme, appropriately, is Older But Wiser. Four years ago, presenters at the first Metricon discussed software security, benchmarking, identity management, enterprise case studies and many other topics. Since then, researchers and enterprises have continued to investigate new techniques. What have we learned? Given that we are trying to measure, measuring the security metrics field (and the success or failures of our own efforts) is also our responsibility.
The program is organized along three temporal perspectives:
- Metrics Past. Which metrics techniques from 2006 worked, and which did not? And how can knowledge of the past inform the present and future?
- Metrics Present. What is the state of the art as practiced today' by leading corporations, consultants and researchers?
- Metrics Future. What new strategies for measuring security will emerge in the future?
Metricon 5 will be a one-day event, Tuesday, August 10th, 2010, co-located with the 19th USENIX Security Symposium in Washington, DC (http://www.usenix.org/events/sec10/). Metricon will begin bright and early in the morning, continue through a catered lunch in meeting room, and extend into the evening with informal discussion. Attendance will be by invitation. Capacity is limited to 60 participants.
Program
| Time | Track |
| 0800–0900h | Breakfast |
| 0900–0930h | Andrew Jaquith, Forrester Research, Welcome address and ''Five Years of Security Metrics: A Look Back'' |
| 0930–1000h | Richard Seiersen, Kaiser Permanente, ''Practical Security Metrics in the 4th Dimension'' |
| 1000–1030h | RH Powell, Akamai, ''Weathering Storms in the Cloud: Analyzing Massive Distributed Denial of Service Attacks to Better Prepare for the Future'' |
| 1030–1100h | Morning break |
| 1100–1130h | John S Quarterman, Quarterman Creations/CREC at the UT Austin School of Business, ''Spam Reputation as Output Measure of Infosec'' |
| 1130–1200h | Gina Fisk, Los Alamos National Laboratories, ''Optimizing Performance Management using Adaptive Metrics, Fitness Functions, and the Balanced Score Card'' |
| 1200–1230h | Fabio Massacci, Universita' di Trento, ''Which is the Right Source for Vulnerability Studies? An Empirical Analysis on Mozilla Firefox'' |
| 1230–1345h | Lunch |
| 1345–1415h | Elizabeth Nichols, PlexLogic: Security Metrics, ''Security Metrics: What’s Hot and What’s Not'' |
| 1415–1445h | Laura Glowick, Federal Home Loan Bank of Boston, ''Enterprise Security Dashboard'' also: FHLB's metrics catalog |
| 1445–1515h | Afternoon break |
| 1515–1545h | Alex Hutton, Verizon Security Intelligence, ''Bridging Risk Modeling, Threat Modeling, and Operational Metrics With the VERIS Framework'' |
| 1545–1615h | Michael Smith, Fish Catchers Heavy Industries, ''Meta-Metrics: Building a Scorecard for the Evaluation of Security Management and Control Frameworks'' |
| 1615–1730 | Rump session: open-mic discussion of current research and topics of shared interest |
| 1730– | Beer! Sponsored by Blue Canopy |
Venue
Metricon 5 will be held at the Marriott Woodman Park Hotel, 2660 Woodley Road Northwest, Washington, DC, on August 10th, 2010. It is co-located with the USENIX Security 2010 Symposium.
Attendance
Attendance is by invitation only. If you would like to attend, send an e-mail to metricon5
at securitymetrics
dot org.
All participants will be expected to "come with findings" and be willing to contribute to group discussions. Politeness will be praised; questions, encouraged; lurkers, flushed out.
The proceedings of all past meetings are available here:
For speakers
- Deadline for final presentation: July 30th, 2010
Conference chairs
- Andrew Jaquith, Forrester Research
- Khalid Kark, Forrester Research
Program committee members
- Jennifer Bayuk, Stevens Institute of Technology
- Dan Geer, In-Q-Tel
- Chris Walsh, SurePayroll
- Wade Baker, Verizon Risk Intelligence
- Ray Kaplan, Ray Kaplan & Associates
- Michael Smith, Akamai Technologies
- Daniel Arista, Syracuse Research Corporation
Mini MetriCon 4.5
Mini MetriCon 4.5 was held Monday, March 1, 2010, in SanFrancisco, California, adjacent to the USA RSA 2010 Conference. The presentations are posted as embedded links in the
agenda
; the
original CFP
remains available as well.
MetriCon 4.0
MetriCon 4.0 was held Tuesday, August 11, 2009, in Montreal, Quebec, co-located with the
USENIX Security Symposium
. See the
MetriCon 4.0
page for the details of the meeting, including its CFP, the final agenda, and the meeting's Digest.
Mini MetriCon 3.5
Mini MetriCon 3.5 was held Monday, April 20, 2009, in SanFrancisco, California, adjacent to the USA RSA 2009 Conference. The
presentations are posted as embedded links in the
agenda
; the
original CFP
remains available as well. Sadly, no Digest was ever completed.
MetriCon 3.0
The
MetriCon 3.0 presentations and digest are available as attachments to the final
agenda
Mini MetriCon 2.5 Presentations
The
MiniMetriCon 2.5 presentations are available as attachments to the final
agenda
.
Metrics Catalog Project:
The
Metrics Catalog Project was officially launched in June 2008. A major revision has been made available as of April 2009. To see the catalog on-line you can visit:
General information about the Metrics Catalog can be found in the following documents:
BEWARE: You will need a Javascript and Java enabled browser to optimally experience the content on these sites. Due to circumstances beyond our control, we cannot support any browser on Vista.
--Elizabeth Nichols
, 3-July-2009
Logged in? Add a New entry to this blog!
November 29, 2005 3:18 PM
The Natives are Restless
Many readers know that my day job is as a security technology analyst for Yankee Group. Well, it's about that time of year where we start to wind down our research calendar. One of the things we're getting out the door is the 2005 Yankee Group Security Leaders and Laggards Survey, in which we ask a statistically relevant number of enterprises (500+) about their spending habits, preferred security suppliers, and future plans.
This year, we added a special set of questions designed to force choices between several competing alternatives. For example, we asked whether enterprises preferred to work with resellers, or direct with security vendors. Probably the most interesting (read: mischievous) question we asked was this one:
- Tell us what outcome is desirable: 1) backporting future Vista security features to older Windows versions (XP, 2000) versus. 2) Enterprise migration to Vista
Now, you might think that companies would be jazzed up about the security improvements Microsoft has promised for Vista, and that upgrading would be something companies would prefer to do. Our data shows exactly the opposite:
- 5% of customers found upgrading "extremely desirable": 12% said it was "desirable"
- 30% were "neutral"
- An even 26% said that backporting was either "desirable"; ditto with "extremely desirable"
But wait, there's more!
When we look only at what we call security "leaders" those companies that spend the highest percentage of their IT budgets on security the differences are even more pronounced. Fully 40% of Leaders felt that backporting was extremely desirable; after adding in the "desirable" percentage, the total favoring backporting is a sky-high 65%. That is a stunning number.
Now consider the additional fact that Vista won't run acceptably on hardware older than about a year. Consider also Joe Wilcox' observation that Microsoft has missed as hardware upgrade cycle
. When you put all of those things together, it tells me that customers don't want a forklift upgrade to a more secure operating system forced on them.
Does this just reflect common sense? Probably. I just didn't expect the numbers to work out quite this unequivocally.
November 14, 2005 11:14 AM
The Devil's Information Security Dictionary
Just saw the very funny Devil's InfoSec Dictionary
on the CSO site. Of course, I had to add a few definitions of my own:
- Blended threat
- a hemlock smoothie
- Process, Security Is A
- a throw-away line that explains why security measurement is impossible
- Risk management
- a repeated process around the Hamster Wheel of Pain that vendors use to enumerate vulnerabilities you didn't know you had, followed by serial remediation of same. See "remediation"
- Remediation
- furious arm-flapping and showy activity designed to convince bosses that something is actually being done about vulnerabilities identified by third parties
- Spear phishing
- a sport undertaken by illiterate anglers
November 9, 2005 11:24 PM
Making the wrong development choices
I hate to be a curmudgeon about this, but this fellow needs a beat-down:
Fixing AJAX: XmlHttpRequest Considered Harmful"
I offer this as exhibit A (as in AJAX) about why application security may well be intractable, in part because we've got mainstream technical outlets teaching techniques to evade well-founded security principles.
- AJAX applications wouldn't be possible (or, at least, wouldn't be nearly as cool) without the XMLHttpRequest object that lets your JavaScript application make GET, POST, and other types of HTTP requests from within the confines of a web browser.... But the kind of AJAX examples that you don't see very often (are there any?) are ones that access third-party web services, such as those from Amazon, Yahoo, Google, and eBay. That's because all the newest web browsers impose a significant security restriction on the use of XMLHttpRequest. That restriction is that you aren't allowed to make XMLHttpRequests to any server except the server where your web page came from... Too bad -- your application is on www.yourserver.com, but their web service is on webservices.amazon.com (for Amazon). The XMLHttpRequest will either fail or pop up warnings, depending on the browser you're using.
(quick cut to Andy spitting up his coffee in disbelief at what he thinks he's about to read)
- On Microsoft's IE 5 and 6, such requests are possible provided your browser security settings are low enough (though most users will still see a security warning that they have to accept before the request will proceed). On Firefox, Netscape, Safari, and the latest versions of Opera, the requests are denied...
(sounds like good security engineering to me... what's the problem?)
- There is hope, or rather, there are gruesome hacks, that can bring the splendor of seamless cross-browser XMLHttpRequests to your developer palette. The three methods currently in vogue are:
(Danger, Will Robinson! Glad I didn't refill the coffee cup...)
- Application proxies. Write an application in your favorite programming language that sits on your server, responds to XMLHttpRequests from users, makes the web service call, and sends the data back to users.
- Apache proxy. Adjust your Apache web server configuration so that XMLHttpRequests can be invisibly re-routed from your server to the target web service domain.
- Script tag hack with application proxy (doesn't use XMLHttpRequest at all). Use the HTML script tag to make a request to an application proxy (see #1 above) that returns your data wrapped in JavaScript. This approach is also known as On-Demand JavaScript.
- The basic idea of all three hacks is the same: fool your user's web browser into thinking that the data is coming from the same domain as the web page.
(Excellent. A good summation of potential threat vectors. But I can't believe we're about to read a serious discussion, signed-off by a serious publisher, about how to evade security protections.)
- A word of caution here: there is a good reason why XMLHttpRequests are restricted. Allowing them to freely access any domain from within a web page opens up users to potential security exploitation. Not surprisingly, these three hacks, which offload the request to your web server, potentially threaten to disparage your web server's identity, if not its contents. Caution is advised before deploying them.
(The obligatory "Kids, don't try this at home" message.... followed by the Snake River motorcycle jump.)
November 7, 2005 10:38 AM
Graphical Integrity, Part I
The folks at the NY Times have put together a nifty interactive graphic that diagrams the various data breach cases that have been disclosed since January. It breaks down when each incident occurred, and categorizes the incidents by industrial sector and geography of the disclosed parties:
Stolen Identites: data breaches since January 2005
The graphic is interesting for two reasons. First, there's the obvious one: it's gives us a sense of the problem, quantitatively speaking.
For example not surprisingly, financial services appears to have suffered the most: 46m, or 12% of the total account disclosures. With regard to type of account, most of the problems were related to credit cards, phone records and the like. E-mail/online payment accounts only comprised about 14% of the issues.
The second reason this graphic is interesting and this is, in fact, the real point of this post is the third graphic ("Case By Case"). The incidents are arranged horizontally, left-to-right, with shaded bubbles placed on the timelines at their respective dates. Cases with larger numbers of affected records have bigger bubbles. This is a pretty good time-series technique, in my opinion. Nice, but do you want to know what's even better than that?
Here's where our friends at the Times get all sorts of bonus points. As I mentioned, the bubbles are sized in proportion to the number of affected account records. But the bubble's size is NOT a simple linear mapping of records to diameter but to AREA. Thus, for a given number of records, the radius of the circle is sqrt( records / ( n x Pi ) ), where n is a scaling factor.
Per Edward Tufte
, scaling in proportion to area is a much more honest and appropriate technique to use than by diameter, due to the way the eye perceives such things. When @stake introduced its infamous 2x2 bubble chart back in early 2000, we scaled the "business impact" ratings in exactly the same way, using area scaling. We wanted issues scored a "5" to be perceived as 5x bigger than the "1" issues, not 25x.
On a side note, Tufte has been known to consult for the NY Times on occasion. This graphic would seem to have his fingerprints on it; I wonder if he had a hand in it. Anybody know?
November 1, 2005 11:13 PM
Fun with Spam
Collecting Hamster Wheels of Pain
is certainly a fun hobby. So is collecting the rather amusing e-mail addresses chosen by spammers to evade e-mail filters. Here are some good'uns from the 305 spam-grams from the past week:
- Amyroh (x2; aka the name of a talented Sun Java engineer; clearly mined from apache.org dev lists)
- brian@domain.com (x2)
- C. Michael Patterson, M.D. ("Dear valued patient: get the safest prescription at the lowest cost here.")
- Deandre Pryor ("Curious? Come explore")
- Donny Cervantes (""Inexpensive tablets here"... yes, but are the windmills cheap too?)
- drew.varner@oracle.com (subject: "Status", plus viral payload. It's not from Mary Ann...)
- Earthlink Customer Support (x100, self-congratulatory e-mails about blocked viruses; legitimate sender)
- eBay ("fraud alert", with link to a not-very-disguised Australian server)
- Elmo Glass (subject: "you travel as horsemeat", with inline image advertising diet/sex/memory pills)
- java2d-interest@sun.com (with Lovegate virus stripped out by Earthlink)
- jimmy@allaire.com (subject: "owmfj")
- Lyle Kramer (subject: "Hello Football Raunchy Blond Mom Gets Hardcore C**tbang")
- Micah Becker (subject: "Re: on begin to tether")
- Parker Hardy (admittedly this is very funny, at least for those of you who have a memory for late 70's primetime TV)
- Santos McGee (Wavmokqlq@optonline.com; advertising cheap mortgages; as if anyone with a brain would entrust financial matters with someone willing to disguise their identity. Assume it was a real name: interesting wedding that must have been...)
- Sherry Comer (subject: "That run an fondu")
- trebor.a.rude@lmco.com (Interesting. With my A&D background, I interpret this as "Trebor A. Rude at Lockheed Martin Corporation)
Yeeeeargh.
There aren't any obvious patterns here, other than to note that we've got plenty of evasion methods on display: random dictionary constructions, website impersonation, old-fashioned come-ons and white-page-concatenation.
From a metrics standpoint, over the one-week span I've received 305 spams in my e-mail inbox, plus another 512 blocked by my ISP (since October 28; which equates to about 1000 weekly), for a total of about 1300 spams per week. I have perhaps received (max) 100 legitimate e-mails in the same period, and probably closer to 50. That means my e-mail inbox is over 96% spam. Swell.
Weblog archives:
This site is not affillated with any organization, and the opinions expressed on this website are strictly those of the authors themselves.
To log in to the Securitymetrics.org website, create a profile
first.