Securitymetrics.org

Measuring security effectiveness.

welcome

Welcome to securitymetrics.org, a community website for security practitioners. Securitymetrics.org offers a community blog (this website) and a members-only mailing list.

topics

aggregation · benchmarking · catalog project · definitions · empirical studies · metricon · modeling · ROI · visualization

contributing

Review the proceedings from the Metricon 8 conference, which was held on March 1st, 2013 at the RSA Conference in San Francisco.

Join the mailing list.

Mini-Metricon 4.5

- - posted in metricon | Comments

Mini-Metricon 4.5 was held Monday, March 1, 2010, in San Francisco, California, adjacent to the USA RSA 2010 Conference. The presentations are posted links in the this page; the original CFP is here as well.

Metrics Catalog Project

- - posted in catalog | Comments

This page provides information on the Metrics Catalog Project that was announced at the MiniMetricon 2.5 Meeting in SanFrancisco, CA on 7 April 2008.

There are two documents on the Metrics Catalog available at this time:

You can find more documents at the MetricsCenter website. The Metrics Catalog Project consists of three primary components:

  • MetricsCenter Google Group. You can subscribe by sending a request to support@metricscenter.net. Along with your request to join, please provide a bit of background about yourself and your interest in the security metrics. As of June 2008, there are approximately 100 individual participating in this community.
  • MetricsCenter.org website. MetricsCenter hosts the catalog. A preview site is up and running now
  • Securitymetrics.org Web Site for posting news and information about the project

The following paragraphs describe each of the above.

Security Metrics Catalog Overview

The Security Metrics Catalog is an open, public catalog for storing, organizing and sharing metrics definitions. It is one of several free services that is hosted at MetricsCenter.

The catalog is based on open source technology and is based upon a metrics management platform developed by PlexLogic, LLC.

The catalog supports the following features:

  • Public Metrics Catalog: A database of structured and unstructured information that completely and unambiguously defines a metric.
  • Catalog Explorer: A web UI that allows one to navigate the set of stored metric definitions
  • Metric Editor: A web UI that allows one to submit a new metric definition or propose a change to an existing one.
  • Metric Versioning: A function that tracks changes to metric definitions and supports a workflow that takes a metric from initial proposed inclusion in the catalog, through reviews, revisions, approval, and publication—followed by periodic updates.
  • Catalog Search: Structured search via contexts and unstructured Google-like search based upon the words used to describe the metric. In addition on can edit associations between metrics and “nodes” within context hierarchies.
  • Metric Rating: Users can assign a rating to a metric and the catalog will compute an overall score that is displayed as zero to five stars (like NetFlix movie ratings)
  • Metric Licensing: In the event that a contributor wishes to treat the metric definition as intellectual property whose usage is governed by one of the widely-used open source licenses, this can be specified as part of the metric definition.

The Catalog contains two primary objects: Metric Defintions and Contexts.

Metric Definitions

Metric Definitions, sometimes called simply Metrics, are a collection of named attributes that are designed to provide a complete and unambiguous specification for a Metric. Ideally, these attributes could be handed to two implementers who would develop code that would yield identical results. In addition to this, the metric definition can provide guidance and use cases for the metric. This includes success stories, unexpected side effects and interjpretation of results. This is what we mean by “complete and unambiguous” specification.

Contexts

Contexts are hierarchies of topics that are typically (but not necessarily) business oriented. A context can be:

  • A regulation, e.g. SOX or HIPAA
  • An industry requirement, e.g. PCI
  • A standard, e.g. ISO 27002-5
  • A best practice, e.g. ITIL or COBIT or CISWG
  • A functional de-composition of a process
  • Or almost anything else that is of general utility

Documents that describe various aspects of the MetricsCatalog are regularly published and provided on the Securitymetrics.org and the MetricsCenter.org websites.

MetricsCenter

MetricsCenter.org is the website that hosts the public Metrics Catalog. Some introductory information about the site—how to use it, what works now, what is planned, and specific requests for feedback—can be found on the Catalog Preview Page.

PlexLogic developed the software for MetricsCenter™ and is the founding lead for the SecurityMetrics.org Catalog Project. By contributing some of its resources to the creation and initial population of a Security Metrics Catalog, PlexLogic hopes to kick-start the process of identifying and defining a common repository of practical and useful metrics for the purposes of corporate governance, risk and compliance management.

In addition to working on the Metrics Center, PlexLogic provides additional services in the area of metrics. Visit http://www.plexlogic.com for more information. You can contact PlexLogic at info@plexlogic.com.

Metrics Catalog Preview

- - posted in catalog | Comments

A free and open site for the Metrics Catalog is up and running for your review and comment. You will need a browser with Javascript and Java enabled to view the Metrics Catalog.

Note that this web site is designed to provide three services:

  • A catalog of metric definitions (no measured results)
  • Dashboards of metric results derived from public sources and
  • A collection of useful resources for security metrics.//

Please provide feedback/suggestions about each of these services. If you are more interested in one, then don’t feel any obligation to look at or comment on the other.

The forum for comments is the MetricsCenter Google Group. A few key points, as you explore the MetricsCenter.org site:

  • You can look but not change metric definitions. If you want to create your own catalog or modify the definitions of existing metrics in the catalog, you will need to obtain a free trial account on www.metricscenter.net.
  • While you can look at the catalog, resources and dashboards without logging in, you will need to log in to edit metric definitions, create surveys to collect metric results. and compose your own dashboards
  • In the MetricsCatalog UI, you need to double-click on a metric that is listed in order to zoom into its full definition.

The whole reason we are doing this is to get your feedback on the utility of such a site. We want feedback earlier rather than later. We also want some indication from you—our intended audience—that this effort is worthwhile. Comments, reactions, emails are all signs that we are doing something that has value—or, that people care enough to review and suggest improvement.

Enjoy and please provide your feedback. We will listen, I promise.

Elizabeth A. Nichols, Ph.D., CTO for Metrics, PlexLogic