Benchmarking generally refers to the process of ranking or scoring security against an established standard measure. Benchmarks can be absolute or cross-sectional.
Comparative Application Security
- The Security of Applications: Not All Are Created Equal (February 2002), Andrew Jaquith. This study examples the security practices of 45 web applications, and finds that the most secure e-business applications have one-quarter as many security defects as the worst — and eighty percent less risk.
Benchmarking Goodness Criteria
Established by the DBench Project.
|Representativeness||how well inputs like workloads corresponds to real system characteristics|
|Repeatability||statistically equivalent results when run multilple times in the same environment|
|Reproducability||degree to which another party obtains statistically equivalent results when the benchmark is implemented from the same specifications|
|Portability||range of target systems to which benchmark specification applies to allow comparision|
|Non-Intrusiveness||requires minimum changes to target system and does not affect results|
|Scalability||ability to evaluate systems of different sizes|
|Time||time required to obtain the result|
|Cost||cost required to obtain result compared to value|
Contributed by Sami Saydjari