How to get CVSS right

For anyone dealing with software vulnerabilities, the CVE and CVSS are often their first stops in finding out the scope and details, and just about everything else they need to know about the specific vulnerability.

Launched in 2007, the Common Vulnerability Scoring System (CVSS) is a free and open industry standard for assessing the severity of computer system security vulnerabilities. Currently in version 2, with an update in version 3 in development, CVSS attempts to establish a measure of how much concern a vulnerability warrants, compared to other vulnerabilities, so efforts can be prioritized. The scores are based on a series of measurements, called metrics. The scores range from 0 to 10. High vulnerabilities are those with a base score in the range 7.0-10.0, medium in 4.0-6.9 and 0-3.9 are low.

Most commercial vulnerability management tools use CVSS as a baseline. In turn, enterprises will often base much of their vulnerability management programs on the CVSS score. CVSS can be a worthwhile way to quickly prioritize and identify vulnerabilities. But that speed comes at the cost of customization.

While CVSS can be a powerful indicator, it like all generic values is generalized. For the best efficacy, it needs to be customized to the specific entity using it. But the reality is that most organizations don't do that. They will simply use the information from Rapid7, Qualys, and Tenable without tailoring it to their specific risks and environment.

For example, security analytics firm Rapid7 is upfront when it notes that base CVSS metrics measure only the potential risk (likelihood plus impact) of a given vulnerability, not requiring temporal or environmental metrics to calculate its score. As such, base metric CVSS scores do not consider the whole context of the identified vulnerability to the organization.

Strictly speaking, CVSS doesn't actually represent likelihood of an event. It only represents the likelihood of compromise if attacked.

I attended a talk at the recent Infosec World conference on risks associated with security investments, presented by Jack Jones, President of CXOWare and co-author of Measuring and Managing Information Risk: A FAIR Approach. In the talk, Jones (an outspoken critic of CVSS) mentioned parenthetically that CVSS has potential but is poorly understood and inappropriately used by most organizations that rely on it.

Jones is not the only CVSS critic. Carsten Eiram and Brian Martin wrote an open letter to FIRST on CVSS shortcomings, faults and failures in formulation, while Patrick Toomey, formerly of Neohapsis, writes that CVSS over-complicates the issue of assigning risk to vulnerabilities.

Another issue with CVSS is that it is often used as a scoring mechanism for vulnerabilities, which in turn is combined with risk measurement. The result is that resources are wasted and organizations can't identify and focus on their most important problems.

Jones' main concern with CVSS involves its use of weighted values. The rationale behind the weighted values in CVSS is not documented, so you are using values that someone else has made up for you without understanding their basis. In Jones' experience, weighted values are rarely developed with significant rigor and are often tightly coupled to a set of assumptions that may only apply to specific conditions. He also believes that they introduce ambiguity, limit the scope of where the analysis can be applied, and can in some cases completely invalidate results. At the very least, if weighted values are going to be used, some well-reasoned rationale should be provided so that users can make an informed choice about whether they agree with the weighted values.

CVSS like all statistical indicators is only ever as good as its design and implementation. In an interesting new book, Statistics Done Wrong: The Woefully Complete Guide, author Alex Reinhart notes that statistical analysis is tricky to get right, even for the best and brightest. It's also surprising how many scientists are doing it wrong. His words need to be heeded for anyone using the CVSS.

To fix that, the CVSS calculator lets you refine the CVSS base score as you see fit for your organization. But my experience is that most organizations will use the standard CVSS weights that the vendor defined rather than customizing it for themselves. The truth is that each organization needs to determine its own weights and values rather than rely on best practices. If that's too much to undertake, then it should start with customizing the temporal and environmental factors as directed in the CVSS standard, and then it can worry about evaluating the weights later.

Getting CVSS right

CVSS can be a powerful tool that can provide a lot of value, and for those needing a quick, dirty, and generally effective rough scoring mechanism for vulnerabilities, it certainly fits the bill. But quick and dirty information security should be the rare exception rather than the rule. Vulnerability management should be customized for each organization. Generic best practices may work, but they won't be optimized.

With that, consider the following in order to make CVSS usage more effective:

  • understand the organization's approach to risk. Only then it is possible to make sense of CVSS and tie it into a vulnerability management program.
  • determine the organization's loss exposure. Ultimately the resources and impact associated with patching and fixing deficiencies have to be justified by its reduction on loss exposure not just whether it's the worst deficiency. Focus on business impact. For example, an easily exploitable vulnerability found on a web-facing system with access to sensitive data should probably be handled more urgently than the same vulnerability found internally with no external exposure and only limited, if any, internal exposure.
  • it's critical that the organization does not rely on generic CVSS results; rather, customize the temporal and environmental factors in order to get a complete score.
  • what if the company has a vulnerability with a high CVSS score with no exploit for it, but also another vulnerability with a lower CVSS score that does have an exploit. Which takes preference?

The more an organization can customize CVSS to its vulnerability management program, the better it will be. With the CVSS, mileage may indeed vary. CVSS off the shelf' is OK, but with limited mileage. CVSS customized' is useful and will allow companies to maximize their mileage as best they can.

Ben Rothke CISSP is a Senior eGRC Consultant with Nettitude, Inc. and the author of Computer Security: 20 Things Every Employee Should Know.

Join the CSO newsletter!

Error: Please check your email address.

Tags vulnerability managementCVSsecurityapplication securityAccess control and authentication

More about FIRSTInc.indeedNeohapsisQualysRapid7Tenable

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Ben Rothke

Latest Videos

  • 150x50

    CSO Webinar: Will your data protection strategy be enough when disaster strikes?

    Speakers: - Paul O’Connor, Engagement leader - Performance Audit Group, Victorian Auditor-General’s Office (VAGO) - Nigel Phair, Managing Director, Centre for Internet Safety - Joshua Stenhouse, Technical Evangelist, Zerto - Anthony Caruana, CSO MC & Moderator

    Play Video

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

More videos

Blog Posts

Market Place