4 security metrics that matter

Today, management demands metrics to get a clearer view of security. Here are four metrics that deliver actionable insight -- and a few others with less value

As security gains greater visibility in boardrooms and C-suites, security professionals are increasingly asked to provide metrics to track the current state of a company's defenses. But which numbers really matter?

More often than not, senior management doesn't know what kind of questions it should be asking -- and may concentrate too much on prevention and too little on mitigation. Metrics like the mean cost to respond to an incident or the number of attacks stopped by the firewall seem reasonable to a nonsecurity person, but they don't really advance an organization's security program.

Instead, experts recommend focusing on metrics that influence behavior or change strategy.

"What would you do differently now that you have this metric?" asks Caroline Wong, security initiative director at Cigital, a security software and consulting firm. Metrics like mean cost to mitigate vulnerabilities and mean time to patch are helpful if the organization has mature and highly optimized processes, but that doesn't apply to 95 percent of organizations today, she said.

Metrics that measure participation, effectiveness, and window of exposure, however, offer information the organization can use to make plans and improve programs.

Security metric No. 1: Program participation levels

Participation metrics look at coverage within the organization. They may measure how many business units regularly conduct penetration testing or how many endpoints are currently being updated by automated patching systems. According to Wong, this basic information helps organizations assess security control adoption levels and identify potential gaps.

For example, while it would be nice to be able to say an organization has 100 percent of its systems patched within a month of new updates being available, that isn't a realistic goal because patching may introduce operational risk to some systems. Looking at participation helps exclude systems that don't fall under the normal patching rules -- and focuses attention on those that should be patched.

Security metric No. 2: Duration of attack

Dwell time, or how long an attacker is in the network, also delivers valuable insight. Attack duration information helps security pros prepare for, contain, and control threats, as well as minimize damage.

Surveys have shown attackers spend several months on average inside a company's network before being discovered. They spend the time learning the infrastructure, performing reconnaissance activities, moving around the network, and stealing information.

The goal should be to reduce dwell time as much as possible, so the attacker has less opportunity to achieve lateral movement and remove critical data, Douglas said. Knowing dwell time helps security teams figure out how to handle vulnerability mitigation and incident response.

"The longer attackers are in your network, the more information they can obtain, and the more damage they can inflict," Douglas said.

Security metric No. 3: Code defect density

Defect density, or the number of issues found in every thousand (or million, depending on the codebase) lines of code, helps organizations assess the security practices of its development teams.

Context is key, however. If an application is at an early stage of development, then a high defect density means all the issues are being found. That's good. On the other hand, if an application is in maintenance mode, the defect density should be lower -- and trending downward -- to show the application is getting more secure over time. If not, there's a problem.

Security metric No. 4: Windows of exposure

An organization may identify defects in the application, but until they've been addressed, the application remains vulnerable. The window of exposure looks at how many days in a year an application remains vulnerable to known serious exploits and issues. The "goal is to have zero days in a year during which serious defects found are known and have not yet been addressed," Wong said.

Misleading indicators

Management in general likes to focus on security incident prevention, in part due to the legacy notion that organizations can stop all attacks at the perimeter. For example, it might make everyone feel good to see the number of intrusion attempts that were blocked, but there's nothing actionable about that information -- it won't help security teams figure out which attacks were not blocked. "You're not fixing anything," says Joshua Douglas, CTO of Raytheon/Websense.

Mean response time, or how quickly the issue was found and mitigated, is another metric that may be less than helpful. Response time ignores the fact that attackers tend to move laterally through the network. You may fix one issue, but if no one tries to determine what else the attacker may have done, a different system compromised by that same attacker may go unnoticed. Focusing on individual issues alone and not on security as a whole leaves environments vulnerable.

"It's not one and done, it's one and understand," Douglas said.

Another common metric tracked is reduction in vulnerabilities, but it isn't so useful on its own. If a lot of low-level vulnerabilities have been fixed, the organization's risk remains the same while critical issues remain open. Some vulnerabilities mean more than others.

Only 28 percent of executives surveyed in a recent Raytheon/Websense survey felt the security metrics used in their organizations were "completely effective," compared to the 65 percent who felt they were "somewhat effective." Security practitioners need to explain to senior management how to focus on security questions that help accomplish well-defined goals. Otherwise, too much attention is wasted on information that doesn't actually reduce risk or improve security.

"Is that really the best place for you to be spending your limited time and money?" asks Wong.

Join the CSO newsletter!

Error: Please check your email address.

More about Websense

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Fahmida Y. Rashid

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts

Market Place