Using security metrics to measure human awareness

Free tools offer security practitioners a way to measure the effectiveness of awareness programs

It's been said that security is hard to measure. Producing measurable results around a lack of problems or incidents is challenging. But the field of security metrics has evolved considerably in recent years, giving security managers more resources to make the case for investing in security programs and technologies.

Now the SANS Institute, through their Securing the Human Program, is offering a set of free metric tools designed to give security leaders the ability to track and measure the impact of their own security awareness programs.

[Security Metrics: Critical Issues]

According to Lance Spitzner, training director for the program, the tools can be used to improve training, demonstrate return on investment, or compare an organization's human risk to other organizations in an industry. All resources are free, developed by the community for the community, said Spitzner.

The tools include:

Metrics Matrix -- A spreadsheet that identifies and documents different options for measuring a security awareness program. It includes metrics for both measuring impact (change in behavior) and for tracking compliance.

Measuring Human Risk Survey -- The newest addition to the tools that is still in development, the twenty-five question survey helps determine the human risk in an organization. Each question and its respective answers have different levels of risk associated with them. Depending on how employees respond, answers can be totaled to determine a quantitative value of your human risk.

Phishing Assessments Planning Package -- Phishing assessments are not only a simple and effective way to measure the impact of your awareness program, but a very powerful way to reinforce key training concepts. This package helps you step by step plan, build and implement a successful phishing assessment program, including several templates, said Spitzner.

CSO spoke with Spitzner about using the metric tools.

CSO: What was the mission in creating these metric-gathering tools?

Spitzner: The tools were developed out of need by the security awareness community. I run a private mail list of about 200 professionals who are all involved in, or lead the security awareness program for their organization. People post what they are looking for, and then, we as a group develop resources that help solve that problem.

One of the first challenges we solved was creating the Security Awareness Maturity Model that helps identify how mature your awareness program is and then how you want to build on that. As a group we then developed the Security Awareness Roadmap that explains in detail how to reach each maturity level. There was a repeated request and need for metrics.

What are the challenges of using security awareness metrics?

As always there are several challenges with metrics, security awareness metrics are no different. A couple of points to keep in mind:

  • Ultimately, metrics are a tool used to measure the effectiveness of your security awareness program and how to improve it. Sometimes organizations get so caught up in their metrics that the metrics become more important then the program itself, they forget about what their ultimate goal is. As such the best approach is to focus only on a few, very good metrics.
  • Unfortunately good metrics are hard. They have to be easy to measure (preferably automated), they have to be measured consistently (in other words even if different people measure they get the same result) and they have to something you can take action on. Classic example of a bad metric is the top ten most infected countries. What value does that metric have? What action are you supposed to take based on that?

This is one of the reasons we developed the security awareness metrics matrix, it has a list of over 15 metrics organizations can choose from, depending on which metric has the most value to them.

What is different about awareness metrics from other types of security metrics?

You are attempting to measure the human element, specifically peoples' behaviors and awareness. Technology is bits and bytes, which can be easier to measure (number of attacks detected, number of ports scans blocked by the firewall, etc). The other challenge is root cause analysis. Quite often incidents are caused by humans but organizations do not realize it because they never do a root cause analysis.

The classic example is infected systems. If your security team did a root cause analysis of infected systems, they would most likely discover that the vast majority of infections are not a technical issue but a human issue. Unfortunately many organizations fail to do any type of root cause analysis of incidents, thus hiding the fact that the human is most often the issue, not technology.

Why are you passionate about this topic?

Because I passionately believe this is where we can have the greatest impact. In the past 15 years I've been in information security our community has focused almost entirely on using technology to secure technology, and we have gotten very good at it. As a result, most operating systems have become very difficult to hack into, except that we have done nothing to secure the human element, what I like to call the HumanOS. They HumanOS has never been trained, as result they have no firewall, they have all their services on by default, there is no patching. All the classic mistakes we made 15 years ago are still happening with people today. This is why the human has become the primary attack vector. By investing some basic resources into people you can have a tremendous impact in reducing risk, just like we have in other operating systems today.

What types of folks do you envision using the SANS tools and what kind of benefit do you hope it will provide?

Absolutely any organization can benefit from our free resources, and not just organizations by ordinary people in their personal lives. Think about it, about 70-80 percent of any security awareness program applies both to the organization and employees personal life; topics such as email, mobile devices, social networking or passwords. Our approach is not to just make people aware and change their behaviors at work, but change those same behaviors at home as they face the same risks. As such, security becomes part of their DNA.

Feedback on how to improve the awareness metric resources can be sent to

Join the CSO newsletter!

Error: Please check your email address.
Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Joan Goodchild

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts