Opinion: Risk Assessments Are Not Worth the Risk

Why spend money implementing mitigations where there is unlikely to be any risk?

For years, professionals of the information security industry have been advising and using risk-based approaches to securing organisations and their information assets.  This has been the received wisdom for so long that this is now encompassed in industry standards, such as ISO 27001, FIPS 200, etc.  This seems to make sense – there is no point in spending money implementing mitigations where there is unlikely to be any risk.  However, is this the right approach?

A paper published in November 2010, Understanding and Managing Risk in Security Systems for the DOE Nuclear Weapons Complex , may have some new insights into the problem.  Although the research performed was based around physical security, and nuclear weapons, the approach taken was remarkably similar to that taken by the information security industry: Probability-based Risk Assessment as the basis of security design.  The recommendations were stark:

The committee advises against the use of probabilistic risk assessment (PRA) in designing security for the DOE nuclear weapons complex at this time.

The problems with using probability to model security include:

• To prepare a risk assessment correctly, the assumption is made that the likelihood or impact of an event (or threat occurrence) happening in the past can be used to predict the likelihood or impact of the event in the future.  This requires that there is a statistically valid history of such events is available for such analysis.  As no two organisations have exactly the same exposure to threats, or the same impacts resulting from those threat occurrences (events), it is impossible to build such a history that is statistically valid.  Consequently, likelihood and impact are subjectivity compiled by analysts and the stakeholders providing input.

• Such risk models break down for the boundary state of low likelihood and high impact.  By definition, low likelihood events occur so infrequently that they do not occur often enough for their likelihood to be predicted accurately, and their impact be objectively predicted.  The ratings from a risk assessment in these cases often lead to weak mitigations that do not protect the organisation when such an event occurs.

• Risk assessments assume that each event occurs singularly.  The impact of several low likelihood and low impact events occurring within a very short or the same time is not examined.  However, the impact of such event together may be greater than their sum, and become a major impact to the organisation.

• Probability-based risk assessments do not work for malicious environments.  An attacker is looking for weaknesses, and is not going to avoid attacking a weakness because the likelihood of the event occurring is assessed by an organisation as low.  Consequently, the attacker is able to “manipulate” likelihood ratings to benefit the success of the attack.

Brian Snow, the ex-technical director of information assurance for the National Security Agency in the US, was on the report’s committee.  He expanded on the findings of the report for the information security industry in an interview on .  He stated, “The bad guy does not act on probability, he acts on intent”

However, let us not throw the baby out with the bathwater.  ISO 31010:2009, the international standard for risk assessments techniques does have some validity in its approach, even if we are no longer interested in the final “risk result”.  The standard calls for a tight scoping of the area under examination (called “context” in the standard).  It also requires that the risks are clearly identified through a formal methodology.

These two steps allow the analyst to start thinking as an attacker: what is available to attack, and how can I do it?  Once this threat-based analysis approach is taken, an organisation has a chance to appropriately defend against malicious attack.

Neither should we throw out the formal governance around information security.  The ISO 27001 requirement for an Information Security Management System (ISMS), a framework of documents and procedures to define and continuously improve the security posture of an organisation is always going to be important.  The inputs of the governance processes should be adjusted from risk assessment to threat based analysis.

Charles Wale CISSP, CISA, CRISC, QSA is a freelance information security consultant with 15 years’ experience in the industry.  He is a Director of Lee Douglas  & Associates.

Read more industry opinion pieces:

Is your data safe and sound?

Information Security - Shaping the Future

Fighting the botnet threat

Mark Ames discusses "Value for Policy"

Breadth first hacking

Join the CSO newsletter!

Error: Please check your email address.

Tags mitigationsinformation securityrisk managementphysical securityInformation Security Management System (ISMS)threats

More about IPSISONational Security AgencySecurity Systems

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Charles Wale

Latest Videos

  • 150x50

    CSO Webinar: Will your data protection strategy be enough when disaster strikes?

    Speakers: - Paul O’Connor, Engagement leader - Performance Audit Group, Victorian Auditor-General’s Office (VAGO) - Nigel Phair, Managing Director, Centre for Internet Safety - Joshua Stenhouse, Technical Evangelist, Zerto - Anthony Caruana, CSO MC & Moderator

    Play Video

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

More videos

Blog Posts

Market Place