Big data security context

I just finished up a lengthy tour through Latin America and Asia, as described in many of my latest blogs. Most recently I was in Australia and New Zealand (ANZ). I had the opportunity to work with various government agencies, organizations within critical infrastructure and general enterprise businesses across ANZ. Their primary topic of interest: big data. More specifically, they were interested in determining what needs to be part of a successful big data security strategy.

Years ago some organizations throughout ANZ viewed cyber security in the same way they viewed physical security in response to nation-state threats. Because ANZ has a land and sea gap physically separating them from other countries, there was a feeling of separation and protection from the nefarious activities that might be happening around the world. Of course others realized, as almost all do today, that cyber attacks have grater range than a jet fighter or ICBM regardless of whether they're perpetrated by nation-states, cyber criminals or activists. To address this issue, organizations are trying to optimize their use of big data security by letting the machines do the heavy lifting and allowing the humans to manage by exception.

Big Data

Big data has already proven its value outside of security across many areas such as space exploration, sports, retail and insurance.

When we think of big data, it doesn't get much bigger than space. Big data analytics have lead to corrections--rest in peace Pluto--and countless discoveries such as:

Consider the value it afforded baseball as portrayed in the 2011 movie Moneyball. We've moved from just relying upon visceral reactions by scouts and gut feel to also incorporating math and science.

Think of your latest online shopping experience.   Chances are that the webpage the retailer displays to you has been customized for your interests based on a variety of factors ranging from age and gender to purchase history and geography. And consider how this experience will mature with mobile devices, the Internet of Things, and apps when you visit a brick-and-mortar establishment or drive past a location that has a sale on a brand you like and as such you are alerted with a map, item photo, sale price, inventory, etc.

Finally, remember when getting car insurance was a few simple questions like make, model and year of vehicle, driving record and age? Now questions include marital status, number of children, your highest level of education and home ownership, because they can be measured against a statistical model to help develop a risk score and ultimately determine what you should be charged.

Big Data Security

Before the term "big data" became common nomenclature in the security industry, there was a trend largely inspired by SIEM and log management solutions. This trend resulted in the mass collection and storage of log data. This helped placate auditors and make storage vendors a lot of money, but without capabilities like threat intelligence feeds, automation, analytics such as correlation, anomaly detection, pattern discovery and prioritization, their effectiveness was limited. Simple collection and storage isn't enough. Today, with big data being measured at levels never before operationalized, such as the Yottabytes of storage that some military-level data centers are being built to handle and the Undecillion IP addresses in IPv6.

Perhaps the most important variable, so that security can be managed by exception in the face of staggering data volumes, velocity and variety, is context.

Big Data Security Context

Folks I spoke with in ANZ want to move beyond thinking of data, regardless of that data being logs, alerts, packet captures, metadata, flows, threat feeds, malware detonation outputs and the like, in terms of what they can collect and store. They want to automatically extract value from it. They want machines to:

  • Evaluate all data sources across traditional IT, cloud and mobile
  • Illustrate root cause
  • Visualize the attack sequences
  • Associate identity information
  • Weigh the incident against historic knows
  • Consider the attacker source and attack type
  • Associate target system intelligence such as operating system, applications, data, regulatory mandates, etc.
  • Prioritize output
  • Incorporate incident workflow
  • Allow for human analytics from a single pane of glass
  • Offer mitigation solutions with weighted impact relevance

More simply put, they want to have context delivery automated so security analysts are given a prioritized list of "stories" to review as opposed to some sentence fragments that they need to piece together.


In ANZ--and frankly everywhere in the world--deriving this level of context is a bit of a utopia at least today. All the pieces of the puzzle are being provided at some level by disparate solutions. Some of these solutions are even integrated. But having a unified, inclusive solution made up of all the necessary best-in-breed technologies that's scalable and effective and will allow security analysts to truly mange by exception is still a ways off but certainly worth striving for.

As organizations begin to embrace big data security, or are already starting to tune their program, context must be at the core of the requirements list. Without context, the simple math of the problem will introduce far too much complexity to be of value and big data security will become be a big waste.

Join the CSO newsletter!

Error: Please check your email address.

Tags applicationssoftwareBlackholedata protection

More about ANZ Banking GroupGalaxyNASA

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Brian Contos

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts