Underprovisioned security-analytics tools, skills hinder big-data adoption, expert warns

Most companies building security-analytics infrastructure see their platforms triple in size within the first 18 months of operation, a data-analytics expert has noted while warning that organic internal growth in use cases often quickly exposes shortcomings in planning the necessary big-data infrastructure.

That infrastructure was often being designed to cope with twice the volume of collected security-log data expected within the first year – but this capacity is often exhausted within months of go-live, MapR chief application architect Ted Dunning told CSO Australia, as internal business units rapidly push for new types of data to be collected and new analyses run.

“People usually start small and want to see some value come out of there,” he explained. “But the best customers are always the ones who are fairly loose about where they think the value might come; they are aware that there will be serendipity, and that they will identify new use cases more valuable than the first one.”

That growth – particularly in the information-security space, where big-data techniques are being applied with often stunning success to large volumes of security data that might otherwise go unprocessed – had driven the use of security analytics to fever pitch this year.

The key in turning big-data analytics techniques into real business results lay in the skills of employees in thinking laterally – and delving into the analytics space often without clear indications of what might be observed.

Insight often came with surprising speed once adequate volumes of data had been funneled into the security-analytics environment: “often these security problems become enormously simpler at scale,” Dunning explained, “because you see very subtle fingerprints and footprints in the data.”

“You see it in the timing, the patterning, the accessing on the network – and it goes from being essentially impossible to detect these attacks, to so embarrassingly easy that you wouldn't want to explain what you do.”

A recent Ovum research note warned users to stop just collecting data and start diving into it to extract new insights.

Yet this can still be difficult in organisations where traditional notions of data ownership and control were proving difficult to dispel. Security-analytics data might be valuable in many ways across the organisation, Dunning said, but it often existed as a subset of the company's larger big-data investment – and ended up under the control of project sponsors in one particular part of the business.

Read more: Software Vulnerability Management, 2016 Predictions

A more sustainable long-term vision could be realised by centralising the data and pushing for open access by skilled big-data staff as well as security experts that may not be well-versed in big-data techniques, Dunning said, noting that the trend was toward “hyperconvergence into a single platform” and that effective security analytics required ongoing support from many people.

“You need a mix of skills,” he explained. “The attacks are insidious and clever and odd, and you don't just walk into the security area and work it out. You need to look at the very human endeavour of trying to defeat security, and look at the very human endeavour of trying to improve security, and to understand what to be looking for.”

Participate in CSO and Gigamon's survey on Security Priorities today!

Go into the draw for a chance to win an Apple iWatch Sports or the equivalent of $500 Visa Cashcard.

For full terms and conditions click here.

Read more: IDC: Mobile device chaos creating heightened security demand

Start survey NOW!

Join the CSO newsletter!

Error: Please check your email address.

Tags MapR chief application architectinfrastructureTed Dunningsecurity-analyticsbig-data adoption

More about AppleCSOGigamonInsightOvumVisa

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by David Braue

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts