Most companies building security-analytics infrastructure see their platforms triple in size within the first 18 months of operation, a data-analytics expert has noted while warning that organic internal growth in use cases often quickly exposes shortcomings in planning the necessary big-data infrastructure.
That infrastructure was often being designed to cope with twice the volume of collected security-log data expected within the first year – but this capacity is often exhausted within months of go-live, MapR chief application architect Ted Dunning told CSO Australia, as internal business units rapidly push for new types of data to be collected and new analyses run.
“People usually start small and want to see some value come out of there,” he explained. “But the best customers are always the ones who are fairly loose about where they think the value might come; they are aware that there will be serendipity, and that they will identify new use cases more valuable than the first one.”
That growth – particularly in the information-security space, where big-data techniques are being applied with often stunning success to large volumes of security data that might otherwise go unprocessed – had driven the use of security analytics to fever pitch this year.
The key in turning big-data analytics techniques into real business results lay in the skills of employees in thinking laterally – and delving into the analytics space often without clear indications of what might be observed.
Insight often came with surprising speed once adequate volumes of data had been funneled into the security-analytics environment: “often these security problems become enormously simpler at scale,” Dunning explained, “because you see very subtle fingerprints and footprints in the data.”
“You see it in the timing, the patterning, the accessing on the network – and it goes from being essentially impossible to detect these attacks, to so embarrassingly easy that you wouldn't want to explain what you do.”
A recent Ovum research note warned users to stop just collecting data and start diving into it to extract new insights.
Yet this can still be difficult in organisations where traditional notions of data ownership and control were proving difficult to dispel. Security-analytics data might be valuable in many ways across the organisation, Dunning said, but it often existed as a subset of the company's larger big-data investment – and ended up under the control of project sponsors in one particular part of the business.
A more sustainable long-term vision could be realised by centralising the data and pushing for open access by skilled big-data staff as well as security experts that may not be well-versed in big-data techniques, Dunning said, noting that the trend was toward “hyperconvergence into a single platform” and that effective security analytics required ongoing support from many people.
“You need a mix of skills,” he explained. “The attacks are insidious and clever and odd, and you don't just walk into the security area and work it out. You need to look at the very human endeavour of trying to defeat security, and look at the very human endeavour of trying to improve security, and to understand what to be looking for.”
Participate in CSO and Gigamon's survey on Security Priorities today!Go into the draw for a chance to win an Apple iWatch Sports or the equivalent of $500 Visa Cashcard.
For full terms and conditions click here.