How security pros are handling data overload

The majority of IT and business professionals in large companies are no more than somewhat confident their security systems can detect a threat before it becomes a real problem, a study shows.

About one in 10 of the 200 pros in Enterprise Management Associates' survey were neither confident nor doubtful that threats could be detected, while 7% reported more doubt than confidence. More than half were only "somewhat" confident at best.

The survey, released in May, found that most security pros in large companies were struggling to make sense of the log data gathered from security-related systems. A third of the respondents, all of who worked in companies with 1,000 employees or more worldwide, found it too difficult to distinguish legitimate from malicious activity, while almost three in 10 were equally successful as unsuccessful in correlating security data to business impact. Worse, 4 percent of the pros said they were mostly unsuccessful.

More than twice a month, almost six in 10 of the respondents have to devote unplanned time to respond to security incidents that occur outside normal investigative activities. A third are doing additional work at least every week, and about one in eight everyday.

The findings point to organizations being overwhelmed with the security data they currently collect. Almost 60% of the respondents knowledgeable about security log and event data management said they collect 50 gigabytes or more of data from routers, firewalls, gateways and other security-related systems each day. This translates into more than 166 million events daily, EMA said.

While this is already an overwhelming amount of information, almost three-quarters of the respondents said they would collect even more security-related data, or a wider variety of data, if they could make use of it.

So why would organizations want to build an even higher mountain of data? Study author and EMA researcher Scott Crawford said part of the reason is fear of the growing likelihood of an attacker's success, given the improvements in technology used to sniff out system vulnerabilities. "Organizations are recognizing that attackers may be far more successful than we have been openly acknowledging," Crawford said.

A second factor is organizations know they have to do better with the data they collect, so they are exploring higher-performing analytical tools and techniques, which can process more information.

Finally, security management practices have been based in part on fear, uncertainty and doubt, which have left overtaxed organizations feeling like they need to do more. "Strategists would like to get a more objective handle on their (data) management priorities," Crawford said.

To shore up defenses, roughly four in 10 of the respondents said they were spending more money on better security data management and analytic technologies. An additional 40% said they would spend more on similar technologies in the next one to three years.

The most popular technology was security information and event management, which was used by nearly eight in 10 of the respondents. Ã'Â Half said they were using data warehouses to store and analyze information, while nearly the same percentage was using analytical databases.

Business intelligence was used by 55% of the respondents, making it the most popular analytic tool. Other popular tools included analytic platforms, risk analysis or modeling and data mining.

Given the large amounts of data that has to be continuously analyzed, almost four in 10 of the respondents used service providers for security data management.



Read more about application security in CSOonline's Application Security section.

Join the CSO newsletter!

Error: Please check your email address.
Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Antone Gonsalves

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts