Rethinking IDS

After the Nachi worm hit last year, Joe Granneman, manager of networks and PC services at Rockford Health System, knew it was time for a change. "It only took three infected machines to bring down our dual processor firewall," he marvels. "Without our Internet connection we couldn't process claims or do much of anything."

Earlier that year a DoS attack disabled a VPN concentrator. A network IDS (intrusion detection system) detected the attack and sent an alert, but not before frustrated users barraged Granneman by phone calls, whom he calls his most reliable SNMP alert.

Monitoring IDS alerts was also taking up more and more of Granneman's time. "I came in early before my meetings every day to check the IDS logs, spent my lunchtime inspecting IDS logs on my notebook from the cafeteria, and dialed in constantly over Christmas because of the terrorist threat. Those logs became my bible."

Granneman's experience with IDS isn't unusual. Firewalls are still a necessary first line of defense, but traditional stateful inspection firewalls are only adept at stopping network level attacks, and so are generally helpless in the face of worms and sophisticated application level attacks that exploit open ports such as 80 (http) and 443 (https). Intrusion detection systems use sensors that sit passively on the LAN inspecting traffic for signs of malicious activity. They use a number of signature- and anomaly-based technologies to help detect many application-level attacks, but they generally do not block them. By the time the administrator is alerted, it's often too late to prevent widespread damage.

A fog of false alerts

IDS has also been prone to endless streams of false alerts. "Our IDS was a mess, alerting us on absolutely everything" says a network security specialist at an electric utility, who asked not to be named for security reasons. "In fact, I can't even remember a single legitimate alert. We never had the time or manpower to monitor it all."

Selim Nart, network architect for global networking at Vignette Software Corp., agrees that false alerts are a management headache. "It can take you 20 hours to investigate two hours worth of alerts."

In fact, the management and performance drawbacks of IDS are so notorious that a Gartner Information Security Hype Cycle report published in June 2003 declared the category a market failure. Instead Gartner recommended that organizations hold off investing in IDS and shift resources to vulnerability scanning, server hardening, and newer, deep-packet inspection firewalls, which are more adept than standard firewalls at detecting and stopping application-level attacks. More recently, Gartner recommended new kinds of IPSes (intrusion prevention systems), available from traditional IDS and security vendors such as Internet Security Systems Inc. (ISS), Netscreen Technologies Inc., and Network Associates Inc., as well as from upstarts such as TippingPoint Technologies Inc., StillSecure, and Top Layer Networks Inc.

Unlike IDS, which simply monitors the network and sends out alerts, network IPS sits inline to block attacks as they happen. Host-based IPSes, such as those from Entercept (now part of Network Associates) and Okena (now part of Cisco), sit directly on application servers, intercepting system calls and looking for alterations to critical system files, changes in file permissions, and other signs of attack.

But is IDS really dead? And are these newer product categories ready to replace it? Most of the analysts, as well as IDS and even IPS vendors we spoke with agreed that the answer is "not yet."

IT should instead invest in IPS and application level firewalls as another layer in the enterprise security strategy, and relegate IDS to a security audit and forensics function. "IDS and IPS use essentially the same detection techniques," says Joel McFarland, manager of security products at Cisco Systems Inc.'s VPN and Security Business Unit. "So both are plagued by the challenge of accuracy. And when you're looking at a network of hundreds or thousands of hosts, you have to be wary about shooting first and asking questions later."

Paul Proctor, vice president of Security and Risk Strategies at the Meta Group Inc., agrees. "Most of the clients we work with run IPS largely in IDS (monitoring only) mode, because in the wild and woolly world of enterprise apps, a false positive from an IPS can give you a self-inflicted denial of service attack. IPS is most appropriate for blocking the narrow band of attacks you know are bad and don't fit into the application detection capabilities of the firewall. For all that other traffic, which may or may not be bad, but which you still want to know about, IDS still makes more sense."

Bob Walder, director of the NSS Group, a UK- and French-based independent network and security testing organization, is not as concerned with the performance and accuracy of today's IPS products. NSS has run extensive testing on both types of solutions, and has found IDS and IPS products that can handle gigabit environments, and that, properly implemented, deftly avoid false positives.

"Sure, if you take every IDS and IPS product and turn on every filter, you'll have a torrid time with false positives," Walder says. "These take some work to deploy and an awareness of where you place the device and what traffic is passing through."

Fine-tune settings

Walder points out, however, that IDS and IPS vendors are getting better at automating the tuning process, which can be lengthy and painful. For example, TippingPoint's UnityOne line has a recommended settings feature that can be configured in minutes. And IDS vendors, including Cisco, Symantec, and ISS now offer system audit and correlation features that can suppress irrelevant alerts, such as an Apache exploit targeting an IIS server or even an IIS exploit targeting an IIS server that has been properly patched. "Tuning our Cisco IDS was hell at first," Nart says. "But after we purchased Cisco Threat Response (Cisco's correlation option), we were able to reduce false positives with very little tuning."

But Walder points out another problem with IPS. "Most of these products are very expensive compared to IDS products, and even if they come down in price, IDSes will come down further as well and will most likely take on additional capabilities. IPSes make sense for protecting the perimeter, the DMZ (demilitarized zone), and perhaps one or two critical subnets, but in a network with 400 subnets, you probably can't afford to put an IPS on all of them."

In fact, a side benefit of using an IPS to block well-known worms and other attacks at the perimeter is that the number of alerts generated by an internal IDS is reduced even further. The need to constantly monitor IDS alerts virtually disappears as IDS becomes more of an auditing tool to confirm your security strategy and monitor less-critical subnets than as a way of stopping critical attacks, a function for which it was never well-suited.

For example, Granneman now uses Top Layer's Attack Mitigator IPS to protect the gateway and the datacenter. "I slowly turned on each filter to make sure we wouldn't block any legitimate business processes." Attack Mitigator IPS typically sits outside the firewall and its behavior-based techniques are particularly adept at blocking DoS attacks. Granneman has kept his IDSes in place but spends much less time looking at IDS logs.

Similarly, John Penrod, director of Network Architecture at the Weather Channel, has replaced his former open source Snort IDS installation with TippingPoint UnityOne IPS appliances at the perimeter and Lancope's behavior-based StealthWatch IDS on a number of internal segments. "I've reduced the volume of IDS alarms by 99 percent," Penrod says. "I had hired someone to be responsible for security and maintenance, but he was spending the whole day looking at IDS logs. Now he has time for maintenance."

Other alternatives

Aside from IPS, another category to consider for specifically protecting Web servers and other DMZ applications is a Web application firewall, such as KaVaDo InterDo, NetContinuum NC-1000 Web Security Gateway, Sanctum AppShield, and Teros Secure Application Gateway . These all focus on Web application exploits typically missed by stateful inspection firewalls and some that may be missed by IPSes, which are hard-pressed to cover every application.

Host-based intrusion prevention software can also provide additional protection for public-facing applications and critical internal servers. Check Point and NetScreen have also added some deep packet inspection capabilities to their firewall products, but unlike many IPS and Web application firewall appliances that implement algorithms in silicon, Check Point's and NetScreen's application firewall features are software-based.

As IPS earns users' trust and as the products come down in price, it may some day replace IDS. Eventually firewalls and intrusion detection and prevention may be subsumed in a single solution, along with anti-virus, anti-spam, and other categories. But with today's security challenges, most analysts and vendors agree a layered defense, with firewalls, IPSes, and IDSes targeted for what they do best, is still the best way to protect a network.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.
Follow our new CSO Australia LinkedIn
Follow our new social and we'll keep you in the loop for exclusive events and all things security!
Have an opinion on security? Want to have your articles published on CSO? Please contact CSO Content Manager for our guidelines.

More about ApacheCiscoGartnerGatewayInternet Security SystemsIPSISS GroupKavadoLancopeMeta GroupNetScreenNetScreen TechnologiesOkenaRockfordSanctumSecurity SystemsSymantecTippingPointTop Layer NetworksVignetteVignette SoftwareWeb Security

Show Comments

Featured Whitepapers

Editor's Recommendations

Brand Page

Stories by Leon Erlanger

Latest Videos

More videos

Blog Posts