As a company that generates 95 percent of its revenues online, Australia-based Wotif.com has paid particular attention to ensuring its operations – whose 500 staff span 19 countries on five continents – are resistant to the depredations of malicious online hackers and well-meaning internal staff alike.
Delivering a consistent user experience across a range of product categories – including hotels, flights, events and most recently holiday rentals – had proved to be a challenging task, but one that was core to the company’s brand as it sought to deliver a customer-focused experience rather than simply turfing users off to a range of different travel sites.
That strategy had kept the company doing extensive inhouse development that included two-way software integration with a multitude of travel providers – and complicated its software architecture by requiring a security framework that was at once strict enough to meet the requirements of PCI DSS card-processing compliance, and flexible enough to accommodate a broad range of input methods.
The security challenge had become even more pointed in recent years, architect Brett Dargin told attendees at the recent IBM Pulse conference, as Wotif grew rapidly by acquisition – and inherited a mishmash of security attitudes, technologies and exposures.
“Each different company that we purchased had a lot of duplicate systems to do the same kinds of things,” he said, explaining that the company had inherited “quite a lot of diverse technologies, different data centre locations, and different views on security. We’ve got growing complexity and keep adding new applications – and throughout all this, we’ve had to continually evolve our security measures.”
The security of knowing
While penetration defences are naturally important, Wotif faces added challenges in the need to detect and defend against surges of traffic from competitors, whose price-comparison bots repeatedly hit the site trying to scrape its discounted prices.
Ironically, in some cases the company found itself “DDoSing ourselves”, Dargin said, noting that co-ordinating software testing across eight internal software-development teams and myriad partners sometimes created huge surges of traffic over short time periods. “Whether it’s load testing that comes across our major site, or it’s partners that start testing in production and don’t have any throttling, we keep relearning that lesson.”
Aiming to get better insight into its ever-changing security posture, Wotif recently began weighing its options for security-intelligence tools that would improve the analysis and correlation of its event logging.
Because it didn’t have a big enough IT department to have a dedicated security team, it needed a way for its “virtual team” to be able to filter out the large volumes of noise from its security tools, and to focus on the areas that needed the most attention.
“Over time, it was becoming really clear that we had some large gaps,” Dargin explained. “Even though we had invested in security for some time, we still weren’t doing it efficiently enough. Over time, we really wanted to get to get to a place where compliance isn’t going to get broken – but there was this general angst that we were too reactive in security.”
With security events typically spawning an intense period of retrospective review, however, this problem had become hard to shake. Massive number-crunching exercises typically ran for many hours, by which point the exercise was more of a post-mortem than a concerted security response.
“The ability to go back in time is not very proactive,” he said. “We wanted to be more proactive, and to get ahead of the game – and to do this, we wanted meaningful data. But because we have a small network team, they can’t be wasting their time nursing the reporting.”
Aiming to get better insight into its ever-changing security posture, Wotif went to market for a security intelligence and event management (SIEM) system that would combine capabilities such as log management, network flow analysis, a range of data-capture capabilities, compatibility with a range of custom log-data formats, and the ability to handle what can often be over 10,000 hits per second.
Wotif ultimately chose the IBM Security QRadar solution, and worked through an implementation that would allow it to continue accommodating a multitude of data formats from different systems; this was a crucial feature given Wotif’s history of systems diversity and growth through acquisition, and had not been an option with many of the other SIEM platforms the company evaluated.
With SIEM running against a range of activity logs, the company was quickly able to move to fix the system of a user that had accidentally clicked on a zero-day exploit, linking their computer to a botnet on a known-bad IP address.
“The symptoms were readily known, and an external IP address on a certain port was being hit,” Dargin said. “It took us around five minutes to track down which machines were infected, and who they belonged to – and we were able to go around and clean their machines.
Better security intelligence has remained a key part of Wotif’s defence mechanisms, with monitoring of traffic patterns highlighting the effects of changes in traffic because of partner’ activities.
Filtering rules are constantly being tweaked, known and blocked IP addresses updated, reports generated, and anomalies quickly detected by looking at aggregate analyses such as the top sources of IP traffic. This makes it easy, for example, to find out what applications are experiencing long response times – and how this affects other elements of Wotif’s application infrastructure.
Effectively using the tool requires a fair bit of attention: “When you turn it on you get a high number of alerts initially,” Dargin explained, “but you have to teach the system about your network priorities. If you really want to get the best benefits out of the system, you’ve got to tend to it.”
This includes, for example, telling the system what network zones are in place, what traffic is allowed in each of them, where the company backup server is, and which servers are allowed to be doing ping sweeps and moving SMB traffic between hosts.
“The real-time performance is fantastic,” Dargin continued, noting that continual updates had allowed the team to establish performance baselines and eliminate false-positives over time.
“We’re aiming to get less than 10 [alerts] per day, although we’re not quite there yet. But it’s important not to let them pile up, and this is an interactive process: you’re constantly going around and tuning and changing things. We’re giving our operations team training in the art of triage and of defences: since you’ve got such an all-encompassing view of what’s going on, we want to be able to support an investigation if we needed to.”