Adobe learns that security is a numbers game

It turns out that trench warfare is a really dumb idea

"Never waste a crisis," says Adobe's product security chief Brad Arkin. He says that with a smile today, but when he joined the company in late 2008 the crisis was very real and the outcome unknowable.

Through the first decade of the 21st century, Microsoft had steadily improved the security of Windows and Office — the direct result of Bill Gates' trustworthy computing memo and a new software development process called the Secure Development Lifecycle (SDL). So the bad guys just moved up the stack.

Adobe had the "good fortune", as Arkin wryly puts it, to have two of the most widely-deployed non-Microsoft applications: Reader and Acrobat. Both had an old, complicated code base. Both had serious security flaws.

Adobe became the new target.

It was Arkin's job to deal with that.

"PDFs were really attractive because [attackers] could email a link or IM a link, and then you get the web-hosted attack vector, and then you have the email attachment attack vector. These things made it really perfect for attacking corporate types," Arkin told CSO Online.

These techniques would eventually be used in the high-profile Operation Aurora attacks against Google, Juniper Networks and even Adobe itself.

"We were experiencing a lot of pain, and we had a plan, but at the time we didn't know if it was going to work. It was merely the best we could come up with. And that was a really scary thing because there were moments where it felt like the challenges we were up against were too great and the speed at which bad guys were changing tactics was always going to stay a step ahead of us," Arkin said.

"In hindsight it's easy to just pay attention to the choices we made that were effective, and then we tend to forget about all the dumb ideas that we wasted time on."

One of those dumb ideas was trying to fix all the bugs.

"If we can do that faster than the bad guys are finding them and exploiting them then we'll be all set," Arkin said.

"It just doesn't work."

Chasing all the bugs is simply trench warfare. For all that time spent on industrial-scale automated bug-finding tools called fuzzers, for every bug fixed there's still another latent bug waiting to be found.

Arkin blames his own consulting background for that mistake. It's the perfect job for a consultant because it never ends. There's always more bugs to find.

"The definition of a crisis isn't people running around screaming, but it's a turning point where things are decided for better or for worse," Arkin said. "So you leverage that moment of crisis and you say, 'Well, this is our chance to make sure things are better going forward.'"

What made things better was forgetting about all the bugs and concentrating on driving up the cost for attackers to exploit those bugs to the point where it wasn't worth their while.

That in turn meant making better use of the available programming tools and developing the sandbox that was first introduced in Reader and Acrobat version X in November 2010.

"The bad guys have a certain MO where they want to get on the machine and install a trojan of some sort and then move laterally, and version X made it really hard to install software," Arkin said. "Other types of things that might still be straightforward are not interesting and so they just moved on."

The improvements in version X are holding up, said Arkin.

"I was hoping we could make it the first 24 hours without anyone pointing out an obvious goof or like, 'Oh, what were they thinking?' And then we made it through the first day and then the first week, the first month. I said, 'OK, this is pretty good.' Nothing glaring was left open."

"We've been very happy, and also a little surprised, that there hasn't been any bad-guy activity against version X. We made it as hard as we could to attack, but... there's certain things our design didn't seek to prevent... To date we still haven't seen any malware that is affective against X, and not just installing trojans but anything at all."

Adobe's problem now is getting everyone who's still running old versions of the software to upgrade.

"We've pretty much scooped up just about everyone who is awake and paying attention, and now we're stuck on this machine replacement rate," Arkin said. "We're not trying to be sneaky, but we're trying to get the people who aren't paying attention."

That's a legal problem. Adobe can't just upgrade people to the new version automatically because in many jurisdictions that would be a cybercrime.

"We're not making much progress there," Arkin said.

Nevertheless, the problems with Reader and Acrobat are slowly going away. Adobe's next challenge will be bringing the same security improvements to Flash Player.

That's more complex. Flash Player has to deal with many different web browser environments as well as different operating systems. Adobe's experience fixing Reader and Acrobat should make things easier — as will the newer Pepper plug-in architecture that replaces the old Netscape Plugin API standard.

"It's fun now, because I can give this advice to other people, and help them avoid my mistakes," Arkin said.

And then comes Adobe's move into the cloud.

"You've got all these other layers you've got to worry about. There's all sorts of new ways to screw up."

Contact Stilgherrian at or follow him on Twitter at @stilgherrian


CSO Announcement

Register Today. Hear from Rob Livingstone, Michael Barnes, Steve Quane and Dave Asprey amongst others on the Evolution, Trends, Solutions and the Future of Cloud Security, limited seats register today through CSO.


Join the CSO newsletter!

Error: Please check your email address.

More about Adobe SystemsBillGoogleJuniperJuniperMicrosoftSDL

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Stilgherrian

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts