Defcon: The security penetration testing quagmire

LAS VEGAS -- The relationship between CISOs and security penetration testers is anything but clear-cut and raises ethical issues for both parties, a Defcon crowd heard from a former CISO.

Whether penetration testers should come in looking for the place where they can spectacularly break into the network or instead assess it clinically and point out potential vulnerabilities is the big decision CISOs have to make, says a CISO-turned penetration tester identified only as Shrdlu.

SELF TESTING: Metasploit 4.0 sets the stage for mass penetration testing

And the choice is the CISO's, she says, because the CISO is paying the bills. "It's not about your satisfaction," she told a crowd that included many penetration testers.

She says that often penetration tests are mandated by regulations, and the network must pass in order to comply. In that case, she prefers a light touch by the tester, telling her informally about technical security shortcomings but not including them in the formal report that goes to the compliance auditor. "Tell me verbally what's wrong and don't write it down," she says.

For example, if the help desk prompts users that they can't login because they've gotten their username wrong, that's a violation. But, she says, doing so saves a lot of help desk and employee time and is a good risk-business tradeoff. She doesn't consider the practice a major breach of good practice.

"There are things I do on purpose and are not high-impact," she says.

That drew protests from audience members, one of whom said it was unethical not to include security problems he finds and is possibly illegal because it is essentially lying to compliance auditors. "It sounds like avoiding regulatory scrutiny," he said.

"That's very fair," Shrdlu responded. But she says most compliance regulations are vague enough that reports can be vague as well, indicating an unspecified problem without detailing it. She says penetration testers can prepare two reports, one for her use and a second for the auditor.

She says these dual reports are useful for public organizations where the reports may become public record. The vague one that doesn't detail specific problems can be the public version and the detailed one can be called a working document and so avoid public scrutiny.

Another audience member said her approach could cause problems for penetration testers if a problem found but not mentioned is exploited. The tester would have no documentation that he'd done his job properly. Again, she fell back on the dual report, where the vague reference to the problem would provide cover for the penetration tester.

She says she's found frustration with penetration testers who haven't worked in corporate security and had to shore up problems testers have found. Often the problems present less of a risk to the organization than the time it would take to fix them is worth, she says. "I'm impatient with penetration testers that have never been on the fixing side," she says. They need to be more aware of the big impact and the business impact of remedies. "There are things that just plain aren't going to be fixed."

She says she doesn't want a broad penetration test that lists all the ills her network harbors. Rather, she wants to hear about the significant ones that she doesn't know about yet so she can create an internal list of what she wants to fix. "Don't tell me what I already know," Shrdlu says.

Read more about wide area network in Network World's Wide Area Network section.

Join the CSO newsletter!

Error: Please check your email address.

Tags DefconsecurityIT managementregulatory compliance

More about ING AustraliaLAN

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Tim Greene

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts