Asking these big questions will help you predict future compromise

Over the past few weeks, I've had arguments with friends in the information security echo chamber about whether it was prudent of me to make public comments about the security the beleaguered website when I had not actually performed a formal assessment of it. My answer -- that I'd assessed all I needed to reach my conclusions -- failed to satisfy some.

[Incident response matters]

Some of this disagreement stemmed from the fact that I was speaking of strong indicators, not evidence, of trouble. Did I go too far in my conclusions? Time will tell.

Could I have been less abrasive about how I stated my conclusions? Always.

But the experience raised a much larger question: I was assuming that incident assessment was more universally understood than it is. My statements therefore seemed black-box and arbitrary, and that's never my intent.

Lest I re-kindle the debate, let me move completely away from and into general incident response and how I walk into a place where I've never been and help figure out (a) if there is a problem; (b) if there is, how big it is, and (c) where do we start to fix the most broken stuff the fastest.

In the past, when an organization asked me to help them understand a compromise, I started by asking questions designed to disqualify avenues of investigation. I often start with two big ones. Of course, these are not the only questions I ask, but they are among the first ones, because the wrong response is so strongly correlated with other horrible practices and habits of security guttersnipes:

Can we take a look at your network logs, flow records or analysis and/or traffic capture for the last few days?

Can I see your latest network scan results?

The answers to these questions are often telling. In many cases, I'm far less interested in what the logs and captures say than the fact that they exist in a form that is accessible by someone in a reasonable amount of time, and that they can in fact be accessed and presented in a form that allows them to be analyzed.

You would be shocked at just how many organizations fail completely at this request.

The "latest scan results" is asking the all-telling question, "Do you know, within a 25% margin of guesstimation, how many endpoints are connected to your network?"

[Business continuity and disaster recovery planning: The basics]

If the company fails these two basic questions, I am certain to find a major mess. Those things didn't cause the mess, but their presence indicates an inattention to security detail, lack of procedural integrity, failure to backstop and a directionality of spending that historically have shown to be conditions present in an organization that has been compromised.

Do I know it, factually? Of course not. But I know it enough to bet the client a case of Dublin Dr. Pepper that I'll find it out right quick.

[Understanding incident response: 5 tips to make IR work for you]

Many incident responders and penetration testers have similar big-picture disqualification questions they ask. Dave Kennedy, CEO of TrustedSec, goes one level deeper into the logs on his first question: "Have you got logs of DNS requests from the past week or two?"

That's a question I usually ask too, once we've got other evidence that something is up. Once we know that there's something on the network, the DNS logs are a great first place to look for where it's calling to (and, inferentially or specifically, then, what it is and who put it there).

There are a whole bunch of other questions you can ask once you start digging, and most of them are designed to disqualify a line of questioning to avoid getting into it. Eric Olson, VP at Cyveillance, tells the story of how his firm asks a single question to determine whether an email was a possible banking phish: "Does the email contain the FDIC symbol?"

Now, not all email with the FDIC symbol is a banking phish, but pretty much no banking phish doesn't have an FDIC symbol. With one question, then, Cyveillance is able to disqualify 97% of the email it sifts through on a daily basis, allowing them to concentrate on the three per cent that may be bank phishing.

Here's another example: on a recent incident we asked about a certain flow analysis product. The first question was highly disqualifying: "You're not using [product] now, are you?"

"Oh, yes I am," came the response, which was in itself highly interesting.

"Oh," we said. "So how often are you using it? Weekly? Monthly?"

"Oh, every day," came the reply. "We love it."

"Oh, I see, that's great," we said. "So when you used it today, did everything look okay?"

"Yeah, you know, things looked pretty normal."

I bet they did. Earlier that morning, I had personally seen the box in question sitting, unplugged, on a cardboard box in the data center.

An engineer told me it had been unplugged for more than a month.

[Beyond breach prevention: The need for adequate response]

In this case, it was clear I didn't need to do any further digging to understand that no analysis had been done. It was also clear that, if the person would cheerfully lie about that, he would cheerfully lie about other things.

It was therefore no big intuitive leap for me to conclude that previous things the guy had signed off on had likely also not been done. Did I know this factually to be true? Not yet.

But I had a wicked-strong indicator. Later we confirmed it, but the indicator itself was strong enough for me to base certain statements and actions.

This stuff is not rocket science. By asking high-level, disqualifying questions, one can easily make some broad assessments. Then it's simply a matter of drilling down into the indicated areas, and finding the problems.

Kennedy offers a few more questions, like, "Are you testing for security threats and doing things like external/internal penetration tests and social-engineering efforts to test your controls and your incident response?" and, "Have you ever done a source code analysis or dynamic testing of applications to determine what risks they pose?"

A number of us are working together to make a list of the top ten questions that can be asked in any organization. We feel that this is a great way to share knowledge and experience with the community. Like that idea? Drop me a line, using the contact form at

Nick Selby is CEO and co-founder of StreetCred Software. A veteran information security consultant, Nick has provided incident response services for F500 companies.

Join the CSO newsletter!

Error: Please check your email address.

Tags security

More about Cyveillance

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Nick Selby

Latest Videos

  • 150x50

    CSO Webinar: Will your data protection strategy be enough when disaster strikes?

    Speakers: - Paul O’Connor, Engagement leader - Performance Audit Group, Victorian Auditor-General’s Office (VAGO) - Nigel Phair, Managing Director, Centre for Internet Safety - Joshua Stenhouse, Technical Evangelist, Zerto - Anthony Caruana, CSO MC & Moderator

    Play Video

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

More videos

Blog Posts

Market Place