Researchers add software bugs to reduce the number of… software bugs

A new strategy for training bug-finding tools could help catch more vulnerabilities.

Researchers are adding bugs to experimental software code in order to ultimately wind up with programs that have fewer vulnerabilities.

The idea is to insert a known quantity of vulnerabilities into code, then see how many of them are discovered by bug-finding tools.

By analyzing the reasons bugs escape detection, developers can create more effective bug-finders, according to researchers at New York University in collaboration with others from MIT’s Lincoln Laboratory and Northeastern University.

They created large-scale automated vulnerability addition (LAVA), which is a low-cost technique that adds the vulnerabilities. “The only way to evaluate a bug finder is to control the number of bugs in a program, which is exactly what we do with LAVA,” says Brendan Dolan-Gavitt, a computer science and engineering professor at NYU’s Tandon School of Engineering.

+ MORE: Lenovo ThinkPwn UEFI exploit also affects products from other vendors +

The research showed that the bug-finding tools they tested had a dismal aggregate detection rates -- 2%. Not only that, they often found bugs that weren’t even there, creating unnecessary work for quality assurance teams trying to clean up software before it’s released.

The team inserted into programs a known number of what it calls synthetic vulnerabilities that mimic the attributes of actual vulnerabilities found in the wild. Creation of these synthetic vulnerabilities was automated and carried out by making “judicious edits” to the source code of actual programs. Their automated platform was far less expensive than the alternative of custom-designed vulnerabilities that can sport price tags of tens of thousands of dollars.

By carefully placing the bugs, they could see how well bug finders discovered them in various segments of the code. In addition, they limited the number of inputs affected by the synthetic bugs so the test programs wouldn’t shut down entirely.

One major challenge was creating hundreds of thousands of unique vulnerabilities that could not have been seen before by the bug-finding tools so the researchers could accurately assess how well the tools worked.

The research team is planning a competition for this summer in which developers of bug-finding software receive a score based on how many vulnerabilities their tools detect in a piece of software made vulnerable by LAVA. The idea is to help the developers produce better products.

“Developers can compete for bragging rights on who has the highest success rate,” Dolan-Gavitt says.

Join the CSO newsletter!

Error: Please check your email address.

More about CreationLenovoMITYork University

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by By IDG News Service staff

Latest Videos

  • 150x50

    CSO Webinar: Will your data protection strategy be enough when disaster strikes?

    Speakers: - Paul O’Connor, Engagement leader - Performance Audit Group, Victorian Auditor-General’s Office (VAGO) - Nigel Phair, Managing Director, Centre for Internet Safety - Joshua Stenhouse, Technical Evangelist, Zerto - Anthony Caruana, CSO MC & Moderator

    Play Video

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

More videos

Blog Posts

Market Place