Malwarebytes questions poor showing in anti-malware protection-evaluation lab testing

Dennis Technology Labs (DTL), which tests anti-malware products for effectiveness in protection, for the first time included the free version of the Malwarebytes software in the labs' competitive evaluation along with nine other vendor products, both paid and free. The results published by DTL today reveal Malwarebytes Anti-Malware Free had a poor showing, with only Microsoft Security Essentials doing worse in terms of effectiveness of protection.

In terms of DTL's "protection" ratings that score for accuracy in protecting against malware, the free version of Malwarebytes was scored at 63 based on how many times it prevented a threat from compromising the system. Only Microsoft Security Essentials did worse at a score of 56. By contrast, the products that scored the best in the DTL tests, Kaspersky Internet Security 2014 and Norton Internet Security, each scored 100. Other high scorers on "protection" included BitDefender Internet Security which earned a score of 94 as well as Avast! Free Antivirus and ESET Smart Security 7 which each scored 93.

+More on Network World: The Worst Security SNAFUs this year (so far!)+

Simon Edwards , technical director at the U.K.-based lab, said DTL decided to include the free version of the Malwarebytes software for the first time in this round of testing mainly because the marketing message from Malwarebytes about it appears to claim "not only is this an anti-malware product but it actually catches stuff antivirus misses."

But the free version of the Malwarebytes software has apparent differences from most of the other anti-malware products tested in that it doesn't have the ability to block malicious websites before they load, which contributed to the relatively low score, says Edwards.

Except for Malwarebytes Anti-Malware Free, the other products in the test "use real-time protection to detect an infection as it happens and attempt to prevent it from succeeding," he pointed out. He adds the free version of the Malwarebytes software seems to be "designed to be used after an infection has happened and lacks this real-time protection."

Pedro Bustamante, director of Special Projects at Malwarebytes, said, "Our score in this test is clearly skewed by the fact that Malwarebytes Anti-Malware Free is the only product being tested without real-time protection. It is specifically designed to clean-up threats, as opposed to prevent them occurring. With proactive protection contributing to a significant portion of the score, it is evident we would never feature highly. This is something the authors of the report have acknowledged."

The DTL report, "Home Anti-Virus Protection, April to June 2014" also states Malwarebytes claims its free version "to be the most popular security product installed by users" along with claims it protects them from "new online threats that antivirus can't detect," and the lab was simply investigating this claim. The DTL test, which was not underwritten by any vendors, did not cover the paid version of Malwarebytes, called Malwarebytes Anti-Malware Premium.

Edwards says Malwarebytes, learning of the upcoming DTL report, contacted him to express concern that their free cleanup tool was put up against free and commercial anti-malware products that have more protection features. Edwards contends his purpose was simply testing Malwarebytes Anti-Malware Free against the claims currently found on the Malwarebytes website, namely that it is a "detection and removal tool." But in the discussion with Malwarebytes, Edwards said he did agree to add a few comments to the report to clarify things.

The added remarks to the DTL test report say: "This test is designed to measure the overall protection provided by anti-malware products. It allows the products to use any and all layers of available protection to block or neutralize web-based threats. The Malwarebytes Anti-Malware Free product is designed to clean up infections after they occur and not to prevent them from occurring. As such, it is never going to be able to score as strongly as products that include real-time protection and/or malicious website blocking. This type of protection is included in the Malwarebytes Anti-Malware Premium product, which requires a license that costs money."

Join the CSO newsletter!

Error: Please check your email address.

Tags MicrosoftsecurityAvastMalwarebytesnortonbitdefender

More about AvastBitDefenderKasperskyMalwarebytesMicrosoftNortonSmartTechnology

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by Ellen Messmer

Latest Videos

  • 150x50

    CSO Webinar: Will your data protection strategy be enough when disaster strikes?

    Speakers: - Paul O’Connor, Engagement leader - Performance Audit Group, Victorian Auditor-General’s Office (VAGO) - Nigel Phair, Managing Director, Centre for Internet Safety - Joshua Stenhouse, Technical Evangelist, Zerto - Anthony Caruana, CSO MC & Moderator

    Play Video

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

More videos

Blog Posts

Market Place