Microsoft Security Essentials struggles in antivirus tests

Zero-day detection mediocre, finds

Microsoft's popular free antivirus program Security Essentials has put in a mediocre showing in the latest quarterly tests from German test outfit, finishing second bottom out of 22 products.

In Q1 2011 Security Essentials 2.0 (MSE) performed well at the least demanding test, that of spotting malware drawn from the industry-agreed Wildlist selection, scoring 100 per cent. It also put in a good performance against a large group of recent malware samples selected by AV-Test itself, with a creditable score of 97 per cent detection.

However, the product's performance deteriorated sharply when pitted against 107 recent zero-day malware web and email malware attacks, described by AV-Test as 'real-world' testing', spotting only half. The product's performance in 'dynamic detection testing' - noticing malware on or post-execution - was also modest at only 45 per cent.

For context, the test average for real-world and dynamic testing was 84 per cent and 62 per cent respectively.

The top-scoring product in the tests was BitDefender's Internet Security Suite 2011, with a maximum weighted score of 6.0 across all tests, ahead of BullGuard Internet Security 10, F-Secure Internet Security 2011, and Kaspersky Internet Security 2011, all on 5.5. MSE scored 2.5, ahead of only one product, CA Internet Security Suite 2011.

AV-Test also looked at the impact of antivirus software on the performance of the PC. By this measure, often rated as important for many consumer users, MSE did relatively well, scoring 162 (lower being better) against the average of 171. This test showed a surprising degree of performance difference between suites, with BitDefender against doing well with a score of 111 against BullGuard's dismal 539.

Security Essentials was in the end awarded a 'pass' certification under the AV-Test assessment for making the grade in at least 11 of the 18 tests, putting it ahead of five products that failed altogether. In addition to CA's suite, these were Norman Security Suite Pro 8.0, McAfee Total Protection 2011, PC Tools Internet Security 2011, and Comodo Internet Security Premium 5.0/5.3.

Do the zero-day tests matter in everyday conditions? Arguably, yes. A common attack method is to hit users with zero-day exploits and so the ability to spot this challenging category of malware is crucial. According to AV-Test's quarterly results, MSE's performance in this test has also deteriorated quarter-on-quarter, dropping from around 75 per cent to Q1's 50 per cent.

"Microsoft is offering a free of charge virus scanner: MSE. The product is missing effective email and web protection and also dynamic detection/protection technologies, so the product performs worse when compared with other free or paid AV/ISS offering," said Andreas Marx of AV-Test by email to Techworld.

"That's the big problem with this tool - the majority of the other products tested includes such protection features, so they are performing better in our tests. And we expect that they are performing better in the 'real world' as well, which is the focus of our tests."

An individual user's exposure to a zero-day attack will depend on a number of factors, including the range of applications used and how assiduously a PC is patched.

As Marx noted, MSE is a free product - many of the rival suites charge upwards of £20 ($33) a year for a license. However, the dividing line isn't necessarily whether a product is free or not; several rival products offered in free versions did better than MSE. It is possible that free programs now need to include a wider range of detection features than they might have done in the past.

Version 2.0 of MSE was launched in December 2010 and anecdotal evidence suggests it has only enhanced the program's huge popularity. By September 2010, the software was said by Microsoft to have been installed on 31 million PCs globally, including 1.7 million in the UK.

The most interesting message of these tests is that a product can drop in effectiveness quite quickly, before in all likelihood rising again as a new version appears that adds new security elements.

Join the CSO newsletter!

Error: Please check your email address.

Tags Personal TechMicrosoftsecurityf-secureSMEbitdefenderantivirus

More about BitDefenderBullGuard AustraliaCA TechnologiesComodoF-SecureISS GroupKasperskyKasperskyMcAfee AustraliaMicrosoftNormanNormanPC Tools

Show Comments

Featured Whitepapers

Editor's Recommendations

Solution Centres

Stories by John E Dunn

Latest Videos

  • 150x50

    CSO Webinar: The Human Factor - Your people are your biggest security weakness

    ​Speakers: David Lacey, Researcher and former CISO Royal Mail David Turner - Global Risk Management Expert Mark Guntrip - Group Manager, Email Protection, Proofpoint

    Play Video

  • 150x50

    CSO Webinar: Current ransomware defences are failing – but machine learning can drive a more proactive solution

    Speakers • Ty Miller, Director, Threat Intelligence • Mark Gregory, Leader, Network Engineering Research Group, RMIT • Jeff Lanza, Retired FBI Agent (USA) • Andy Solterbeck, VP Asia Pacific, Cylance • David Braue, CSO MC/Moderator What to expect: ​Hear from industry experts on the local and global ransomware threat landscape. Explore a new approach to dealing with ransomware using machine-learning techniques and by thinking about the problem in a fundamentally different way. Apply techniques for gathering insight into ransomware behaviour and find out what elements must go into a truly effective ransomware defence. Get a first-hand look at how ransomware actually works in practice, and how machine-learning techniques can pick up on its activities long before your employees do.

    Play Video

  • 150x50

    CSO Webinar: Get real about metadata to avoid a false sense of security

    Speakers: • Anthony Caruana – CSO MC and moderator • Ian Farquhar, Worldwide Virtual Security Team Lead, Gigamon • John Lindsay, Former CTO, iiNet • Skeeve Stevens, Futurist, Future Sumo • David Vaile - Vice chair of APF, Co-Convenor of the Cyberspace Law And Policy Community, UNSW Law Faculty This webinar covers: - A 101 on metadata - what it is and how to use it - Insight into a typical attack, what happens and what we would find when looking into the metadata - How to collect metadata, use this to detect attacks and get greater insight into how you can use this to protect your organisation - Learn how much raw data and metadata to retain and how long for - Get a reality check on how you're using your metadata and if this is enough to secure your organisation

    Play Video

  • 150x50

    CSO Webinar: How banking trojans work and how you can stop them

    CSO Webinar: How banking trojans work and how you can stop them Featuring: • John Baird, Director of Global Technology Production, Deutsche Bank • Samantha Macleod, GM Cyber Security, ME Bank • Sherrod DeGrippo, Director of Emerging Threats, Proofpoint (USA)

    Play Video

  • 150x50

    IDG Live Webinar:The right collaboration strategy will help your business take flight

    Speakers - Mike Harris, Engineering Services Manager, Jetstar - Christopher Johnson, IT Director APAC, 20th Century Fox - Brent Maxwell, Director of Information Systems, THE ICONIC - IDG MC/Moderator Anthony Caruana

    Play Video

More videos

Blog Posts