New test results highlight Windows security struggles

The latest results are in from AV-Test.org, an independent organization that tests consumer-security suites. Except for a few bright spots, the numbers are a bloodbath of lower scores.

Today might be Friday, September 14, but to PC security companies, it probably feels like Friday the 13th. AV-Test.org's latest regularly published results on home security suites have been released, and the vast majority of the best performing suites slipped a full point or more.

You can read the report at AV-Test's site.

AV-Test looks at three categories in its tests, and scores them each on a six-point scale. Protection gauges how well a suite blocks threats, Repair evaluates how well a suite removes malware, and Usability includes testing for false positives. Eleven points total are required to receive AV-Test.org certification.

The organization also regularly changes the Windows system it tests on. The previous test was on Windows 7. This test was conducted on a 32-bit Windows XP computer running Service Pack 3, and run during July and August 2012.

The attrition wasn't pretty.

In the results published today, two suites that had passed the previous test, published in June 2012, failed it this time around. AhnLab V3 Internet Security 8.0 slipped from 11 out of 18 to 10, and Lavasoft Ad-Aware 10.2 dropped 1.5 points to 10.5.

More notable, though, was the number of suites that had scored above 15 in June but dropped at least one full point. Three lost two points each: AVG Anti-Virus Free 2012 and Webroot SecureAnywhere Complete 8.0 both crashed 15.5 to 13.5; F-Secure Internet Security 2013 dropped from 16.5 to 14.5.

A fourth suite was the biggest loser of them all. Trend Micro Titanium Maximum Security 2012 and 2013 got shaved by 2.5 points, from 14 to 11.5.

G Data Internet Security 2013 got knocked down from 15.5 to 14, and Microsoft's much-touted Security Essentials 4 went from 13.5 to 12.5. Symantec's Norton Internet Security 2012 and BitDefender Internet Security 2012 lost only half a point each, to 15 and 16.5 respectively. Panda Cloud Antivirus 1.5.2 and 2.0 also weakened by 0.5, to 11.5; and Eset Smart Security 5.2 fell from 12.5 to 12.

Meanwhile, a select few held steady or did better than before. CheckPoint ZoneAlarm Free Antivirus + Firewall 10.2 treaded water at 16; Avast Free Antivirus 7.0 (2012) and Avira Internet Security 2012 both bumped up from 14.5 to 15; and Kaspersky Internet Security 2012 gained 0.5 to 17, making it the top-rated suite by AV-Test in this report. Avast Chief Technical Officer Ondrej Vlcek said in an e-mail to CNET that while Avast's score is better than those of many competitors, "we still plan to improve, especially on the detection side -- and we have some good ammo rolling out later in the fall."

Av-Test.org's Andreas Marx emphasized the change in test bed from Windows 7 to Windows XP but also pointed to the way his organization tests software as a possible explanation for the change in scores. "With some changes on the average, or the worst or best performing product, the entire calculation will shift slightly. In general, we cannot see any noticeable performance increase or decrease in the test," he wrote in an e-mail to CNET.

And Milos Korenko, Avast's marketing director, applauded his company's score but criticized AV-Test in general for using too small a sample set. He explained that while AV-Test tests against 100 samples or so per month, Avast sees more than 500,000 unique infected domains during the same period.

While it's true that real-world test results vary depending on the particular kinds of infections that are attacking or on the rise at a given time, it's unusual for so many suites from the major publishers to all slip significantly at the same time. It could be because of the test computer: Windows XP is a decade-old operating system and has notoriously worse default security than Windows 7.

Natalie Severino, Trend Micro's director of consumer product marketing, didn't think much of these results. She explained that the results for Trend Micro Titanium, as well as several other vendors, are a mix of this and last year's versions, and therefore don't accurately reflect the improvements to this year's model.

It also could be because of a rise in a previously less-prevalent form of malware, or an entirely new form of malware that few of the suites were prepared for. Another reasonable explanation would be that most of the suites are now optimized for Windows 7, and people still on Windows XP will continue to suffer ever-greater security-related injustices as their archaic operating system plods along.

What this doesn't mean, though, is that it's time immediately to drop your current security suite. If there were several tests in a row with below-expected results, or several tests at the same time from different test organizations that showed poor results, then it might be worth considering a change. But one bad test does not mean your suite will fail you.

Update, September 14 at 2 p.m. PT: Adds comments from several security suite vendors and from AV-Test.

CNET Top 5
Companies Apple could buy with their billions
Apple's sitting on a massive pile of cash. Here are five interesting ways they could spend it.
Play Video
 

Member Comments