Antivirus and antimalware developers have been in the spotlight for the last month or so and have been the focus of malware developers for much longer over the plan to run the Race to Zero contest at this year's DefCon in Las Vegas. Now, it might be the turn of companies that produce and promote 'This Site is Safe from Hackers'-style certification and coverage for their clients to share the spotlight.

Since at least late 2006 there have been various small groups of interested Information Security researchers that have turned their attention to the quality (or lack thereof) of service provided to Web sites. Unfortunately for the vendors, the results have been just as embarrassing as the protection coverage provided by common antivirus tools -- great at identifying issues that are fairly old and well known, but deficient when it comes to current vulnerabilities.

Worsening the case for the vendors are accusations that their tools are inconsistent across the same class of vulnerability (XSS or SQL Injection, to name two). These accusations have been backed up with numerous examples where the certification fails to deliver.

Since the end of April there has been an increasing chorus of voices speaking out about the poor performance and sometimes downright misleading marketing associated with these products. With noted Web Security researchers such as Ronald van den Heetkamp, Nate McFeters, Jeremiah Grossman, and Jericho publicly airing their grievances with the state of these tools, more people are beginning to sit up and take note of the difference between reality and marketing for the current state of this technology.

It isn't just these tools under the spotlight, with SiteAdvisor, in-browser malicious site alerts, and other similar tools having similar accusations levelled against them, complete with examples where alerts of malicious activity have been misdirected or completely missed. Even then the tools suggested to address the problems have their own limitations, suggesting that the underlying technological problems still have not been addressed properly.

One of the biggest problems that all tools like this face is that the entire lifecycle of an attack against a site and its users can be complete before the list of 'bad' sites or technology can be updated. This means that users trusting in the tick of approval will be at risk of compromise from a site marked safe and others will avoid a safe site due to an out of date list (even if it is only a few hours old, it is enough). When PayPal publicly had an XSS vulnerability disclosed at the end of last week (with no notice of resolution), at least SiteAdvisor still finds it safe.

That might be the least of the problems for Safari users, though, after "Carpet Bombing" was disclosed earlier this week. Carpet Bombing is being used to describe Safari's automated downloading of files without the user's consent via a newly disclosed technique. Placing files in a known position on a user's system is the first step to system compromise in a number of blended attacks (attacks using more than one vulnerability to achieve the desired result). After Apple declared it a non-security issue, the researcher behind the discovery released it publicly along with another problem, where Safari happily runs scripts from local files. This last issue seems very similar to a zero-day code execution vulnerability for Internet Explorer released last week. In both cases, it would take intentional effort from the user for a system to be affected, but it points to continuing serious security problems for browser developers.

With a rapidly changing online security environment, where threats from attackers and vulnerabilities in browsers can be discovered and globally attacked in hours, tools like ScanAlert, SiteAdvisor, and others in their class will always be reactive to what is known. As the gulf between threat emergence and vendor awareness grows, vendors are always going to be playing a game of catchup. Users should be aware of this when they use the output from these tools in determining if a site is safe or not.

Just in case you were wondering where you should look for guidance on how to keep your site at least relatively safe and secure, or if you are just looking for guidance on what is a threat, OWASP is a good place to start, especially with its Top Ten Guide to web vulnerabilities.