On any given day cybercriminals and nation states are in possession of as many as 100 zero-day software exploits known only to them, NSS Labs has calculated using the commercial vulnerability market as a baseline.
NSS Labs research director Dr. Stefan Frei reached this startling conclusion after studying at up to ten years' worth of software vulnerability data from the two firms that pioneered the market for purchasing flaws from researchers, iDefense (which started its programme in 2002) and TippingPoint (which started in 2005 and is now owned by HP).
NSS found that iDefense's Vulnerability Contributor Program (VCP) and HP TippingPoint's Zero Day Initiative (ZDI) have from birth to late September 2013 published a total of 2,392 vulnerabilities with an average time from purchase to public disclosure of 133 days for the VCP and 174 days for the ZDI.
In Frei's view, this confirms the conventional wisdom that serious zero-day flaws are remaining private and potentially exploitable in attacks for long periods of time; if legitimate vendors take an average of 153 days or five months to make flaws public, cybercriminals are surely able to keep them secret for even longer.
In the case of iDefense and HP TippingPoint, the timescales are dictated by internal rules on disclosing the flaws they buy to affected vendors. However, one might also uncharitably conclude that the software industry is still dragging its feet when it comes to issuing patches.
As an interesting aside, Frei's research offers some detail on the significant influence these two firms have on the flaws being fed into public domain patching cycles which serve as a partial vindication of their once-controversial programmes.
Microsoft for example received 390 flaws from the pair, equivalent to 14 percent of its total over the ten years looked at, with the equivalent percentages for Apple over the same period being 10 percent, Adobe 17 percent, SAP 13 percent, Symantec 18 percent, HP 19 percent and EMC 38 percent, to pick only ones that jump out.
Put another way, the vulnerability programmes of only two small firms have brought to light a remarkably high percentage of unknown flaws. There were considerable differences in how quickly each affacted vendor reacted to such disclosures with most firms taking months to issue a patch.
Frei then turns to the thorny issue of what all this might tell us about the 'known unknown' of the zero-day flaws that are discovered by or sold to criminals groups or nations looking to hack their rivals.
His approach was to use the commercial vulnerability programmes as a best case for calculating the number of non-disclosed flaws that might exist at any one moment in time. Taking 1 August 2012 as a test example in the case of the VCP this turns out to be 20 purchased but undisclosed flaws while the ZDI had 93 in its queue.
Averaged over the last three years for only major software vendors, the figure on any given day was 58.
Extrapolating these numbers to the entire universe of serious undisclosed flaws is tricky not least because other firms such as Google, Mozilla, Facebook and more recently Microsoft and Yahoo also now pay researchers for critical flaws, but it is a reasonable inference that only a small part of the iceberg is visible.
"It is NSS' belief that the figures represent only a minimum estimate of the number of 'known unknowns' and of the amount of time that users are exposed to them" said Frei, who added that he believed the number of flaws not known about on any given day was around 100.
"Some of the parties involved in the exploitation of vulnerabilities have no desire to coordinate vulnerability information with the affected vendors, potentially using this information for offensive operations."
Not all of these entities are criminal and includes smaller boutique research and software broker firms running their own paid and reverse-engineering programmes, defence contractors and of course government agencies such as the NSA. Some of these flaws will come to the notice of the affected vendor through other channels, while many others will surely not.
"It is safe to assume that cyber criminals and government agencies primarily purchase vulnerabilities and exploits that target prevalent products from major vendors. Therefore, these "known unknowns" pose a real and present threat to the security of corporate and private software users," concluded Frei.
His recommendations are that the scale of the vulnerability and zero-day problem is now so vast that enterprises can't simply rely on patching cycles to dig them out of trouble. Cybercriminals are simply too far head on vulnerabilities and firms should assume they will fall prey to unknown vulnerabilities and direct their effort to spotting the results of breaches once they happen.
It would also be unwise to assume that the greatest threat comes from nation states which are certainly not the only entities with money to spend buying zero-days from black hat researchers, according to Frei.
As for software vendors, all would probably benefit from offering bug bounty programmes and should start viewing them as a necessary part off their business model.