The title of this C|Net article declares that Firefox was "buggier, but issued fixes quicker" than Internet Explorer, Opera and Safari last year.
Mozilla reported more vulnerabilities in its Firefox Web browser last year than Internet Explorer, Safari, and Opera combined, but Mozilla dealt with those flaws quicker than Microsoft, according to a new a report by vulnerability-testing company Secunia.
Firefox had 115 reported flaws in 2008, nearly four times as many as every other popular browser, and nearly twice as many as Microsoft and Apple combined, according to browser vulnerability research (PDF) released this week. In comparison, Microsoft reported 31 flaws in IE, Apple reported 32 in Safari, and Opera reported 30.
115 vs. a total of 93. Looks bad, right? But note the key word there ... "reported".
This study doesn't say how many bugs there were in each browser. It can't. You can only count the bugs you find. Unknown bugs are ... well, unknown. You can't count them until you find them ... and then they're not unknown any more.
So, you can look at this two ways.
You can do as the C|Net headline does, and say, "Firefox had more bugs last year than the other three major browsers combined, but fixed them faster."
Or, you can look at it and say, "Firefox found and fixed more bugs last year than the other three major browsers combined, and fixed them faster."
(Neither one, of course, tells you anything about how many unreported bugs still exist in each of those products. As we mentioned above, they can't. It's unknown, and will remain so until and unless some genius comes up with a way to mathematically prove the correctness of a given body of code.)
Take a look at the sample table excerpt in that article. (The Secunia report, unfortunately, does not contain the full table, just that sample.) Of the three Firefox bugs in that excerpt, the longest fix took 86 days, on a bug rated "not critical". The quickest fix of the three was a "less critical" bug, fixed in fifteen days. Of the six Internet Explorer bugs shown, one "less critical" bug from March and two non-critical bugs from May 2008 were still unfixed by December 31.
That's still not the full story. If you read the full Secunia report, there's other revealing information, too. For instance, looking at vulnerabilities in browser plug-ins, there were zero vulnerabilities reported in Opera widgets, one in a Firefox extension, 19 in Flash, 30 in QuickTime, 54 in Java ... and 366 in ActiveX. Microsoft's ActiveX technology, developed by Microsoft after it failed in its bid to take over Java and — in Microsoft's own words — "Kill cross-platform Java by grow[ing] the polluted Java market" by adding Windows-specific extensions in an attempt to lock Java users into Windows, has been a major security hole from day one ... and it clearly still is. An ActiveX plugin for the all-but defunct Netscape Navigator exists, but it's not used much even by the 0.66% of Web users who still use Netscape. Even if every Netscape user used the ActiveX plugin, it would still be insignificant in the market. To all practical purposes, ActiveX means MSIE ... and that means that when you combine browser vulnerabilities and browser-specific-plugin vulnerabilities, Firefox-plus-extensions had 116 total vulnerabilities in 2008, while MSIE-plus-ActiveX had 397.
Opera was still the safest browser on the basis of vulnerability count alone, with 30 total browser-plus-browser-specific-extension-technology vulnerabilities (not counting multi-platform, multi-browser plugins) ... but, far from Firefox being buggier — by a small margin — than the other three combined, MSIE had around two and a half times as many total browser-plus-extension-technology vulnerabilities as all its competitors put together, and more than three times as many as Firefox.
That's a bit of a different perspective, isn't it?
Read beyond the headlines. Think about what's being said. And think about whether the spin really represents the truth of the underlying facts. There are no more effective ways to mislead than by telling part of the truth, or by presenting technically-true — and therefore verifiable — information in a way that makes it appear to say what you want it to say.