Sign up for our newsletter - Navigating the horizon of business technology​
Cybersecurity / CVEs

We analysed 18 years of software vulnerabilities: Here’s what we found

Software is eating the world, and it’s riddled with holes…

Tech Monitor analysis of 123,413 unique software vulnerabilities (“Common Vulnerabilities and Exposures”, or CVEs) dating back almost two decades reveals a rising tsunami of bugs – an increasingly critical challenge for IT leaders in the face of ever-more professionalised and well-resourced cybercrime syndicates that are capable of reverse-engineering patches to exploit such flaws in a highly condensed timescale.

In 2016 just 5,579 bugs were allocated CVEs (an open standard that provides identifiers for cybersecurity vulnerabilities). By 2019 the figure had hit 13,988. It is on track to climb above 20,000 in 2020.

The analysis suggests the rise in security vulnerabilities is, in part, driven by a proliferation of digital products: 2020 is on course to record the most individual vulnerable products in a year — with some 5,635 unique vendor products found to be hackable. (A sample).

To conduct the analysis we pulled all historical CVE data from the National Institute of Standards and Technology’s (NIST) National Vulnerability Database (NVD), which provides JSON files dating back to 2002. We downloaded these using an R script, then extracted, and ran them through a function which parsed the JSON data into human-readable tables. This allowed us to query and drill down into the data.

Here’s what we found.

The two companies with the highest number of reported vulnerabilities in NIST’s database (as per the chart above) are Oracle, with 6,983, and Microsoft, with 6,952, reflecting their scale as well – a critic would argue – gaps in their quality assurance processes. (A more charitable view would be that well-oiled processes are in place at both companies to identify and resolve vulnerabilities in a quasi-public and standardised manner).

Software vulnerabilities seeing a rise in severity…

NIST defines a vulnerability as: “A weakness in the computational logic (e.g. code) found in software and hardware components that, when exploited, results in a negative impact to confidentiality, integrity, or availability.”

Exploits are graded using the CVSS — which scores them based on the type of access needed, whether it needs user interaction or user or administrator privileges, the power it gives the attacker, and the overall severity of the exploit, among other metrics.

Recent years have seen updates to the categorisation system — so for some features older years have been excluded from the analysis. Looking at overall severity, the proportion of those receiving a ‘high’ or ‘critical’ categorisation has increased since 2014.

Part of this is down to the introduction of CVSSv3, which introduced the ‘critical’ category. However, even after excluding critical vulnerabilities, the data shows an increase in overall severity. This means that not only are vulnerabilities becoming more commonplace, they are also becoming more serious. (When analysing time series data longer than a few years, CVSSv3 was used wherever it existed, otherwise CVSSv2 was used.)

The most common way an attacker can exploit a software vulnerability has consistently been through a network — 76% of vulnerabilities so far this year have been remotely exploitable. This followed by local vulnerabilities, where the attacker needs access to the system (21% of vulnerabilities this year).

More often than not, attackers can exploit a software vulnerability in a system without the unwitting help of a human user. However, as the graph below shows, 38% of the time action on the part of a human is necessary (for example, a system administrator installing some software).

In general, most vulnerabilities are “easily” exploitable in terms of CVSS ranking. Since 2016 only 4,797 flaws have had a high attack complexity — meaning an attacker might require perfect timing or other circumstances that make repeating the exploit more difficult. In comparison, a chunky 51,134 vulnerabilities have had a low complexity, meaning the exploit can, in theory, be easily duplicated.

Recent years have seen the integrity impact of software vulnerabilities worsen. Since 2016, the number of exploits that result in a ‘high’ loss of integrity (previously described as ‘complete’) has numbered in the 6,000s each year.

This jump is likely to do with the change in CVSS specification.

Over the past two decades, the same group of vendors have consistently seen the highest number of tech vulnerabilities. Microsoft has never dropped out of the top ten — and is expected to see its annual total top 1,000 for the first time this coming “Patch Tuesday”.

Recent years have also seen Google rise up the rankings. Security researchers have found 723 vulnerabilities in 2020 thus far. Oracle has also been persistently buggy, with the number of vulnerabilities rising since its acquisition of Sun microsystems in 2010: Oracle has recorded 651 so far this year.

Analysis of individual “products” shows that Linux is ahead with 4,111 vulnerabilities, followed by Android with 3,121: perhaps affirming Linus’s law. (Various editions of Windows are classed separately – when added together they would total 3,453, coming in second place.)

Note: The analysis looks only at unique CVEs — meaning figures for individual products will sum to greater than that of their vendors. This is because an individual exploit may apply to more than one product — for example, a recently discovered flaw in Windows 10 may also be a flaw in Windows XP, and so on.

Stepping back from the individual details of the analysis, the broader deluge of patching requirements places huge pressure on IT teams, particularly with the risk of breaking changes and need for hard-reboots with many patches. And while vulnerability management processes may have kept pace — so IT teams can in theory rapidly find those issues in both traditional IT assets and in newer services like cloud or containers — IT operations may not have matured as well.

 

Ed Targett

Editor

Ed Targett was editor of Tech Monitor until December 2020 and previously led editorial at CBR.

Michael Goodier is a data journalist with the New Statesman Media Group .