OWASP Top Ten 2021 April Update

  • By Brian Glas
  • •
  • 25 Apr, 2021
  • •
We wanted to send everyone updates on our progress related to data analysis, survey, and format of the OWASP Top Ten 2021.

The Data

We are wrapping up the analysis of the data submissions, which is more intensive this cycle than before. Previous data calls for the Top Ten were limited to requesting ~30 CWEs with an option to write in more. While this would make analysis more straightforward, it has the potential not to show what’s going on in the wild. For this Top Ten cycle, we changed it up and asked for all the CWEs that organizations would map to their findings. Boy, did we get what we asked for; instead of 30 CWEs of data to process, we have data for ~380 unique CWEs to review and analyze. In 2017 we had data submitted that represented testing of ~144k applications; for 2021, we are looking at ~500k applications worth of testing data.

Similar to the last Top Ten, we are looking at incidence rate instead of frequency. We do this for two reasons. The frequency will allow a small number of risk categories to dominate the data analysis and hide more impactful issues. For example, if we have a population of 1000 applications and ten of them have an average of 4,000 instances of Cross-site Scripting (XSS) and 200 applications with an average of five instances of SQL Injection the frequency data might look like this:

 Cross-site Scripting: 97.6% of vulnerabilities

 SQL Injection: 2.4% of vulnerabilities

If we look at incidence rate, or how many applications in the population have a particular vulnerability, this provides a very different perspective (and arguably more accurate for measuring risk).

 SQL Injection: 20% incidence rate

 Cross-site Scripting: 1% incidence rate

One other item we are adding to this year’s analysis is the concept of “confidence.” When each organization sent us data, we had them send it in the form of: 

 “X apps were tested for Y CWE, and Z apps were positive.” 

 We can calculate an incidence rate for that population tested. However, we also have a larger population overall, and not all of these applications may have been tested for that type of vulnerability; this is where “confidence” factors into the equation. If we have a CWE with an incidence rate of 50%, but the total number of the overall population tested for it is just 1%, we determine there is lower confidence for that incidence rate.  On the other side, if we have a CWE with an incidence rate of .67%, but the total number of the overall population tested for it is 92%, we have high confidence that it’s accurately representing what’s in the wild.

Side note: Yes, you can argue against this by bringing up factors like quality of test cases, differences in testing between organizations, and so on. However, trying to get to that level of detail is far beyond what a group of volunteers can be reasonably expected to undertake at this time.

The Survey

Many thanks to everyone that took a little time from their day and completed the survey for the Top Ten! Without you, there are no results to analyze :-D.  We finished with a total of 437 responses, which is close to the 516 we had in 2017. Reviewing the timeline of submissions, nothing looks strange or out of the ordinary, so no real worries about someone trying to game the survey.

As we have mentioned before, the reason for the survey is that solely relying on testing data has some limitations and blind spots. We will only get a volume of data on vulnerabilities found once we figure out how to manually test for them, convert that to automated testing, and scale it beyond a few organizations. As a result, we have a time lag, and looking at just the data will always be looking at some point in the past. The time lag is why we create the survey for people in the front lines to share what they believe are essential categories based on their experiences. We will use the survey results to pull in up to two categories that we don’t have data (yet) to represent.

 I would like to look at some of the metadata of who completed the survey, as I think it holds a lot of insight and value. All the metadata questions were optional; the only required question was for ranking the top four categories one thinks are worthy of consideration in the Top Ten; thus, not all of these answers will have 437 responses.

For the first question about experience, the distribution is reasonably balanced; we can see the industry is aging which is no surprise. One takeaway from this distribution that we should pay attention to is the 28.5% of 0-3 years. The longer we are in the industry, the further from our origins we get, and we tend to forget that new people are joining our industry daily. We need to make it a priority to build paths to help group and mature people joining our ranks; otherwise, we will not be able to make the progress we dearly need to help improve security.

Looking at the position of people completing the survey, in-house security is the largest group, with consultants at number two. We support the write-in option for most of these questions because the industry is still a long way from maturing, so we can’t possibly provide every viable option to select. 6.6% of responses wrote in something other than the two options. We have representation from both Professors and Students, people who wear multiple hats, and others.

The primary role question usually receives the most write-in options, and for a good reason, we don’t have a good role standardization in our industry. There is quite a laundry list of submitted roles that people are assigned.

Results from 2021 Survey
Results from 2017
Looking at the position of people completing the survey, in-house security is the largest group, with consultants at number two. We support the write-in option for most of these questions because the industry is still a long way from maturing, so we can’t possibly provide every viable option to select. 6.6% of responses wrote in something other than the two options. We have representation from both Professors and Students, people who wear multiple hats, and others.
The primary role question usually receives the most write-in options, and for a good reason, we don’t have a good role standardization in our industry. There is quite a laundry list of submitted roles that people are assigned.

Advisor / Analyst

Developer / Engineer

Tester

Monitor / Responder

Management

Architect

Researcher

Consultant

Advisor, Tester, Management, Respon…

Audit

Vuln Researcher

Secure Coding Educator

Vendor / Advisor

security architect

Hacker

Multiple

Technical Evangelist - advisory and training fit most closely

Security Architect

Information Security Architect

Security engineer

Red Team

The last metric to look at is how the Top Ten impacts work. The respondents are allowed to select all that apply, as the Top Ten can be used for many things and explains why it doesn’t add up to 100%.  The number one answer is “It helps provides structure for standards, requirements, security tests, test results, etc.,” with 72.8% of the respondents selecting that option.  Number two is “Mostly for the education of developers,” and number three is “We build processes around it.”

Formatting

For this round of updates for the OWASP Top Ten, we plan to focus on a more mobile friendly version of the output. We also plan to update graphics and provide more supporting content and links from the Top Ten.  More updates to follow in the near future.

The Release of the OWASP Top 10:2021

By Brian Glas • September 24, 2021
Release of the OWASP Top 10:2021

OWASP Top Ten September Update

By Brian Glas • September 12, 2021
The draft release of the OWASP Top 10 2021 has been published for review:  https://owasp.org/Top10

Feedback, comments, issues can all be filed in our GitHub project:  https://github.com/OWASP/Top10/issues

A mammoth THANK YOU to everyone that contributed data, time, thoughts, and anything else.

Hundreds of hours went into the data collection, analysis, and initial draft. Here is a high level overview of what is in the draft.

What's changed in the Top 10 for 2021

There are three new categories, four categories with naming and scoping changes, and some consolidation in the Top 10 for 2021.

  • A01:2021-Broken Access Control  moves up from the fifth position; 94% of applications were tested for some form of broken access control. The 34 CWEs mapped to Broken Access Control had more occurrences in applications than any other category. 
  • A02:2021-Cryptographic Failures  shifts up one position to #2, previously known as Sensitive Data Exposure, which was broad symptom rather than a root cause. The renewed focus here is on failures related to cryptography which often leads to sensitive data exposure or system compromise.
  • A03:2021-Injection  slides down to the third position. 94% of the applications were tested for some form of injection, and the 33 CWEs mapped into this category have the second most occurrences in applications. Cross-site Scripting is now part of this category in this edition.
  • A04:2021-Insecure Design  is a new category for 2021, with a focus on risks related to design flaws. If we genuinely want to "move left" as an industry, it calls for more use of threat modeling, secure design patterns and principles, and reference architectures.
  • A05:2021-Security Misconfiguration  moves up from #6 in the previous edition; 90% of applications were tested for some form of misconfiguration. With more shifts into highly configurable software, it's not surprising to see this category move up. The former category for XML External Entities (XXE) is now part of this category.
  • A06:2021-Vulnerable and Outdated Components  was previously titled Using Components with Known Vulnerabilities and is #2 in the industry survey, but also had enough data to make the Top 10 via data analysis. This category moves up from #9 in 2017 and is a known issue that we struggle to test and assess risk. It is the only category not to have any CVEs mapped to the included CWEs, so a default exploit and impact weights of 5.0 are factored into their scores.
  • A07:2021-Identification and Authentication Failures  was previously Broken Authentication and is sliding down from the second position, and now includes CWEs that are more related to identification failures. This category is still an integral part of the Top 10, but the increased availability of standardized frameworks seems to be helping.
  • A08:2021-Software and Data Integrity Failures  is a new category for 2021, focusing on making assumptions related to software updates, critical data, and CI/CD pipelines without verifying integrity. One of the highest weighted impacts from CVE/CVSS data mapped to the 10 CWEs in this category. Insecure Deserialization from 2017 is now a part of this larger category.
  • A09:2021-Security Logging and Monitoring Failures  was previously Insufficient Logging & Monitoring and is added from the industry survey (#3), moving up from #10 previously. This category is expanded to include more types of failures, is challenging to test for, and isn't well represented in the CVE/CVSS data. However, failures in this category can directly impact visibility, incident alerting, and forensics.
  • A10:2021-Server-Side Request Forgery  is added from the industry survey (#1). The data shows a relatively low incidence rate with above average testing coverage, along with above-average ratings for Exploit and Impact potential. This category represents the scenario where the industry professionals are telling us this is important, even though it's not illustrated in the data at this time.

We will be accepting feedback as long as we can and plan to release the final version as part of the OWASP 20th Anniversary on September 24, 2021. 

OWASP Top Ten 2021 August Update

By Brian Glas • August 19, 2021

All told for the data collection; we have thirteen contributors and a grand total of 515k applications represented as non-retests (we have additional data marked as retest, so it's not in the initial data for building the Top 10, but will be used to look at trends and such later).

 We asked ourselves whether we wanted to go with a single CWE for each "category" in the OWASP Top 10. Based on the contributed data, this is what it could have looked something like:

1.     Reachable Assertion

2.     Divide by Zero

3.     Insufficient Transport Layer Encryption

4.     Clickjacking

5.     Known Vulns

6.     Deployment of the Wrong Handler

7.     Infinite Loop

8.     Known Vulns

9.     File or Dir Externally Accessible

10.  Missing Release of Resources

 

And that is why we aren't doing single CWEs from this data. It's not helpful for awareness, training, baselines, etc. So we confirmed that we are building risk categories of groups of related CWEs. As we categorized CWEs, we ran into a decision point, focusing more on  Root Cause  or  Symptom ? 

For example,  Sensitive Data Exposure  is a symptom, and Cryptographic  Failure  is a root cause. Cryptographic Failure can likely lead to Sensitive Data Exposure, but not the other way around. Another way to think about it is a sore arm is a symptom; a broken bone is the root cause for the soreness. Grouping by  Root Cause  or  Symptom  isn't a new concept, but we wanted to call it out. Within the CWE hierarchy, there is a mix of  Root Cause  and  Symptom  weaknesses.  After much thought, we focused on mapping primarily to  Root Cause  categories as possible, understanding that sometimes it's just going to be a  Symptom  category because it isn't classified by root cause in the data. A benefit of grouping by  Root Cause  is that it can help with identification and remediation as well.

We spent a few months grouping and regrouping CWEs by categories and finally stopped. We could have kept going but needed to stop at some point. We have ten categories with an average of almost 20 CWEs per category. The smallest category has one CWE, and the largest category has 40 CWEs. We've received positive feedback related to grouping like this as it can make it easier for training and awareness programs to focus on CWEs that impact a targeted language or framework. Previously we had some Top 10 categories that simply no longer existed in some languages or frameworks, and that would make training a little awkward.

 

Finding Impact (via Exploit and Impact in CVSS)

In 2017, once we defined  Likelihood  using incidence rate from the data, we spent a good while discussing the high-level values for Exploitability , Detectability , and Technical Impact . While four of us used decades of experience to agree, we wanted to see if it could be more data-driven this time around. (We also decided that we couldn't get  Detectability  from data so we are not going to use it for this iteration.)

We downloaded OWASP Dependency Check and extracted the CVSS Exploit and Impact scores grouped by related CWEs. It took a fair bit of research and effort as all the CVEs have CVSSv2 scores, but there are flaws in CVSSv2 that CVSSv3 should address. After a certain point in time, all CVEs are assigned a CVSSv3 score as well. Additionally, the scoring ranges and formulas were updated between CVSSv2 and CVSSv3.   

In CVSSv2, both Exploit and Impact could be up to 10.0, but the formula would knock them down to 60% for Exploit and 40% for Impact. In CVSSv3, the theoretical max was limited to 6.0 for Exploit and 4.0 for Impact. We analyzed the average scores for CVSSv3 after the changes to weighting are factored in; and the Impact scoring shifted higher, almost a point and a half on average, and exploitability moved nearly half a point lower on average. 

There are 125k records of a CVE mapped to a CWE in the NVD data extracted from OWASP Dependency Check at the time of extract, and there are 241 unique CWEs mapped to a CVE. 62k CWE maps have a CVSSv3 score, which is approximately half of the population in the data set. 

For the Top Ten, we calculated average exploit and impact scores in the following manner. We grouped all the CVEs with CVSS scores by CWE and weighted both exploit and impact scored by the percentage of the population that had CVSSv3 + the remaining population of CVSSv2 scores to get an overall average. We mapped these averages to the CWEs in the dataset as Exploit and Impact scoring for the other half of the risk equation. 

We agreed that we would use the high watermark of the incidence rate for each grouping to help set the order of the 2021 Top 10. The results of this will be released shortly as our target release date is Sept 24, 2021, to align with the OWASP 20th Anniversary.

 

 

OWASP Top Ten 2021 April Update

By Brian Glas • April 25, 2021
We wanted to send everyone updates on our progress related to data analysis, survey, and format of the OWASP Top Ten 2021.

Building the 2021 Top Ten Survey

By Brian Glas • February 2, 2021
As we're putting together the survey for the next Top Ten so that you can help pick two vulnerability categories or risks for inclusion, we face the challenge of what to include in the survey.
Share by: