Security Code Review Methodology Overview
We start the security code review by using a suite of automated tools including open source static analysis tools, internally developed scripts, and commercial static analysis products. Automated static analysis has shown itself to be fairly effective at finding bugs due to their syntactical nature which make up approximately 50 percent of all software vulnerabilities. The results from these scans are used in creating a prioritized list of security mechanisms to review and potential security vulnerabilities to investigate. This prioritized list will be used in creating a test plan that will ensure complete and efficient coverage of the application and the areas of concern. Because automated scans are not as labor intensive as manual code inspection, automated tools provide organizations the ability to scale up the coverage of an application security program and provide at least some minimum of secure code analysis across an enterprise. In this way, automated static analysis tools have an advantage in their ability to quickly identify "low hanging fruit" across large sets of applications.
However, NSA studies have shown even if a software security team leveraged all static analysis tools available on the market today, the combined results would identify less than 40 percent of the security bugs within an application. Moreover, static analysis tools are incapable of finding application flaws and business logic vulnerabilities which require context and application understanding to identify. Subsequently, Praetorian experts manually validate every issue found and manually inspect the code to overcome the limitations of automated tools. This allows us to apply our knowledge of the business logic, use and abuse cases, and extensive prior experience in the identification of vulnerabilities to reduce the likelihood of false positives and false negatives. Unfortunately, manual methods are also labor intensive and expensive.
For these reasons, the process of automated review combined with a manual review is the best approach. Using both methods together enables our consultants to identify more software security vulnerabilities in both an efficient and cost effective manner.
In addition, when assessing larger applications of 100,000 lines of code or more, we recommend a threat model in conjunction with the security code review. The threat model helps us to understand the application's functionality, technical design, and existing security threats and countermeasures. For large code bases where a threat model is warranted, the threat model helps us to focus our review efforts on the key components of the code and can reduce the amount of code that needs to be reviewed by as much as 70 percent.