When to perform Software security analysis and tests?
Most of the software security practitioners would agree that the common practice of postponing security analysis and tests after the software implementation phase and even after it has been deployed (i.e., during its acceptance phase), makes it extremely difficult to address in a cost-effective, timely manner any vulnerabilities and weaknesses discovered during the analysis and testing process.
Figure [1] illustrates the relation between cost and time in security testing process which may be doubled or tripled due to lack of this testing coverage during its proper time.
 |  |
Figure [1][i] Security testing cost vs. time – The cost of fixing software bugs
Source: OSSTMM – Open Source Security Testing Methodology Manual
Security testing involves in software developing lifecycle to ensure the implementation of security requirements. It is worth mentioning that Security testing is not a phase only in SDLC but it involves also in many system components and processes as illustrated in figure [2] below.
Source: OSSTMM – Open Source Security Testing Methodology Manual
Therefore, each component of the system has different methodologies and techniques to assure the security, while our focus here on the software development lifecycle. The Figure [3] and Figure[4] below illustrate the security testing existence at SDLC
Source: Information Assurance Technology Analysis Center (IATAC)
[ii]Figure [4] illustrates the software security touch-points (a set of best practices) and shows how software practitioners can apply the touch-points to the various software artifacts produced during software development.
These best practices first appeared as a set in 2004 in IEEE Security & Privacy magazine. Since then, they have been adopted (and in some cases adapted) by the U.S. government in the National Cyber Security Task Force report, by Cigital, by the U.S. Department of Homeland Security, and by Ernst and Young.
So here in the below table, a range of security reviews, analysis, and tests can be mapped to the different software lifecycle phases starting with the requirements phase:
| Life Cycle Phase | Reviews/tests |
| Requirements | Security review of requirements and abuse/misuse cases |
| Architecture/Product Design | Architectural risk analysis (including external reviews) |
| Detailed Design | Security review of the design. Development of test plans, including security tests. |
| Coding/Unit Testing | Code review (static and dynamic analysis), white box testing |
| Assembly/Integration Testing | Black box testing (fault injection, fuzz testing) |
| System Testing | Black box testing, vulnerability scanning |
| Distribution/Deployment | Penetration testing (by software testing expert), vulnerability scanning, impact analysis of patches |
| Maintenance/support | (Feedback loop into previous phases), impact analysis of patches and updates |
Security testing in software test plan
The security test plan should be included in the overall software test plan, and should define:
- Security test cases or scenarios (based on misuse and abuse cases)
- Test data, including attack patterns
- Test oracle
- Test tools (white box and black box, static and dynamic)
- Analysis to be performed to interpret, correlate, and synthesize the results of the various tests and outputs from the various tools.
The security test plan should acknowledge that the security assumptions that were valid when the software’s requirements were specified; will probably have changed by the time the software is deployed. The threat environment in which the software will actually operate is unlikely to have remained static. New threats and attack patterns are continually emerging. Also, emerging has new versions of non-developmental components and patches to those components. All these changes have the potential to invalidate at least some of the security assumptions under which the original requirements were specified.
[i] http://www.agitar.com/solutions/why_unit_testing.html
[ii] http://www.swsec.com/resources/touchpoints/
Help to do more!
The content you read is available for free. If you’ve liked any of the articles at this site, please take a second to help us write more and more articles based on real experiences and maintain them for you and others. Your support will make it possible for us.
$10.00
Also published on Medium.
Your blog is awesome. You have shared the most valuable information about Software Security Testing and we need to look out for every possible bug in the software testing program.
Great post but I was wanting to know if you could write a litte more on this subject?
I’d be very thankful if you could elaborate a little bit further.
Thank you!
Hi there! I hope you do not mind but I decided to publish your blog:
http://melsatar.wordpress.com/2012/03/02/software-security-testing-in-sdlc/ to my online directory.
I used, “Software security testing in SDLC? | Mohamed Sami” as your weblog headline.
I hope this is ok with you. In the event that you’d like me to change the title or perhaps remove it entirely, email me at feliciaalmond@web.de. Thank you so much.
Sure, but please, make a redirection link to the blog