Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Background
Source of Test Cases
Test Case Generation Method
Test Case Granularity
Test Case Generation
Test Case Execution
References
All Topics
Computer Science
Cybersecurity

Automatic Testing of Program Security Vulnerabilities

Profile image of hossain shahriarhossain shahriar

2009

https://doi.org/10.1109/COMPSAC.2009.191
visibility

…

description

6 pages

link

1 file

Abstract

Vulnerabilities in applications and their widespread exploitation through successful attacks are common these days. Testing applications for preventing vulnerabilities is an important step to address this issue. In recent years, a number of security testing approaches have been proposed. However, there is no comparative study of these work that might help security practitioners select an appropriate approach for their needs. Moreover, there is no comparison with respect to automation capabilities of these approaches. In this work, we identify seven criteria to analyze program security testing work. These are vulnerability coverage, source of test cases, test generation method, level of testing, granularity of test cases, testing automation, and target applications. We compare and contrast prominent security testing approaches available in the literature based on these criteria. In particular, we focus on work that address four most common but dangerous vulnerabilities namely buffer overflow, SQL injection, format string bug, and cross site scripting. Moreover, we investigate automation features available in these work across a security testing process. We believe that our findings will provide practical information for security practitioners in choosing the most appropriate tools.

Related papers

Security Testing of Web Applications: Issues and Challenges
Gaurav Raj

International Journal of Computer Applications, 2014

Due to the increasing complexity of web systems, security testing has become indispensable and critical activity of web application development life cycle. Security testing aims to maintain the confidentiality of the data, to check against any information leakage and to maintain the functionality as intended. It checks whether the security requirements are fulfilled by the web applications when they are subjected to malicious input data. Due to the rising explosion in the security vulnerabilities, there occurs a need to understand its unique challenges and issues which will eventually serve as a useful input for the security testing tool developers and test managers for their relative projects.

View PDFchevron_right
Exploring Software Security Test Generation Techniques: Challenges and Opportunities
Hamid A Basit

International journal of education and information technologies, 2021

Ensuring the security of the software has raised concerns from the research community which triggered numerous approaches that tend to eliminate it. The process of ensuring the security of software includes the introduction of processes in the Software Development Life Cycle where one of them is testing after the software is developed. Manually testing software for security is a labor-intensive task. Therefore, it is required to automate the process of testing by generating test cases by automated techniques. In this paper, we review various software security test case generation approaches and techniques. We try to explore and classify the most eminent techniques for test case generation. The techniques are summarized and presented briefly to covers all researches work that has been done in the targeted classification. Moreover, this paper aims to depict the sound of security in the current state of the art of test case generation. The findings are summarized and discussed where the opportunities and challenges are revealed narratively. Although the paper intends to provide a comprehensive view of the research in test case generation, there was a noticeable lack in the test case generation from the security perspectives.

View PDFchevron_right
SOFTWARE SECURITY TESTING A PERTINENT FRAMEWORK
Rajeev Kumar

Different surveys point toward that over the past several years’ software security has raised its precedence for many software organizations. Software security testing is upgraded with the help of security attributes, security models, testing tools and most importantly test case used in testing. Software security testing was once considered a technical obligation performed by software developers. At that time, software security was much important during the test phase of software development life cycle. Present time an emergent number of security incidents and a growing awareness among business owners about invalidated software due to security issues have moved security testing into software world of testers. It is often impossible to find all the errors in a program. This basics problem in testing thus arise an open question, as to what would be the policy that we would adopt for testing. Thus now security testing is important in each phase of development life cycle. Here is a proposed framework is finalized for that security testing.

View PDFchevron_right
Security Testing Web Applications throughout Automated Software Tests
Buat Akun

owasp.org

Testing software during the development phase has become an important part of the development lifecycle and is key to agile methodologies. Code quality and maintainability is increased by adopting an integrated testing strategy that stresses unit tests, integration tests and acceptance tests throughout the project. But these tests are typically only focused on the functional requirements of the application, and rarely include security tests. Implementing security in the unit testing cycle means investing more in developer awareness of security and how to test for security issues, and less in specialised external resources. This is a long-term investment that can vastly improve the overall quality of software, and reduce the number of vulnerabilities in web applications, and consequently, the associated risks.

View PDFchevron_right
Detection and Mitigation of Web Application Vulnerabilities Based on Security Testing
Namje Park

2012

Abstract The paper proposes a security testing technique to detect known vulnerabilities of web applications using both static and dynamic analysis. We also present a process to improve the security of web applications by mitigating many of the vulnerabilities revealed in the testing phase, and address a new method for detecting unknown vulnerabilities by applying dynamic black-box testing based on a fuzzing technique.

View PDFchevron_right
Comparative Analysis And Evaluation Of Software Vulnerabilities Testing Techniques
Zia Paracha

2013

Software and applications are subjected to serious and damaging security threats, these threats are increasing as a result of increased number of potential vulnerabilities. Security testing is an indispensable process to validate software security requirements and to identify security related vulnerabilities. In this paper we analyze and compare different available vulnerabilities testing techniques based on a pre defined criteria using analytical hierarchy process (AHP). We have selected five testing techniques which includes Source code analysis, Fault code injection, Robustness, Stress and Penetration testing techniques. These testing techniques have been evaluated against five criteria which include cost, thoroughness, Ease of use, effectiveness and efficiency. The outcome of the study is helpful for researchers, testers and developers to understand effectiveness of each technique in its respective domain. Also the study helps to compare the inner working of testing techniques a...

View PDFchevron_right
MCP: A Security Testing Tool Driven by Requirements
Fabrizio Pastore

2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 2019

We present MCP, a tool for automatically generating executable security test cases from misuse case specifications in natural language (i.e., use case specifications capturing the behavior of malicious users). MCP relies on Natural Language Processing (NLP), a restricted form of misuse case specifications, and a test driver API implementing basic utility functions for security testing. NLP is used to identify the activities performed by the malicious user and the control flow of misuse case specifications. MCP matches the malicious user's activities to the methods of the provided test driver API in order to generate executable security test cases that perform the activities described in the misuse case specifications. MCP has been successfully evaluated on an industrial case study.

View PDFchevron_right
CRAXweb: Automatic Web Application Testing and Attack Generation
Shih-Kun Huang

2013 IEEE 7th International Conference on Software Security and Reliability, 2013

This paper proposes to test web applications and generate the feasible exploits automatically, including cross-site scripting and SQL injection attacks. We test the web applications with initial random inputs by detecting symbolic queries to SQL servers or symbolic responses to HTTP servers. After symbolic outputs detected, we are able to generate attack strings and reproduce the results, emulating the manual attack behavior. In contrast with other traditional detection and prevention methods, we can determine the presence of vulnerabilities and prove the feasibility of attacks. This automatic generation process is based on a dynamic software testing method-symbolic execution by S 2 E. We have applied this automatic process to several known vulnerabilities on large-scale open source web applications, and generated the attack strings successfully. Our method is web platform independent, covering PHP, JSP, Rails, and Django due to the supports of the whole system environment of S 2 E.

View PDFchevron_right
Abhishek Nandula 1 A Survey paper on Security Testing AIT 682-Network and Systems Security
abhishek yadav

Identifying vulnerabilities and ensuring security by testing is a widely applied measure to evaluate and improve the security of any software. Due to the openness of modern software-based systems, applying appropriate security testing techniques is of growing importance and essential to perform effective and efficient security testing. Therefore, an overview of actual security testing techniques is of high value both for researchers to evaluate and refine the techniques and for practitioners to apply and disseminate them. This survey paper fulfills the need and provides an overview of recent security testing techniques. It first summarizes the required background of testing and security engineering. Then, basics and recent developments of security testing techniques applied during the secure software development lifecycle, i.e. model-based security testing, code-based testing and static analysis, penetration testing and dynamic analysis, as well as security regression testing are discussed. Finally, the security testing techniques are illustrated.

View PDFchevron_right
State of the Art: Automated BlackBox Web Application Vulnerability Testing
kal lam

2010

Black-box web application vulnerability scanners are automated tools that probe web applications for security vulnerabilities. In order to assess the current state of the art, we obtained access to eight leading tools and carried out a study of: (i) the class of vulnerabilities tested by these scanners, (ii) their effectiveness against target vulnerabilities, and (iii) the relevance of the target vulnerabilities to vulnerabilities found in the wild. To conduct our study we used a custom web application vulnerable to known and projected vulnerabilities, and previous versions of widely used web applications containing known vulnerabilities. Our results show the promise and effectiveness of automated tools, as a group, and also some limitations. In particular, "stored" forms of Cross Site Scripting (XSS) and SQL Injection (SQLI) vulnerabilities are not currently found by many tools. Because our goal is to assess the potential of future research, not to evaluate specific vendors, we do not report comparative data or make any recommendations about purchase of specific tools.

View PDFchevron_right

References (39)

  1. Common Vulnerabilities and Exposures, http://cve.mitre.org
  2. Open Source Vulnerability Database, http://osvdb.org.
  3. The Open Web Application Security Project (OWASP), http://www.owasp.org/index.php/Top_10_2007.
  4. Aleph One, "Smashing the Stack for Fun and Profit", Phrack Magazine, Volume 7, Issue 49, Nov 1996. http://www.phrack.org/archives/49/P49-14
  5. W. G. Halfond, J. Viegas, and A. Orso, "A Classification of SQL-Injection Attacks and Countermeasures", In Proc. of the Intern. Symposium on Secure Software Engineering (ISSSE 2006), March 2006.
  6. Scut/team teso, "Exploiting Format String Vulnerabilities", 2001, http://doc.bughunter.net/format-string/exploit-fs.html
  7. G. Zuchlinski, "The Anatomy of Cross Site Scripting", November 2003.
  8. M. Dowd, J. McDonald, and J. Schuh, The Art of Software Security Assessment, Addision-Wesley publications, 2007.
  9. A. Mathur, Foundations of Software Testing, First edition, Pearson Education, 2008.
  10. R. Xu, P. Godefroid, and R. Majumdar, "Testing for Buffer Overflows with Length Abstraction", Proceedings of the International Symposium on Software Testing and Analysis, Seattle, WA, July 2008, pp. 27-38.
  11. P. Vilela, M. Machado, and E. Wong, "Testing for Security Vulnerabilities in Software", Proceeding of Software Engineering and Applications (SEA 2002), Cambridge, USA, November 2002.
  12. O. Tal, S. Knight, and T. Dean, "Syntax-based Vulnerabilities Testing of Frame-based Network Protocols", In Proceedings of the 2nd Annual Conference on Privacy, Security and Trust, Fredericton, October 2004, pp. 155-160
  13. Tappenden, A. Beatty, P. Miller, J. Geras, A., and Smith, M., "Agile Security Testing of Web-based Systems via HTTPUnit", Proceedings of Agile Development Conference (ADC), Denver, Colorado, July 2005, pp. 29-38.
  14. Pari Salas, Krishnan, K.J Ross, "Model-Based Security Vulnerability Testing", In Proceedings of Australian Software Engineering Conference, Australia, 2007, pp. 284-296
  15. Yao-Wen Huang; Chung-Hung Tsai; Lee, D.T.; Sy-Yen Kuo; "Non-Detrimental Web Application Security Scanning", Proceedings of the 15 th International Symposium on Software Reliability Engineering, France, Nov. 2004, pp. 219-230.
  16. S. Kals, E. Krida, C. Kruegel, and N. Jovanovic, "SecuBat: A Web Vulnerability Scanner", Proceedings of the 15 th International Conference on World Wide Web, Edinburgh, Scotland, May 2006, pp. 247-256.
  17. Ben Breech and Lori Pollock, "A Framework for Testing Security Mechanisms for Program-based Attacks", Proceedings of the 2005 Workshop on Software Engineering for Secure Systems-Building Trustworthy Applications, St. Louis, Missouri, pp. 1-7.
  18. Yao-Wen Huang; Chung-Hung Tsai; Lee, D.T.; Sy-Yen Kuo; "Non-Detrimental Web Application Security Scanning", Proceedings of the 15 th International Symposium on Software Reliability Engineering, France, Nov. 2004 pp. 219-230.
  19. Y. Kosuga, K. Kono, M. Hanaoka, M. Hishiyama, Y. Takahama, "Sania: Syntactic and Semantic Analysis for Automated Testing against SQL Injection", In Proceedings of the 23 rd Annual Computer Security Applications Conference, 2007, Miami, December 2007, pp. 107-117.
  20. Adam Kieżun, Philip J. Guo, Karthick Jayaraman, and Michael D. Ernst, "Automatic creation of SQL injection and cross-site scripting attacks", MIT Computer Science and Artificial Intelligence Laboratory technical report MIT- CSAIL-TR-2008-054, Cambridge, MA, September, 2008.
  21. Offutt, J.; Wu, Ye.; Du, X.; Huang, H., "Bypass Testing of Web Applications", In Proceedings of the 15 th International Symposium on Software Reliability Engineering, France, November 2004, pp. 187-197.
  22. Sean Mcallister, Engin Kirda, and Christopher Kruegel, "Leveraging User Interactions for In-Depth Testing of Web Applications", Proceedings of the 11 th International Symposium on Recent Advances in Intrusion Detection, 2008, Massachusetts, USA, pp. 191-210.
  23. E. Fong, R. Gaucher, V. Okun, P. Black, "Building A Test Suite for Web Application Scanners", Proceedings of the 41 st Hawaii International. Conference on System Sciences (HICSS'08), Hawaii, January 2008, pp. 478-485.
  24. Jose Fonseca, Marco Vieira, and Henrique Madeira, "Testing and Comparing Web Vulnerability Scanning Tools for SQL Injection and XSS Attacks", In Proceedings of the 13 th Pacific Rim International Symposium on Dependable Computing, Australia, December 2007, pp. 365-372.
  25. G. Vigna, W. Robertson, D. Balzarotti, "Testing Network- based Intrusion Detection Signature Using Mutant Exploits", In Proceedings of the ACM Conference on Computer and Communication Security (ACM CCS), October 2004, Washington DC, pp. 21-30.
  26. FlawFinder, Available: http://www.dwheeler.com/flawfinder/
  27. D. Evans and D. Larochelle, "Improving Security Using Extensible Lightweight Static Analysis", IEEE Software, 19(1):42-51, 2002.
  28. U. Shankar, K. Talwar, J. Foster, and D. Wagner, "Detecting Format String Vulnerabilities with Type Qualifiers", In Proceedings of 10 th USENIX Security Symposium, August 2001, Washington, pp. 201-218.
  29. G. Wassermann and Z. Su, "Static Detection of Cross-site Scripting Vulnerabilities", Proceedings of the 30 th ICSE, Leipzig, Germany, May 2008, pp. 171-180.
  30. V. Benjamin Livshits and Monica S. Lam, "Finding Security Vulnerabilities in Java Applications with Static Analysis", Proceedings of the 14 th Conference on USENIX Security Symposium, Baltimore, MD, August 2005, pp 18-18.
  31. W. Halfond, and A. Orso, "AMNESIA: Analysis and Monitoring for NEutralizing SQL-Injection Attacks", Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering (ASE 2005), Nov 2005, Long Beach, CA, USA, pp.174-183.
  32. G. A. Di Lucca, A. R. Fasolino, M. Mastoianni, and P. Tramontana, "Identifying Cross Site Scripting Vulnerabilities in Web Applications", Proceedings of the Sixth International Workshop on Web Site Evolution (WSE 2004), Chicago, September 2004, pp. 71-80.
  33. S. Thomas and L. Williams, "Using Automated Fix Generation to Secure SQL Statements", Third International Workshop on Software Engineering for Secure Systems (SESS'07), Minneapolis, 2007, pp. 9-14.
  34. J. Lin and J. Chen, "An Automatic Revised Tool for Anti- Malicious Injection", In Proceedings of the 6 th International Conference on Computer and Information Technology (CIT2006), Seoul, Korea, September 2006, pp. 164-169.
  35. H. Shahriar and M. Zulkernine, "Mutation-based Testing of Buffer Overflow Vulnerabilities", To appear in the Proceedings of the Second International Workshop on Security in Software Engineering (IWSSE 2008), IEEE CS Press, Turku, Finland, July 2008, pp. 979-984.
  36. H. Shahriar and M. Zulkernine, "MUSIC: Mutation-based SQL Injection Vulnerability Checking". Proceedings of the Eighth International Conference on Quality Software (QSIC 2008), IEEE CS Press, London, August 2008, pp. 77-86.
  37. H. Shahriar and M. Zulkernine, "Mutation-based Testing of Format String Bugs", Proceedings of 11 th High Assurance Systems Engineering Symposium (HASE 2008), IEEE CS Press, Nanjing, China, December 2008, pp. 229-238
  38. H. Shahriar and M. Zulkernine, "MUTEC: Mutation-based Testing of Cross Site Scripting," To appear in the Proceedings of the 5 th International Workshop on Software Engineering for Secure Systems (SESS), Vancouver, Canada, May 2009.
  39. Felix "FX" Lindner, "Software security is software reliability", Communications of the ACM, Volume 49 , Issue 6, June 2006, pp. 57-61.

Related papers

Automatic web security unit testing
Mahmoud Mohammadi

Proceedings of the 11th International Workshop on Automation of Software Test - AST '16, 2016

Integrating security testing into the workflow of software developers not only can save resources for separate security testing but also reduce the cost of fixing security vulnerabilities by detecting them early in the development cycle. We present an automatic testing approach to detect a common type of Cross Site Scripting (XSS) vulnerability caused by improper encoding of untrusted data. We automatically extract encoding functions used in a web application to sanitize untrusted inputs and then evaluate their effectiveness by automatically generating XSS attack strings. Our evaluations show that this technique can detect 0-day XSS vulnerabilities that cannot be found by static analysis tools. We will also show that our approach can efficiently cover a common type of XSS vulnerability. This approach can be generalized to test for input validation against other types injections such as command line injection.

View PDFchevron_right
Advancements in Security Testing: A Comprehensive Review of Methodologies and Emerging Trends in Software Quality Engineering
Shravan Pargaonkar

INTERNATIONAL JOURNAL OF SCIENCE AND RESEARCH, 2023

In an era dominated by digital interactions and sensitive data exchange, ensuring the security of software applications has become a paramount concern. This article provides an extensive exploration of security testing methodologies and emerging trends that play a pivotal role in safeguarding applications against evolving cyber threats. The article begins by emphasizing the critical importance of security testing in identifying vulnerabilities, mitigating risks, and protecting sensitive user information. It delineates the multifaceted nature of security testing, which encompasses a spectrum of techniques aimed at uncovering vulnerabilities ranging from code-level weaknesses to intricate architectural flaws. Previous decades witnessed the use of various analyzing methods, but they often focused solely on the views of single stakeholders, leading to significant limitations in the development process [2] A comprehensive overview of security testing methodologies is presented, covering diverse approaches such as penetration testing, vulnerability scanning, code reviews, and threat modeling. Each methodology is dissected to elucidate its purpose, scope, and potential benefits, equipping practitioners with a holistic understanding of their applicability and limitations. The article delves into the incorporation of automated tools and technologies in security testing, highlighting the role of dynamic analysis, static analysis, and interactive application security testing (IAST) in efficiently detecting vulnerabilities across various stages of the software development lifecycle. Furthermore, emerging trends in security testing are explored, encompassing areas such as DevSecOps integration, continuous security testing, and threat intelligence sharing. The article underscores the significance of seamlessly integrating security testing into the development pipeline, enabling early detection and remediation of vulnerabilities, and fostering a proactive security posture. Challenges inherent to security testing are addressed, including the dynamic threat landscape, the complexity of modern applications, and the balance between automated scanning and manual analysis. Mitigation strategies are discussed, emphasizing the amalgamation of human expertise with automated tools to achieve comprehensive security assessments. In conclusion, this article serves as a comprehensive reference for practitioners and researchers in the realm of security testing. By synthesizing methodologies, tools, trends, and challenges, it aims to guide the effective implementation of security testing strategies and contribute to the development of resilient and secure software applications in an increasingly interconnected digital ecosystem.

View PDFchevron_right
A Categorized Review on Software Security Testing
NEHA MAHENDRA

International Journal of Computer Applications, 2016

The main objective of security testing is to check the weaknesses of the implemented security mechanism. It is done for finding the vulnerabilities of a system and to determine whether the system is protected from intruders or not. Security testing can be done prior to production or after the production of the system. But, if the security testing is done after the production, then cost will be more and the huge amount of rework will be required to remove the problems. Also the time between the vulnerability is get known and the malicious attack against it, is becoming less. Therefore it is required to include the security testing in the early phases of software development life cycle. The present paper deals with the review of software security testing approaches and techniques proposed so far. The review is presented in a categorized way and tabulated for the last one and half decade (2000-2015).

View PDFchevron_right
A testing framework for Web application security assessment
Elim Chung

Computer Networks, 2005

The rapid development phases and extremely short turnaround time of Web applications make it difficult to eliminate their vulnerabilities. Here we study how software testing techniques such as fault injection and runtime monitoring can be applied to Web applications. We implemented our proposed mechanisms in the Web Application Vulnerability and Error Scanner (WAVES)-a black-box testing framework for automated Web application security assessment. Real-world situations are used to test WAVES and to compare it with other tools. Our results show that WAVES is a feasible platform for assessing Web application security.

View PDFchevron_right
A Review: Software Security Testing
T Pushparaj

International Journal of Computer Science and Engineering, 2017

Software security testing is an essential means to ensure software security and trustiness. Through the developingdifficulty of today's software applications order with the increasing modest pressure has pushed the quality assurance of developed software towards new heights. Software testing is apredictable part of the software development lifecycle, and possession in line with its criticality in the pre and post development procedure makes it something that should be provided with improved and efficient methodologies and techniques.Most technologists acknowledge this responsibility'ssignificance, but themessential some help in understanding how to tackle it. This new section aims to deliver that help by exploring software security best practices.Finally, the paper points out future focus and development ways of software security testing technology.

View PDFchevron_right

Related topics

  • Sql Injection
  • Buffer Overflow
  • Application Software

    • Cited by

    Mitigating program security vulnerabilities: Approaches and challenges
    hossain shahriar

    ACM Computing Surveys, 2012

    Programs are implemented in a variety of languages and contain serious vulnerabilities which might be exploited to cause security breaches. These vulnerabilities have been exploited in real life and caused damages to related stakeholders such as program users. As many security vulnerabilities belong to program code, many techniques have been applied to mitigate these vulnerabilities before program deployment. Unfortunately, there is no comprehensive comparative analysis of different vulnerability mitigation works. As a result, there exists an obscure mapping between the techniques, the addressed vulnerabilities, and the limitations of different approaches. This article attempts to address these issues. The work extensively compares and contrasts the existing program security vulnerability mitigation techniques, namely testing, static analysis, and hybrid analysis. We also discuss three other approaches employed to mitigate the most common program security vulnerabilities: secure programming, program transformation, and patching. The survey provides a comprehensive understanding of the current program security vulnerability mitigation approaches and challenges as well as their key characteristics and limitations. Moreover, our discussion highlights the open issues and future research directions in the area of program security vulnerability mitigation.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved