RAW: runtime automatic workarounds

2011

Web APIs, characterized by their relative simplicity and their natural suitability for the Web, have become increasingly dominant in the world of services on the Web. Despite their popularity, Web APIs are so heterogeneous in terms of the underlying principles adopted and the means used for publishing them that discovering, understanding and notably invoking Web APIs is nowadays more an art than a science. In this paper, we present our work towards supporting the automated invocation of Web APIs.

IEEE Transactions on Software Engineering, 2022

Automated test case generation for web APIs is a thriving research topic, where test cases are frequently derived from the API specification. However, this process is only partially automated since testers are usually obliged to manually set meaningful valid test inputs for each input parameter. In this article, we present ARTE, an approach for the automated extraction of realistic test data for web APIs from knowledge bases like DBpedia. Specifically, ARTE leverages the specification of the API parameters to automatically search for realistic test inputs using natural language processing, search-based, and knowledge extraction techniques. ARTE has been integrated into RESTest, an open-source testing framework for RESTful APIs, fully automating the test case generation process. Evaluation results on 140 operations from 48 real-world web APIs show that ARTE can efficiently generate realistic test inputs for 64.9% of the target parameters, outperforming the state-of-the-art approach SAIGEN (31.8%). More importantly, ARTE supported the generation of over twice as many valid API calls (57.3%) as random generation (20%) and SAIGEN (26%), leading to a higher failure detection capability and uncovering several real-world bugs. These results show the potential of ARTE for enhancing existing web API testing tools, achieving an unprecedented level of automation. Index Terms-Test data generation, automated testing, web APIs, Web of Data ! 1 INTRODUCTION 1 W EB Application Programming Interfaces (APIs) allow 2 heterogeneous software systems to talk to each other 3 over the network [1], [2]. Modern web APIs typically adhere 4 to the REpresentational State Transfer (REST) architectural 5 style, being referred to as RESTful web APIs [3]. RESTful 6 web APIs typically allow applications to interact by ex-7 changing JSON messages sent over HTTP. In practice, this 8 allows, for example, checking the result of a football match 9 (BeSoccer API [4]), posting a tweet (Twitter API [5]), booking 10 a hotel room (Amadeus API [6]), translating a text (DeepL 11 API [7]), or finding a route between two locations (Open-12 route API [8]). RESTful APIs are commonly described using 13 languages such as the OpenAPI Specification (OAS) [9].

2017

Nowadays, invoking third party code increasingly involves calling web services via their web APIs, as opposed to the more traditional scenario of downloading a library and invoking the library's API. However, there are also new challenges for developers calling these web APIs. In this paper, we highlight a broad set of these challenges and argue for resulting opportunities for software engineering research to support developers in consuming web APIs. We outline two specific research threads in this context: (1) web API specification curation, which enables us to know the signatures of web APIs, and (2) static analysis that is capable of extracting URLs, HTTP methods etc. of web API calls. Furthermore, we present new work on how we combine (1) and (2) to provide IDE support for application developers consuming web APIs. As web APIs are used broadly, research in supporting the consumption of web APIs offers exciting opportunities.

2013

Web application security is an important problem in today’s internet. A major cause of this status is that many program-mers do not have adequate knowledge about secure coding, so they leave applications with vulnerabilities. An approach to solve this problem is to use source code static analysis to find these bugs, but these tools are known to report many false positives that make hard the task of correcting the applica-tion. This paper explores the use of a hybrid of methods to detect vulnerabilities with less false positives. After an initial step that uses taint analysis to flag candidate vulnerabilities, our approach uses data mining to predict the existence of false positives. This approach reaches a trade-off between two ap-parently opposite approaches: humans coding the knowledge about vulnerabilities (for taint analysis) versus automatically obtaining that knowledge (with machine learning, for data mining). Given this more precise form of detection, we do au-tomatic code co...

The International Journal of Software Engineering & Applications (IJSEA), 2024

In the continually evolving domain of software development, guaranteeing the dependability and functionality of Application Programming Interfaces (APIs) is of utmost importance. Traditional approaches to API testing frequently encounter difficulties in keeping up with the dynamic nature of APIs, resulting in inefficiencies and overlooked defects. This research paper investigates the transformative potential of Artificial Intelligence (AI) in API testing, ushering in a new era of intelligent testing. Intelligent API testing harnesses the capabilities of AI to enhance the efficiency, precision, and adaptability of the testing process. API driven techniques enable the production of diverse and realistic test data, ensuring comprehensive test coverage. Furthermore, AI-powered algorithms can anticipate potential issues, identify anomalies, and optimize test case selection, all while adapting to evolving API schemas. This research paper delves into the various aspects of intelligent API testing, encompassing data generation, tools and technologies, benefits and impact, challenges, and real-world use cases. We illustrate how AI empowers testers to discover subtle defects, streamline testing endeavors, and enhance the overall quality of APIdriven applications. As we navigate the era of digital transformation, intelligent API testing emerges as an essential tool in the software development toolkit, enabling organizations to deliver robust and resilient APIs that fulfill the demands of contemporary applications. Embracing AI in API testing not only holds the promise of expediting the development lifecycle but also ensures that APIs remain agile and reliable in an ever-changing digital landscape.

ACM SIGSOFT Software Engineering Notes, 2004

This paper presents a new compile-time analysis that enables a testing methodology for white-box coverage testing of error recovery code (i.e., exception handlers) in Java web services using compiler-directed fault injection. The analysis allows compiler-generated instrumentation to guide the fault injection and to record the recovery code exercised. (An injected fault is experienced as a Java exception.) The analysis (i) identifies the exception-flow 'def-uses' to be tested in this manner, (ii) determines the kind of fault to be requested at a program point, and (iii) finds appropriate locations for code instrumentation. The analysis incorporates refinements that establish sufficient context sensitivity to ensure relatively precise def-use links and to eliminate some spurious def-uses due to demonstrably infeasible control flow. A runtime test harness calculates test coverage of these links using an exception def-catch metric. Experiments with the methodology demonstrate th...

Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2021

Testing web APIs automatically requires generating input data values such as addressess, coordinates or country codes. Generating meaningful values for these types of parameters randomly is rarely feasible, which means a major obstacle for current test case generation approaches. In this paper, we present ARTE, the first semantic-based approach for the Automated generation of Realistic TEst inputs for web APIs. Specifically, ARTE leverages the specification of the API under test to search for meaningful test inputs for the API parameters in knowledge bases like DBpedia. Our approach has been integrated into RESTest, a state-of-the-art tool for API testing, achieving an unprecedented level of automation which allows to generate up to 100% more valid API calls than existing fuzzing techniques, 30% on average. Evaluation results on a set of 26 real-world APIs show that ARTE can generate realistic inputs for 7 out of every 10 parameters, outperforming related approaches. CCS CONCEPTS • Software and its engineering → Software testing and debugging; • Information systems → RESTful web services.

Proceedings of the 2008 international symposium on Software testing and analysis - ISSTA '08, 2008

Web script crashes and malformed dynamically-generated Web pages are common errors, and they seriously impact usability of Web applications. Current tools for Web-page validation cannot handle the dynamically-generated pages that are ubiquitous on today's Internet. In this work, we apply a dynamic test generation technique, based on combined concrete and symbolic execution, to the domain of dynamic Web applications. The technique generates tests automatically, uses the tests to detect failures, and minimizes the conditions on the inputs exposing each failure, so that the resulting bug reports are small and useful in finding and fixing the underlying faults. Our tool Apollo implements the technique for PHP. Apollo generates test inputs for the Web application, monitors the application for crashes, and validates that the output conforms to the HTML specification. This paper presents Apollo's algorithms and implementation, and an experimental evaluation that revealed 214 faults in 4 PHP Web applications.

2009 IEEE International Conference on Web Services, 2009

Developing robust web services is a difficult task. Field studies show that a large number of web services are deployed with robustness problems (i.e., presenting unexpected behaviors in the presence of invalid inputs). Several techniques for the identification of robustness problems have been proposed in the past. This paper proposes a mechanism that automatically fixes the problems detected. The approach consists of using robustness testing to detect robustness issues and then mitigate those issues by applying inputs verification based on well-defined parameter domains, including domain dependencies between different parameters. This integrated and fully automatable methodology has been used to improve three different implementations of the TPC-App web services. Results show that this tool can be easily used by developers to improve the robustness of web services implementations.

Proceeding of the 3rd international workshop on Principles of engineering service-oriented systems - PESOS '11, 2011

We address the problem of interface-based test coverage for Web services. We suggest an approach to analyze the Application Programming Interface (API) of Web services, calculate the number of possible input combinations and compare it to the number of actual historical invocations. Such API coverage metrics are an indicator to which extent the service has been used. Measuring API coverage is a key concern for assessing the significance of Verification and Validation (V&V) techniques; on the other hand, API coverage metrics can also yield interesting usage reports for a service-based system in production use. The coverage metrics rely on the exact specification of service interfaces, and we provide a mechanism to specify restrictions for data types in the Java Web services framework (JAX-WS). As full enumeration of all possible inputs is often infeasible, we allow the definition of custom coverage metrics by means of domain partitioning: the user divides domain ranges into subsets, and a coverage of 100% means that the logged invocations contain at least one sample for each subset. Based on a prototype implementation, we evaluate different aspects of our approach.