Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Analysis and Decomposition
Initialization Input Data Generation
Input Data Generation
Results and Discussion
Related Work
Conclusion
References
All Topics
Computer Science
Embedded Systems

Let's get less optimistic in measurement-based timing analysis

Profile image of Michael ZoldaMichael Zolda

2011, 2011 6th IEEE International Symposium on Industrial and Embedded Systems

Abstract

Measurement-based timing analysis (MBTA) is a hybrid approach that combines execution time measurements with static program analysis techniques to obtain an estimate of the worst-case execution time (WCET) of a program. In order to minimize the chance that the WCET estimate is below the real WCET, the set of representative execution-time measurements has to be selected advisedly. We present an input data generation technique that uses a combination of model checking and genetic algorithms in order to heuristically optimize the set of measurements in terms of safety.

Related papers

A comparison of static analysis and evolutionary testing for the verification of timing constraints
Joachim Wegener

Proceedings. Fourth IEEE Real-Time Technology and Applications Symposium (Cat. No.98TB100245), 1998

This paper contrasts two methods to verify timing constraints of real-time applications. The method of static analysis predicts the worst-case and best-case execution times of a task's code by analyzing execution paths and simulating processor characteristics without ever executing the program or requiring the program's input. Evolutionary testing is an iterative testing procedure, which approximates the extreme execution times within several generations. By executing the test object dynamically and measuring the execution times the inputs are guided yielding gradually tighter predictions of the extreme execution times. We examined both approaches on a number of real world examples. The results show that static analysis and evolutionary testing are complementary methods, which together provide upper and lower bounds for both worst-case and best-case execution times.

View PDFchevron_right
Measurement-based timing analysis
Raimund Kirner

Leveraging Applications of …, 2009

View PDFchevron_right
Automatic timing model generation by CFG partitioning and model checking
Raimund Kirner

Proceedings of the …, 2005

View PDFchevron_right
Metamoc: Modular execution time analysis using model checking
Mads Chr. Olesen

… Workshop on Worst- …

Safe and tight worst-case execution times (WCETs) are important when scheduling hard realtime systems. This paper presents METAMOC, a modular method, based on model checking and static analysis, that determines safe and tight WCETs for programs running on platforms featuring caching and pipelining. The method works by constructing a UPPAAL model of the program being analysed and annotating the model with information from an inter-procedural value analysis. The program model is then combined with a model of the hardware platform and model checked for the WCET. Through support for the platforms ARM7, ARM9 and ATMEL AVR 8-bit, the modularity and retargetability of the method are demonstrated, as only the pipeline needs to be remodelled. Hardware modelling is performed in a state-of-the-art graphical modelling environment. Experiments on the Mälardalen WCET benchmark programs show that taking caching into account yields much tighter WCETs than without modelling caches, and that METAMOC is a sufficiently fast and versatile approach for WCET analysis.

View PDFchevron_right
aiT: Worst-Case Execution Time Prediction by Static Program Analysis
Christian Arinze Ferdinand

IFIP International Federation for Information Processing, 2004

View PDFchevron_right
Automatic Generation of Timing Models for Timing Analysis of High-Level Code
Björn Lisper

Traditional timing analysis is applied only in the late stages of embedded system software development, when the hardware is available and the code is compiled and linked. However, preliminary timing estimates are often needed already in early stages of system development, both for hard and soft real-time systems. If the hardware is not yet fully accessible, or the code is not yet ready to compile or link, then the timing estimation must be done for the source code rather than for the binary. This paper describes how source-level timing models can be derived automatically for given combinations of hardware architecture and compiler. The models are identified from measured execution times for a set of synthetic "training programs" compiled for the hardware platform in question. The models can be used to derive source-level WCET estimates, as well as for estimating the execution times for single program runs. Our experiments indicate that the models can predict the execution times of the final, compiled code with a deviation up to 20%.

View PDFchevron_right
Computing worst case execution time (WCET) by Symbolically Executing a time-accurate Hardware Model
Bruno Monsuez

To ensure that a program will respect all its timing constraints we must be able to compute a safe estimation of its worst case execution time (WCET). However with the increasing sophistication of the processors, computing a precise estimation of the WCET becomes very difficult. In this paper, we propose a novel formal method to compute a precise estimation of the WCET that can be easily parameterized by the hardware architecture. Assuming that there exists an executable timed model of the hardware, we first use symbolic execution to precisely infer the execution time for a given instruction flow. Then we merge the states relying on the loss of precision we are ready to accept.

View PDFchevron_right
Computing Worst Case Execution Time by Symbolically Executing a Time-accurate Hardware Model
Bruno Monsuez

2011

To ensure that a program will respect all its timing constraints we must be able to compute a safe estimation of its worst case execution time (WCET). However with the increasing sophistication of the processors, computing a precise estimation of the WCET becomes very difficult. In this paper, we propose a novel formal method to compute a precise estimation of the WCET that can be easily parameterized by the hardware architecture. Assuming that there exists an executable timed model of the hardware, we first use symbolic execution to precisely infer the execution time for a given instruction flow. Then we merge the states relying on the loss of precision we are ready to accept.

View PDFchevron_right
8th International Workshop on Worst-Case Execution Time Analysis
Raimund Kirner

2008

Following the successful WCET Tool Challenge in 2006, the second event in this series was organized in 2008, again with support from the ARTIST2 Network of Excellence. The WCET Tool Challenge 2008 (WCC'08) provides benchmark programs and poses a number of "analysis problems" about the dynamic, runtime properties of these programs. The participants are challenged to solve these problems with their programanalysis tools. Two kinds of problems are defined: WCET problems, which ask for bounds on the execution time of chosen parts (sub programs) of the benchmarks, under given constraints on input data; and flowanalysis problems, which ask for bounds on the number of times certain parts of the benchmark can be executed, again under some constraints. We describe the organization of WCC'08, the benchmark programs, the participating tools, and the general results, successes, and failures. Most participants found WCC'08 to be a useful test of their tools. Unlike the 2006 Challenge, the WCC'08 participants include several tools for the same target (ARM7, LPC2138), and tools that combine measurements and static analysis, as well as pure staticanalysis tools.

View PDFchevron_right
New Directions in Worst-Case Execution Time analysis
Dimitar Kazakov

2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence), 2008

Most software engineering methods require some form of model populated with appropriate information. Realtime systems are no exception. A significant issue is that the information needed is not always freely available and derived it using manual methods is costly in terms of time and money. Previous work showed how machine learning information derived during software testing can be used to derive loop bounds as part of the Worst-Case Execution Time analysis problem. In this paper we build on this work by investigating the issue of branch prediction.

View PDFchevron_right

References (37)

  1. A. Burns and A. J. Wellings, Real-Time Systems and Programming Languages, 2nd ed. Addison Wesley, 1996.
  2. R. Wilhelm, J. Engblom, A. Ermedahl, N. Holsti, S. Thesing, D. Whal- ley, G. Bernat, C. Ferdinand, R. Heckmann, T. Mitra, F. Mueller, I. Puaut, P. Puschner, J. Staschulat, and P. Stenström, "The worst-case execution-time problem-overview of methods and survey of tools," ACM Trans. Embed. Comput. Syst., vol. 7, no. 3, pp. 1-53, 2008.
  3. R. Kirner, I. Wenzel, B. Rieder, and P. Puschner, Intelligent Systems at the Service of Mankind. Augsburg, Germany: UBooks Verlag, Jan. 2006, vol. 2, ch. Using Measurements as a Complement to Static Worst- Case Execution Time Analysis, pp. 205-226, iSBN: 3-86608-052-2.
  4. S. Stattelmann and F. Martin, "On the use of context information for precise measurement-based execution-time estimation," in Proceedings of 10th International Workshop on Worst-Case Execution Time (WCET) Analysis, B. Lisper, Ed. Austrian Computer Society, July 2010, pp. 68-79.
  5. I. Wenzel, R. Kirner, B. Rieder, and P. Puschner, "Measurement- based timing analysis," in Proc. 3rd Int'l Symposium on Leveraging Applications of Formal Methods, Verification and Validation, Porto Sani, Greece, Oct. 2008.
  6. M. Zolda, S. Bünte, and R. Kirner, "Towards adaptable control flow segmentation for measurement-based execution time analysis," in Proc. 17th International Conference on Real-Time and Network Systems (RTNS), Paris, France, Oct. 2009.
  7. A. Betts and G. Bernat, "Tree-based WCET analysis on instrumentation point graphs," in Proc. 9th IEEE International Symposium on Object- oriented Real-time distributed Computing, Gyeongju, Korea, Apr. 2006.
  8. P. P. Puschner and A. V. Schedl, "Computing maximum task execution times -a graph-based approach," Real-Time Systems, vol. 13, no. 1, pp. 67-91, July 1997.
  9. Y.-T. S. Li and S. Malik, "Performance analysis of embedded software using implicit path enumeration," in DAC '95: Proceedings of the 32nd annual ACM/IEEE Design Automation Conference. New York, NY, USA: ACM, 1995, pp. 456-461.
  10. P. Puschner and C. Koza, "Calculating the maximum execution time of real-time programs," Real-Time Syst., vol. 1, no. 2, pp. 159-176, 1989.
  11. M. Zolda, S. Bünte, and R. Kirner, "Context-sensitivity in IPET for measurement-based timing analysis," in 4th International Symposium On Leveraging Applications of Formal Methods, Verification and Validation (ISoLA'10), Oct. 2010.
  12. C. Darwin, On the Origin of Species by Means of Natural Selection. London: Murray, 1859.
  13. J. H. Holland, Adaptation in Natural and Artificial Systems. Ann Arbor, MI, USA: University of Michigan Press, 1975.
  14. J. Wegener, "Evolutionärer Test von Realzeit-Systemen," Ph.D. disser- tation, Humboldt-Universität zu Berlin, 2001.
  15. D. E. Goldberg, Genetic Algorithms in Search, Optimization and Ma- chine Learning, 1st ed. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc., 1989.
  16. T. Blickle and L. Thiele, "A comparison of selection schemes used in genetic algorithms," ETH Zürich, Switzerland, Tech. Rep. 11, 1995.
  17. J. E. Baker, "Reducing bias and inefficiency in the selection algorithm," in Proceedings of the Second International Conference on Genetic Algorithms on Genetic algorithms and their application. Hillsdale, NJ, USA: L. Erlbaum Associates Inc., 1987, pp. 14-21. [Online]. Available: http://portal.acm.org/citation.cfm?id=42512.42515
  18. W. M. Spears and K. A. D. Jong, "An analysis of multi-point crossover," in Proceedings of the First Workshop on Foundations of Genetic Algo- rithms, G. J. E. Rawlins, Ed. Morgan Kaufmann, 1991, pp. 301-315.
  19. S. Bünte, M. Zolda, M. Tautschnig, and R. Kirner, "Improving the confidence in measurement-based timing analysis," in Proc. 14th IEEE International Symposium on Object/Component/Service-oriented Real- time Distributed Computing (ISORC'11), Mar. 2011, To appear.
  20. A. Holzer, C. Schallhart, M. Tautschnig, and H. Veith, "Fshell: Sys- tematic test case generation for dynamic analysis and measurement," in Proceedings of the 20th International Conference on Computer Aided Verification (CAV 2008), ser. Lecture Notes in Computer Science, vol. 5123. Princeton, NJ, USA: Springer, July 2008, pp. 209-213.
  21. --, "Query-driven program testing," in Proceedings of the Tenth International Conference on Verification, Model Checking, and Abstract Interpretation (VMCAI 2009), ser. Lecture Notes in Computer Science, N. D. Jones and M. Müller-Olm, Eds., vol. 5403. Savannah, GA, USA: Springer, January 2009, pp. 151-166.
  22. E. Clarke, D. Kroening, and F. Lerda, "A tool for checking ANSI-C programs," in Tools and Algorithms for the Construction and Analysis of Systems (TACAS 2004), ser. Lecture Notes in Computer Science, K. Jensen and A. Podelski, Eds., vol. 2988. Springer, 2004, pp. 168- 176.
  23. A. Holzer, C. Schallhart, M. Tautschnig, and H. Veith, "How did you specify your test suite ?" in Proceedings of the 25th IEEE/ACM International Conference on Automated Software Engineering (ASE 2010), Sep. 2010.
  24. U. Khan and I. Bate, "WCET analysis of modern processors using multi- criteria optimisation," in Proceedings of the 1st International Symposium on Search Based Software Engineering (SSBSE '09), 2009, pp. 103-112.
  25. P. Atanassov, "Experimental assessment of worst-case program execu- tion times," Ph.D. dissertation, Technische Universität Wien, Institut für Technische Informatik, Treitlstr. 3/3/182-1, 1040 Vienna, Austria, 2003.
  26. M. Schoeberl, W. Puffitsch, R. U. Pedersen, and B. Huber, "Worst-case execution time analysis for a Java processor," Software: Practice and Experience, vol. 40/6, pp. 507-542, 2010.
  27. P. Puschner and R. Nossal, "Testing the results of static worst-case execution-time analysis," in Real-Time Systems Symposium, 1998. Pro- ceedings., The 19th IEEE, dec 1998, pp. 134 -143.
  28. J. Wegener, M. Grochtmann, and B. Jones, "Testing temporal correctness of real-time systems by means of genetic algorithms," in International Software Quality Week (QW'97), May 1997.
  29. N. Tracey, "A search-based automated test-data generation framework for safety-critical software," Ph.D. dissertation, University of York, Department of Computer Science, 2000.
  30. H.-G. Gross, B. Jones, and D. Eyres, "Evolutionary algorithms for the verification of execution time bounds for real-time software," in Applicable Modelling, Verification and Analysis Techniques for Real- Time Systems (Ref. No. 1999/006), IEE Colloquium on, Jan. 1999, pp. 8/1 -8/8.
  31. I. Bate and U. Khan, "WCET analysis of modern processors using multi- criteria optimisation," Empirical Software Engineering, vol. 16, pp. 5- 28, February 2011.
  32. N. Tracey, J. Clark, and K. Mander, "The way forward for unifying dynamic test case generation: The optimisation-based approach," in In International Workshop on Dependable Computing and Its Applications, 1998, pp. 169-180.
  33. J. Wegener, H. Sthamer, B. F. Jones, and D. E. Eyres, "Testing real-time systems using genetic algorithms," Software Quality Control, vol. 6, pp. 127-135, October 1997.
  34. J. Wegener and F. Mueller, "A comparison of static analysis and evolutionary testing for the verification of timing constraints," Real-Time Systems, vol. 21, no. 3, pp. 241-268, November 2001.
  35. H.-G. Gross, "A prediction system for evolutionary testability applied to dynamic execution time analysis," Information and Software Technology, vol. 43, no. 14, pp. 855 -862, 2001.
  36. J. Kennedy and R. Eberhart, "Particle swarm optimization," in Neural Networks, 1995. Proceedings., IEEE International Conference on, vol. 4, Aug. 2002, pp. 1942-1948.
  37. --, "A discrete binary version of the particle swarm algorithm," in Systems, Man, and Cybernetics, 1997. 'Computational Cybernetics and Simulation'., 1997 IEEE International Conference on, vol. 5, Oct. 1997, pp. 4104 -4108 vol.5.

Related papers

Measurement-based worst-case execution time analysis using automatic test-data generation
Raimund Kirner

Proc. 4th Euromicro International …, 2004

View PDFchevron_right
Measurement-based worst-case execution time analysis
Raimund Kirner

2005

View PDFchevron_right
Using a model checker to determine worst-case execution time
Stephen A. Edwards

2009

Hard real-time systems use worst-case execution time (WCET) estimates to ensure that timing requirements are met. The typical approach for obtaining WCET estimates is to employ static program analysis methods. While these approaches provide WCET bounds, they struggle to analyze programs with loops whose iteration counts depend on input data. Such programs mandate user-guided annotations. We propose a hybrid approach by augmenting static program analysis with model-checking to analyze such programs and derive the loop bounds automatically. In addition, we use model-checking to guarantee repeatable timing behaviors from segments of program code. Our target platform is a precision timed architecture: a SPARC-based architecture promising predictable and repeatable timing behaviors. We use CBMC and illustrate our approach on Euclidean's greatest common divisor algorithm (for WCET analysis) and a VGA controller (for repeatable timing validation).

View PDFchevron_right
COMPILER SUPPORT FOR MEASUREMENT-BASED TIMING ANALYSIS
Raimund Kirner

Workshop Program, 2011

Measurement-based timing analysis (MBTA) techniques have been developed as a complimentary to static WCET analysis, in order to exploit worst-case execution time (WCET) analysis at early stages of system development. The direct advantage of MBTA is that, in contrast to static WCET analysis, no timing model of the hardware platform has to be developed. Instead the timing model is generated automatically by performing systematic execution time measurements. MBTA provides high retargetability, as the test suite used for execution time measurements is typically derived from the source code of the program. In order to provide an accurate WCET estimate, the test suite has to provide a sufficient coverage of the temporal system behavior. Here also the compilation tool chain is important as the compiler may introduce additional control flow that is not visible at the source code. In this paper we present FORTAS, an MBTA tool that systematically generates test data using a range of different techniques, like heuristics and model checking. Furthermore, we show how compilersupport for MBTA can provide code optimization while preserving the code coverage achieved by the MBTA test suite at source-code level. First evaluations indicate that the performance penalty for ensuring coverage preservation of the test suite is low.

View PDFchevron_right
Improving the confidence in measurement-based timing analysis
Raimund Kirner

… /Service-Oriented Real- …, 2011

Measurement-based timing analysis (MBTA) is a hybrid approach that combines execution-time measurements with static program analysis techniques to obtain an estimate of the worst-case execution time (WCET) of a program. The most challenging part of MBTA is test data generation. Choosing an adequate set of test vectors determines safety and efficiency of the overall analysis. So far, there are no feasible criteria that determine how well the worst-case temporal behavior of program parts is covered by a given test-suite.

View PDFchevron_right

Related topics

  • Computer Science
  • Real Time Systems
  • Static timing analysis
  • Worst Case Execution Time
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved