Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Background
Experimental Evaluation
Experimental Setup
Detailed Performance Analysis
Related Work
Conclusion
References

Vectorization technology to improve interpreter performance

Profile image of David YusteDavid Yuste

2013, ACM Transactions on Architecture and Code Optimization

https://doi.org/10.1145/2400682.2400685
visibility

…

description

22 pages

link

1 file

Abstract

In the present computing landscape, interpreters are in use in a wide range of systems. Recent trends in consumer electronics have created a new category of portable, lightweight software applications. Typically, these applications have fast development cycles and short life spans. They run on a wide range of systems and are deployed in a target independent bytecode format over Internet and cellular networks. Their authors are untrusted third-party vendors, and they are executed in secure managed runtimes or virtual machines. Furthermore, due to security policies or development time constraints, these virtual machines often lack just-in-time compilers and rely on interpreted execution. At the other end of the spectrum, interpreters are also a reality in the field of high-performance computations because of the flexibility they provide.

Related papers

The Raw Compiler Project
Rajeev Barua

Compilers today are capable of inferring detailed information about program parallelism and analyzing whole-program behavior. However, the traditional interface between the compiler and the processor, as defined by the instruction set architecture (ISA), is unable to communicate much of the compiler knowledge to the processor. The approach taken by modern processors such as superscalars is to incorporate purely run-time algorithms in their hardware to perform analyses and optimizations such as detection of instruction-level parallelism. However, these complex hardware implementations can only exploit a small fraction of the parallelism information available to the compiler.

View PDFchevron_right
Runtime Vectorization Transformations of Binary Code
Nabil Hallou

International Journal of Parallel Programming, 2016

View PDFchevron_right
The Structure and Performance of Ecient Interpreters
David Gregg

2003

Interpreters designed for high general-purpose performance typically perform a large number of indirect branches (3.2%{13% of all executed instructions in our benchmarks). These branches consume more than half of the run-time in a number of congurations we simulated. We evaluate how accurate various existing and proposed branch prediction schemes are on a number of interpreters, how the mispredictions aect the performance of the interpreters and how two dieren t interpreter implementation techniques perform with various branch predictors. We also suggest various ways in which hardware designers, C compiler writers, and interpreter writers can improve the performance of interpreters.

View PDFchevron_right
Running parallel bytecode interpreters on heterogeneous hardware
Juan Fumero, Athanasios Stratikopoulos

Conference Companion of the 4th International Conference on Art, Science, and Engineering of Programming, 2020

Since the early conception of managed runtime systems with tiered JIT compilation, several research attempts have been made to accelerate the bytecode execution. In this paper, we extend prior attempts by performing an initial analysis of whether heterogeneous hardware accelerators in the form of Graphics Processing Units (GPUs) and Field Programmable Gate Arrays (FPGAS) can help towards achieving higher performance during the bytecode interpreter mode. To answer this question, we implemented a simple parallel Java bytecode interpreter written in OpenCL and executed it across a plethora of devices, including GPUs and FPGAs. Our preliminary evaluation shows that under specific workloads, hardware acceleration can yield up to 17x better performance compared to traditional optimized interpreters running on Intel CPUs and up to 214x compared to ARM CPUs.

View PDFchevron_right
Split compilation: an application to just-in-time vectorization
Andrea C Ornstein

2007

In a world of ubiquitous, heterogeneous parallelism, achieving portable performance is a challenge. It requires finely tuned coordination, from the programming language to the hardware, through the compiler and multiple layers of the run-time system. This document presents our work in split compilation and parallelization. Split compilation relies on automatically generated semantical annotations to enrich the intermediate format, decoupling costly offline analyzes from lighter, online or just-in-time program transformations.

View PDFchevron_right
sEc: A portable interpreter optimizing technique for embedded Java virtual machine
Geetha Manjunath

Usenix Java Virtual Machine …, 2002

This paper describes a radical approach to aggressively optimize an embedded Java virtual machine interpretation in a portable way. We call this technique Semantically Enriched Code (sEc). The sEc technique can improve the speed of a JVM by orders of ...

View PDFchevron_right
Compiler Analysis and Optimizations: What is New?
Uday Khedker

2003

Traditional compiler analyses and back-end optimizations, which play an important role in generating efficient code for modern high-performance processors, are quite mature, well understood, and have been widely used in production compilers. However, recent advances in high-performance (general purpose) processor architecture, emergence of novel architectural paradigms, emphasis on application-specific processors and embedded systems, and the increasing trend on compiling applications directly onto silicon present several interesting challenges and opportunities in high performance compilation techniques. In this paper we discuss the trends that are emerging to meet the above challenges. In particular, we discuss recent advances in data flow analyses, compiling techniques for embedded and DSP processors, and compiling techniques that reduce power consumption.

View PDFchevron_right
Performance measurement of interpreted, just-in-time compiled, and dynamically compiled executions
Tia Newhall

1999

With the increasing popularity of Java, interpreted, just-in-time (JIT) compiled and dynamically compiled executions are becoming a common way for application programs to be executed. As a result, performance profiling tools for these types of executions are in greater demand by program developers. In this thesis, we present techniques for measuring and representing performance data from interpreted, just-in-time and dynamically compiled program executions. Our techniques solve problems related to the unique characteristics of these executions that make performance measurement difficult, namely that there is an interdependence between the interpreter virtual machine program and the application program, and that application program code is transformed at run-time by just-in-time and dynamic compilers. Our solution is a representational model for describing performance data that allows for a concrete description of behaviors in interpreted, JIT compiled and dynamically compiled execut...

View PDFchevron_right
SUIF: An Infrastructure for Research on Parallelizing and Optimizing Compilers
Saman Amarasinghe

ACM Sigplan …, 1994

Compiler infrastructures that support experimental research are crucial to the advancement of high-performance computing. New compiler technology must be implemented and evaluated in the context of a complete compiler, but developing such an infrastructure requires a huge investment in time and resources. We have spent a number of years building the SUIF compiler into a powerful, flexible system, and we would now like to share the results of our efforts.

View PDFchevron_right
a synergy between efficient interpretation and fast selective dynamic compilation for the acceleration of embedded Java virtual machines
Hamdi Yahyaoui

We propose, in this paper, a technique for the acceleration of embedded Java virtual machines. The technique relies on an established synergy between efficient interpretation and selective dynamic compilation. Actually, efficient interpretation is achieved by a generated threaded interpreter that is made of a pool of codelets. The latter are native code units efficiently implementing the dynamic semantics of a given bytecode. Besides, each codelet carries out the dispatch to the next bytecode eliminating therefore the need for a costly centralized traditional dispatch mechanism. The acceleration technique described in this paper advocates the use of a selective dynamic compiler to translate performance-critical methods to native code. The translation process takes advantage of the threaded interpreter by reusing most of the previously mentioned codelets. This tight collaboration between the interpreter and the dynamic compiler leads to a fast and lightweight (in terms of footprint) ...

View PDFchevron_right

References (28)

  1. AMDAHL, G. M. 1967. Validity of the single processor approach to achieving large scale computing capabilities. In Proceedings of the Spring Joint Computer Conference (AFIPS'67). ACM, New York, 483-485.
  2. BAILEY, D., BARSZCZ, E., BARTON, J., BROWNING, D., CARTER, R., DAGUM, L., FATOOHI, R., FINEBERG, S., FREDERICKSON, P., LASINKI, T., SCHREIBER, R., SIMON, H., VENKATAKRISHNAN, V., AND WEERATUNGA, S. 1994. The NAS parallel benchmarks. Tech. rep. RNR-94-007, NASA.
  3. BELL, J. R. 1973. Threaded code. Comm. ACM 16, 370-372.
  4. BROWNE, S., DONGARRA, J., GARNER, N., LONDON, K., AND MUCCI, P. 2000. A scalable cross-platform infrastruc- ture for application performance tuning using hardware counters. In Proceedings of the Conference on Supercomputing.
  5. CAMPANONI, S., AGOSTA, G., CRESPI REGHIZZI, S., AND DI BIAGIO, A. 2010. A highly flexible, parallel virtual machine: Design and experience of ILDJIT. Softw. Pract. Experi. 7, 2, 177-207.
  6. CLINGMAN, D., KENDALL, S., AND MESDAGHI, S. 2004. Practical Java Game Programming. Jenifer Niles.
  7. COSTA, R., ORNSTEIN, A. C., AND ROHOU, E. 2007. CLI back-end in GCC. In Proceedings of the GCC Developers' Summit. 111-116.
  8. DE ICAZA, M. 2008. Mono's SIMD support: Making mono safe for gaming. http://tirania.org/blog/archive/ 2008/Nov-03.html.
  9. ECMA International 2006. Common Language Infrastructure (CLI) Partitions I to IV, 4th ed. ECMA Inter- national.
  10. ERTL, M. AND GREGG, D. 2001. The behavior of efficient virtual machine interpreters on modern architectures. In Euro-Par Parallel Processing, Lecture Notes in Computer Science, vol. 2150. 403-413.
  11. ERTL, M. A. AND GREGG, D. 2003. Optimizing indirect branch prediction accuracy in virtual machine inter- preters. In Proceedings of Conference on Programming Language Design and Implementation (PLDI). ACM, New York, 278-288.
  12. ERTL, M. A. AND GREGG, D. 2004. Combining stack caching with dynamic superinstructions. In Proceedings of the Workshop on Interpreters, Virtual Machines and Emulators (IVME'04). ACM, New York, USA, 7-14.
  13. FRUMKIN, M. A. AND SHABANOV, L. 2003. Arithmetic data cube as a data intensive benchmark. Tech. rep. NAS-03-005, NASA. February.
  14. GUTHAUS, M. R., RINGENBERG, J. S., ERNST, D., AUSTIN, T. M., MUDGE, T., AND BROWN, R. B. 2001. MiBench: A free, commercially representative embedded benchmark suite. In IEEE 4th Annual Workshop on Workload Characterization. MONO. The mono project. http://www.mono-project.com. MOONLIGHT 2009. Moonlight. http://www.go-mono.com/moonlight.
  15. NAUMANN, A. AND CANAL, P. 2008. The role of interpreters in high performance computing. In Proceedings of the XII Advanced Computing and Analysis Techniques in Physics Research Conference. 65.
  16. NIE, J., CHENG, B., LI, S., WANG, L., AND LI, X.-F. 2010. Vectorization for java. In Network and Parallel Computing. 3-17.
  17. NUZMAN, D., DYSHEL, S., ROHOU, E., ROSEN, I., WILLIAMS, K., YUSTE, D., COHEN, A., AND ZAKS, A. 2011. Vapor SIMD: Auto-Vectorize once, run everywhere. In Proceedings of the International Symposium on Code Generation and Optimization (CGO).
  18. NUZMAN, D. AND HENDERSON, R. 2006. Multi-Platform auto-vectorization. In Proceedings of the International Symposium on Code Generation and Optimization (CGO).
  19. NUZMAN, D. AND ZAKS, A. 2006. Autovectorization in GCC-Two years later. In GCC Developer's Summit. ORACLE. 2010. Frequently asked questions about the java hotspot vm. http://www.oracle.com/technetwork/ java/hotspotfaq-138619.html.
  20. PALECZNY, M., VICK, C., AND CLICK, C. 2001. The java hotspot TM server compiler. In Proceedings of the Sym- posium on Java TM Virtual Machine Research and Technology Symposium (JVM'01). Vol. 1. USENIX Association, Berkeley.
  21. PAPA, J. AND KINNEY, A. 2010. Silverlight 4 Overview -Technical Features. Microsoft.
  22. POUCHET, L.-N., BONDHUGULA, U., BASTOUL, C., COHEN, A., RAMANUJAM, J., AND SADAYAPPAN, P. 2010. Combined iterative and model-driven optimization in an automatic parallelization framework. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis.
  23. POZO, R. AND MILLER, B. 2000. Scimark. version 2.1c. http://math.nist.gov/scimark2/.
  24. ROHOU, E., DYSHEL, S., NUZMAN, D., ROSEN, I., WILLIAMS, K., COHEN, A., AND ZAKS, A. 2011. Speculatively vectorized bytecode. In Proceedings of the International Conference on High Performance and Embedded Architectures and Compilers (HiPEAC).
  25. SEZNEC, A. AND MICHAUD, P. 2006. A case for (partially) TAgged GEometric history length branch prediction. J. Instruc. Level Parall. 8.
  26. STALLMAN, R. M. AND THE GCC DEVELOPER COMMUNITY. 2011. Using the GNU Compiler Collection -For GCC 4.6.1. Free Software Foundation, Inc.
  27. WIEPRECHT, E., BRUMFIT, J., BAKKER, J., DE CANDUSSIO, N., GUEST, S., HUYGEN, R., DE JONGE, A., MATTHIEW, J. J., OSTERHAGE, S., OTT, S., SIDDIQUI, H., VANDENBUSSCHE, B., DE MEESTER, W., WETZSTEIN, M., WIEZORREK, E., AND ZAAL, P. 2004. The HERSCHEL/PACS common software system as data reduction system. In Astronomical Data Analysis Software and Systems Conference (ADASS) XIII.
  28. XAMARIN. 2011. MonoTouch limitations. http://ios.xamarin.com/documentation/limitations. Received July 2011; revised May 2012; accepted August 2012

Related papers

Interpreter Register Autolocalisation: Improving the Performance of Efficient Interpreters
Nahuel Palumbo

Proceedings of the 6th International Conference on the Art, Science, and Engineering of Programming

Language interpreters are generally slower than (JIT) compiled implementations because they trade off simplicity for performance and portability. However, they are still an important part of modern Virtual Machines (VMs) as part of mixed-mode execution schema. The reasons behind their importance are many. On the one hand, not all code gets hot and deserves to be optimized by JIT compilers. Examples of cold code are tests, command-line applications, and scripts. On the other hand, compilers are more difficult to write and maintain, thus interpreters are an attractive solution because of their simplicity and portability. In the context of this paper, we will center on bytecode interpreters. Interpreter performance has been a hot topic for a long time, where several solutions have been proposed with different ranges of complexity and portability. On the one hand, some work proposes to optimize language-specific features in interpreters such as type dispatches using static type predictions, quickening [3] or type specializations [18]. On the other hand, many solutions focus on improving general interpreter behavior by minimizing branch miss-predictions of interpreter dispatches and stack caching. Solutions to branch mis-predictions propose variants of code threading [1, 4, 6, 7, 10] and improving it further with selective inlining [14]. Some solutions aim for minimizing branch miss-predictions by modifying the intermediate code (e.g., bytecode) design with super-instructions [15] and register-based instructions [9, 16]. Stack caching [5] proposes to optimize the access of operands by caching the top of the stack. interpreter registers are also related to stack caching: interpreter variables that are critical to the efficient execution of the interpreter loop. Examples of such variables are the instruction pointer (IP), the stack pointer (SP), and the frame pointer (FP). Interpreter registers put pressure on the overall design and implementation of the interpreter: Req1: Value access outside the interpreter loop. VM routines outside of the interpreter loop may require access to interpreter registers. For example, this is the case of garbage collectors that need to traverse the stack to find root objects, routines that unwind or reify the stack, or give access to stack values to native methods. Req2: Efficiency. Interpreter registers are used on each instruction to manipulate the instruction stream and the stack. Under-efficient implementations have negative impacts on performance. These two requirements are opposing in the sense that Req1 demands that register values are stored on globally accessible memory, either in the heap or the data sections of the process, while Req2 would benefit from putting those values in registers and avoid memory accesses at all.

View PDFchevron_right
The structure and performance of efficient interpreters
David Gregg

2003

Interpreters designed for high general-purpose performance typically perform a large number of indirect branches (3.2%-13% of all executed instructions in our benchmarks). These branches consume more than half of the run-time in a number of configurations we simulated. We evaluate how accurate various existing and proposed branch prediction schemes are on a number of interpreters, how the mispredictions affect the performance of the interpreters and how two different interpreter implementation techniques perform with various branch predictors. We also suggest various ways in which hardware designers, C compiler writers, and interpreter writers can improve the performance of interpreters.

View PDFchevron_right
Dynamic native optimization of interpreters
Saman Amarasinghe

2003

For domain specific languages, "scripting languages", dynamic languages, and for virtual machine-based languages, the most straightforward implementation strategy is to write an interpreter. A simple interpreter consists of a loop that fetches the next bytecode, dispatches to the routine handling that bytecode, then loops. There are many ways to improve upon this simple mechanism, but as long as the execution of the program is driven by a representation of the program other than as a stream of native instructions, there will be some "interpretive overhead".

View PDFchevron_right
The Behavior of Efficient Virtual Machine Interpreters on Modern Architectures
David Gregg

Euro-Par 2001 Parallel Processing, 2001

Romer et al (ASPLOS 96) examined several interpreters and concluded that they behave much like general purpose integer programs such as gcc. We show that there is an important class of interpreters which behave very differently. Efficient virtual machine interpreters perform a large number of indirect branches (3.2%-13% of all executed instructions in our benchmarks, taking up to 61%-79% of the cycles on a machine with no branch prediction). We evaluate how various branch prediction schemes and methods to reduce the mispredict penalty affect the performance of several virtual machine interpreters. Our results show that for current branch predictors, threaded code interpreters cause fewer mispredictions, and are almost twice as fast as switch based interpreters on modern superscalar architectures.

View PDFchevron_right
Optimizing interpreters by tuning opcode orderings on virtual machines for modern architectures
David Gregg

Proceedings of the 9th International Conference on Principles and Practice of Programming in Java - PPPJ '11, 2011

Virtual machines (VMs) are commonly used to implement programming languages such as Java, Python and Lua. VMs are typically implemented using an interpreter, a JIT compiler, or some combination of the two. A long-standing question in the design of VM interpreters is whether it is worthwhile to reorder the cases in the main interpreter loop to improve code locality. We investigate this phenomenon using an iterative, feedback-directed approach. We show that the ordering of the cases in the interpreter loop has a significant impact on performance on recent processors. Using hardware performance counters, we demonstrate that the performance improvement is primarily the result of indirect branch prediction, not instruction cache locality. We propose a number of strategies to achieve better orderings, and evaluate these strategies in the Python and Lua virtual machine interpreters. We show speedups of up to 40%.

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved