Recency-based TLB preloading

ACM Transactions on Architecture and Code Optimization, 2013

Translation Lookaside Buffers (TLBs) are critical to overall system performance. Much past research has addressed uniprocessor TLBs, lowering access times and miss rates. However, as Chip MultiProcessors (CMPs) become ubiquitous, TLB design and performance must be reevaluated. Our article begins by performing a thorough TLB performance evaluation of sequential and parallel benchmarks running on a real-world, modern CMP system using hardware performance counters. This analysis demonstrates the need for further improvement of TLB hit rates for both classes of application, and it also points out that the data TLB has a significantly higher miss rate than the instruction TLB in both cases. In response to the characterization data, we propose and evaluate both Inter-Core Cooperative (ICC) TLB prefetchers and Shared Last-Level (SLL) TLBs as alternatives to the commercial norm of private, per-core L2 TLBs. ICC prefetchers eliminate 19% to 90% of Data TLB (D-TLB) misses across parallel work...

ACM SIGPLAN Notices, 2010

Translation Lookaside Buffers (TLBs) are commonly employed in modern processor designs and have considerable impact on overall system performance. A number of past works have studied TLB designs to lower access times and miss rates, specifically for uniprocessors. With the growing dominance of chip multiprocessors (CMPs), it is necessary to examine TLB performance in the context of parallel workloads. This work is the first to present TLB prefetchers that exploit commonality in TLB miss patterns across cores in CMPs. We propose and evaluate two Inter-Core Cooperative (ICC) TLB prefetching mechanisms, assessing their effectiveness at eliminating TLB misses both individually and together. Our results show these approaches require at most modest hardware and can collectively eliminate 19% to 90% of data TLB (D-TLB) misses across the surveyed parallel workloads. We also compare performance improvements across a range of hardware and software implementation possibilities. We find that wh...

2011 IEEE 17th International Symposium on High Performance Computer Architecture, 2011

Translation Lookaside Buffers (TLBs) are critical to processor performance. Much past research has addressed uniprocessor TLBs, lowering access times and miss rates. However, as chip multiprocessors (CMPs) become ubiquitous, TLB design must be re-evaluated. This paper is the first to propose and evaluate shared last-level (SLL) TLBs as an alternative to the commercial norm of private, per-core L2 TLBs. SLL TLBs eliminate 7-79% of system-wide misses for parallel workloads. This is an average of 27% better than conventional private, per-core L2 TLBs, translating to notable runtime gains. SLL TLBs also provide benefits comparable to recently-proposed Inter-Core Cooperative (ICC) TLB prefetchers, but with considerably simpler hardware. Furthermore, unlike these prefetchers, SLL TLBs can aid sequential applications, eliminating 35-95% of the TLB misses for various multiprogrammed combinations of sequential applications. This corresponds to a 21% average increase in TLB miss eliminations compared to private, per-core L2 TLBs. Because of their benefits for parallel and sequential applications, and their readily-implementable hardware, SLL TLBs hold great promise for CMPs.

IEEE Transactions on Computers, 2006

While runahead execution is effective at parallelizing independent long-latency cache misses, it is unable to parallelize dependent long-latency cache misses. To overcome this limitation, this paper proposes a novel hardware technique, addressvalue delta (AVD) prediction. An AVD predictor keeps track of the address (pointer) load instructions for which the arithmetic difference (i.e., delta) between the effective address and the data value is stable. If such a load instruction incurs a long-latency cache miss during runahead execution, its data value is predicted by subtracting the stable delta from its effective address. This prediction enables the pre-execution of dependent instructions, including load instructions that incur long-latency cache misses. We analyze why and for what kind of loads AVD prediction works and describe the design of an implementable AVD predictor. We also describe simple hardware and software optimizations that can significantly improve the benefits of AVD prediction and analyze the interaction of AVD prediction with runahead efficiency techniques and stream-based data prefetching. Our analysis shows that AVD prediction is complementary to these techniques. Our results show that augmenting a runahead processor with a simple, 16-entry AVD predictor improves the average execution time of a set of pointer-intensive applications by 14.3%.

Proceedings of the 16th international conference on Supercomputing - ICS '02, 2002

A processor must know a load instruction's latency to schedule the load's dependent instructions at the correct time. Unfortunately, modern processors do not know this latency until well after the dependent instructions should have been scheduled to avoid pipeline bubbles between themselves and the load. One solution to this problem is to predict the load's latency, by predicting whether the load will hit or miss in the data cache. Existing cache hit/miss predictors, however, can only correctly predict about 50% of cache misses. This paper introduces a new hit/miss predictor that uses a Bloom Filter to identify cache misses early in the pipeline. This early identification of cache misses allows the processor to more accurately schedule instructions that are dependent on loads and to more precisely prefetch data into the cache. Simulations using a modified SimpleScalar model show that the proposed Bloom Filter is nearly perfect, with a prediction accuracy greater than 99% for the SPECint2000 benchmarks. IPC (Instructions Per Cycle) performance improved by 19% over a processor that delayed the scheduling of instructions dependent on a load until the load latency was known, and by 6% and 7% over a processor that always predicted a load would hit the cache and with a counter-based hit/miss predictor respectively. This IPC reaches 99.7% of the IPC of a processor with perfect scheduling.

Object/Component/Service-Oriented …, 2011

Today's embedded systems are considering cache as inherent part of their design. Unfortunately, cache memory behavior heavily depends on the past references which model a large execution history and makes WCET analysis impractical. This paper presents a novel prefetch memory mechanism that simplifies the prediction of cache hits/misses because the memory access times are independent of the execution history. We use local prefetching into on-chip memory together with a customdesigned prefetch controller instead of cache memories to provide for time-predictable memory accesses. To be competitive in code execution time, our approach relies on a special organization of main memory and on a modified compiler that generates code layouts to allow for parallel prefetching from different memory banks. The proposed solution is still in a conceptual phase. The paper discusses design decisions and parameters to be explored.

2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2021

The last level TLB (LLT) and the last level cache (LLC) play a critical role in the overall performance of memoryintensive applications. While management of LLC content has received significant attention, the same may not be true for LLT. In this work, we first explore the well-known concept of dead blocks in caches for TLBs. We find that dead pages are fairly common in the LLT. Different from dead blocks in LLCs, dead pages in LLTs are most often dead-on-arrival, i.e., they produce zero hits in the TLB. We design a storage-efficient dead page predictor that works with a fraction of storage compared to typical dead block predictors. This is important since an LLT itself requires only a few KBs of storage compared to MBs in LLC. We then leverage the dead page information to guide a simple dead block predictor in LLC. This is driven by the observation that dead blocks are often concentrated within dead pages. In effect, we designed a dead page predictor and a correlating dead block predictor with a total storage overhead of only 11KB to bypass predicted dead pages and dead blocks in LLTs and LLCs, respectively. Together, these predictors help improve the IPC of a set of 14 memory-intensive workloads by 8.3%, on average.

IEEE Design & Test, 2016

This paper aims to tackle two fundamental memory bottlenecks: limited off-chip bandwidth and long access latency. Our approach exploits the inherent error resilience of a wide range of applications through an approximation technique, called Rollback-Free Value Prediction (RFVP). When certain safe-toapproximate load operations miss in the cache, RFVP predicts the requested values. However, RFVP does not check for or recover from load value mispredictions, hence avoiding the high cost of pipeline flushes and re-executions. RFVP mitigates long memory access latencies by enabling the execution to continue without stalling for these accesses. To mitigate the limited off-chip bandwidth, RFVP drops a fraction of load requests which miss in the cache after predicting their values. The drop rate then becomes a knob to control the tradeoff between performance/energy efficiency and output quality. Our extensive evaluations show that RFVP, when used in GPUs, yields significant performance improvements and energy reductions for a wide range of quality loss levels.

1999

As the gap between memory and processor speeds continues to widen, cache efficiency is an increasingly important component of processor performance. Compiler techniques have been used to improve instruction and data cache performance for virtually indexed caches by mapping code and data with temporal locality to different cache blocks. In this paper we examine the performance of compiler and hardware approaches for reordering pages in physically addressed caches to eliminate cache misses. The software approach provides a color mapping at compile-time for code and data pages, which can then be used by the operating system to guide its allocation of physical pages. The hardware approach works by adding a page remap field to the TLB, which is used to allow a page to be remapped to a different color in the physically indexed cache while keeping the same physical page in memory. The results show that software page placement provided a 28% speedup and hardware page placement provided a 21% speedup on average for a superscalar processor. For a 4 processor single-chip multiprocessor, the miss rate was reduced from 8.7% down to 7.2% on average.

Advances in Computer Systems Architecture, 2003