Makalu: fast recoverable allocation of non-volatile memory

2020

After nearly a decade of anticipation, scalable nonvolatile memory DIMMs are finally commercially available with the release of Intel's 3D XPoint DIMM. This new nonvolatile DIMM supports byte-granularity accesses with access times on the order of DRAM, while also providing data storage that survives power outages. Researchers have not idly waited for real nonvolatile DIMMs (NVDIMMs) to arrive. Over the past decade, they have written a slew of papers proposing new programming models, file systems, libraries, and applications built to exploit the performance and flexibility that NVDIMMs promised to deliver. Those papers drew conclusions and made design decisions without detailed knowledge of how real NVDIMMs would behave or how industry would integrate them into computer architectures. Now that 3D XPoint NVDIMMs are actually here, we can provide detailed performance numbers, concrete guidance for programmers on these systems, reevaluate prior art for performance, and reoptimize pe...

ACM SIGOPS Operating Systems Review, 1993

found in situations involving the meta-data of storage Recoverable virtual memory refers to regions of a virtual repositories. Thus RVM can benefit a wide range of address space on which transactional guarantees are applications from distributed file systems and databases, to offered. This paper describes RVM, an efficient, portable, and easily used implementation of recoverable virtual object-oriented repositories, CAD tools, and CASE tools. memory for Unix environments. A unique characteristic RVM can also provide runtime support for persistent of RVM is that it allows independent control over the programming languages. Since RVM allows independent transactional properties of atomicity, permanence, and control over the basic transactional properties of atomicity, serializability. This leads to considerable flexibility in the permanence, and serializability, applications have use of RVM, potentially enlarging the range of considerable flexibility in how they use transactions. applications than can benefit from transactions. It also simplifies the layering of functionality such as nesting and It may often be tempting, and sometimes unavoidable, to distribution. The paper shows that RVM performs well use a mechanism that is richer in functionality or better over its intended range of usage even though it does not integrated with the operating system. But our experience benefit from specialized operating system support. It also has been that such sophistication comes at the cost of demonstrates the importance of intra-and interportability, ease of use and more onerous programming transaction optimizations. constraints. Thus RVM represents a balance between the system-level concerns of functionality and performance,

ArXiv, 2018

With the availability of hybrid DRAM-NVRAM memory on the memory bus of CPUs, a number of file systems on NVRAM have been designed and implemented. In this paper we present the design and implementation of a file system on NVRAM called DurableFS, which provides atomicity and durability of file operations to applications. Due to the byte level random accessibility of memory, it is possible to provide these guarantees without much overhead. We use standard techniques like copy on write for data, and a redo log for metadata changes to build an efficient file system which provides durability and atomicity guarantees at the time a file is closed. Benchmarks on the implementation shows that there is only a 7 %degradation in performance due to providing these guarantees.

Emerging non-volatile memory technologies (NVM) offer fast and byte-addressable access, allowing to rethink the durability mechanisms of in-memory databases. In this paper, we present Hyrise-NV, a database storage engine that maintains table and index structures on NVM. Our architecture updates the database state and index structures transactionally consistent on NVM using multi-version data structures, allowing to instantly recover databases independent of their size. For index structures, we present nvBTree using multi-versioning to provide failure-atomic tree updates on NVM. We evaluate Hyrise-NV both on DRAM and with hardware-based emulation of NVM using the TPC-C benchmark. Hyrise-NV recovers databases independent of their size, allowing the recovery of a table with 10 million rows in less than 100ms.

Proceedings of the 6th ACM & IEEE International conference on Embedded software - EMSOFT '06, 2006

Portable systems such as cell phones and portable media players commonly use non-volatile RAM (NVRAM) to hold all of their data and metadata, and larger systems can store metadata in NVRAM to increase file system performance by reducing synchronization and transfer overhead between disk and memory data structures. Unfortunately, wayward writes from buggy software and random bit flips may result in an unreliable persistent store. We introduce two orthogonal and complementary approaches to reliably storing file system structures in NVRAM. First, we reinforce hardware and operating system memory consistency by employing page-level write protection and error correcting codes. Second, we perform on-line consistency checking of the filesystem structures by replaying logged file system transactions on copied data structures; a structure is consistent if the replayed copy matches its live counterpart. Our experiments show that the protection mechanisms can increase fault tolerance by six orders of magnitude while incurring an acceptable amount of overhead on writes to NVRAM. Since NVRAM is much faster and consumes far less power than disk-based storage, the added overhead of error checking leaves an NVRAM-based system both faster and more reliable than a disk-based system. Additionally, our techniques can be implemented on systems lacking hardware support for memory management, allowing them to be used on lowend and embedded systems without an MMU.

Persistence, in relation to computer memories, refers to the ability to retain the data in time, without the need of any power supply. Until now, persistence has been always supplied by hard disks or Flash memory drives. Regarding memories, persistence has been until now conceived as a slow service, while volatility has been thought in relation to speed, as it happens in DRAM and in SRAM. Such a dichotomy represents a bottleneck hard to bypass. The panorama of memory devices is changing: new memory technologies are being currently developed, and are expected to be ready for commercialization in the next years. These new technologies will offer features that represent a major qualitative change: these memories will be fast and persistent. This work aims to understand how these new technologies will integrate into operating systems, and to which extent they have the potential to change their current design. Therefore, in the intent to gain this understanding, I have followed these goals throughout the work: - to analyze the economical and technological causes that are triggering these qualitative changes; - to present the new technologies, along with a classification and a description of their features; - to analyze the effects that these technologies might have on models that are currently used in the design of operating systems; - to present and summarize both the opportunities and the potential issues that operating system designers will have to manage in order to use conveniently such new memory devices; - to analyze the proposals found in scientific literature to exploit these new technologies. Following the structure of the title, the first chapter is focused mainly on memory devices, whereas the second chapter will be centered on operating systems. The first chapter, initially, tries to grasp the causes of the expected technological change, beginning with economical observations. Subsequently, the chapter contains some considerations about how different but complementary aspects of the economical relation are urging the semiconductor industry to find new memory technologies, able to satisfy the increasing demand of features and performances. Afterwards, the paper shifts its focus on current technologies and their features. After a brief summary of each specific technology, a short description about the issues shared among all current charge-based technologies follows. Then, the reader finds a presentation of each of the new memory technologies, presented following the order of the ITRS taxonomy related to memory devices: firstly are presented the ones in a prototypical development stage (MRAM, Fe-RAM, PCRAM), then followed by those in an emerging development stage (Ferroelectric RAM, ReRAM, Mott Memory, Macromolecular and molecular memory). The second chapter aims, in its first part, to understand the extent to which current funding models (Von Neumann model and the memory hierarchy) are influenced by the new technologies. As far as the computational model (fetch-decode-execute) does not change, the validity of the Von Neumann model seems to hold. Conversely, as far as it concerns the memory hierarchy, the changes might be extensive: two new layers should be added near to DRAM. After these considerations, some additional observations will be made about how persistence is just a technological property, and how a specific model would be necessary to explicit how an operating system uses it. Afterwards, there will be a description of the use of non-volatile memory technologies such as Phase Change RAM inside fast SSDs. Even if this approach is quite traditional, the scientific literature explains how faster devices would require a deep restructuring of the I/O stack. Such a restructuring is required because the current I/O stack has been developed concentrating on functionality, not efficiency. Fast devices would instead require a high efficiency. This second chapter will then present the most appealing use of persistent memories: as storage class memory, either in replacement of common DRAM, either in tandem with DRAM on the same memory bus. This approach has per se a higher level of complexity, and under the umbrella of SCM there are many viable declinations of use. Firstly some preliminary observations common with all the approaches are made. Then, two easier approaches are presented (no-change and Whole System Persistence). Finally, the approaches that aim to develop a persistent-memory aware operating system will be introduced: most of them uses the file system paradigm to exploit persistence into main memory. The paper proceeds in presenting firstly some peculiarities of the current I/O path used in the Linux operating systems, remarking how caching already moved persistence into main memory; afterwards, some other considerations about consistency are made. Those observations then are used to understand the main differences between standard I/O in respect with memory operations. After a brief presentation of some incomplete approaches proposed by the author, a framework to classify the thoroughness used by the different approaches follows. The paper continues by reporting the efforts of the Linux community and then introduces each specific approach found in literature: Quill, BPFS, PRAMFS, PMFS, SCMFS. Concluding the part about file system, there will be some remarks about integration, a mean to use both file system services and memory services from the same persistent memory. Finally, persistent-memory awareness into user applications, along with a brief introduction of the two main proposals coming from two academic research groups will be presented.

Microprocessors and Microsystems, 1998

In this paper we consider mechanisms for supporting recoverable-persistence in a single address space memory model. In particular, we consider a memory management system, MMS, for a persistent single address space and show how the operations MMS supplies can be exploited by applications to make their state recoverable-persistent. To this end, we present detailed examples of application-level management of recoverable-persistence. The resulting scenario appears to be particularly attractive because of its modularity. Moreover, additional complexity is not introduced at the lower layers of the system and applications that do not need recoverable-persistence need not be penalised. ᭧ 1998 Elsevier Science B.V.

2020

Michael L. Scott University of Rochester, NY, USA scott@cs.rochester.edu Abstract The recent emergence of fast, dense, nonvolatile main memory suggests that certain long-lived data structures might remain in their natural, pointer-rich format across program runs and hardware reboots. Operations on such structures must be instrumented with explicit write-back and fence instructions to ensure consistency in the wake of a crash. Techniques to minimize the cost of this instrumentation are an active topic of current research. We present what we believe to be the first general-purpose approach to building buffered durably linearizable persistent data structures, and a system, Montage, to support that approach. Montage is built on top of the Ralloc nonblocking persistent allocator. It employs a slow-ticking epoch clock, and ensures that no operation appears to span an epoch boundary. If a crash occurs in epoch e, all work performed in epochs e and e− 1 is lost, but all work from prior epoc...

Software: Practice and Experience, 2019

SummaryCurrent computer systems separate main memory from storage, and programming languages typically reflect this distinction using different representations for data in memory and storage. However, moving data back and forth between these different layers and representations compromise both programming and execution efficiency. To remedy this, the concept of orthogonal persistence (OP) was proposed in the early 1980s advocating that, from a programmer's standpoint, there should be no differences in the way that short‐term and long‐term data are manipulated. However, at that time, the underlying implementations still had to cope with the complexity of moving data across memory and storage. Today, recent nonvolatile memory (NVM) technologies, such as resistive RAM and phase‐change memory, allow main memory and storage to be collapsed into a single layer of persistent memory, opening the way for more efficient programming abstractions for handling persistence. In this work, we r...

1991

Reachability-based persistence imposes new requirements for main memory management in general, and garbage collection in particular. After a brief introduction to the characteristics and requirements of reachability-based persistence, we present the design of a run-time storage manager for Persistent Smalltalk and Persistent Modula-3, which allows the reclamation of storage from both temporary objects and buffered persistent objects.