Data structures resilient to memory faults

Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms, 2010

Several existing cache-oblivious dynamic dictionaries achieve O(log B N) (or slightly better O(log B N M)) memory transfers per operation, where N is the number of items stored, M is the memory size, and B is the block size, which matches the classic B-tree data structure. One recent structure achieves the same query bound and a sometimes-better amortized update bound of O 1 B Θ(1/(log log B) 2) log B N + 1 B log 2 N memory transfers. This paper presents a new data structure, the xDict, implementing predecessor queries in O(1 ε log B N M) worstcase memory transfers and insertions and deletions in O 1 εB 1−ε log B N M amortized memory transfers, for any constant ε with 0 < ε < 1. For example, the xDict achieves subconstant amortized update cost when N = M B o(B 1−ε) , whereas the B-tree's Θ(log B N M) is subconstant only when N = o(M B), and the previously obtained Θ 1 B Θ(1/(log log B) 2) log B N + 1 B log 2 N is subconstant only when N = o(2 √ B). The xDict attains the optimal tradeoff between insertions and queries, even in the broader external-memory model, for the range where inserts cost between Ω(1 B lg 1+ε N) and O(1/ lg 3 N) memory transfers.

2004

Abstract We investigate the design of algorithms resilient to memory faults, ie, algorithms that, despite the corruption of some memory values during their execution, are able to produce a correct output on the set of uncorrupted values. In this framework, we consider two fundamental problems: sorting and searching. In particular, we prove that any O (nlog n) comparison-based sorting algorithm can tolerate at most O ((nlog n) 1/2) memory faults.

ArXiv, 2019

Data structures used in software development have inbuilt redundancy to improve software reliability and to speed up performance. Examples include a Doubly Linked List which allows a faster deletion due to the presence of the previous pointer. With the introduction of Persistent Memory, storing the redundant data fields into persistent memory adds a significant write overhead, and reduces performance. In this work, we focus on three data structures - Doubly Linked List, B+Tree and Hashmap, and showcase alternate partly persistent implementations where we only store a limited set of data fields to persistent memory. After a crash/restart, we use the persistent data fields to recreate the data structures along with the redundant data fields. We compare our implementation with the base implementation and show that we achieve speedups around 5-20% for some data structures, and up to 165% for a flush-dominated data structure.

2009

Abstract We address the problem of sorting in the presence of faults that may arbitrarily corrupt memory locations, and investigate the impact of memory faults both on the correctness and on the running times of mergesort-based algorithms. To achieve this goal, we develop a software testbed that simulates different fault injection strategies, and perform a thorough experimental study using a combination of several fault parameters.

2020

Data management applications store their data using structured files in which data are usually sorted to serve indexing and queries. However, in-place insertions and removals of data are not naturally supported in a file’s address space. To avoid repeatedly rewriting existing data in a sorted file to admit changes in place, applications usually employ extra layers of indirections, such as mapping tables and logs, to admit changes out of place. However, this approach leads to increased access cost and excessive complexity. This paper presents a novel storage engine that provides a flexible address space, where in-place updates of arbitrary-sized data, such as insertions and removals, can be performed efficiently. With this mechanism, applications can manage sorted data in a linear address space with minimal complexity. Extensive evaluations show that a key-value store built on top of it can achieve high performance and efficiency with a simple implementation.

2007

Abstract We investigate the problem of computing in a reliable fashion in the presence of faults that may arbitrarily corrupt memory locations. In this framework, we focus on the design of resilient data structures, ie, data structures that, despite the corruption of some memory values during their lifetime, are nevertheless able to operate correctly (at least) on the set of uncorrupted values.

Information Processing Letters, 1996

1991

Data replication has been widely used to build resilient data objects. These objects normally consist of several replicas stored in stable storage and a replication control protocol managing these replicas. Replicated data objects incur a significant penalty resulting from the increased number of disk accesses. We investigate the feasibility of replicated data objects consisting of several memory-resident replicas and one append-only log maintained on disk.

Theoretical Computer Science, 2007

We present a dynamic comparison-based search structure that supports insertions, deletions, and searches within the unified bound. The unified bound specifies that it is quick to access an element that is near a recently accessed element. More precisely, if w(y) distinct elements have been accessed since the last access to element y, and d(x, y) denotes the rank distance between x and y among the current set of elements, then the amortized cost to access element x is O(min y log[w(y) + d(x, y) + 2]). This property generalizes the working-set and dynamic-finger properties of splay trees.

2012

The emergence of multicore architecture as well as larger, less expensive RAM has made it possible to leverage the performance superiority of main memory for large databases. Increasingly, large scale applications demanding high performance have also made RAM an appealing candidate for primary storage. However, conventional DRAM is volatile, meaning that hardware or software crashes result in the loss of data. The existing solutions to this, such as write-ahead logging and replication, result in either partial loss of data or significant performance reduction. We propose an approach to provide durability to memory databases, with a negligible overhead and a low probability of data loss. We exploit known techniques such as chain replication, write-ahead logging and sequential writes to disk to provide durability while maintaining the high throughput and the low latency of main memory.