Crash-tolerant causal broadcast in O(n) messages

IEEE Transactions on Parallel and Distributed Systems, 1990

We present an innovative approach to the design of faultprocessors agree on exactly the same sequence of broadcast tolerant distributed systems that avoids the several rounds of message exchange required by current protocols for consensus agreement. The messages. approach is based on broadcast communication over a local area It is easy to demonstrate that placing a total order on network, such as an Ethernet or a token ring, and on two novel protocols, broadcast messages, so that every working processor procthe Tram protocol, which provides efficient reliable broadcast communi-esses the same messages in the same order, provides an cation, and the Total protocol, which with high probability promptly immediate solution to the agreement problem. Once this total places a total order on messages and achieves distributed agreement even in the presence of fail-stoo. omission. timing, and communication faults. order is determined, distributed actions can be carried out Reliable distributed operations such as locking, update and commitment, using simple sequential fault-tolerant algorithms. The strategy typically require only a single broadcast message rather than the several is very efficient: for example, locking records in a distributed tens of messages required by current algorithms. database typically requires only a single broadcast message to claim a lock and a single broadcast message to release it.

International Conference on Recent Advances and Innovations in Engineering (ICRAIE-2014), 2014

Causal broadcast is a communication abstraction designed for asynchronous systems. It ensures that the messages broadcast by the processes are delivered in their broadcast causality order, namely, if the broadcast of a message m causally precedes the broadcast of a message m , no process delivers m unless it has previously delivered m. Several algorithms implementing causal broadcast have been proposed for asynchronous systems prone to any number of process crashes. These algorithms rely on an underlying Reliable Broadcast abstraction, whose message cost is n 2. This paper presents a simple causal broadcast algorithm whose cost is n messages per causal broadcast. This is obtained at the cost of protocol messages whose size can be up to n application messages. Hence, the proposed algorithm is particularly interesting for applications whose messages are small.

14th International Conference on Distributed Computing Systems, 1994

The distributed applications require group communications among multiple entities. In the group communication, it is important to discuss in what order each entity in the group can receive data units. In order to realize fault-tolerant systems, the same events have to occur in the same order in each entity. The ordering among the events is known as a causal order. This paper presents a reliable causally ordering broadcast (CO) protocol which provides the same causal ordering of data units for all the entities in the group. In the CO protocol, the data units received are causally ordered by using the sequence numbers of the data units. The CO protocol is based on the fully distributed control scheme, i.e. no master controller, and uses high-speed networks where each entity may fail to receive data units due to the buer overrun. Furthermore, the CO protocol provides asynchronous data transmission for multiple entities in the group.

2000

Several current applications of distributed systems demand a highly scalable support, e.g., cloud computing data centres, grid applications, web search engines like Google, Bing, Yahoo!, etc. In most cases such applications replicate their data or their servers. Some kind of reliable broadcast is needed for ensuring the consistency of such replicas, propagating their updates in a pre-determined order (either FIFO, causal or total) at some time. Since total order demands consensus and implies more than a single communication round for delivering each message, causal order seems to be the ideal candidate for scalable broadcasts that ensure an acceptable degree of consistency. This paper surveys different approaches used in the last decades for enhancing the scalability of this kind of communication service.

ACM SIGOPS Operating Systems Review, 1989

Many distributed and parallel applications can make good use of broadcast communication. In this paper we present a (software) protocol that simulates reliable broadcast, even on an unreliable network. Using this protocol, application programs need not worry about lost messages. Recovery of communication failures is handled automatically and transparently by the protocol. In normal operation, our protocol is more efficient than previously published reliable broadcast protocols. An initial implementation of the protocol on 10 MC68020 CPUs connected by a 10 Mbit/sec Ethernet performs a reliable broadcast in 1.5 msec.

ACM Transactions on Computer Systems, 1985

The design and correctness of a communication facility for a distributed computer system are reported on. The facility provides support for fault-tolerant process groups in the form of a family of reliable multicast protocols that can be used in both local-and wide-area networks. These protocols attain high levels of concurrency, while respecting application-specific delivery ordering constraints, and have varying cost and performance that depend on the degree of ordering desired. In particular, a protocol that enforces causal delivery orderings is introduced and shown to be a valuable alternative to conventional asynchronous communication protocols. The facility also ensures that the processes belonging to a fault-tolerant process group will observe consistent orderings of events affecting the group as a whole, including process failures, recoveries, migration, and dynamic changes to group properties like member rankings. A review of several uses for the protocols in the ISIS system, which supports fault-tolerant resilient objects and bulletin boards, illustrates the significant simplification of higher level algorithms made possible by our approach.

2013

In the process of designing a new fault tolerant run-time for future exascale systems, we discovered that a total order broadcast would be necessary. That is, nodes of a supercomputer should be able to broadcast messages to other nodes even in the face of failures. All messages should be seen in the same order at all nodes. While this is a well studied problem in distributed systems, few researchers have looked at how to perform total order broadcasts at large scales for data availability. Our experience implementing a published total order broadcast algorithm showed poor scalability at tens of nodes. In this paper we present a novel algorithm for total order broadcast which scales logarithmically in the number of processes and is not delayed by most process failures. While we are motivated by the needs of our run-time we believe this primitive is of general applicability. Total order broadcasts are used often in datacenter environments and as HPC developers begins to address fault ...

2002

Replicated services are often required to sustain high loads of multiple concurrent requests. This requirement is hard to balance with strong consistency. Typically, to ensure inter-replica consistency, all replicas should receive all updates. Unfortunately, in this case, a single slow replica may degrade the performance of the whole system. This paper proposes a novel reliable broadcast primitive that uses semantic knowledge to weaken reliable delivery guarantees while, at the same time, ensuring strong consistency at the semantic level. By allowing some obsolete messages to be dropped, the protocol that implements this primitive is able to sustain a higher throughput than a fully reliable broadcast protocol. The usefulness of the primitive and the performance of the protocol are illustrated through a concrete example.

2018 IEEE 37th Symposium on Reliable Distributed Systems (SRDS), 2018

Many distributed protocols and applications rely on causal broadcast to ensure consistency criteria. However, none of causality tracking state-of-the-art approaches scale in large and dynamic systems. This paper presents a new nonblocking causal broadcast protocol suited for dynamic systems. The proposed protocol outperforms state-of-the-art in size of messages, execution time complexity, and local space complexity. Most importantly, messages piggyback control information the size of which is constant. We prove that for both static and dynamic systems. Consequently, large and dynamic systems can finally afford causal broadcast.

HAL (Le Centre pour la Communication Scientifique Directe), 2017

Yet another paper on" the implementation of read/write registers in crash-prone asynchronous messagepassing systems! Yes..., but, differently from its predecessors, this paper looks for a communication abstraction which captures the essence of such an implementation in the same sense that total order broadcast can be associated with consensus, or message causal delivery can be associated with causal read/write registers. To this end, the paper introduces a new communication abstraction, named SCD-broadcast (SCD standing for "Set Constrained Delivery"), which, instead of a single message, delivers to processes sets of messages (whose size can be arbitrary), such that the sequences of message sets delivered to any two processes satisfies some constraints. The paper then shows that: (a) SCD-broadcast allows for a very simple implementation of a snapshot object (and consequently also of atomic read/write registers) in crashprone asynchronous message-passing systems; (b) SCD-broadcast can be built from snapshot objects (hence SCD-broadcast and snapshot objects-or read/write registers-are "computationally equivalent"); (c) SCDbroadcast can be built in message-passing systems where any minority of processes may crash (which is the weakest assumption on the number of possible process crashes needed to implement a read/write register).