From Static Distributed Systems to Dynamic Systems

IEEE Transactions on Parallel and Distributed Systems, 1990

We present an innovative approach to the design of faultprocessors agree on exactly the same sequence of broadcast tolerant distributed systems that avoids the several rounds of message exchange required by current protocols for consensus agreement. The messages. approach is based on broadcast communication over a local area It is easy to demonstrate that placing a total order on network, such as an Ethernet or a token ring, and on two novel protocols, broadcast messages, so that every working processor procthe Tram protocol, which provides efficient reliable broadcast communi-esses the same messages in the same order, provides an cation, and the Total protocol, which with high probability promptly immediate solution to the agreement problem. Once this total places a total order on messages and achieves distributed agreement even in the presence of fail-stoo. omission. timing, and communication faults. order is determined, distributed actions can be carried out Reliable distributed operations such as locking, update and commitment, using simple sequential fault-tolerant algorithms. The strategy typically require only a single broadcast message rather than the several is very efficient: for example, locking records in a distributed tens of messages required by current algorithms. database typically requires only a single broadcast message to claim a lock and a single broadcast message to release it.

Theoretical Aspects of Reasoning About Knowledge, 1986

Proceedings of the 15th International Middleware Conference on - Middleware '14, 2014

The rise of worldwide Internet-scale services demands large distributed systems. Indeed, when handling several millions of users, it is common to operate thousands of servers spread across the globe. Here, replication plays a central role, as it contributes to improve the user experience by hiding failures and by providing acceptable latency. In this thesis, we claim that atomic multicast, with strong and well-defined properties, is the appropriate abstraction to efficiently design and implement globally scalable distributed systems. Internet-scale services rely on data partitioning and replication to provide scalable performance and high availability. Moreover, to reduce user-perceived response times and tolerate disasters (i.e., the failure of a whole datacenter), services are increasingly becoming geographically distributed. Data partitioning and replication, combined with local and geographical distribution, introduce daunting challenges, including the need to carefully order requests among replicas and partitions. One way to tackle this problem is to use group communication primitives that encapsulate order requirements. While replication is a common technique used to design such reliable distributed systems, to cope with the requirements of modern cloud based "alwayson" applications, replication protocols must additionally allow for throughput scalability and dynamic reconfiguration, that is, on-demand replacement or provisioning of system resources. We propose a dynamic atomic multicast protocol which fulfills these requirements. It allows to dynamically add and remove resources to an online replicated state machine and to recover crashed processes. Major efforts have been spent in recent years to improve the performance, scalability and reliability of distributed systems. In order to hide the complexity of designing distributed applications, many proposals provide efficient highlevel communication abstractions. Since the implementation of a productionready system based on this abstraction is still a major task, we further propose to expose our protocol to developers in the form of distributed data structures. B-trees for example, are commonly used in different kinds of applications, including database indexes or file systems. Providing a distributed, fault-tolerant vii xi xii Contents Contents xi List of Figures xvii List of Tables xxi

TURKISH JOURNAL OF ELECTRICAL ENGINEERING & COMPUTER SCIENCES, 2013

One of the important aspects of decision making and management in distributed systems is collecting accurate information about the available resources of the peers. The previously proposed approaches for collecting such information completely depend on the system's architecture. In the server-oriented architecture, servers assume the main role of collecting comprehensive information from the peers and the system. Next, based on the information about the features of the basic activities and the system, an exact description of the peers' status is produced. Accurate decisions are then made using this description. However, the amount of information gathered in this architecture is too large, and it requires massive processing. On the other hand, updating the information takes time, causing delays and undermining the validity of the information. In addition, due to the limitations imposed by the servers, such architecture is not scalable and dynamic enough. The peer-to-peer architecture was introduced to address these concerns. However, due to a lack of complete knowledge of the peers and the system, the decisions are made without a precise description of the peers' status and are only based on the hardware data collected from the peers. Such an abstract and general image of the peers is not adequate for the purpose of decision making. In this paper, a 4-dimensional model is presented for the purpose of information collection and the exact description of the peer's status, including the features of the peer, the basic activity, the time, and the specifications of the system. The proposed model is for a server-oriented architecture, but it also adapts to the peer-to-peer serverless architecture. Based on this model, a new approach is introduced for information collection and an exact description of the peers' status in a peer-to-peer system based on the Latin square concept. We evaluate the model in the server-oriented and serverless situations. The workload is considered as the basic activity in our evaluation. Our evaluation demonstrates that in a server-oriented situation, increasing the size of the system has a direct relation with time. However, a serverless situation does not follow this behavior.

2020

Reliable broadcast is a powerful primitive guaranteeing that, intuitively, all processes in a distributed system deliver the same set of messages. Œere is a twofold reason why this primitive is appealing: (i) we can implement it deterministically in a completely asynchronous environment, unlike stronger primitives like consensus and total-order broadcast, and yet (ii) it is powerful enough to implement numerous useful applications like payment systems. Œe problem we tackle in this paper is that of dynamic reliable broadcast, i.e., enabling processes to join or leave the system. Œis is desirable property for long-lived applications supposed to be highly available, yet has been precluded in previous asynchronous reliable broadcast protocols. We introduce the first specification of a dynamic Byzantine reliable broadcast (dbrb) primitive that is amenable to an asynchronous implementation. Indeed, we present an algorithm that implements this specification in an asynchronous environment. ...

2006

Distributed Computing Theory continues to be one of the most active research fields in Theoretical Computer Science today. Besides its foundational topics (such as consensus and synchronization), it is currently being enriched with many new topics inspired from modern technological advances (e.g., the Internet). In this note, we present eight open problems in Distributed Computing Theory that span a wide range of topics – both classical and modern. 1 Wait-Free Consensus A consensus protocol is a distributed algorithm where n processes collectively arrive at a common decision value starting from individual process inputs. It must satisfyagreement(all processes decide on the same value), validity (the decision value is an input to some process), and termination(all processes eventually decide). A protocol in an asynchronous shared-memory system is wait-freeif each process terminates in a finite number of its own steps regardless of scheduling. From the FLP impossibility result [31, 54...

This paper studies the problem of maintaining eventual consistent group membership without any synchrony assumption inside an unbounded group G of processes that varies over the time (processes may join and leave the group). Eventual consistency means that if at any time all group membership changes cease, processes will converge in a finite time to a single consistent view. Due to the lack of any synchrony assumption, this specification is well suited to large scale peer to peer environments. The paper also presents two impossibility results and two eventual group membership implementations. Last but not least, pointing out a circularity problem, the paper also shows the impossibility of implementing eventual group membership without the existence of a special peer inside the group.

2009

This chapter describes the P2P-MPI project, a software framework aimed at the development of message-passing programs in large scale distributed networks of computers. Our goal is to provide a light-weight, self-contained software package that requires minimum effort to use and maintain. P2P-MPI relies on three features to reach this goal: i) its installation and use does not require administrator privileges, ii) available resources are discovered and selected for a computation without intervention from the user, iii) program executions can be fault-tolerant on user demand, in a completely transparent fashion (no checkpoint server to configure). P2P-MPI is typically suited for organizations having spare individual computers linked by a high speed network, possibly running heterogeneous operating systems, and having Java applications. From a technical point of view, the framework has three layers: an infrastructure management layer at the bottom, a middleware layer containing the services, and the communication layer implementing an MPJ (Message Passing for Java) communication library at the top. We explain the design and the implementation of these layers, and we discuss the allocation strategy based on network locality to the submitter. Allocation experiments of more than five hundreds peers are presented to validate the implementation. We also present how fault-management issues have been tackled. First, the monitoring of the infrastructure itself is implemented through the use of failure detectors. We theoretically evaluate several candidate protocols for these detectors to motivate our choice for the gossiping protocol called binary round robin. A variant of this protocol is also proposed for a greater reliability. Finally, the system scalability and the theoretical findings are validated through experiments. The second aspect of fault management concerns program executions. Fault-tolerance is provided by the communication library through replication of processes. We describe the underlying protocol and the properties that need to be met in order to insure the correctness of execution. We then discuss how to choose the number of replicas by quantifying how much more robust is an application using replication, depending on the failure model parameters.

2004

This paper presents two main contributions: semi-passive replication and Lazy Consensus. The former is a replication technique with parsimonious processing. It is based on the latter; a variant of Consensus allowing the lazy evaluation of proposed values. Semi-passive replication is a replication technique with parsimonious processing. This means that, in the normal case, each request is processed by only one single process. The most significant aspect of semi-passive replication is that it requires a weaker system model than existing techniques of the same family. For semi-passive replication, we give an algorithm based on the Lazy Consensus. Lazy Consensus is a variant of the Consensus problem that allows the lazy evaluation of proposed values, hence the name. The main difference with Consensus is the introduction of an additional property of laziness. This property requires that proposed values are computed only when they are actually needed. We present an algorithm based on Chandra and Toueg's Consensus algorithm for asynchronous distributed systems with a ♦S failure detector.

We propose a highly available replication control protocol tailored to environments where network partitions are always the result of a gateway failure. Our protocol divides nodes holding replicas into local nodes that can communicate directly with each other and non-local nodes that communicate with other nodes through one or more gateways. While local nodes are assumed to remain up to date as long as they don't crash, non-local nodes are required to maintain a volatile witness on the same network segment as the local nodes and must poll this witness before answering any user request. To speed up recovery from a total failure, each site maintains a list of replicas that were available the last time the data were updated or a replica recovered from a crash. Markov models are used to compare the performance of our protocol with that of the dynamic-linear voting protocol (DLV), which is the best replication control protocol tolerating communication failures. We also observe that ...