Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
ZZ Design
ZZ Design Overview
Design Challenges
ZZ Protocol
Experimental Evaluation
Trade-Offs and Discussion
Agreement Protocol Performance
Limitations & Potential for Optimization
Related Work
Conclusions
References
All Topics
Computer Science
Distributed Systems

ZZ and the art of practical BFT execution

Profile image of Rahul SinghRahul Singh

2011, Proceedings of the sixth conference on Computer systems

https://doi.org/10.1145/1966445.1966457
visibility

…

description

15 pages

link

1 file

Abstract

The high replication cost of Byzantine fault-tolerance (BFT) methods has been a major barrier to their widespread adoption in commercial distributed applications. We present ZZ, a new approach that reduces the replication cost of BFT services from 2f + 1 to practically f + 1. The key insight in ZZ is to use f + 1 execution replicas in the normal case and to activate additional replicas only upon failures. In data centers where multiple applications share a physical server, ZZ reduces the aggregate number of execution replicas running in the data center, improving throughput and response times. ZZ relies on virtualization-a technology already employed in modern data centers-for fast replica activation upon failures, and enables newly activated replicas to immediately begin processing requests by fetching state on-demand. A prototype implementation of ZZ using the BASE library and Xen shows that, when compared to a system with 2f + 1 replicas, our approach yields lower response times and up to 33% higher throughput in a prototype data center with four BFT web applications. We also show that ZZ can handle simultaneous failures and achieve sub-second recovery.

References (12)

  1. Abd-El-Malek 2005] Michael Abd-El-Malek, Gregory R. Ganger, Garth R. Goodson, Michael K. Reiter, and Jay J. Wylie. Fault- scalable Byzantine Fault-Tolerant Services. SIGOPS Oper. Syst. Rev., 39(5):59-74, 2005. ISSN 0163-5980.
  2. Adya 2002] Atul Adya, William J. Bolosky, Miguel Castro, Ger- ald Cermak, Ronnie Chaiken, John R. Douceur, Jon Howell, Ja- cob R. Lorch, Marvin Theimer, and Roger P. Wattenhofer. FAR- SITE: Federated, Available, and Reliable Storage for an Incom- pletely Trusted Environment. In Proc. of the 5th Symposium on Operating Systems Design and Implementation (OSDI), 2002. [Castro 1999] M. Castro and B. Liskov. Practical Byzantine Fault Tolerance. In Proceedings of the Third Symposium on Operating Systems Design and Implementation, February 1999. [Castro 2002] Miguel Castro and Barbara Liskov. Practical Byzan- tine Fault Tolerance and Proactive Recovery. ACM Transactions on Computer Systems (TOCS), 20(4), November 2002. [Clement 2009] A. Clement, M. Marchetti, E. Wong, L. Alvisi, and M. Dahlin. Making Byzantine Fault Tolerant Systems Tolerate Byzantine Faults. In 6th USENIX Symposium on Networked Systems Design and Implementation (NSDI), April 2009. [Cowling 2006] James Cowling, Daniel Myers, Barbara Liskov, Rodrigo Rodrigues, and Liuba Shrira. HQ Replication: A Hy- brid Quorum Protocol for Byzantine Fault Tolerance. In Pro- ceedings of the Seventh Symposium on Operating Systems De- sign and Implementations (OSDI), Seattle, Washington, Novem- ber 2006.
  3. Brendan Cully, Geoffrey Lefebvre, Dutch Meyer, Mike Feeley, Norm Hutchinson, and Andrew Warfield. Remus: High Availability via Asynchronous Virtual Machine Replica- tion. In NSDI, 2008. [Distler 2011a] Tobias Distler and Rüdiger Kapitza. Increasing Per- formance in Byzantine Fault-Tolerant Systems with On-Demand Replica Consistency. In European Chapter of ACM SIGOPS, editor, Proceedings of the EuroSys 2011 Conference (EuroSys '11), 2011.
  4. Distler 2011b] Tobias Distler, Rüdiger Kapitza, Ivan Popov, Hans P. Reiser, and Wolfgang Schröder-Preikschat. SPARE: Replicas on Hold. In Internet Society (ISOC), editor, Proceed- ings of the 18th Network and Distributed System Security Sym- posium (NDSS '11), 2011. [Dwork 1988] Cynthia Dwork, Nancy Lynch, and Larry Stock- meyer. Consensus in the Presence of Partial Synchrony. Journal of the ACM, 35(2), 1988.
  5. Fischer 1985] Michael J. Fischer, Nancy A. Lynch, and Michael S. Paterson. Impossibility of Distributed Consensus with One Faulty Process. J. ACM, 32(2):374-382, 1985. ISSN 0004-5411.
  6. Garfinkel 2003] Tal Garfinkel, Ben Pfaff, Jim Chow, Mendel Rosenblum, and Dan Boneh. Terra: a Virtual Machine-based Platform for Trusted Computing. In SOSP '03: Proceedings of the nineteenth ACM symposium on Operating systems princi- ples, pages 193-206, New York, NY, USA, 2003. ACM Press. [Guerraoui 2010] Rachid Guerraoui, Nikola Knežević, Vivien Quéma, and Marko Vukolić. The Next 700 BFT Protocols. In EuroSys '10: Proceedings of the 5th European conference on Computer systems, pages 363-376, New York, NY, USA, 2010. ACM. ISBN 978-1-60558-577-2.
  7. Kihlstrom 1998] Kim Potter Kihlstrom, L. E. Moser, and P. M. Melliar-Smith. The SecureRing Protocols for Securing Group Communication. In HICSS '98: Proceedings of the Thirty-First Annual Hawaii International Conference on System Sciences- Volume 3, Washington, DC, USA, 1998.
  8. Kotla 2007] Ramakrishna Kotla, Lorenzo Alvisi, Mike Dahlin, Allen Clement, and Edmund Wong. Zyzzyva: Speculative Byzantine Fault Tolerance. In SOSP '07: Proceedings of twenty- first ACM SIGOPS Symposium on Operating Systems Principles, New York, NY, USA, 2007. ACM. [Lamport 1998] L. Lamport. Part Time Parliament. ACM Transac- tions on Computer Systems, 16(2), May 1998. [Lamport 1982] L. Lamport, R. Shostack, and M. Pease. The Byzantine Generals Problem. ACM Transactions on Program- ming Languages and Systems, 4(3):382-401, 1982. [Lamport 1978] Leslie Lamport. Time, Clocks, and the Ordering of Events in a Distributed System. Commun. ACM, 21(7), 1978. [Lamport 2004] Leslie Lamport and Mike Massa. Cheap Paxos. In DSN '04: Proceedings of the 2004 International Conference on Depen dable Systems and Networks, page 307, Washington, DC, USA, 2004. IEEE Computer Society. ISBN 0-7695-2052-9. [Oki 1988] Brian M. Oki and Barbara H. Liskov. Viewstamped Replication: a General Primary Copy. In PODC '88: Proceed- ings of the seventh annual ACM Symposium on Principles of dis- tributed computing, New York, NY, USA, 1988. ACM. [Reiter 1995] Michael K. Reiter. The Rampart Toolkit for Building High-integrity Services. In Selected Papers from the Interna- tional Workshop on Theory and Practice in Distributed Systems, London, UK, 1995. Springer-Verlag.
  9. Rodrigues 2001] Rodrigo Rodrigues, Miguel Castro, and Barbara Liskov. BASE: Using Abstraction to Improve Fault Tolerance. In Proceedings of the eighteenth ACM symposium on Operating systems principles, New York, NY, USA, 2001. [Singh 2008] Atul Singh, Tathagata Das, Petros Maniatis, Peter Druschel, and Timothy Roscoe. BFT Protocols Under Fire. In NSDI '08: Proceedings of the Usenix Symposium on Networked System Design and Implementation, 2008.
  10. Sousa 2007] Paulo Sousa, Alysson N. Bessani, Miguel Correia, Nuno F. Neves, and Paulo Verissimo. Resilient Intrusion Toler- ance Through Proactive and Reactive Recovery. In Proceedings of the 13th Pacific Rim International Symposium on Dependable Computing, Washington, DC, USA, 2007.
  11. Vandiver 2007] Ben Vandiver, Hari Balakrishnan, Barbara Liskov, and Sam Madden. Tolerating Byzantine Faults in Database Systems Using Commit Barrier Scheduling. In Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP), Stevenson, Washington, USA, October 2007. [Wood 2011] Timothy Wood, Rahul Singh, Arun Venkataramani, Prashant Shenoy, and Emmanuel Cecchet. ZZ and the Art of Practical BFT. Technical report, University of Massachusetts Amherst, Feb. 2011.
  12. Yin 2003] J. Yin, J.P. Martin, A. Venkataramani, L. Alvisi, and M. Dahlin. Separating Agreement from Execution for Byzan- tine Fault Tolerant Services. In Proceedings of the 19th ACM Symposium on Operating Systems Principles, October 2003. [ZFS 2004 ] ZFS. The Last Word in File Systems. http://www.sun.com/2004-0914/feature/, 2004.

Related papers

Zeno: Eventually Consistent Byzantine-Fault Tolerance
rodrigo rodrigues

2009

Many distributed services are hosted at large, shared, geographically diverse data centers, and they use replication to achieve high availability despite the unreachability of an entire data center. Recent events show that non-crash faults occur in these services and may lead to long outages. While Byzantine-Fault Tolerance (BFT) could be used to withstand these faults, current BFT protocols can become unavailable if a small fraction of their replicas are unreachable. This is because existing BFT protocols favor strong safety guarantees (consistency) over liveness (availability).

View PDFchevron_right
Practical byzantine fault tolerance and proactive recovery
Miguel Castro

ACM Transactions on Computer Systems, 2002

Our growing reliance on online services accessible on the Internet demands highly available systems that provide correct service without interruptions. Software bugs, operator mistakes, and malicious attacks are a major cause of service interruptions and they can cause arbitrary behavior, that is, Byzantine faults. This article describes a new replication algorithm, BFT, that can be used to build highly available systems that tolerate Byzantine faults. BFT can be used in practice to implement real services: it performs well, it is safe in asynchronous environments such as the Internet, it incorporates mechanisms to defend against Byzantine-faulty clients, and it recovers replicas proactively. The recovery mechanism allows the algorithm to tolerate any number of faults over the lifetime of the system provided fewer than 1/3 of the replicas become faulty within a small window of vulnerability. BFT has been implemented as a generic program library with a simple interface. We used the library to implement the first Byzantine-fault-tolerant NFS file system, BFS. The BFT library and BFS perform well because the library incorporates several important optimizations, the most important of which is the use of symmetric cryptography to authenticate messages. The performance results show that BFS performs 2% faster to 24% slower than production implementations of the NFS protocol that are not replicated. This supports our claim that the BFT library can be used to build practical systems that tolerate Byzantine faults.

View PDFchevron_right
ZZ: Cheap practical BFT using virtualization
Rahul Singh

2008

Despite numerous efforts to improve their performance and scalability, Byzantine fault-tolerance (BFT) techniques remain expensive, and few commercial systems use BFT today. We present ZZ, a novel approach to construct general BFT services with a replication cost of practically f + 1, halving the 2f + 1 or higher cost incurred by state-of-the-art approaches. The key insight in ZZ is to use f + 1 execution replicas in the normal case and to activate additional replicas only upon failures. ZZ uses virtual machines for fast replica activation and several novel mechanisms for rapid recovery of these replicas such as using filesystem snapshots to reduce checkpointing overhead, replaying state updates instead of full requests, and an amortized state transfer mechanism that fetches state on-demand. We have implemented ZZ using the BASE library, Xen virtual machines and the ZFS file system. Our experimental evaluation shows that the recovery time of ZZ replicas is independent of the application disk state, taking less than 4s for 400MB of disk state, at the expense of a small increase in request latency in the fault-mode.

View PDFchevron_right
Scrooge: Reducing the costs of fast Byzantine replication in presence of unresponsive replicas
Neeraj Suri

2010 IEEE/IFIP International Conference on Dependable Systems & Networks (DSN), 2010

Byzantine-Fault-Tolerant (BFT) state machine replication is an appealing technique to tolerate arbitrary failures. However, Byzantine agreement incurs a fundamental trade-off between being fast (i.e. optimal latency) and achieving optimal resilience (i.e. 2f + b + 1 replicas, where f is the bound on failures and b the bound on Byzantine failures [10]). Achieving fast Byzantine replication despite f failures requires at least f + b − 2 additional replicas . In this paper we show, maybe surprisingly, that fast Byzantine agreement despite f failures is practically attainable using only b − 1 additional replicas, which is independent of the number of crashes tolerated. This makes our approach particularly appealing for systems that must tolerate many crashes (large f ) and few Byzantine faults (small b). The core principle underlying our approach is to have the correct replicas agree on a quorum of responsive replicas before agreeing on requests. This is key to circumventing the resilience lower bound of fast Byzantine agreement .

View PDFchevron_right

Related topics

  • Computer Science
  • Design
  • Data Centers

    • Cited by

    Antifragile and Resilient Geographical Information System Service Delivery in Fog Computing
    Alina Eugenia Pascual Barrera

    Sensors

    The demand for cloud computing has drastically increased recently, but this paradigm has several issues due to its inherent complications, such as non-reliability, latency, lesser mobility support, and location-aware services. Fog computing can resolve these issues to some extent, yet it is still in its infancy. Despite several existing works, these works lack fault-tolerant fog computing, which necessitates further research. Fault tolerance enables the performing and provisioning of services despite failures and maintains anti-fragility and resiliency. Fog computing is highly diverse in terms of failures as compared to cloud computing and requires wide research and investigation. From this perspective, this study primarily focuses on the provision of uninterrupted services through fog computing. A framework has been designed to provide uninterrupted services while maintaining resiliency. The geographical information system (GIS) services have been deployed as a test bed which requi...

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved