Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Works
Technical Background and Preliminaries
Related Work
Secure Generalized Polydot Code: The S < T Case
Discussion and Concluding Remarks
Chapter 4 Concluding Remarks and Future Directions
References
All Topics
Computer Science
Cryptography

Coding against stragglers in distributed computation scenarios

Profile image of MALIHE ALIASGARIMALIHE ALIASGARI

2020

Abstract

Data and analytics capabilities have made a leap forward in recent years. The volume of available data has grown exponentially. The huge amount of data needs to be transferred and stored with extremely high reliability. The concept of coded computing , or a distributed computing paradigm that utilizes coding theory to smartly inject and leverage data/computation redundancy into distributed computing systems, mitigates the fundamental performance bottlenecks for running large-scale data analytics. In this dissertation, a distributed computing framework, first for input files distributedly stored on the uplink of a cloud radio access network architecture, is studied. It focuses on that decoding at the cloud takes place via network function virtualization on commercial off-the-shelf servers. In order to mitigate the impact of straggling decoders in this platform, a novel coding strategy is proposed, whereby the cloud re-encodes the received frames via a linear code before distributing ...

Related papers

Service Rate Region: A New Aspect of Coded Distributed System Design
Fatemeh Kazemi

IEEE Transactions on Information Theory, 2021

Erasure coding has been recognized as a powerful method to mitigate delays due to slow or straggling nodes in distributed systems. This work shows that erasure coding of data objects can flexibly handle skews in the request rates. Coding can help boost the service rate region, that is, increase the overall volume of data access requests that the system can handle. This paper aims to postulate the service rate region as an important consideration in the design of erasure-coded distributed systems. We highlight several open problems that can be grouped into two broad threads: 1) characterizing the service rate region of a given code and finding the optimal request allocation, and 2) designing the underlying erasure code for a given service rate region. As contributions along the first thread, we characterize the rate regions of maximum-distance-separable, locally repairable, and Simplex codes. We show the effectiveness of hybrid codes that combine replication and erasure coding in terms of code design. We also discover fundamental connections between multi-set batch codes and the problem of maximizing the service rate region. Index Terms erasure coded storage, coded computing, resource allocation, distributed systems, batch codes I. INTRODUCTION The emergence of flexible and affordable cloud storage and computing has resulted in an exponential growth in the amount of data that is stored and processed in cloud data centers. This increase in data is accompanied by a similar rapid increase in the volume of users accessing it, resulting in frequent contention for shared cloud resources. A simple way to handle more requests in a fast and reliable fashion is to replicate data at multiple M. Aktaş is with The MathWorks Inc.

View PDFchevron_right
Coded Distributed Computing with Node Cooperation Substantially Increases Speedup Factors
Petros ELIA

2018 IEEE International Symposium on Information Theory (ISIT), 2018

This work explores a distributed computing setting where K nodes are assigned fractions (subtasks) of a computational task in order to perform the computation in parallel. In this setting, a well-known main bottleneck has been the internode communication cost required to parallelize the task, because unlike the computational cost which could keep decreasing as K increases, the communication cost remains approximately constant, thus bounding the total speedup gains associated to having more computing nodes. This bottleneck was substantially ameliorated by the recent introduction of coded MapReduce techniques which allowed each node -at the computational cost of having to preprocess approximately t times more subtasks -to reduce its communication cost by approximately t times. In reality though, the associated speed up gains were severely limited by the requirement that larger t and K necessitated that the original task be divided into an extremely large number of subtasks. In this work we show how node cooperation, along with a novel assignment of tasks, can help to dramatically ameliorate this limitation. The result applies to wired as well as wireless distributed computing, and it is based on the idea of having groups of nodes compute identical parallelization (mapping) tasks and then employing a here-proposed novel D2D coded caching algorithm.

View PDFchevron_right
On Batch-Processing Based Coded Computing for Heterogeneous Distributed Computing Systems
Kejie Lu

IEEE Transactions on Network Science and Engineering, 2021

In recent years, coded distributed computing (CDC) has attracted significant attention, because it can efficiently facilitate many delay-sensitive computation tasks against unexpected latencies in distributed computing systems. Despite such a salient feature, many design challenges and opportunities remain. In this paper, we focus on practical computing systems with heterogeneous computing resources, and design a novel CDC approach, called batch-processing based coded computing (BPCC), which exploits the fact that every computing node can obtain some coded results before it completes the whole task. To this end, we first describe the main idea of the BPCC framework, and then formulate an optimization problem for BPCC to minimize the task completion time by configuring the computation load. Through formal theoretical analyses, extensive simulation studies, and comprehensive real experiments on the Amazon EC2 computing clusters, we demonstrate promising performance of the proposed BPCC scheme, in terms of high computational efficiency and robustness to uncertain disturbances.

View PDFchevron_right
Hierarchical Coded Elastic Computing
THARINDU ADIKARI

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021

Elasticity is offered by cloud service providers to exploit under-utilized computing resources. The low-cost elastic nodes can leave and join any time during the computation cycle. The possibility of elastic events occurring together with the problem of slow nodes, referred to as stragglers, increases the uncertainty of the system, leading to computation delay. Recent results have shown that coded computing can be used to reduce the negative effect of elasticity and stragglers. In this paper, we propose two hierarchical coded elastic computing schemes that can further speed up the system by exploiting stragglers and effectively allocating tasks among available nodes. In our simulations, our scheme realizes 45% improvement in average finishing time compared to the state-of-the-art coded elastic computing scheme.

View PDFchevron_right
Analysis of virtualized Turbo-decoder implementation for Cloud-RAN systems
Dirk Wübben

2016 9th International Symposium on Turbo Codes and Iterative Information Processing (ISTC), 2016

The virtualization of radio access network functions using centralized cloud platforms will enable advanced joint processing approaches and offers the ability to improve the utilization efficiency of the computational resources. However, the tight timing constraints caused by the protocol stack makes the implementation of physical layer functionality on a cloud-platform a challenging task. As the Turbo decoder is the computational most demanding part of the LTE uplink processing, we analyze its implementation on a typical cloud platform, discuss the rate-complexity tradeoff, and analyze the benefit of computational aware link adaptation.

View PDFchevron_right
Performance Analysis of Layered Application Codes in High Performance Distributed Computing
Michela Taufer

2002

View PDFchevron_right
Distributed cloud storage using network coding
Sipos Márton, Daniel Lucani

2014 IEEE 11th Consumer Communications and Networking Conference (CCNC), 2014

Distributed storage is usually considered within a cloud provider to ensure availability and reliability of the data. However, the user is still directly dependent on the quality of a single system. It is also entrusting the service provider with large amounts of private data, which may be accessed by a successful attack to that cloud system or even be inspected by government agencies in some countries. This paper advocates a general framework for network coding enabled distributed storage over multiple commercial cloud solutions, such as, Dropbox, Box, Skydrive, and Google Drive, as a way to address these reliability and privacy issues. By means of theoretical analysis and reallife implementations, we show not only that our framework constitutes a viable solution to increase the reliability of stored data and to ensure data privacy, but it also provides a way to reduce the storage costs and to increase the download speed significantly. Our measurements show that the download time could be reduced up to six fold in some scenarios exploiting four commercial cloud solutions.

View PDFchevron_right
Implementation and analysis of forward error correction decoding for Cloud-RAN systems
Dirk Wübben

2015 IEEE International Conference on Communication Workshop (ICCW), 2015

In future 5G mobile networks, radio access network functions will be virtualized and implemented on centralized cloud platforms. In principle, this allows for more advanced algorithms of joint processing and offers the ability to balance the computational load. However, the shift of functionality on a cloudplatform also imposes challenges on the design of the applied algorithms. In this paper, we analyse the implementation of forward error correction decoding algorithms on general purpose hardware, which draws the main computational burden of signal processing in the uplink. Although the protocol stack introduces strict timing requirements we demonstrate by numerical results the feasibility of such implementation.

View PDFchevron_right
Reading fMultimedia Cloud Computing IEEE Signal Processing
小凡 赵

T his article introduces the principal concepts of multimedia cloud computing and presents a novel framework. We address multimedia cloud computing from multimedia-aware cloud (media cloud) and cloud-aware multimedia (cloud media) perspectives. First, we present a multimedia-aware cloud, which addresses how a cloud can perform distributed multimedia processing and storage and provide quality of service (QoS) provisioning for multimedia services. To achieve a high QoS for multimedia services, we propose a media-edge cloud (MEC) architecture, in which storage, central processing unit (CPU), and graphics processing unit (GPU) clusters are presented at the edge to provide distributed parallel processing and QoS adaptation for various types of devices. Then we present a cloudaware multimedia, which addresses how multimedia services and applications, such as storage and sharing, authoring and mashup, adaptation and delivery, and rendering and retrieval, can optimally utilize cloud-computing resources to achieve better quality of experience (QoE). The research directions and problems encountered are presented accordingly.

View PDFchevron_right
On the Combination of Multi-Cloud and Network Coding for Cost-Efficient Storage in Industrial Applications
Josu Legarreta Bilbao

Sensors, 2019

The adoption of both Cyber–Physical Systems (CPSs) and the Internet-of-Things (IoT) has enabled the evolution towards the so-called Industry 4.0. These technologies, together with cloud computing and artificial intelligence, foster new business opportunities. Besides, several industrial applications need immediate decision making and fog computing is emerging as a promising solution to address such requirement. In order to achieve a cost-efficient system, we propose taking advantage from spot instances, a new service offered by cloud providers, which provide resources at lower prices. The main downside of these instances is that they do not ensure service continuity and they might suffer from interruptions. An architecture that combines fog and multi-cloud deployments along with Network Coding (NC) techniques, guarantees the needed fault-tolerance for the cloud environment, and also reduces the required amount of redundant data to provide reliable services. In this paper we analyze ...

View PDFchevron_right

References (114)

  1. European Telecommunications Standards Institute. Network function virtualisation (NFV); report on models and features for end-to-end reliability. Technical Report GS NFV-REL 003, Apr., 2016.
  2. European Telecommunications Standards Institute. Cloud RAN and MEC: A perfect pairing. ISBN No. 979-10-92620-17-7, Feb., 2018.
  3. Martin Abadi, Joan Feigenbaum, and Joe Kilian. On hiding information from an oracle. Journal of computer and system sciences, 39(1):21-50, Aug., 1989.
  4. Mehmet Fatih Aktas, Pei Peng, and Emina Soljanin. Effective straggler mitigation: Which clones should attack and when? ACM SIGMETRICS Performance Evaluation Review, 45(2):12-14, Sep., 2017.
  5. Ali Al-Shuwaili, Osvaldo Simeone, Joerg Kliewer, and Petar Popovski. Coded network function virtualization: Fault tolerance via in-network coding. IEEE Wireless Communications Letters, 5(6):644-647, Dec., 2016.
  6. Malihe Aliasgari, Jörg Kliewer, and Osvaldo Simeone. Coded computation against processing delays for virtualized cloud-based channel decoding. IEEE Transaction on Communication, 67(1):28-38, Jan., 2019.
  7. Malihe Aliasgari, Jörg Kliewer, and Osvaldo Simeone. Coded computation against straggling decoders for network function virtualization. In Proceeding IEEE International Symposium on Information Theory (ISIT), pages 711-715, Jun., 2018.
  8. Malihe Aliasgari, Osvaldo Simeone, and Jörg Kliewer. Distributed and private coded matrix computation with flexible communication load. In Proceeding IEEE International Symposium Information Theory (ISIT), pages 1092-1096, Jul., 2019.
  9. Islam Alyafawi, Eryk Schiller, Torsten Braun, Desislava Dimitrova, Andre Gomes, and Navid Nikaein. Critical issues of centralized and cloudified LTE-FDD radio access networks. In IEEE International Conference on Communications (ICC), pages 5523-5528. IEEE, Jun., 2015.
  10. Gene M Amdahl. Validity of the single processor approach to achieving large scale computing capabilities. In Proceedings Federation of Information Processing Societies, pages 483-485, Apr., 1967.
  11. Ganesh Ananthanarayanan, Ali Ghodsi, Scott Shenker, and Ion Stoica. Effective straggler mitigation: Attack of the clones. In Proceeding of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI), volume 13, pages 185-198, Apr., 2013.
  12. Ganesh Ananthanarayanan, Srikanth Kandula, Albert G Greenberg, Ion Stoica, Yi Lu, Bikas Saha, and Edward Harris. Reining in the outliers in Map-Reduce clusters using mantri. In Proceeding of the 10th USENIX Symposium on Operating Systems Design and Implementation, volume 10, page 24, Oct., 2010.
  13. Navid Azizan-Ruhi, Farshad Lahouti, Amir Salman Avestimehr, and Babak Hassibi. Distributed solution of large-scale linear systems via accelerated projection- based consensus. IEEE Transactions on Signal Processing, 67(14):3806-3817, Dec., 2019.
  14. Karim Banawan and Sennur Ulukus. The capacity of private information retrieval from coded databases. IEEE Transaction on Information Theory, 64(3):1945- 1956, Mar., 2018.
  15. Donald Beaver, Joan Feigenbaum, Joe Kilian, and Phillip Rogaway. Locally random reductions: Improvements and applications. Journal of Cryptology, 10(1):17- 36, Sep., 1997.
  16. Amos Beimel, Yuval Ishai, Eyal Kushilevitz, and Ilan Orlov. Share conversion and private information retrieval. In IEEE 27th Conference on Computational Complexity, pages 258-268, Jun., 2012.
  17. Yitzhak Birk and Tomer Kol. Coding on demand by an informed source (iscod) for efficient broadcast of different supplemental data to caching clients. IEEE Transactions on Information Theory, 52(6):2825-2830, Jan., 2006.
  18. George Blakley. Safeguarding cryptographic keys. In International Workshop on Managing Requirements Knowledge (MARK), pages 313-318. IEEE, Jun., 1979.
  19. Béla Bollobás. Modern graph theory, volume 184. Springer Science & Business Media, 2013.
  20. Rowland Leonard Brooks. On colouring the nodes of a network. Mathematical Proceedings of the Cambridge Philosophical Society, 37(02):194-197, Jul., 1941.
  21. Lynn Elliot Cannon. A cellular computer to implement the Kalman filter algorithm. PhD thesis, Montana State University-Bozeman, College of Engineering, Aug., 1969.
  22. Wei-Ting Chang and Ravi Tandon. On the capacity of secure distributed matrix multiplication. arXiv preprint, arXiv:1806.00469, 2018.
  23. Wei-Ting Chang and Ravi Tandon. On the upload versus download cost for secure and private matrix multiplication. arXiv preprint, arXiv:1906.10684, 2019.
  24. Manmohan Chaubey and Erik Saule. Replicated data placement for uncertain scheduling. In IEEE International Parallel and Distributed Processing Symposium Workshop, pages 464-472. IEEE, May., 2015.
  25. Jaeyoung Choi, David W Walker, and Jack J Dongarra. Pumma: Parallel universal matrix multiplication algorithms on distributed memory concurrent computers. Concurrency: Practice and Experience, 6(7):543-570, Oct., 1994.
  26. Benny Chor, Oded Goldreich, Eyal Kushilevitz, and Madhu Sudan. Private information retrieval. In Proceedings of IEEE 36th Annual Foundations of Computer Science, pages 41-50, Oct., 1995.
  27. Anindya B Das, Aditya Ramamoorthy, and Namrata Vaswani. Random convolutional coding for robust and straggler resilient distributed matrix computation. arXiv preprint, arXiv:1907.08064, 2019.
  28. Jeffrey Dean and Luiz André Barroso. The tail at scale. Communications of the ACM, 56(2):74-80, Feb., 2013.
  29. Jeffrey Dean and Sanjay Ghemawat. MapReduce: simplified data processing on large clusters. Communication of the ACM, 51(1):107-113, Feb., 2008.
  30. Robert H Dennard, Fritz H Gaensslen, V Leo Rideout, Ernest Bassous, and Andre R LeBlanc. Design of ion-implanted mosfet's with very small physical dimensions. IEEE Journal of Solid-State Circuits, 9(5):256-268, Dec., 1974.
  31. Alexandros G Dimakis, Kannan Ramchandran, Yunnan Wu, and Changho Suh. A survey on network codes for distributed storage. Proceedings of the IEEE, 99(3):476-489, 2011.
  32. Rafael GL D'Oliveira, Salim El Rouayheb, and David Karpuk. GASP codes for secure distributed matrix multiplication. arXiv preprint, arXiv:1812.09962, 2018.
  33. Jack Dongarra, Thomas Herault, and Yves Robert. Fault tolerance techniques for high-performance computing. In Computer Communications and Networks, pages 3-85. Springer, Jul., 2015.
  34. Uwe Dötsch, Mark Doll, Hans-Peter Mayer, Frank Schaich, Jonathan Segel, and Philippe Sehier. Quantitative analysis of split base station processing and determination of advantageous architectures for LTE. Bell Labs Technical Journal, 18(1):105-128, Dec., 2013.
  35. Sanghamitra Dutta, Ziqian Bai, Haewon Jeong, Tze Meng Low, and Pulkit Grover. A unified coded deep neural network training strategy based on generalized polydot codes for matrix multiplication. arXiv preprint, arXiv:1811.10751, 2018.
  36. Sanghamitra Dutta, Viveck Cadambe, and Pulkit Grover. Short-dot: Computing large linear transforms distributedly using coded short dot products. In Advances In Neural Information Processing Systems, pages 2100-2108, Dec., 2016.
  37. Sanghamitra Dutta, Mohammad Fahim, Farzin Haddadpour, Haewon Jeong, Viveck Cadambe, and Pulkit Grover. On the optimal recovery threshold of coded matrix multiplication. arXiv preprint, arXiv:1801.10292, 2018.
  38. Mohammad Fahim and Viveck R Cadambe. Numerically stable polynomially coded computing. arXiv preprint, arXiv:1903.08326, 2019.
  39. Mohammad Fahim, Haewon Jeong, Farzin Haddadpour, Sanghamitra Dutta, Viveck Cadambe, and Pulkit Grover. On the optimal recovery threshold of coded matrix multiplication. In Proceeding 55th Allerton Conference on Communication, Control, and Computing, IL, USA, pages 1264-1270, Oct., 2017.
  40. Joan Feigenbaum. Encrypting problem instances. In Conference on the Theory and Application of Cryptographic Techniques, pages 477-488. Springer, Aug., 1985.
  41. R Freij-Hollanti, O. W. Gnilke, C Hollanti, and D. A. Karpuk. Private information retrieval from coded databases with colluding servers. SIAM Journal on Applied Algebra and Geometry, 1(1):647-664, Nov., 2017.
  42. Kristen Gardner, Samuel Zbarsky, Sherwin Doroudi, Mor Harchol-Balter, and Esa Hyytia. Reducing latency via redundant requests: Exact analysis. ACM SIGMETRICS Performance Evaluation Review, 43(1):347-360, Jun., 2015.
  43. William Gasarch. A survey on private information retrieval. Bulletin of the EATCS, 82(113):72-107, Feb., 2004.
  44. Yael Gertner, Yuval Ishai, Eyal Kushilevitz, and Tal Malkin. Protecting data privacy in private information retrieval schemes. Journal of Computer and System Sciences, 60(3):592-629, Jun., 2000.
  45. Thomas Herault and Yves Robert. Fault-tolerance techniques for high-performance computing. Springer, Jul., 2015.
  46. Juliver Gil Herrera and Juan Felipe Botero. Resource allocation in NFV: A compre- hensive survey. IEEE Transactions on Network and Service Management, 13(3):518-532, Mar., 2016.
  47. Wassily Hoeffding. A class of statistics with asymptotically normal distribution. In Breakthroughs in Statistics, pages 308-334. Springer, 1992.
  48. Wassily Hoeffding. Probability inequalities for sums of bounded random variables. In The Collected Works of Wassily Hoeffding, pages 409-426. Springer, 1994.
  49. Kuang-Hua Huang and Jacob A Abraham. Algorithm-based fault tolerance for matrix operations. IEEE Transaction on Computers, 100(6):518-528, Jun., 1984.
  50. Svante Janson. Large deviations for sums of partly dependent random variables. Random Structures & Algorithms, 24(3):234-248, Mar., 2004.
  51. Majid Janzamin, Hanie Sedghi, and Anima Anandkumar. Beating the perils of non-convexity: Guaranteed training of neural networks using tensor methods. arXiv preprint, arXiv:1506.08473, 2015.
  52. Zhuqing Jia and Syed A Jafar. Cross subspace alignment codes for coded distributed batch matrix multiplication. arXiv preprint, arXiv:1909.13873, 2019.
  53. Zhuqing Jia and Syed A Jafar. On the capacity of secure distributed matrix multiplication. arXiv preprint, arXiv:1908.06957, 2019.
  54. Gauri Joshi, Yanpei Liu, and Emina Soljanin. On the delay-storage trade-off in content download from coded distributed storage systems. IEEE Journal on Selected Areas in Communications, 32(5):989-997, Dec., 2014.
  55. Gauri Joshi, Emina Soljanin, and Gregory Wornell. Efficient redundancy techniques for latency reduction in cloud systems. ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), 2(2):12, Sep., 2017.
  56. Jaber Kakar, Seyedhamed Ebadifar, and Aydin Sezgin. On the capacity and straggler- robustness of distributed secure matrix multiplication. IEEE Access, 7:45783- 45799, Apr., 2019.
  57. Jinkyu Kang, Osvaldo Simeone, and Joonhyuk Kang. On the trade-off between computational load and reliability for network function virtualization. IEEE Communications Letters, 21:1767-1770, Dec., 2017.
  58. Fatemeh Kazemi, Esmaeil Karimi, Anoosheh Heidarzadeh, and Alex Sprintson. Private information retrieval with private coded side information: The multi-server case. arXiv preprint, arXiv:1906.11278, 2019.
  59. Fatemeh Kazemi, Esmaeil Karimi, Anoosheh Heidarzadeh, and Alex Sprintson. Single-server single-message online private information retrieval with side information. In Proceeding IEEE International Symposium on Information Theory (ISIT), pages 350-354, Jul., 2019.
  60. Shahrouz Khalili and Osvaldo Simeone. Uplink HARQ for cloud RAN via separation of control and data planes. IEEE Transactions on Vehicular Technology, 66(5):4005-4016, Mar., 2017.
  61. Minchul Kim and Jungwoo Lee. Private secure coded computation. arXiv preprint, arXiv:1902.00167, 2019.
  62. Ger Koole and Rhonda Righter. Resource allocation in grid computing. Journal of Scheduling, 11(3):163-173, Aug., 2008.
  63. Jack Kosaian, KV Rashmi, and Shivaram Venkataraman. Learning a code: Machine learning for approximate non-linear coded computation. arXiv preprint, arXiv:1806.01259, 2018.
  64. Hsiang-Tsung Kung. Fast evaluation and interpolation. Carnegie Mellon University, Tech. Rep., 2009.
  65. Kangwook Lee, Maximilian Lam, Ramtin Pedarsani, Dimitris Papailiopoulos, and Kannan Ramchandran. Speeding up distributed machine learning using codes. Proceeding IEEE International Symposium on Information Theory, pages 1143-1147, Jul., 2016.
  66. Kangwook Lee, Maximilian Lam, Ramtin Pedarsani, Dimitris Papailiopoulos, and Kannan Ramchandran. Speeding up distributed machine learning using codes. IEEE Transation on Information Theory, 64(3):1514-1529, Mar., 2018.
  67. Kangwook Lee, Ramtin Pedarsani, and Kannan Ramchandran. On scheduling redundant requests with cancellation overheads. IEEE/ACM Transactions on Networking, 25(2):1279-1290, Apr., 2017.
  68. Kangwook Lee, Changho Suh, and Kannan Ramchandran. High-dimensional coded matrix multiplication. In Proceeding IEEE International Symposium Information Theory (ISIT), pages 2418-2422, Jun., 2017.
  69. Mu Li, David G Andersen, Jun Woo Park, Alexander J Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J Shekita, and Bor-Yiing Su. Scaling distributed machine learning with the parameter server. In Proceeding of the 11th USENIX Conference on Operating Systems Design and Implementation, OSDI, volume 14, pages 583-598, Oct., 2014.
  70. Songze Li, Mohammad Ali Maddah-Ali, and A Salman Avestimehr. A unified coding framework for distributed computing with straggling servers. In Globecom Workshops (GC Wkshps), 2016 IEEE, pages 1-6. IEEE, Dec., 2016.
  71. Songze Li, Mohammad Ali Maddah-Ali, and A Salman Avestimehr. Coded MapReduce. In Communication, Control, and Computing (Allerton), 2015 53rd Annual Allerton Conference on, pages 964-971. IEEE, Oct., 2015.
  72. Songze Li, Mohammad Ali Maddah-Ali, and A Salman Avestimehr. Coded distributed computing: Straggling servers and multistage dataflows. In Communication, Control, and Computing (Allerton), 2016 54th Annual Allerton Conference on, pages 164-171. IEEE, Oct., 2016.
  73. Songze Li, Mohammad Ali Maddah-Ali, Qian Yu, and A Salman Avestimehr. A fundamental tradeoff between computation and communication in distributed computing. IEEE Transactions on Information Theory, 64(1):109-128, May., 2018.
  74. Songze Li, Mohammad Ali Maddah-Ali, Qian Yu, and A Salman Avestimehr. A fundamental tradeoff between computation and communication in distributed computing. IEEE Transaction on Information Theory, 64(1):109-128, Sep., 2017.
  75. Jiajia Liu, Zhongyuan Jiang, Nei Kato, Osamu Akashi, and Atsushi Takahara. Reliability evaluation for NFV deployment of future mobile broadband networks. IEEE Wireless Communications, 23(3):90-96, Apr., 2016.
  76. Ankur Mallick, Malhar Chaudhari, and Gauri Joshi. Rateless codes for near-perfect load balancing in distributed matrix-vector multiplication. arXiv preprint, arXiv:1804.10331, 2018.
  77. Rashid Mijumbi, Joan Serrat, Juan-Luis Gorricho, Niels Bouten, Filip De Turck, and Raouf Boutaba. Network function virtualization: State-of-the-art and research challenges. IEEE Communications Surveys & Tutorials, 18(1):236-262, Dec., 2016.
  78. Navid Nikaein. Processing radio access network functions in the cloud: Critical issues and modeling. In Proceedings of the 6th International Workshop on Mobile Cloud Computing and Services,, pages 36-43. ACM, Apr., 2015.
  79. Navid Nikaein, Raymond Knopp, Florian Kaltenberger, Lionel Gauthier, Christian Bonnet, Dominique Nussbaum, and Riadh Ghaddab. OpenAirInterface: an open LTE network in a PC. In Proceedings of the 20th annual international conference on Mobile computing and networking, pages 305-308. ACM, Sep., 2014.
  80. Hanzaleh Akbari Nodehi and Mohammad Ali Maddah-Ali. Secure coded multi-party computation for massive matrix operations. arXiv preprint, arXiv:1908.04255, 2019.
  81. Hanzaleh Akbari Nodehi and Mohammad Ali Maddah-Ali. Limited-sharing multi- party computation for massive matrix operations. In Proceeding IEEE International Symposium on Information Theory (ISIT), pages 1231-1235, Jun., 2018.
  82. Linus Nyman and Mikael Laakso. Notes on the history of fork and join. IEEE Annals of the History of Computing, 38(3):84-87, 2016.
  83. Yury Polyanskiy, H Vincent Poor, and Sergio Verdú. Channel coding rate in the finite blocklength regime. IEEE Transactions on Information Theory, 56(5):2307- 2359, Dec., 2010.
  84. Amirhossein Reisizadehmobarakeh, Saurav Prakash, Ramtin Pedarsani, and Salman Avestimehr. Coded computation over heterogeneous clusters. [Online] www.arxiv.org, arXiv:1701.05973 [cs.IT], 2017.
  85. Francesco Ricci, Lior Rokach, and Bracha Shapira. Introduction to recommender systems handbook. Springer, Oct., 2010.
  86. David P Rodgers. Improvements in multiprocessor system design. ACM SIGARCH Computer Architecture News, 13(3):225-231, Jun., 1985.
  87. Veronica Quintuna Rodriguez and Fabrice Guillemin. Towards the deployment of a fully centralized cloud-RAN architecture. In Wireless Communications and Mobile Computing Conference (IWCMC), 2017 13th International, pages 1055-1060, Valencia, Spain, Jun., 2017.
  88. Veronica Quintuna Rodriguez and Fabrice Guillemin. Cloud-ran modeling based on parallel processing. IEEE Journal on Selected Areas in Communications, 36(3):457-468, Nov., 2018.
  89. Sheldon M Ross. Introduction to Probability Models. Academic Press, 2014.
  90. Peter Rost and Athul Prasad. Opportunistic hybrid arqenabler of centralized-RAN over nonideal backhaul. IEEE Wireless Communications Letters, 3(5):481- 484, Dec., 2014.
  91. Abdón Sánchez-Arroyo. Determining the total colouring number is NP-hard. Discrete Mathematics, 78(3):315-319, 1989.
  92. Albin Severinson, Alexandre Graell i Amat, and Eirik Rosnes. Block-diagonal coding for distributed computing with straggling servers. In Information Theory Workshop (ITW), pages 464-468, Nov., 2017.
  93. Nihar B Shah, Kangwook Lee, and Kannan Ramchandran. When do redundant requests reduce latency? IEEE Transactions on Communications, 64(2):715- 722, Sep., 2015.
  94. Adi Shamir. How to share a secret. Communications of the ACM, 22(11):612-613, Nov., 1979.
  95. Claude E Shannon. A mathematical theory of communication. Bell system technical journal, 27(3):379-423, Jul., 1948.
  96. Konstantin Shvachko, Hairong Kuang, Sanjay Radia, and Robert Chansler. The hadoop distributed file system. In IEEE 26th symposium on mass storage systems and technologies (MSST), pages 1-10, May., 2010.
  97. Edgar Solomonik and James Demmel. Communication-optimal parallel 2.5 d matrix multiplication and lu factorization algorithms. In European Conference on Parallel Processing, pages 90-109. Springer, Aug., 2011.
  98. Adarsh M Subramaniam, Anoosheh Heidarzadeh, and Krishna R Narayanan. Random khatri-rao-product codes for numerically-stable distributed matrix multiplication. arXiv preprint, arXiv:1907.05965, 2019.
  99. Hua Sun and Syed Ali Jafar. The capacity of private information retrieval. IEEE Transaction on Information Theory, 63(7):4075-4088, Jul., 2017.
  100. Behrooz Tahmasebi and Mohammad Ali Maddah-Ali. Private sequential function computation. arXiv preprint, arXiv:1908.01204, 2019.
  101. Rashish Tandon, Qi Lei, Alexandros Dimakis, and Nikos Karampatziakis. Gradient coding: Avoiding stragglers in synchronous gradient descent. [Online] www.arxiv.org arXiv:1612.03301 [cs.IT], 2016.
  102. Rashish Tandon, Qi Lei, Alexandros G Dimakis, and Nikos Karampatziakis. Gradient coding: Avoiding stragglers in distributed learning. In International Conference on Machine Learning, pages 3368-3376, Aug., 2017.
  103. Henk C Tijms. A First Course in Stochastic Models. John Wiley and Sons, Jul., 2003.
  104. Robert A Van De Geijn and Jerrell Watts. Summa: Scalable universal matrix multipli- cation algorithm. Concurrency: Practice and Experience, 9(4):255-274, Oct., 1997.
  105. Richard Walker. Implementing discrete mathematics: combinatorics and graph theory with mathematica. The Mathematical Gazette, 76(476):286-288, Jul., 1992.
  106. Da Wang, Gauri Joshi, and Gregory Wornell. Using straggler replication to reduce latency in large-scale parallel computing. ACM SIGMETRICS Performance Evaluation Review, 43(3):7-11, Jun., 2015.
  107. Heecheol Yang and Jungwoo Lee. Secure distributed computing with straggling servers using polynomial codes. IEEE Transaction on Information Forensics and Security, 14(1):141-150, Jan., 2019.
  108. Yaoqing Yang, Malhar Chaudhari, Pulkit Grover, and Soummya Kar. Coded iterative computing using substitute decoding. arXiv preprint, arXiv:1805.06046, 2018.
  109. Yaoqing Yang, Pulkit Grover, and Soummya Kar. Computing linear transformations with unreliable components. IEEE Transactions on Information Theory, 63(6):3729-3756, Mar., 2017.
  110. Sergey Yekhanin. Private information retrieval. Commun. ACM, 53(4):68-73, Apr., 2010.
  111. Qian Yu, Mohammad Maddah-Ali, and Salman Avestimehr. Polynomial codes: An optimal design for high-dimensional coded matrix multiplication. In Advances in Neural Information Processing Systems, pages 4403-4413, Dec., 2017.
  112. Qian Yu, Mohammad Ali Maddah-Ali, and A Salman Avestimehr. Straggler mitigation in distributed matrix multiplication: Fundamental limits and optimal coding. arXiv preprint, arXiv:1801.07487, 2018.
  113. Qian Yu, Netanel Raviv, Jinhyun So, and A Salman Avestimehr. Lagrange coded computing: Optimal design for resiliency, security and privacy. arXiv preprint, arXiv:1806.00939, 2018.
  114. Matei Zaharia, Mosharaf Chowdhury, Michael J Franklin, Scott Shenker, and Ion Stoica. Spark: Cluster computing with working sets. Proceeding of the 2nd USENIX Conference on Hot topics in Cloud Computing, pages 10-10, Jun., 2010.

Related papers

Coded Computation Against Processing Delays for Virtualized Cloud-Based Channel Decoding
MALIHE ALIASGARI

IEEE Transactions on Communications, 2019

The uplink of a cloud radio access network architecture is studied in which decoding at the cloud takes place via network function virtualization on commercial off-the-shelf servers. In order to mitigate the impact of straggling decoders in this platform, a novel coding strategy is proposed, whereby the cloud re-encodes the received frames via a linear code before distributing them to the decoding processors. Transmission of a single frame is considered first, and upper bounds on the resulting frame unavailability probability as a function of the decoding latency are derived by assuming a binary symmetric channel for uplink communications. Then, the analysis is extended to account for random frame arrival times. In this case, the tradeoff between average decoding latency and the frame error rate is studied for two different queuing policies, whereby the servers carry out per-frame decoding or continuous decoding, respectively. Numerical examples demonstrate that the bounds are useful tools for code design and that coding is instrumental in obtaining a desirable compromise between decoding latency and reliability.

View PDFchevron_right
Coded Computation Against Straggling Decoders for Network Function Virtualization
MALIHE ALIASGARI

2018 IEEE International Symposium on Information Theory (ISIT), 2018

The uplink of a cloud radio access network architecture is studied in which decoding at the cloud takes place via network function virtualization (NFV) on commercial offthe-shelf (COTS) servers. In order to mitigate the impact of straggling decoders in the cloud computing platform, a novel coding strategy is proposed, whereby the cloud re-encodes the received frames via a linear code before distributing them to the decoding processors. Upper bounds on the resulting frame unavailability probability (FUP) as a function of the decoding latency are derived by assuming a binary symmetric channel for uplink communications. The bounds leverage large deviation results for correlated variables, and depend on the properties of both the uplink linear channel code adopted at the user and the NFV linear code applied at the cloud. Numerical examples demonstrate that the bounds are useful tools for code design, and that coding is instrumental in obtaining a desirable tradeoff between FUP and decoding latency.

View PDFchevron_right
Fed-DIC: Diagonally Interleaved Coding in a Federated Cloud Environment
Vana Kalogeraki

Lecture Notes in Computer Science, 2020

Coping with failures in modern distributed storage systems that handle massive volumes of heterogeneous and potentially rapidly changing data, has become a very important challenge. A common practice is to utilize fault tolerance methods like Replication and Erasure Coding for maximizing data availability. However, while erasure codes provide better fault tolerance compared to replication with a more affordable storage overhead, they frequently suffer from high reconstruction cost as they require to access all available nodes when a data block needs to be repaired, and also can repair up to a limited number of unavailable data blocks, depending on the number of the code's parity block capabilities. Furthermore, storing and placing the encoded data in the federated storage system also remains a challenge. In this paper we present Fed-DIC, a framework which combines Diagonally Interleaved Coding on client devices at the edge of the network with organized storage of encoded data in a federated cloud system comprised of multiple independent storage clusters. The erasure coding operations are performed on client devices at the edge while they interact with the federated cloud to store the encoded data. We describe how our solution integrates the functionality of federated clouds alongside erasure coding implemented on edge devices for maximizing data availability and we evaluate the working and benefits of our approach in terms of read access cost, data availability, storage overhead, load balancing and network bandwidth rate compared to popular Replication and Erasure Coding schemes.

View PDFchevron_right
Optimal Coding Scheme and Resource Allocation for Distributed Computation with Limited Resources
Shu-Jie CAO

ArXiv, 2021

A central issue of distributed computing systems is how to optimally allocate computing and storage resources and design data shuffling strategies such that the total execution time for computing and data shuffling is minimized. This is extremely critical when the computation, storage and communication resources are limited. In this paper, we study the resource allocation and coding scheme for the MapReduce-type framework with limited resources. In particular, we focus on the coded distributed computing (CDC) approach proposed by Li et al.. We first extend the asymmetric CDC (ACDC) scheme proposed by Yu et al. to the cascade case where each output function is computed by multiple servers. Then we demonstrate that whether CDC or ACDC is better depends on system parameters (e.g., number of computing servers) and task parameters (e.g., number of input files), implying that neither CDC nor ACDC is optimal. By merging the ideas of CDC and ACDC, we propose a hybrid scheme and show that it...

View PDFchevron_right
Coding-theoretic approaches to distributed caching, storage and computing
nitish mital

2021

The next-generation communication networks aim to bring data storage and computation closer to the end users and Internet of things (IoT) endpoints by exploiting edge servers and/or memory available at edge devices. This greatly improves the efficiency and economics of network applications. Traditionally, coding techniques have been applied to communication problems with terrific gains in performance. In this thesis, we apply novel coding techniques to edge storage and computation problems, and show similar significant gains in terms of storage costs, communication costs, computation costs, and privacy. In the first part of the thesis, we study a wireless distributed network of small base stations (SBS) which serve cache-enabled users. We propose a novel scheme of coded caching in a multi-server system with a random connectivity pattern, and study the fundamental limits of the trade-off between the storage capacity at the servers, the cache capacity at the users, and the delivery la...

View PDFchevron_right

Related topics

  • Applied Mathematics
  • Private Information Retrieval
  • Secret Sharing
  • Network Function Virtualization
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved