Orchestration in Fog Computing: A Comprehensive Survey
2023, ACM Computing Surveys
https://doi.org/10.1145/3486221Abstract
Fog computing is a paradigm that brings computational resources and services to the network edge in the vicinity of user devices, lowering latency and connecting with cloud computing resources. Unlike cloud computing, fog resources are based on constrained and heterogeneous nodes whose connectivity can be unstable. In this complex scenario, there is a need to define and implement orchestration processes to ensure that applications and services can be provided, considering the settled agreements. Although some publications have dealt with orchestration in fog computing, there are still some diverse definitions and functional intersection with other areas, such as resource management and monitoring. This article presents a systematic review of the literature with focus on orchestration in fog computing. A generic architecture of fog orchestration is presented, created from the consolidation of the analyzed proposals, bringing to light the essential functionalities addressed in the lit...
References (159)
- Mohammad Aazam and Eui Nam Huh. 2015. Fog computing micro datacenter based dynamic resource estimation and pricing model for IoT. Proceedings -International Conference on Advanced Information Networking and Applications, AINA 2015-April, March (2015), 687-694.
- Mohammad Aazam, Sherali Zeadally, and Khaled A. Harras. 2018. Offloading in fog computing for IoT: Review, enabling technologies, and research opportunities. Future Generation Computer Systems 87 (2018), 278-289.
- N. Agrawal, J. Rellermeyer, and A. Y. Ding. 2019. IoT resource-aware orchestration framework for edge computing. In CoNEXT 2019 Companion -Proceedings of the 15th International Conference on Emerging Networking EXperiments and Technologies. 62-64. https://doi.org/10.1145/3360468.3368179
- F. Al-Doghman, Z. Chaczko, A. R. Ajayan, and R. Klempous. 2016. A review on fog computing technology. In 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). 001525-001530. https://doi.org/10.1109/SMC. 2016.7844455
- Brian Amento, Bharath Balasubramanian, Robert J. Hall, Kaustubh Joshi, Gueyoung Jung, and K. Hal Purdy. 2016. FocusStack: Orchestrating edge clouds using location-based focus of attention. In 2016 IEEE/ACM Symposium on Edge Computing (SEC). IEEE, 179-191. https://doi.org/10.1109/SEC.2016.22
- John L. Anderson. 1983. Autonomous systems intelligence. In Proceedings of the 1983 Annual Conference on Computers: Extending the Human Resource. 229-233.
- Hamid Reza Arkian, Abolfazl Diyanat, and Atefe Pourkhalili. 2017. MIST: Fog-based data analytics scheme with cost- efficient resource provisioning for IoT crowdsensing applications. Journal of Network and Computer Applications 82 (2017), 152-165.
- A. Asensio, X. Masip-Bruin, R.J. Durán, I. de Miguel, G. Ren, S. Daijavad, and A. Jukan. 2020. Designing an efficient clustering strategy for combined fog-to-cloud scenarios. Future Generation Computer Systems (2020).
- Cosmin Avasalcai and Schahram Dustdar. 2018. Latency-aware decentralized resource management for IoT applica- tions. In Proceedings of the 8th International Conference on the Internet of Things. 1-4.
- Julian Bellendorf and Zoltán Ádám Mann. 2020. Classification of optimization problems in fog computing. Future Generation Computer Systems 107 (2020), 158-176.
- Flavio Bonomi, Rodolfo Milito, Jiang Zhu, and Sateesh Addepalli. 2012. Fog computing and its role in the Internet of Things. In Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing (MCC'12). ACM, New York, NY. 13-16. https://doi.org/10.1145/2342509.2342513
- A. Buzachis, A. Galletta, A. Celesti, L. Carnevale, and M. Villari. 2019. Towards osmotic computing: A blue-green strategy for the fast re-deployment of microservices. In 2019 IEEE Symposium on Computers and Communications (ISCC), Vol. 2019-June. 1-6. https://doi.org/10.1109/ISCC47284.2019.8969621
- Cambridge English Dictionary. ORCHESTRATION | meaning. 2021. Retrieved Feb 17, 2021 from https://dictionary. cambridge.org/dictionary/english/orchestration.
- Andrew Campbell, Geoff Coulson, Francisco Garcia, and David Hutchison. 1992. A continuous media transport and orchestration service. In Conference Proceedings on Communications Architectures & Protocols. 99-110.
- Gabriele Castellano, Flavio Esposito, and Fulvio Risso. 2019. A service-defined approach for orchestration of hetero- geneous applications in cloud/edge platforms. IEEE Transactions on Network and Service Management 16, 4 (2019), 1404-1418. https://doi.org/10.1109/TNSM.2019.2941639
- B. Cheng, E. Kovacs, A. Kitazawa, K. Terasawa, T. Hada, and M. Takeuchi. 2018. FogFlow: Orchestrating IoT services over cloud and edges. NEC Technical Journal 13, 1 (2018), 48-53.
- Konstantinos Christidis and Michael Devetsikiotis. 2016. Blockchains and smart contracts for the Internet of Things. IEEE Access 4 (2016), 2292-2303.
- Cisco. Fog Computing and the Internet of Things: Extend the Cloud to Where the Things Are. 2021. Retrieved Feb 17, 2021 from https://www.cisco.com/c/dam/en-us/solutions/trends/iot/docs/computing-overview.pdf.
- Breno GS Costa, Marco Antonio Sousa Reis, Aletéia PF Araújo, and Priscila Solis. 2018. Performance and cost analysis between on-demand and preemptive virtual machines. In CLOSER. 169-178.
- V. Cozzolino, J. Ott, A. Y. Ding, and R. Mortier. 2020. ECCO: Edge-cloud chaining and orchestration framework for road context assessment. In Proceedings -5th ACM/IEEE Conference on Internet of Things Design and Implementation, IoTDI 2020. 223-230. https://doi.org/10.1109/IoTDI49375.2020.00029
- Wided Ben Daoud, Mohammad S. Obaidat, Amel Meddeb-Makhlouf, Faouzi Zarai, and Kuei-Fang Hsiao. 2019. TACRM: Trust access control and resource management mechanism in fog computing. Human-centric Computing and Information Sciences 9, 1 (2019), 1-18.
- Amir Vahid Dastjerdi, Harshit Gupta, Rodrigo Neves Calheiros, Soumya K. Ghosh, and Rajkumar Buyya. 2016. Fog computing: Principles, architectures, and applications. CoRR abs/1601.02752 (2016).
- G. Davoli, D. Borsatti, D. Tarchi, and W. Cerroni. 2020. FORCH: An orchestrator for fog computing service deploy- ment. In 2020 IFIP Networking Conference (Networking). 677-678.
- Mathias Santos de Brito, Saiful Hoque, Thomas Magedanz, Ronald Steinke, Alexander Willner, Daniel Nehls, Oliver Keils, and Florian Schreiner. 2017. A service orchestration architecture for fog-enabled infrastructures. IEEE, 345 E 47th Street, New York, NY, 127-132.
- Leonardo Rebouças de Carvalho and Aleteia Patricia Favacho de Araujo. 2020. Performance comparison of terraform and cloudify as multicloud orchestrators. In 2020 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID). IEEE, 380-389.
- Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, 337-340.
- Nathan F. Saraiva de Sousa, Danny A. Lachos Perez, Raphael V. Rosa, Mateus A.S. Santos, and Christian Esteve Rothenberg. 2019. Network service orchestration: A survey. Computer Communications 142 (2019), 69-94.
- J. Díaz-De-Arcaya, R. Miñon, and A. I. Torre-Bastida. 2019. Towards an architecture for big data analytics leverag- ing edge/fog paradigms. In ACM International Conference Proceeding Series, Vol. 2. 173-176. https://doi.org/10.1145/ 3344948.3344987
- Docker. Docker -Empowering App Development for Developers. 2021. Retrieved Feb 17, 2021 from https://www. docker.com/.
- Docker Swarm. Swarm mode overview | Docker Documentation. 2021. Retrieved Feb 17, 2021 from https://docs. docker.com/engine/swarm/.
- Koustabh Dolui and Soumya Kanti Datta. 2017. Comparison of edge computing implementations: Fog computing, cloudlet and mobile edge computing. GIoTS 2017 -Global Internet of Things Summit, Proceedings (2017).
- Bruno Donassolo, Ilhem Fajjari, Arnaud Legrand, and Panayotis Mertikopoulos. 2019. Fog based framework for IoT service provisioning. In 2019 16th IEEE Annual Consumer Communications and Networking Conference (CCNC). 345 E. 47th Street, New York, NY. 1-6. https://doi.org/10.1109/CCNC.2019.8651835
- Rob Enns, Martin Bjorklund, and Juergen Schoenwaelder. 2006. NETCONF Configuration Protocol. Technical Report. RFC 4741, December.
- Masoumeh Etemadi, Mostafa Ghobaei-Arani, and Ali Shahidinejad. 2021. A cost-efficient auto-scaling mechanism for IoT applications in fog computing environment: A deep learning-based approach. Cluster Computing (2021), 1-16.
- Ali Fahs and Guillaume Pierre. 2019. Proximity-aware traffic routing in distributed fog computing platforms.
- Chih Tien Fan, Zong You Wu, Che Pin Chang, and Shyan Ming Yuan. 2017. Web resource cacheable edge device in fog computing. Proceedings -15th International Symposium on Parallel and Distributed Computing, ISPDC 2016 November 2010 (2017), 432-439.
- Yaoling Fan, Qiliang Zhu, and Yang Liu. 2018. Cloud/fog computing system architecture and key technologies for south-north water transfer project safety. Wireless Communications and Mobile Computing 2018 (2018).
- Soodeh Farokhi, Pooyan Jamshidi, Drazen Lucanin, and Ivona Brandic. 2015. Performance-based vertical memory elasticity. In 2015 IEEE International Conference on Autonomic Computing. IEEE, 151-152.
- Nicolas Ferry, Phu Nguyen, Hui Song, Pierre-Emmanuel Novac, Stéphane Lavirotte, Jean-Yves Tigli, and Arnor Sol- berg. 2019. Genesis: Continuous orchestration and deployment of smart IoT systems. In 2019 IEEE 43rd Annual Com- puter Software and Applications Conference (COMPSAC), Vol. 1. IEEE, 870-875.
- A. Galis, H. Abramowicz, M. Brunner, D. Raz, P. Chemouil, J. Butler, C. Polychronopoulos, S. Clayman, H. de Meer, T. Coupaye, et al. 2009. Management and service-aware networking architectures for future internet position paper: System functions, capabilities and requirements. In IEEE 2009 Fourth International Conference on Communications and Networking in China (ChinaCom09).
- Aram Galstyan, Karl Czajkowski, and Kristina Lerman. 2004. Resource allocation in the grid using reinforcement learning. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems- Volume 3. IEEE Computer Society, 1314-1315.
- Jerry Gao, Volker Gruhn, Jingsha He, George Roussos, Wei-Tek Tsai, et al. 2013. Mobile cloud computing research- issues, challenges and needs. In 2013 IEEE Seventh International Symposium on Service-Oriented System Engineering. IEEE, 442-453.
- Gartner. "Gartner Says the Internet of Things Installed Base Will Grow to 26 Billion Units by 2020". 2015. Retrieved Feb 17, 2021 from http://www.gartner.com/newsroom/id/2636073.
- J. Gedeon, S. Zengerle, S. Alles, F. Brandherm, and M. Mühlhäuser. 2020. Sunstone: Navigating the way through the fog. In 2020 IEEE 4th International Conference on Fog and Edge Computing (ICFEC). 49-58. https://doi.org/10.1109/ ICFEC50348.2020.00013
- Mostafa Ghobaei-Arani, Alireza Souri, and Ali A. Rahmanian. 2019. Resource management approaches in fog com- puting: A comprehensive review. Journal of Grid Computing (2019), 1-42.
- Sukhpal Singh Gill, Shreshth Tuli, Minxian Xu, Inderpreet Singh, Karan Vijay Singh, Dominic Lindsay, Shikhar Tuli, Daria Smirnova, Manmeet Singh, Udit Jain, Haris Pervaiz, Bhanu Sehgal, Sukhwinder Singh Kaila, Sanjay Misra, Mohammad Sadegh Aslanpour, Harshit Mehta, Vlado Stankovski, and Peter Garraghan. 2019. Transformative effects of IoT, blockchain and Artificial Intelligence on cloud computing: Evolution, vision, trends and open challenges. Internet of Things 8, October (2019), 100118.
- Spyridon V. Gogouvitis, Harald Mueller, Sreenath Premnadh, Andreas Seitz, and Bernd Bruegge. 2020. Seamless computing in industrial systems using container orchestration. Future Generation Computer Systems 109 (2020), 678-688. https://doi.org/10.1016/j.future.2018.07.033
- Narasimha Raju Gottumukkala, Chokchai Box Leangsuksun, Narate Taerat, Raja Nassar, and Stephen L. Scott. 2007. Reliability-aware resource allocation in HPC systems. In 2007 IEEE International Conference on Cluster Computing. IEEE, 312-321.
- Laura Grit, David Irwin, Aydan Yumerefendi, and Jeff Chase. 2006. Virtual machine hosting for networked clusters: Building the foundations for "autonomic" orchestration. In First International Workshop on Virtualization Technology in Distributed Computing (VTDC 2006). IEEE, 7-7.
- Jayavardhana Gubbi, Rajkumar Buyya, Slaven Marusic, and Marimuthu Palaniswami. 2013. Internet of things (IoT): A vision, architectural elements, and future directions. Future Generation Computer Systems 29, 7 (2013), 1645-1660.
- Pooyan Habibi, Mohammad Farhoudi, Sepehr Kazemian, Siavash Khorsandi, and Alberto Leon-Garcia. 2020. Fog computing : A comprehensive architectural survey. IEEE Access PP (2020), 1.
- Joacim Halen, Stefan Hellkvist, Stephan Baucke, Fetahi Wuhib, and Yagiz Onat Yazir. 2015. Wind: Management and orchestration in the distributed heterogeneous cloud. In IEEE World Congress on Services. 39-46. https://doi.org/10. 1109/SERVICES.2015.15
- Bo Han, Vijay Gopalakrishnan, Lusheng Ji, and Seungjoon Lee. 2015. Network function virtualization: Challenges and opportunities for innovations. IEEE Communications Magazine 53, 2 (2015), 90-97.
- Pieter Hintjens. 2013. ZeroMQ: Messaging for Many Applications. "O'Reilly Media, Inc. "
- Cheol-Ho Hong and Blesson Varghese. 2019. Resource management in fog/edge computing: A survey on architec- tures, infrastructure, and algorithms. ACM Computing Surveys (CSUR) 52, 5 (2019), 97.
- IBM. What is fog computing? 2014. Retrieved Feb 17, 2021 from https://www.ibm.com/blogs/cloud-computing/2014/ 08/25/fog-computing/.
- Michaela Iorga, Larry Feldman, Robert Barton, Michael Martin, Nedim Goren, and Charif Mahmoudi. 2018. The NIST Definition of Fog Computing. Technical Report. National Institute of Standards and Technology.
- Fatemeh Jalali, Timothy Lynar, Olivia J. Smith, Ramachandra Rao Kolluri, Claire Hardgrove, Nick Waywood, and Frank Suits. 2019. DEFT: Dynamic edge fabric environment seamless and automatic switching among resources at the edge of IoT network and cloud. In 2019 IEEE International Conference on Edge Computing (IEEE EDGE). 77-86. https://doi.org/10.1109/EDGE.2019.00028
- S. Javaid, N. Javaid, T. Saba, Z. Wadud, A. Rehman, and A. Haseeb. 2019. Intelligent resource allocation in residential buildings using consumer to fog to cloud based framework. Energies 12, 5 (2019). https://doi.org/10.3390/en12050815
- Yuxuan Jiang, Zhe Huang, and Danny HK Tsang. 2017. Challenges and solutions in fog computing orchestration. IEEE Network 32, 3 (2017), 122-129.
- Lara Lorna Jimenez and Olov Schelen. 2020. HYDRA: Decentralized location-aware orchestration of containerized applications. IEEE Transactions on Cloud Computing (2020).
- João Bachiega Junior, Marco Antonio Sousa Reis, Aleteia PF de Araujo, and Maristela Holanda. 2017. Cost optimiza- tion on public cloud provider for big geospatial data. In Proceedings of the 7th International Conference on Cloud Computing and Services Science. 82-90.
- Evangelia Kalyvianaki. 2009. Resource Provisioning for Virtualized Server Applications. Technical Report. University of Cambridge, Computer Laboratory.
- Naoki Katoh and Toshihide Ibaraki. 1998. Resource allocation problems. In Handbook of Combinatorial Optimization. Springer, 905-1006.
- Barbara Kitchenham, O. Pearl Brereton, David Budgen, Mark Turner, John Bailey, and Stephen Linkman. 2009. Sys- tematic literature reviews in software engineering-a systematic literature review. Information and Software Technol- ogy 51, 1 (2009), 7-15.
- Diego Kreutz, Fernando MV Ramos, Paulo Esteves Verissimo, Christian Esteve Rothenberg, Siamak Azodolmolky, and Steve Uhlig. 2014. Software-defined networking: A comprehensive survey. Proc. IEEE 103, 1 (2014), 14-76.
- Kubernetes. Kubernetes. 2021. Retrieved Feb 17, 2021 from https://kubernetes.io/.
- Imen Ben Lahmar and Khouloud Boukadi. 2020. Resource allocation in fog computing: A systematic mapping study. In 2020 Fifth International Conference on Fog and Mobile Edge Computing (FMEC). IEEE, 86-93.
- A. Lertsinsrubtavee, A. Ali, C. Molina-Jimenez, A. Sathiaseelan, and J. Crowcroft. 2017. Picasso: A lightweight edge computing platform. In 2017 IEEE 6th International Conference on Cloud Networking (CloudNet). 1-7. https://doi.org/ 10.1109/CloudNet.2017.8071529
- Tom H. Luan, Longxiang Gao, Zhi Li, Yang Xiang, Guiyi Wei, and Limin Sun. 2015. Fog computing: Focusing on mobile users at the edge. (2015), 1-11. arXiv:1502.01815 http://arxiv.org/abs/1502.01815.
- Md Mahmud and Rajkumar Buyya. 2016. Fog Computing: A Taxonomy, Survey and Future Directions.
- Redowan Mahmud, Kotagiri Ramamohanarao, and Rajkumar Buyya. 2020. Application management in fog comput- ing environments : A taxonomy, review and future directions. 1, 1 (2020).
- Ayesha Abdul Majeed, Peter Kilpatrick, Ivor Spence, and Blesson Varghese. 2020. Modelling fog offloading perfor- mance. In 2020 IEEE 4th International Conference on Fog and Edge Computing (ICFEC). IEEE, 29-38.
- Zoltán Ádám Mann. 2011. Optimization in Computer Engineering-Theory and Applications. Scientific Research Pub- lishing, Inc. USA.
- Sunilkumar S. Manvi and Gopal Krishna Shyam. 2014. Resource management for infrastructure as a service (IaaS) in cloud computing: A survey. Journal of Network and Computer Applications 41, 1 (2014), 424-440.
- Antonio Manzalini, Diego R. Lopez, Hakon Lonsethagen, Lucian Suciu, Roberto Bifulco, Marie-Paule Odini, Giuseppe Celozzi, Barbara Martini, Fulvio Risso, Jokin Garay, et al. 2017. A unifying operating platform for 5G end-to-end and multi-layer orchestration. In 2017 IEEE Conference on Network Softwarization (NetSoft). IEEE, 1-5.
- Marathon. Marathon: A container orchestration platform for Mesos and DC/OS. 2021. Retrieved Feb 17, 2021 from https://mesosphere.github.io/marathon/.
- John Paul Martin, A. Kandasamy, K. Chandrasekaran, and Christina Terese Joseph. 2019. Elucidating the challenges for the praxis of fog computing: An aspect-based study. International Journal of Communication Systems 32, 7 (2019), e3926.
- X. Masip, E. Marín, J. Garcia, and S. Sánchez. 2020. Collaborative Mechanism for Hybrid Fog-Cloud Scenarios. 7-60. https://doi.org/10.1002/9781119501121.ch2
- X. Masip-Bruin, E. Marín-Tordera, A. J. Ferrer, A. Salis, J. Kennedy, J. Jensen, A. Jukan, A. Bartoli, R. M. Badia, M. Cankar, and M. E. Bégin. 2020. mF2C: The evolution of cloud computing towards an open and coordinated ecosystem of fogs and clouds. Lecture Notes in Computer Science 11997 LNCS (2020), 136-147. https://doi.org/10.1007/978-3-030- 48340-1_11
- Nick McKeown, Tom Anderson, Hari Balakrishnan, Guru Parulkar, Larry Peterson, Jennifer Rexford, Scott Shenker, and Jonathan Turner. 2008. OpenFlow: Enabling innovation in campus networks. ACM SIGCOMM Computer Com- munication Review 38, 2 (2008), 69-74.
- C. Mechalikh, H. Taktak, and F. Moussa. 2020. A fuzzy decision tree based tasks orchestration algorithm for edge computing environments. Advances in Intelligent Systems and Computing 1151 AISC (2020), 193-203. https://doi.org/ 10.1007/978-3-030-44041-1_18
- Peter Mell, Tim Grance, et al. 2011. The NIST definition of cloud computing. (2011).
- Giovanni Merlino, Rustem Dautov, Salvatore Distefano, and Dario Bruneo. 2019. Enabling workload engineering in edge, fog, and cloud computing through openstack-based middleware. ACM Transactions on Internet Technology 19, 2 (2019). https://doi.org/10.1145/3309705
- Merriam-Webster. Orchestration | Definition of Orchestration. 2021. Retrieved Feb 17, 2021 from https://www. merriam-webster.com/dictionary/orchestration.
- Rashid Mijumbi, Joan Serrat, Juan-Luis Gorricho, Niels Bouten, Filip De Turck, and Raouf Boutaba. 2015. Network function virtualization: State-of-the-art and research challenges. IEEE Communications Surveys & Tutorials 18, 1 (2015), 236-262.
- Carla Mouradian, Fereshteh Ebrahimnezhad, Yassine Jebbar, Jasmeen Kaur Ahluwalia, Seyedeh Negar Afrasiabi, Roch H Glitho, and Ashok Moghe. 2020. An IoT platform-as-a-service for NFV-based hybrid cloud/fog systems. IEEE Internet of Things Journal 7, 7 (2020), 6102-6115.
- Carla Mouradian, Diala Naboulsi, Sami Yangui, Roch H. Glitho, Monique J. Morrow, and Paul A. Polakos. 2018. A comprehensive survey on fog computing: State-of-the-art and research challenges. IEEE Communications Surveys and Tutorials 20, 1 (2018), 416-464.
- MQTT. Message Queuing Telemetry Transport. 2021. Retrieved Feb 17, 2021 from https://mqtt.org/.
- Mithun Mukherjee, Rakesh Matam, Lei Shu, Leandros Maglaras, Mohamed Amine Ferrag, Nikumani Choudhury, and Vikas Kumar. 2017. Security and privacy in fog computing: Challenges. IEEE Access 5 (2017), 19293-19304.
- Mithun Mukherjee, Lei Shu, and Di Wang. 2018. Survey of fog computing: Fundamental, network applications, and research challenges. IEEE Communications Surveys and Tutorials 20, 3 (2018), 1826-1857.
- Mohammed Islam Naas, Philippe Raipin Parvedy, Jalil Boukhobza, and Laurent Lemarchand. 2017. IFogStor: An IoT data placement strategy for fog infrastructure. Proceedings -2017 IEEE 1st International Conference on Fog and Edge Computing, ICFEC 2017 May 2018 (2017), 97-104.
- Ranesh Kumar Naha, Saurabh Garg, Dimitrios Georgakopoulos, Prem Prakash Jayaraman, Longxiang Gao, Yong Xiang, and Rajiv Ranjan. 2018. Fog computing: Survey of trends, architectures, requirements, and research directions. IEEE Access 6 (2018), 47980-48009.
- Athanasios Naskos, Emmanouela Stachtiari, Anastasios Gounaris, Panagiotis Katsaros, Dimitrios Tsoumakos, Ioan- nis Konstantinou, and Spyros Sioutas. 2015. Dependable horizontal scaling based on probabilistic model checking. In 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. IEEE, 31-40.
- S. B. Nath, S. Chattopadhyay, R. Karmakar, S. K. Addya, S. Chakraborty, and S. K. Ghosh. 2019. PTC: Pick-test- choose to place containerized micro-services in IoT. In 2019 IEEE Global Communications Conference (GLOBECOM). 1-6. https://doi.org/10.1109/GLOBECOM38437.2019.9013163
- NGSI. NGSI 9/10 Information Model. 2012. Retrieved Feb 17, 2021 from http://www.openmobilealliance.org/release/ NGSI/V1_0-20120529-A/OMA-TS-NGSI_Context_Management-V1_0-20120529-A.pdf.
- Phu Nguyen, Nicolas Ferry, Gencer Erdogan, Hui Song, Stéphane Lavirotte, Jean-Yves Tigli, and Arnor Solberg. 2019. Advances in deployment and orchestration approaches for IoT-a systematic review. In 2019 IEEE International Con- gress on Internet of Things (ICIOT). IEEE, 53-60.
- Phu Hong Nguyen, Nicolas Ferry, Gencer Erdogan, Hui Song, Stéphane Lavirotte, Jean-Yves Tigli, and Arnor Solberg. 2019. A systematic mapping study of deployment and orchestration approaches for IoT. In IoTBDS. 69-82.
- R. Nicholson, T. Ward, D. Baum, X. Tao, D. Conzon, and E. Ferrera. 2019. Dynamic fog computing platform for event- driven deployment and orchestration of distributed Internet of Things applications. In 2019 Third World Conference on Smart Trends in Systems Security and Sustainablity (WorldS4). 239-246. https://doi.org/10.1109/WorldS4.2019.8903975
- OpenFog. 2017. OpenFog Consortium Architecture Working Group. (February 2017), 162 pages.
- Per-Olov Östberg, James Byrne, Paolo Casari, Philip Eardley, Antonio Fernandez Anta, Johan Forsman, John Kennedy, Thang Le Duc, Manuel Noya Marino, Radhika Loomba, et al. 2017. Reliable capacity provisioning for distributed cloud/edge/fog computing applications. In 2017 European Conference on Networks and Communications (EuCNC). IEEE, 1-6.
- C. Pahl, N. E. Ioini, S. Helmer, and B. Lee. 2018. An architecture pattern for trusted orchestration in IoT edge clouds. In 2018 Third International Conference on Fog and Mobile Edge Computing (FMEC). 63-70. https://doi.org/10.1109/ FMEC.2018.8364046
- G. Papathanail, I. Fotoglou, C. Demertzis, A. Pentelas, K. Sgouromitis, P. Papadimitriou, D. Spatharakis, I. Dimolitsas, D. Dechouniotis, and S. Papavassiliou. 2020. COSMOS: An orchestration framework for smart computation offloading in edge clouds. In NOMS 2020 -2020 IEEE/IFIP Network Operations and Management Symposium. 1-6. https://doi.org/ 10.1109/NOMS47738.2020.9110294
- Mike P. Papazoglou and Dimitrios Georgakopoulos. 2003. Introduction: Service-oriented computing. Commun. ACM 46, 10 (2003), 24-28.
- C. Peltz. 2003. Web services orchestration and composition. Computer 36, 10 (2003), 46-52.
- Charith Perera, Yongrui Qin, Julio C. Estrella, Stephan Reiff-Marganiec, and Athanasios V. Vasilakos. 2017. Fog com- puting for sustainable smart cities: A survey. ACM Computing Surveys (CSUR) 50, 3 (2017), 1-43.
- Kai Petersen, Robert Feldt, Shahid Mujtaba, and Michael Mattsson. 2008. Systematic mapping studies in software engineering. In Ease, Vol. 8. 68-77.
- Antonio Pintus, Davide Carboni, Andrea Piras, and Alessandro Giordano. 2010. Connecting smart things through web services orchestrations. In International Conference on Web Engineering. Springer, 431-441.
- Paul Pop, Bahram Zarrin, Mohammadreza Barzegaran, Stefan Schulte, Sasikumar Punnekkat, Jan Ruh, and Wilfried Steiner. 2021. The FORA fog computing platform for industrial IoT. Information Systems (2021), 101727.
- Subhav Pradhan, Abhishek Dubey, Shweta Khare, Saideep Nannapaneni, Aniruddha Gokhale, Sankaran Mahadevan, Douglas C. Schmidt, and Martin Lehofer. 2018. CHARIOT: Goal-driven orchestration middleware for resilient IoT systems. ACM Transactions on Cyber-Physical Systems 2, 3, SI (2018). https://doi.org/10.1145/3134844
- Carlo Puliafito, Enzo Mingozzi, Francesco Longo, Antonio Puliafito, and Omer Rana. 2019. Fog computing for the Internet of Things: A survey. ACM Transactions on Internet Technology (TOIT) 19, 2 (2019), 1-41.
- Ju Ren, Deyu Zhang, Shiwen He, Yaoxue Zhang, and Tao Li. 2019. A survey on end-edge-cloud orchestrated net- work computing paradigms: Transparent computing, mobile edge computing, fog computing, and cloudlet. ACM Computing Surveys (CSUR) 52, 6 (2019), 1-36.
- Wayne Robbins. 1997. Implementation and performance issues in an object-oriented orchestration architecture. In Proceedings of IEEE International Conference on Multimedia Computing and Systems. IEEE, 628-629.
- Rodrigo Roman, Javier Lopez, and Masahiro Mambo. 2018. Mobile edge computing, fog et al.: A survey and analysis of security threats and challenges. Future Generation Computer Systems 78 (2018), 680-698.
- Michel J. F. Rosa, Aletéia P. F. Araüjo, and Felipe L. S. Mendes. 2018. Cost and time prediction for efficient execution of bioinformatics workflows in federated cloud. In 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 1703-1710.
- Farah Ait Salaht, Frédéric Desprez, and Adrien Lebre. 2020. An overview of service placement problem in fog and edge computing. ACM Computing Surveys (CSUR) 53, 3 (2020), 1-35.
- A. Salem and T. Nadeem. 2017. LAMEN: Towards orchestrating the growing intelligence on the edge. In 2016 IEEE 3rd World Forum on Internet of Things, WF-IoT 2016. 508-513. https://doi.org/10.1109/WF-IoT.2016.7845453
- Daniele Santoro, Daniel Zozin, Daniele Pizzolli, Francesco De Pellegrini, and Silvio Cretti. 2017. Foggy: A platform for workload orchestration in a fog computing environment. In 2017 9th IEEE International Conference on Cloud Computing Technology and Science (CLOUDCOM). IEEE, 345 E. 47th St. New York, NY, 231-234. https://doi.org/10. 1109/CloudCom.2017.62
- Jose Santos, Tim Wauters, Bruno Volckaert, and Filip De Turck. 2018. Fog computing: Enabling the management and orchestration of smart city applications in 5G Networks. ENTROPY 20, 1 (2018). https://doi.org/10.3390/e20010004
- Orchestration in Fog Computing: A Comprehensive Survey 29:33
- Subhadeep Sarkar and Sudip Misra. 2016. Theoretical modelling of fog computing: A green computing paradigm to support IoT applications. Iet Networks 5, 2 (2016), 23-29.
- Mahadev Satyanarayanan, Victor Bahl, Ramón Caceres, and Nigel Davies. 2009. The case for vm-based cloudlets in mobile computing. IEEE Pervasive Computing (2009).
- Mahadev Satyanarayanan, Zhuo Chen, Kiryong Ha, Wenlu Hu, Wolfgang Richter, and Padmanabhan Pillai. 2014. Cloudlets: At the leading edge of mobile-cloud convergence. In 6th International Conference on Mobile Computing, Applications and Services. IEEE, 1-9.
- Sukhpal Singh and Inderveer Chana. 2016. Cloud resource provisioning: Survey, status and future research directions. Knowledge and Information Systems 49, 3 (2016), 1005-1069.
- Olena Skarlat, Stefan Schulte, Michael Borkowski, and Philipp Leitner. 2016. Resource provisioning for IoT services in the fog. In 2016 IEEE 9th International Conference on Service-Oriented Computing and Applications (SOCA). IEEE, 32-39.
- OpenFlow Switch Specification. 2013. Open networking foundation. ONF TS-015, Version 1, 3 (2013), 1-164.
- Ivan Stojmenovic and Sheng Wen. 2014. The fog computing paradigm: Scenarios and security issues. In 2014 Federated Conference on Computer Science and Information Systems. IEEE, 1-8.
- Salman Taherizadeh and Vlado Stankovski. 2017. Auto-scaling applications in edge computing: Taxonomy and chal- lenges. In Proceedings of the International Conference on Big Data and Internet of Thing. 158-163.
- Antero Taivalsaari and Tommi Mikkonen. 2017. A roadmap to the programmable world: Software challenges in the IoT era. IEEE Software 34, 1 (2017), 72-80.
- Bo Tang, Zhen Chen, Gerald Hefferman, Tao Wei, Haibo He, and Qing Yang. 2015. A hierarchical distributed fog computing architecture for big data analysis in smart cities. Proceedings of the ASE BigData & Social Informatics 2015 October (2015), 28.
- Klervie Toczé and Simin Nadjm-Tehrani. 2018. A taxonomy for management and optimization of multiple resources in edge computing. Wireless Communications and Mobile Computing 2018 (2018).
- Klervie Tocze and Simin Nadjm-Tehrani. 2019. ORCH: Distributed orchestration framework using mobile edge de- vices. In 2019 IEEE 3rd International Conference on Fog and Edge Computing, ICFEC -Proceedings. 345 E 47th St. New York, NY. 1-10. https://doi.org/10.1109/CFEC.2019.8733152
- Adel Nadjaran Toosi, Redowan Mahmud, Qinghua Chi, and Rajkumar Buyya. 2019. Management and orchestration of network slices in 5G, fog, edge and clouds. Fog and Edge Computing 10 (2019).
- J. Tsai, I. Chuang, J. Liu, Y. Kuo, and W. Liao. 2020. QoS-aware fog service orchestration for industrial Internet of Things. IEEE Transactions on Services Computing (2020), 1. https://doi.org/10.1109/TSC.2020.2978472
- TW van den Berg and A . Cramp. 2018. Container orchestration environments for M&S. In 2018 Winter Simulation Innovation Workshop, SIW 2018, 2018 Winter Simulation Innovation Workshop, SIW 2018, 21 January 2018 through 26 January 2018. SISO-Simulation Interoperability Standards Organization.
- Frank Van Lingen, Marcelo Yannuzzi, Anuj Jain, Rik Irons-Mclean, Oriol Lluch, David Carrera, J. L. Perez, Alberto Gutierrez, Diego Montero, Josep Marti, Ricard Maso, and J. P. Rodriguez. 2017. The unavoidable convergence of NFV, 5G, and Fog: A model-driven approach to bridge cloud and edge. IEEE Communications Magazine 55, 8 (2017), 28-35. https://doi.org/10.1109/MCOM.2017.1600907
- Luis M. Vaquero, Felix Cuadrado, Yehia Elkhatib, Jorge Bernal-Bernabe, Satish N Srirama, and Mohamed Faten Zhani. 2019. Research challenges in nextgen service orchestration. Future Generation Computer Systems 90 (2019), 20-38.
- Luis M. Vaquero and Luis Rodero-Merino. 2014. Finding your way in the fog: Towards a comprehensive definition of fog computing. SIGCOMM Comput. Commun. Rev. 44, 5 (Oct. 2014), 27-32. https://doi.org/10.1145/2677046.2677052
- Luis M. Vaquero, Luis Rodero-Merino, Juan Caceres, and Maik Lindner. 2008. A break in the clouds: Towards a cloud definition. ACM SIGCOMM Computer Communication Review 39, 1 (2008), 50-55.
- Karima Velasquez, David Perez Abreu, Marcio RM Assis, Carlos Senna, Diego F. Aranha, Luiz F. Bittencourt, Nuno Laranjeiro, Marilia Curado, Marco Vieira, Edmundo Monteiro, et al. 2018. Fog orchestration for the Internet of Ev- erything: State-of-the-art and research challenges. Journal of Internet Services and Applications 9, 1 (2018), 14.
- Karima Velasquez, David Perez Abreu, Marilia Curado, and Edmundo Monteiro. 2017. Service placement for latency reduction in the Internet of Things. Annales des Telecommunications/Annals of Telecommunications 72, 1-2 (2017), 105-115. https://doi.org/10.1007/s12243-016-0524-9
- K. Velasquez, D. P. Abreu, D. Goncalves, L. Bittencourt, M. Curado, E. Monteiro, and E. Madeira. 2017. Service or- chestration in fog environments. In Proceedings -2017 IEEE 5th International Conference on Future Internet of Things and Cloud, FiCloud 2017, Vol. 2017-Janua. 329-336. https://doi.org/10.1109/FiCloud.2017.49
- Alvaro Videla and Jason J.W. Williams. 2012. RabbitMQ in Action: Distributed Messaging for Everyone. Manning.
- Alexandre Viejo and David Sánchez. 2019. Secure and privacy-preserving orchestration and delivery of fog-enabled IoT services. Ad Hoc Networks 82 (2019), 113-125.
- Hariharasudhan Viswanathan, Parul Pandey, and Dario Pompili. 2016. Maestro: Orchestrating concurrent application workflows in mobile device clouds. IEEE, 257-262.
- Sebastian Wahle, Thomas Magedanz, and Frank Schulze. 2012. The OpenMTC framework-M2M solutions for smart cities and the Internet of Things. In 2012 IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM). IEEE, 1-3.
- Florian Wamser, Chiara Lombardo, Constantinos Vassilakis, Lam Dinh-Xuan, Paolo Lago, Roberto Bruschi, and Phuoc Tran-Gia. 2018. Orchestration and monitoring in fog computing for personal edge cloud service support. In 2018 IEEE International Symposium on Local and Metropolitan Area Networks (LANMAN). 345 E. 47th St. New York, NY. 91-96. https://doi.org/10.1109/LANMAN.2018.8475113
- Nan Wang, Blesson Varghese, Michail Matthaiou, and Dimitrios S Nikolopoulos. 2017. ENORM: A framework for edge node resource management. IEEE Transactions on Services Computing (2017).
- Denis Weerasiri, Moshe Chai Barukh, Boualem Benatallah, Quan Z. Sheng, and Rajiv Ranjan. 2017. A taxonomy and survey of cloud resource orchestration techniques. ACM Computing Surveys (CSUR) 50, 2 (2017), 1-41.
- Z. Wen, R. Yang, P. Garraghan, T. Lin, J. Xu, and M. Rovatsos. 2017. Fog orchestration for Internet of Things services. IEEE Internet Computing 21, 2 (2017), 16-24. https://doi.org/10.1109/MIC.2017.36
- Zhenyu Wen, Renyu Yang, Peter Garraghan, Tao Lin, Jie Xu, and Michael Rovatsos. 2017. Fog orchestration for Internet of Things services. IEEE Internet Computing 21, 2 (2017), 16-24.
- Cecil Wöbker, Andreas Seitz, Harald Mueller, and Bernd Bruegge. 2018. Fogernetes: Deployment and management of fog computing applications. IEEE, 1-7.
- Zhang Yaoxue. 2004. Transparence computing: Concept, architecture and example. Acta Electronica Sinica 32, 12 A (2004), 169-174.
- Shanhe Yi, Cheng Li, and Qun Li. 2015. A survey of fog computing: Concepts, applications and issues. In Proceedings of the 2015 Workshop on Mobile Big Data. 37-42.
- E. Yigitoglu, L. Liu, M. Looper, and C. Pu. 2017. Distributed orchestration in large-scale IoT systems. In Proceedings -2017 IEEE 2nd International Congress on Internet of Things, ICIOT 2017. 58-65. https://doi.org/10.1109/IEEE.ICIOT. 2017.16
- Emre Yigitoglu, Mohamed Mohamed, Ling Liu, and Heiko Ludwig. 2017. Foggy: A framework for continuous auto- mated IoT application deployment in fog computing. 345 E. 47th St. New York, NY. 38-45. https://doi.org/10.1109/ AIMS.2017.14
- Ashkan Yousefpour, Caleb Fung, Tam Nguyen, Krishna Kadiyala, Fatemeh Jalali, Amirreza Niakanlahiji, Jian Kong, and Jason P. Jue. 2019. All one needs to know about fog computing and related edge computing paradigms: A com- plete survey. Journal of Systems Architecture December 2018 (2019).
- L. Zanzi, F. Giust, and V. Sciancalepore. 2018. M2EC: A multi-tenant resource orchestration in multi-access edge computing systems. In 2018 IEEE Wireless Communications and Networking Conference (WCNC), Vol. 2018-April. 1-6. https://doi.org/10.1109/WCNC.2018.8377292
- Aleksandr Zavodovski, Nitinder Mohan, Suzan Bayhan, Walter Wong, and Jussi Kangasharju. 2019. ExEC: Elastic extensible edge cloud. In Proceedings of the 2Nd International Workshop on Edge Systems, Analytics and Networking. 24-29. Received March 2021; revised August 2021; accepted September 2021