Academia.eduAcademia.edu

Outline

Title
Abstract
Key Takeaways
Introduction
Overview
Service-Generated Big Data
Big Data-As-A-Service
Big Data Infrastructure-As-A-Service
Big Data Platform-As-A-Service
Big Data Analytics Software-As-A-Service
Conclusion and Future Work
References
All Topics
Computer Science
Software Engineering

Service-generated Big Data and Big Data-as-a-Service: An Overview

Profile image of Alain CorreaAlain Correa
https://doi.org/10.1109/BIGDATA.CONGRESS.2013.60
visibility

…

description

8 pages

link

1 file

Abstract

With the prevalence of service computing and cloud computing, more and more services are emerging on the Internet, generating huge volume of data, such as trace logs, QoS information, service relationship, etc. The overwhelming service-generated data become too large and complex to be effectively processed by traditional approaches. How to store, manage, and create values from the service-oriented big data become an important research problem. On the other hand, with the increasingly large amount of data, a single infrastructure which provides common functionality for managing and analyzing different types of service-generated big data is urgently required. To address this challenge, this paper provides an overview of service-generated big data and Big Data-as-a-Service. First, three types of service-generated big data are exploited to enhance system performance. Then, Big Data-as-a-Service, including Big Data Infrastructure-as-a-Service, Big Data Platform-as-a-Service, and Big Data Analytics Software-as-a-Service, is employed to provide common big data related services (e.g., accessing servicegenerated big data and data analytics results) to users to enhance efficiency and reduce cost.

Key takeaways
sparkles

AI

  1. Service-generated big data, including trace logs and QoS data, poses significant processing challenges.
  2. Big Data-as-a-Service (BDaaS) offers essential infrastructure, platform, and analytics services to manage big data effectively.
  3. Volume, velocity, variety, and veracity define the complexities of service-generated big data management.
  4. The global Big Data-as-a-Service market is projected to grow from $2.25 billion in 2015 to $30 billion by 2021.
  5. The paper reviews existing frameworks and proposes future research directions for enhancing service-generated big data analytics.

Related papers

Cloud Based Big-Data Analytics
kamran yousaf

With the bulk usage of informative and comprehensive data, direct streaming, massive web development, humungous web content, and enormous digital data, trustworthy and competent mechanisms can be made use of in order to store, sort and analyze these bulks of digital data.

View PDFchevron_right
Editorial for FGCS special issue: Big Data in the cloud
Muthu Ramachandran

Future Generation Computer Systems, 2016

a b s t r a c t Research associated with Big Data in the Cloud will be important topic over the next few years. The topic includes work on demonstrating architectures, applications, services, experiments and simulations in the Cloud to support the cases related to adoption of Big Data. A common approach to Big Data in the Cloud to allow better access, performance and efficiency when analysing and understanding the data is to deliver Everything as a Service. Organisations adopting Big Data this way find the boundaries between private clouds, public clouds and Internet of Things (IoT) can be very thin. Volume, variety, velocity, veracity and value are the major factors in Big Data systems but there are other challenges to be resolved. The papers of this special issue address a variety of issues and concerns in Big Data, including: searching and processing Big Data, implementing and modelling event and workflow systems, visualisation modelling and simulation and aspects of social media.

View PDFchevron_right
Big Data and Cloud Computing: Trends and Challenges
Hosam El-Sofany

International Journal of Interactive Mobile Technologies (iJIM), 2017

Big data is currently one of the most critical emerging technologies. Big Data are used as a concept that refers to the inability of traditional data architectures to efficiently handle the new data sets. The 4V’s of big data – volume, velocity, variety and veracity makes the data management and analytics challenging for the traditional data warehouses. It is important to think of big data and analytics together. Big data is the term used to describe the recent explosion of different types of data from disparate sources. Analytics is about examining data to derive interesting and relevant trends and patterns, which can be used to inform decisions, optimize processes, and even drive new business models. Cloud computing seems to be a perfect vehicle for hosting big data workloads. However, working on big data in the cloud brings its own challenge of reconciling two contradictory design principles. Cloud computing is based on the concepts of consolidation and resource pooling, but big ...

View PDFchevron_right
The Future of Big Data in Cloud
Emmanuel Oshogwe

Big data describes a large volume of data. This data is used on daily basis, it is not the amount of data that is most important, it is what the data is used for that matters and is most important, especially when it has to do with using the data to increase the efficiency and effectiveness of the business. Big data can be analyzed for insights that lead to better decisions and strategic business moves. The size of this data created is beyond the ability of typical database software tools to capture, store, manage, and analyze. Nowadays Big data ranges from a few dozen of terabytes to multiple petabytes (thousands of terabytes). This data can be measure in the value of data initiatives along the dimensions of the four Vs of big data. Measuring the value of data is an endless process, it is the way we make use of data that allows us to fully recognize its true value and importance to improve our decision making capabilities. These approach place high emphasis on the importance of every individual data item that goes into these systems and, as a result, highlight the importance of every single outcome linking to business impacts delivered. Big Data is now becoming a critically important driver of business success across sectors, but many executives say they don’t think their companies are equipped to make the most of it.

View PDFchevron_right
Towards Cloud-Based Analytics-as-a-Service (CLAaaS) for Big Data Analytics in the Cloud
Frank Bauernöppel, Farhana Zulkernine

2013 IEEE International Congress on Big Data, 2013

Data Analytics has proven its importance in knowledge discovery and decision support in different data and application domains. Big data analytics poses a serious challenge in terms of the necessary hardware and software resources. The cloud technology today offers a promising solution to this challenge by enabling ubiquitous and scalable provisioning of the computing resources. However, there are further challenges that remain to be addressed such as the availability of the required analytic software for various application domains, estimation and subscription of necessary resources for the analytic job or workflow, management of data in the cloud, and design, verification and execution of analytic workflows. We present a taxonomy for analytic workflow systems to highlight the important features in existing systems. Based on the taxonomy and a study of the existing analytic software and systems, we propose the conceptual architecture of CLoud-based Analytics-as-a-Service (CLAaaS), a big data analytics service provisioning platform, in the cloud. We outline the features that are important for CLAaaS as a service provisioning system such as user and domain specific customization and assistance, collaboration, modular architecture for scalable deployment and Service Level Agreement.

View PDFchevron_right
The big data system, components, tools, and technologies: a survey
Ravindara Bhatt

Knowledge and Information Systems, 2018

The traditional databases are not capable of handling unstructured data and high volumes of real-time datasets. Diverse datasets are unstructured lead to big data, and it is laborious to store, manage, process, analyze, visualize, and extract the useful insights from these datasets using traditional database approaches. However, many technical aspects exist in refining large heterogeneous datasets in the trend of big data. This paper aims to present a generalized view of complete big data system which includes several stages and key components of each stage in processing the big data. In particular, we compare and contrast various distributed file systems and MapReduce-supported NoSQL databases concerning certain parameters in data management process. Further, we present distinct distributed/cloud-based machine learning (ML) tools that play a key role to design, develop and deploy data models. The paper investigates case studies on distributed ML tools such as Mahout, Spark MLlib, and FlinkML. Further, we classify analytics based on the type of data, domain, and application. We distinguish various visualization tools pertaining three parameters: functionality, analysis capabilities, and supported development environment. Furthermore, we systematically investigate big data tools and technologies (Hadoop 3.0, Spark 2.3) including distributed/cloud-based stream processing tools in a comparative approach. Moreover, we discuss functionalities of several SQL Query tools on Hadoop based on 10 parameters. Finally, We present some critical points relevant to research directions and opportunities according to the current trend of big data. Investigating infrastructure tools for big data with recent developments provides a better understanding that how different tools and technologies apply to solve real-life applications.

View PDFchevron_right
Data as a Service (DaaS) for Sharing and Processing of Large Data Collections in the Cloud
Marcelle Martins

Data as a Service (DaaS) is among the latest kind of services being investigated in the Cloud computing community. The main aim of DaaS is to overcome limitations of state-ofthe-art approaches in data technologies, according to which data is stored and accessed from repositories whose location is known and is relevant for sharing and processing. Besides limitations for the data sharing, current approaches also do not achieve to fully separate/decouple software services from data and thus impose limitations in inter-operability. In this paper we propose a DaaS approach for intelligent sharing and processing of large data collections with the aim of abstracting the data location (by making it relevant to the needs of sharing and accessing) and to fully decouple the data and its processing. The aim of our approach is to build a Cloud computing platform, offering DaaS to support large communities of users that need to share, access, and process the data for collectively building knowledge from data. We exemplify the approach from large data collections from health and biology domains.

View PDFchevron_right
A Review on Cloud Based Big Data Analytics
International Computer Science and Engineering Society (ICSES)

— Today's computing world is facing tsunami and driving without riding on this tsunami towards next generation computing is no choice. So many IT companies decided to grow up with this tsunami like technology. One of these is cloud computing and another is Big data. Currently more than 5 billion mobile users, nearly same facebook, and other social media user generate this tsunami of data. On another side to deliver this services of big data a model called as cloud computing is spreading everywhere as next generations IT Service model. Both technologies continue to evolve. Ultimately, as a cloud, computing development matures, every top mind of organizations will think for development of efficient and agile cloud environment. At the other side, every cloud provider offers the services to the huge number data processing companies that generate data process data and make decision on cloud infrastructure. Ultimately its today's need to think on futures efficient cloud based Big data analytics In this review paper we are focusing on, how we can club Big data and cloud Computing in one frame of development.

View PDFchevron_right
A New Paradigm of Software Service Engineering in the Era of Big Data and Big Service
Gianmario Motta

ArXiv, 2016

Servitization is one of the most significant trends that reshapes the information world and society in recent years. The requirement of collecting,storing, processing, and sharing of the Big Data has led to massive software resources being developed and made accessible as web-based services to facilitate such process. These services that handle the Big Data come from various domains and heterogeneous networks, and converge into a huge complicated service network (or ecosystem), called the Big Service.The key issue facing the big data and big service ecosystem is how to optimally configure and operate the related service resources to serve the specific requirements of possible applications, i.e., how to reuse the existing service resources effectively and efficiently to develop the new applications or software services, to meet the massive individualized requirements of end-users.Based on analyzing the big service ecosystem, we present in this paper a new paradigm for software servic...

View PDFchevron_right
Coming Together of Big Data and Cloud Computing : A Review
International Journal of Scientific Research in Computer Science, Engineering and Information Technology IJSRCSEIT

International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 2020

Big data stands for sheer amount of data that is growing unceasingly at a rapid pace. Big Data demands high-powered, robust, reliable, fault-tolerant tools and techniques in order to make it convenient to process, analyse and uproot new insights from Big Data. Big data refers to huge, heterogeneous amount of details, facts and data generating at constantly rising rate. The data sets in Big Data are too bulky or extensive, as a result classical data handling application software are not competent enough to administer them. On the other hand, Cloud computing is a resourceful technology providing high computing power, scalability, computing resources as and when required for processing, storage, analytics and visualization of Big Data. Therefore, cloud computing can be regarded as a feasible and applicable technology which promises to handle Big Data challenges and also provides here and now infrastructures with all the mandatory resources. This paper will mainly review processing of big data cloud using Hadoop and spark in cloud, advantages of driving Big Data using cloud computing and applications of Big data in Cloud.

View PDFchevron_right

References (51)

  1. M. A. Beyer and D. Laney, "The importance of 'big data': A definition," Gartner, Tech. Rep., 2012.
  2. D. Austin, "eDiscovery Trends: CGOCs Information Lifecycle Gov- ernance Leader Reference Guide," "http://www.ediscoverydaily.com", May 2012.
  3. The Economist, "A special report on managing information: Data, data everywhere," The Economist, February 2010.
  4. IBM, "What is big data? ł bringing big data to the enterprise," "http://www-01.ibm.com/software/data/bigdata", 2013.
  5. H. Mi, H. Wang, Y. Zhou, M. R. Lyu, and H. Cai, "Towards fine-grained, unsupervised, scalable performance diagnosis for production cloud computing systems," IEEE Transactions on Parallel and Distributed Systems, no. PrePrints, 2013.
  6. B. H. Sigelman, L. A. Barroso, M. Burrows, P. Stephenson, M. Plakal, D. Beaver, S. Jaspan, and C. Shanbhag, "Dapper, a large-scale dis- tributed systems tracing infrastructure," Google, Inc., Tech. Rep., 2010.
  7. S. Han, Y. Dang, S. Ge, D. Zhang, and T. Xie, "Performance debugging in the large via mining millions of stack traces," in Proc. 34th Int'l Conf. on Software Engineering (ICSE'12), 2012, pp. 145-155.
  8. M. Y. Chen, A. Accardi, E. Kiciman, J. Lloyd, D. Patterson, A. Fox, and E. Brewer, "Path-based faliure and evolution management," in Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation. USENIX Association, 2004, pp. 23-23.
  9. C. Lim, N. Singh, and S. Yajnik, "A log mining approach to fail- ure analysis of enterprise telephony systems," in Proceedings of the IEEE International Conference on Dependable Systems and Networks (DSN'08), 2008, pp. 398-403.
  10. E. Thereska, B. Salmon, J. Strunk, M. Wachs, M. Abd-El-Malek, J. Lopez, and G. R. Ganger, "Stardust: tracking activity in a distributed storage system," in ACM SIGMETRICS Performance Evaluation Re- view, vol. 34, no. 1. ACM, 2006, pp. 3-14.
  11. H. Mi, H. Wang, H. Cai, Y. Zhou, M. R. Lyu, and Z. Chen, "P- tracer: Path-based performance profiling in cloud computing systems," in Proceedings of the 36th IEEE Annual Computer Software and Applications Conference (COMPSAC'12). IEEE, 2012, pp. 509-514.
  12. B. M. Cantrill, M. W. Shapiro, A. H. Leventhal et al., "Dynamic instrumentation of production systems," in USENIX Annual Technical Conference, 2004, pp. 15-28.
  13. S. L. Graham, P. B. Kessler, and M. K. Mckusick, "Gprof: A call graph execution profiler," ACM Sigplan Notices, vol. 17, no. 6, pp. 120-126, 1982.
  14. P. Barham, A. Donnelly, R. Isaacs, and R. Mortier, "Using magpie for request extraction and workload modelling," in Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation (OSDI'04), 2004, pp. 18-18.
  15. P. Reynolds, C. Killian, J. L. Wiener, J. C. Mogul, M. A. Shah, and A. Vahdat, "Pip: detecting the unexpected in distributed systems," in Proceedings of the 3rd conference on Networked Systems Design & Implementation (NSDI'06), 2006, pp. 9-9.
  16. E. Thereska and G. R. Ganger, "Ironmodel: robust performance mod- els in the wild," in Proceedings of the International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '08), 2008, pp. 253-264.
  17. M. Chen, E. Kiciman, E. Fratkin, A. Fox, and E. Brewer, "Pinpoint: problem determination in large, dynamic internet services," in Pro- ceedings of the International Conference on Dependable Systems and Networks (DSN'02), pp. 595-604.
  18. W. Xu, L. Huang, A. Fox, D. Patterson, and M. I. Jordan, "Detecting large-scale system problems by mining console logs," in Proceedings of the ACM 22nd Simposium on Operating Systems Principles (SOSP'09), 2009, pp. 117-132.
  19. M. R. Lyu, Software Fault Tolerance. Trends in Software, Wiley, 1995.
  20. Z. Zheng and M. R. Lyu, "A QoS-aware fault tolerant middleware for dependable service composition," in Proc. 39th Int'l Conf. Dependable Systems and Networks (DSN'09), 2009, pp. 239-248.
  21. H. Yang, Z. Xu, I. King, and M. Lyu, "Online learning for group lasso," in International Conference on Machine Learning (ICML'10), 2010.
  22. L. Shao, J. Zhang, Y. Wei, J. Zhao, B. Xie, and H. Mei, "Personalized QoS prediction for Web services via collaborative filtering," in Proc. 5th Int'l Conf. Web Services (ICWS'07), 2007, pp. 439-446.
  23. Z. Zheng, H. Ma, M. R. Lyu, and I. King, "QoS-aware Web service rec- ommendation by collaborative filtering," IEEE Transactions on Service Computing, vol. 4, no. 2, pp. 140-152, 2011.
  24. Z. Zheng, Y. Zhang, and M. R. Lyu, "CloudRank: A QoS-driven component ranking framework for cloud computing," in Proc. Int'l Symp. Reliable Distributed Systems (SRDS'10), 2010, pp. 184-193.
  25. X. Chen, Z. Zheng, X. Liu, Z. Huang, and H. Sun, "Personalized QoS-aware Web service recommendation and visualization," IEEE Transactions on Services Computing, no. PrePrints, 2011.
  26. M. Tang, Y. Jiang, J. Liu, and X. F. Liu, "Location-aware collaborative filtering for qos-based service recommendation," in Pro. IEEE 19th Int'l Conf' on Web Services (ICWS'12), 2012, pp. 202-209.
  27. W. Lo, J. Yin, S. Deng, Y. Li, and Z. Wu, "Collaborative web service qos prediction with location-based regularization," in Pro. IEEE 19th Int'l Conf' on Web Services (ICWS'12), 2012, pp. 464-471.
  28. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels, "Dynamo: amazon's highly available key-value store," in Proc. 21st ACM Sympo- sium on Operating Systems Principles (SOSP'07), 2007, pp. 205-220.
  29. A. Liu, Q. Li, L. Huang, and S. Wen, "Shapley value based impression propagation for reputation management in web service composition," in Pro. IEEE 19th Int'l Conf' on Web Services (ICWS'12), 2012, pp. 58-65.
  30. Y. Kang, Z. Zheng, and M. Lyu, "A latency-aware co-deployment mechanism for cloud-based services," in Proceedings of the IEEE 5th International Conference on Cloud Computing (CLOUD'12), 2012, pp. 630-637.
  31. J. Zhu, Z. Zheng, Y. Zhou, and M. R. Lyu, "Scaling service-oriented applications into geo-distributed clouds," in Pro. IEEE Int'l Workshop on Internet-based Virtual Computing Environment (iVCE'13), 2013.
  32. Q. Zhang, Q. Zhu, M. F. Zhani, and R. Boutaba, "Dynamic service placement in geographically distributed clouds," in Proc. IEEE 32nd Int'l Conf. on Distributed Computing Systems (ICDCS'12), 2012, pp. 526-535.
  33. M. Steiner, B. G. Gaglianello, V. K. Gurbani, V. Hilt, W. D. Roome, M. Scharf, and T. Voith, "Network-aware service placement in a distributed cloud environment," in Proc. ACM SIGCOMM'12, 2012, pp. 73-74.
  34. M. Alicherry and T. V. Lakshman, "Network aware resource allocation in distributed clouds," in Proc. IEEE INFOCOM'12, 2012, pp. 963-971.
  35. S. Lohr, "The age of big data," New York Times, vol. 11, 2012.
  36. "Challenges and opportunities with big data," leading researchers across the United States, Tech. Rep., 2011.
  37. E. Slack, "Storage infrastructures for big data workflows," Storage Switchland, LLC, Tech. Rep., 2012.
  38. C. Lynch, "Big data: How do your data grow?" Nature, vol. 455, no. 7209, pp. 28-29, 2008.
  39. Y. Demchenko, "Bof: Infrastructure issues in big data," "https://tnc2013.terena.org/core/event/15", 2013.
  40. "Big data-as-a-service: A market and technology perspective," EMC Solution Group, Tech. Rep., 2012.
  41. J. Horey, E. Begoli, R. Gunasekaran, S.-H. Lim, and J. Nutaro, "Big data platforms as a service: challenges and approach," in Proceedings of the 4th USENIX conference on Hot Topics in Cloud Ccomputing, ser. HotCloud'12, 2012, pp. 16-16.
  42. R. Sharir, "Cloud database service: The difference between dbaas, daas and cloud storage -what's the difference?" "http://xeround.com/blog/2011/02/dbaas-vs-daas-vs-cloud-storage- difference", 2011.
  43. B. Devlin, S. Rogers, and J. Myers, "Big data comes of age," Tech. Rep.
  44. M. Lenzerini, "Data integration: A theoretical perspective," in Pro- ceedings of the 21st ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems. ACM, 2002, pp. 233-246.
  45. J. LaFayette, "The future of 'big data' is apps, not infras- tructure," "http://venturebeat.com/2013/01/04/the-future-of-big-data-is- apps-not-infrastructure/", 2013.
  46. M. Rouse, "Definition of big data analytics," "http://searchbusinessanalytics.techtarget.com/definition/big-data- analytics", January 2012.
  47. "Why big data analytics as a service?" "http://www.analyticsasaservice.org/why-big-data-analytics-as-a- service/", August 2012.
  48. P. O'Brien, "The future: Big data apps or web services?" "http://blog.fliptop.com/blog/2012/05/12/the-future-big-data-apps- or-web-services/", 2013.
  49. "What is big data? analytics as a service in the cloud," "http://www.analyticsasaservice.org/what-is-big-data-analytics-as-a- service-in-the-cloud/", March 2012.
  50. V. W. Consulting, "Big data, big impact: New possibilities for interna- tional development," The World Economic Forum, Tech. Rep., 2012.
  51. J. Manyika, M. Chui, B. Brown, J. Bughin, R. Dobbs, C. Roxburgh, and A. H. Byers, "Big data: The next frontier for innovation, competition, and productivity," McKinsey Global Institute, pp. 1-137, 2011.

Related papers

Big data analytics as-a-service: Issues and challenges
Ernesto Damiani

2016 IEEE International Conference on Big Data (Big Data), 2016

Big Data domain is one of the most promising ICT sectors with substantial expectations both on the side of market growing and design shift in the area of data storage managment and analytics. However, today, the level of complexity achieved and the lack of standardisation of Big Data management architectures represent a huge barrier towards the adoption and execution of analytics especially for those organizations and SMEs not including a sufficient amount of competences and knowledge. The full potential of Big Data Analytics (BDA) can be unleashed only through the definition of approaches that accomplish Big Data users' expectations and requirements, also when the latter are fuzzy and ambiguous. Under these premises, we propose Big Data Analytics-as-a-Service (BDAaaS) as the next-generation Big Data Analytics paradigm and we discuss issues and challenges from the BDAaaS design and development perspective.

View PDFchevron_right
Big Data Platforms as a Service: Challenges and Approach
Edmon Begoli

usenix.org

Infrastructure-as-a-Service has revolutionized the manner in which users commission computing infrastructure. Coupled with Big Data platforms (Hadoop, Cassandra), IaaS has democratized the ability to store and process massive datasets. For users that need to customize or create new Big Data stacks, however, readily available solutions do not yet exist. Users must first acquire the necessary cloud computing infrastructure, and manually install the prerequisite software. For complex distributed services this can be a daunting challenge. To address this issue, we argue that distributed services should be viewed as a single application consisting of virtual machines. Users should no longer be concerned about individual machines or their internal organization. To illustrate this concept, we introduce Cloud-Get, a distributed package manager that enables the simple installation of distributed services in a cloud computing environment. Cloud-Get enables users to instantiate and modify distributed services, including Big Data services, using simple commands. Cloud-Get also simplifies creating new distributed services via standardized package definitions.

View PDFchevron_right
Big Data Analytics With Service-Oriented Architecture
Asoke Nath

Advances in Business Information Systems and Analytics, 2017

This chapter focuses on Big Data and its relation with Service-Oriented Architecture. We start with the introduction to Big Data Trends in recent times, how data explosion is not only faced by web and retail networks but also the enterprises. The notorious “V's” – Variety, volume, velocity and value can cause a lot of trouble. We emphasize on the fact that Big Data is much more than just size, the problem that we face today is neither the amount of data that is created nor its consumption, but the analysis of all those data. In our next step, we describe what service-oriented architecture is and how SOA can efficiently handle the increasingly massive amount of transactions. Next, we focus on the main purpose of SOA here is to meaningfully interoperate, trade, and reuse data between IT systems and trading partners. Using this Big Data scenario, we investigate the integration of Services with new capabilities of Enterprise Architectures and Management. This has had varying success...

View PDFchevron_right
ARCHITECTURE FOR INTELLIGENT BIG DATA ANALYSIS BASED ON AUTOMATIC SERVICE COMPOSITION
B. T. G. S. Kumara

Big Data contains massive information, which are generating from heterogeneous, autonomous sources with distributed and anonymous platforms. Since, it raises extreme challenge to organizations to store and process these data. Conventional pathway of store and process is happening as collection of manual steps and it is consuming various resources. An automated real-time and online analytical process is the most cognitive solution. Therefore it needs state of the art approach to overcome barriers and concerns currently facing by the Big Data industry. In this paper we proposed a novel architecture to automate data analytics process using Nested Automatic Service Composition (NASC) and CRoss Industry Standard Platform for Data Mining (CRISP-DM) as main based technologies of the solution. NASC is well defined scalable technology to automate multi-disciplined problems domains. Since CRISP-DM also a well-known data science process which can be used as innovative accumulator of multi-dimensional data sets. CRISP-DM will be mapped with Big Data analytical process and NASC will automate the CRISP-DM process in an intelligent and innovative way.

View PDFchevron_right
Toward Model-Based Big Data-as-a-Service: The TOREADOR Approach
Nello Scarabottolo

Advances in Databases and Information Systems, 2017

The full potential of Big Data Analytics (BDA) can be unleashed only by overcoming hurdles like the high architectural complexity and lack of transparency of Big Data toolkits, as well as the high cost and lack of legal clearance of data collection, access and processing procedures. We first discuss the notion of Big Data Analytics-as-a-Service (BDAaaS) to help potential users of BDA in overcoming such hurdles. We then present TOREADOR, a first approach to BDAaaS.

View PDFchevron_right

Related topics

  • Service Oriented Computing
  • Big Data As a Service

    • Cited by

    Supply Chain Management and Logistics Big Data Challenges in Bulgaria
    Luben Boyanov

    LOGI – Scientific Journal on Transport and Logistics

    The article focuses on the challenges of applying big data approaches to logistics and supply chain management in Bulgaria, while also presenting the challenges, opportunities and problems from a broader perspective. This includes a review of big data in logistics and its subsystems in relation to integration and transformation processes, as well as the role of big data and the relevant technologies in supply chain management. The research framework for the study of Bulgarian companies focuses on their experiences, both positive and negative, with big data. Data was collected using an electronic questionnaire. The respondents were from small, medium-sized and large companies alike. The results demonstrate and explain the companies’ experiences with big data and the level of application thereof. The findings showed that the main areas for the use of big data are order processing, communication processes, reporting systems, inventory management and vehicle database management.

    View PDFchevron_right
    BINARY: A framework for big data integration for ad-hoc querying
    Farhana Zulkernine

    2016 IEEE International Conference on Big Data (Big Data), 2016

    Traditional relational database systems are not practical for big data workloads that require scalable architectures for efficient data storage, manipulation, and analysis. Apache Hadoop, one of these big data frameworks, provides distributed storage and processing as well as a central repository for different types of data from different sources. Data integration from various sources is often required before performing analytics. Apache Hive on Hadoop is widely used for this purpose, as well as for data summarization and analysis. It has features such as a SQL-like query language, a Metastore to hold metadata and file formats to support access to various frameworks on Hadoop and beyond. For comprehensive analysis and decision-making, however, a hybrid system is required to integrate Hadoop with traditional relational database management systems in order to access the valuable data stored in relational databases. Current hybrid systems are either expensive proprietary products or require a system to be developed by the user, which requires programming knowledge. In addition these approaches are not sufficiently flexible to be applied to other frameworks. In this thesis, we propose a framework called BINARY (A framework for Big data INtegration for Ad-hoc queRYing). BINARY is a hybrid Software as a Service that provides a web interface supported by a back-end infrastructure for ad-hoc querying, accessing, visualizing and joining data from different data sources, including Relational Database Management Systems and Apache Hive. Our framework uses scalable Hive and HDFS big data storage systems and supports different data sources via back-end resource adapters. There is also a front-end web interface that enables the use of HiveQL to query the data sources. The framework is extendable and allows adding other storage engines (e.g. HBase) and analytics engines (e.g. R) as needed. We used REST software architecture to enable loose connections between the engines and the User Interface programs to facilitate independent updates without affecting the data infrastructure. Our approach is validated with a proof-of-concept prototype implemented on the OpenStack cloud system. iii Acknowledgements First and foremost I would like to sincerely thank my supervisors, Prof. Patrick Martin, and Dr. Farhana Zulkernine. I am grateful to you for your guidance and knowledge, and most of all for your valuable feedback, support and encouragement throughout my research. I would like to thank Wendy Powley for offering her continuous support and helping me during the course of my research. I am very grateful to Shadi Khalifa and Yehia Elshater for friendly technical discussions and constructive criticism during this work. I would like to thank the School of Computing and the School of Graduate Studies at Queen's University for providing me with the opportunity to pursue my master's degree in computing. I am grateful to IBM Canada Ltd. and NSERC for their financial support to the Database Systems Lab at Queen's University. Lastly, but most importantly, I would like to sincerely thank my parents and my dear husband, Alireza Safaee, for their continuous support and encouragement.

    View PDFchevron_right
    Cyber physical systems design, methodology, and integration: the current status and future outlook
    Ashraf Darwish

    Journal of Ambient Intelligence and Humanized Computing, 2017

    Emerging cyber-physical systems (CPS) field will impact on every activity of the daily life in the next few years With the increasing advances in ICT information communication technology (ICT), multidisciplinary fields of research such as cyber-physical systems (CPS) have been wide. The emergence of cyber-physical-social systems (CPS) as a novel paradigm has revolutionized the relationship between humans, computers and the physical environment. This paper aims to present a comprehensive overview of CPS key aspects from different points of views such as design, methodology, and key enabler technologies such as sensor networks, Internet of Things (IoT), cloud computing, and multi-agent systems. Also, some important applications such as smart cities which are considered as one of the important application that will provide new services for healthcare applications, vehicular networks, energy distribution, the monitoring of environmental changes, business and commerce applications, emergency response, and other social activities and industrial systems are explored. Also, this paper considers the requirements and methodologies for CPS integrations such as CloudIoT and CloudIoV systems. Although, there are significant efforts and studies have been made towards CPS application, key challenges and problems

    View PDFchevron_right
    A Hybrid Service Selection and Composition for Cloud Computing Using the Adaptive Penalty Function in Genetic and Artificial Bee Colony Algorithm
    Seyed Salar Sefati, Simona Halunga

    Sensors

    The rapid development of Cloud Computing (CC) has led to the release of many services in the cloud environment. Service composition awareness of Quality of Service (QoS) is a significant challenge in CC. A single service in the cloud environment cannot respond to the complex requests and diverse requirements of the real world. In some cases, one service cannot fulfill the user’s needs, so it is necessary to combine different services to meet these requirements. Many available services provide an enormous QoS and selecting or composing those combined services is called an Np-hard optimization problem. One of the significant challenges in CC is integrating existing services to meet the intricate necessities of different types of users. Due to NP-hard complexity of service composition, many metaheuristic algorithms have been used so far. This article presents the Artificial Bee Colony and Genetic Algorithm (ABCGA) as a metaheuristic algorithm to achieve the desired goals. If the fitnes...

    View PDFchevron_right
    IoT Mashups
    Thomas Devogele

    Proceedings of the International Conference on Future Networks and Distributed Systems, 2017

    Internet of Things (IoT) addresses the challenge to provide a transparent access to a huge number of IoT resources that can be either physical devices or just data resources. Moreover, because of the large number of resource-constrained devices and the dynamic nature of IoT environments, integrating the resulted data becomes a non trivial task. We believe that the use of mashups, a way to compose new services from existing ones, can be a solution to the above challenge if each resource exposes its functionalities as a Web service. In the IoT environment, this will constitute a Web of Things where mashups development will take advantage of the connected physical world. The huge amounts of IoT-generated data from physical devices and data sources, called IoT Big Data, requires new design solutions to speed up data processing, scale up with the data volume and improve data adaptability. Besides existing techniques for IoT data collection, ltering, and analytics, we present in this article a mashup oriented model, called IoT Big Services, for provisioning data-centric IoT services in the context of IOT mashups. These IoT services are organized in tree structure where each node, called an IoT Big Service, acts as an integrator that collects data from lower level, processes them and delivers the results to higher level in the architecture.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved