MLaaS: Machine Learning as a Service

XIAMEN-CUSTIPEN WORKSHOP ON THE EQUATION OF STATE OF DENSE NEUTRON-RICH MATTER IN THE ERA OF GRAVITATIONAL WAVE ASTRONOMY

Future Generation Computer Systems, 2020

As Machine Learning (ML) applications are becoming ever more pervasive, fully-trained systems are made increasingly available to a wide public, allowing end-users to submit queries with their own data, and to efficiently retrieve results. With increasingly sophisticated such services, a new challenge is how to scale up to ever growing user bases. In this paper, we present a distributed architecture that could be exploited to parallelize a typical ML system pipeline. We propose a case study consisting of a text mining service, and discuss how the method can be generalized to many similar applications. We demonstrate the significance of the computational gain boosted by the distributed architecture by way of an extensive experimental evaluation.

2019

With the proliferation of machine learning (ML) libraries and frameworks, and the programming languages that they use, along with operations of data loading, transformation, preparation and mining, ML model development is becoming a daunting task. Furthermore, with a plethora of cloud-based ML model development platforms, heterogeneity in hardware, increased focus on exploiting edge computing resources for low-latency prediction serving and often a lack of a complete understanding of resources required to execute ML workflows efficiently, ML model deployment demands expertise for managing the lifecycle of ML workflows efficiently and with minimal cost. To address these challenges, we propose an end-to-end data analytics, a serverless platform called Stratum. Stratum can deploy, schedule and dynamically manage data ingestion tools, live streaming apps, batch analytics tools, ML-as-a-service (for inference jobs), and visualization tools across the cloud-fog-edge spectrum. This paper describes the Stratum architecture highlighting the problems it resolves.

In traditional SaaS enterprise applications, microservices are an essential ingredient to deploy machine learning (ML) models successfully. In general, microservices result in efficiencies in software service design, development, and delivery. As they become ubiquitous in the redesign of monolithic software, with the addition of machine learning, the traditional applications are also becoming increasingly intelligent. Here, we propose a portable ML microservice framework Minerva (microservices container for applied ML) as an efficient way to modularize and deploy intelligent microservices in traditional "legacy" SaaS applications suite, especially in the enterprise domain. We identify and discuss the needs, challenges and architecture to incorporate ML microservices in such applications. Minerva's design for optimal integration with legacy applications using microservices architecture leveraging lightweight infrastructure accelerates deploying ML models in such applications.

arXiv (Cornell University), 2020

We are witnessing an increasing trend towards using Machine Learning (ML) based prediction systems, spanning across different application domains, including product recommendation systems, personal assistant devices, facial recognition, etc. These applications typically have diverse requirements in terms of accuracy and response latency, that have a direct impact on the cost of deploying them in a public cloud. Furthermore, the deployment cost also depends on the type of resources being procured, which by themselves are heterogeneous in terms of provisioning latencies and billing complexity. Thus, it is strenuous for an inference serving system to choose from this confounding array of resource types and model types to provide low-latency and cost-effective inferences. In this work we quantitatively characterize the cost, accuracy and latency implications of hosting ML inferences on different public cloud resource offerings. In addition, we comprehensively evaluate prior work which tries to achieve cost-effective prediction-serving. Our evaluation shows that, prior work does not solve the problem from both dimensions of model and resource heterogeneity. Hence, we argue that to address this problem, we need to holistically solve the issues that arise when trying to combine both model and resource heterogeneity towards optimizing for application constraints. Towards this, we envision developing a self-managed inference serving system, which can optimize the application requirements based on public cloud resource characteristics. In order to solve this complex optimization problem, we explore the high level design of a reinforcement-learning based system that can efficiently adapt to the changing needs of the system at scale.

International Journal of Advanced Research in Science, Communication and Technology (IJARSCT), 2024

This paper conducts an in-depth examination of Amazon Web Services (AWS) Machine Learning, a collection of tools and services aimed at simplifying the process of building, training, and deploying machine learning models. It starts with an analysis of essential components such as Amazon SageMaker and AWS Deep Learning AMIs, detailing their functionalities and how they integrate into the larger AWS framework. The discussion then shifts to real-world applications in various sectors, including healthcare, finance, retail, and manufacturing, highlighting successful use cases and practical examples. The paper evaluates the strengths and limitations of AWS Machine Learning, considering factors like scalability, userfriendliness, cost, and support for diverse machine learning frameworks, as well as challenges such as the learning curve and reliance on cloud infrastructure. The paper also explores future trends and directions, including improvements in automation, the fusion of AI with Internet of Things (IoT) devices, and the development of new tools to enhance the machine learning lifecycle. These insights are intended to assist organizations in making informed decisions about using AWS for their AI and machine learning projects, enabling them to effectively harness AWS's capabilities to meet their objectives.

International Journal of Advanced Scientific Innovation, 2021

Machine learning is used in enterprise systems for automation, but it can be difficult to integrate many machine learning dependencies and utilities, due to the absence of a common architecture. In certain cases, machine learning frameworks are advanced for systems that are struggling for particular business area usage cases, resulting in redundant effort and incredible reuse. Acumos is a free and open-source framework. Acumos can conveniently bundle machine learning models into scalable containerized microservices that can be shared through the platform's catalogue and implemented into a range of business applications. Via Acumos app, providing a case study of packaging sentiment analysis and classification machine learning models, enabling allocation with others. We show that when adapting machine learning models to business applications, the Acumos framework reduces the technological burden on application developers. I.

Curry, E., Scerri, S., Tuikka, T. (eds) Data Spaces, 2022

The current digital transformation of many businesses and the exponential growth of digital data are two of the key factors of the digital revolution. For the successful meeting of the high expectations, the data platforms need to employ the recent theoretical, technological, and methodological advances in contemporary computing and data science and engineering. This chapter presents an approach to address these challenges by combining logical methods for knowledge processing and machine learning methods for data analysis into a hybrid AI-based framework. It is applicable to a wide range of problems that involve both synchronous operations and asynchronous events in different domains. The framework is a foundation for building the GATE Data Platform, which aims at the application of Big Data technologies in civil and government services, industry, and healthcare. The platform implementation will utilize several recent distributed technologies such as Internet-of-Things, Cloud, and Edge computing and will integrate them into a multi-level service-oriented architecture that supports services along the entire data value chain, while the service orchestration guarantees a high degree of interoperability, reusability, and automation. The platform is designed to be compliant with the open-source software, but its open architecture supports also mixing with commercial components and tools.

2015

In this paper, we start off by summarizing the key evolutionary turning points of machine learning APIs and conclude by laying out our vision for the future of this key enabling component that can power tomorrow’s ubiquitous intelligent systems.

2024

The rapid development of modern information technology (IT), power supply, communication and traffic information systems and so on is resulting in progress in the area of distributed and energy-efficient (if possible, powered by renewable energy sources) smart grid components securely connected to entire smart city management systems. This enables a wide range of applications such as distributed energy management, system health forecasting and cybersecurity based on huge volumes of data that automate and improve the performance of the smart grid, but also require analysis, inference and prediction using artificial intelligence. Data management strategies, but also the sharing of data by consumers, institutions, organisations and industries, can be supported by edge clouds, thus protecting privacy and improving performance. This article presents and develops the authors’ own concept in this area, which is planned for research in the coming years. The paper aims to develop and initially test a conceptual framework that takes into account the aspects discussed above, emphasising the practical aspects and use cases of the Social Internet of Things (SIoT) and artificial intelligence (AI) in the everyday lives of smart sustainable city (SSC) residents. We present an approach consisting of seven algorithms for the integration of large data sets for machine learning processing to be applied in optimisation in the context of smart cities.