Analyzing Traces from a Google Data Center

2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2020

Proceedings of the 2nd …, 2011

Evaluating the performance of large compute clusters requires benchmarks with representative workloads. At Google, performance benchmarks are used to obtain performance metrics such as task scheduling delays and machine resource utilizations to assess changes in application codes, machine configurations, and scheduling algorithms. Existing approaches to workload characterization for high performance computing and grids focus on task resource requirements for CPU, memory, disk, I/O, network, etc. Such resource requirements address how much resource is consumed by a task. However, in addition to resource requirements, Google workloads commonly include task placement constraints that determine which machine resources are consumed by tasks. Task placement constraints arise because of task dependencies such as those related to hardware architecture and kernel version.

Informatica, 2021

Cloud computing consists of an advanced set of technologies that allow cloud providers to offer computing resources such as infrastructure, platforms and applications to be accessible over the Internet as services. Cloud computing relies on virtualization of resources in the cloud data centers, where a set of Virtual Machines (VMs) are deployed on Physical Machines (PMs) to provision and serve user requests. Due to the dynamic nature of cloud environments and complexity of resources virtualization, as well as the diversity of user's requests, developing effective techniques to evaluate and analyze the performance of cloud centers has become highly required. In this paper, we propose the use of probabilistic model checking as an effective framework for the evaluation and the performance analysis of resource provisioning in the cloud. Based on an analytical model for resource provisioning in Infrastructure-as-a-Service (IaaS) cloud, we build a stochastic model using the probabilistic model checker PRISM and analyze it against a useful set of probabilistic and reward properties that help to measure and analyze cloud performance in an efficient way. Povzetek: Analizirane so razne komponente računanja v oblaku, npr. modeliranje in performance virov.

2014

ACM SIGMETRICS Performance Evaluation Review, 2010

The advent of cloud computing promises highly available, efficient, and flexible computing services for applications such as web search, email, voice over IP, and web search alerts. Our experience at Google is that realizing the promises of cloud computing requires an extremely scalable backend consisting of many large compute clusters that are shared by application tasks with diverse service level requirements for throughput, latency, and jitter. These considerations impact (a) capacity planning to determine which machine resources must grow and by how much and (b) task scheduling to achieve high machine utilization and to meet service level objectives. Both capacity planning and task scheduling require a good understanding of task resource consumption (e.g., CPU and memory usage). This in turn demands simple and accurate approaches to workload classification-determining how to form groups of tasks (workloads) with similar resource demands. One approach to workload classification i...

Cloud computing, the long-held dream of computing as a utility, has the potential to transform a large part of the IT industry, making software even more attractive as a service and shaping the way in which hardware is designed and purchased. We review the new cloud computing technologies, and indicate the main challenges for their development in future, among which resource management problem stands out and attracts our attention. Combining the current scheduling theories, we propose cloud scheduling hierarchy to deal with different requirements of cloud services.

2014 IEEE Network Operations and Management Symposium (NOMS), 2014

T h e quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely even t that the author did not send a com plete m anuscript and there are missing pages, these will be noted. Also, if m aterial had to be rem oved, a note will indicate the deletion. Di!ss0?t&iori Publishing UM I 3580411

ArXiv, 2021

Improving datacenter operations is vital for the digital society. We posit that doing so requires our community to shift, from operational aspects taken in isolation to holistic analysis of datacenter resources, energy, and workloads. In turn, this shift will require new analysis methods, and open-access, FAIR datasets with fine temporal and spatial granularity. We leverage in this work one of the (rare) public datasets providing fine-grained information on datacenter operations. Using it, we show strong evidence that fine-grained information reveals new operational aspects. We then propose a method for holistic analysis of datacenter operations, providing statistical characterization of node, energy, and workload aspects. We demonstrate the benefits of our holistic analysis method by applying it to the operations of a datacenter infrastructure with over 300 nodes. Our analysis reveals both generic and ML-specific aspects, and further details how the operational behavior of the data...

Cluster, Cloud and Grid …, 2011

Cloud computing is an emerging infrastructure paradigm that promises to eliminate the need for companies to maintain expensive computing hardware. Through the use of virtualization and resource time-sharing, clouds address with a single set of physical resources a large user base with diverse needs. Thus, clouds have the potential to provide their owners the benefits of an economy of scale and, at the same time, become an alternative for both the industry and the scientific community to self-owned clusters, grids, and parallel production environments. For this potential to become reality, the first generation of commercial clouds need to be proven to be dependable. In this work we analyze the dependability of cloud services. Towards this end, we analyze long-term performance traces from Amazon Web Services and Google App Engine, currently two of the largest commercial clouds in production. We find that the performance of about half of the cloud services we investigate exhibits yearly and daily patterns, but also that most services have periods of especially stable performance. Last, through trace-based simulation we assess the impact of the variability observed for the studied cloud services on three large-scale applications, job execution in scientific computing, virtual goods trading in social networks, and state management in social gaming. We show that the impact of performance variability depends on the application, and give evidence that performance variability can be an important factor in cloud provider selection.