Computer science and engineering

Performance and power issues are becoming increasingly important in the design of large cluster based multitier data centers for supporting a multitude of services. Design and analysis of such large/complex distributed system often suffers from the lack of availability of an adequate physical infrastructure and the cost constraints especially in the academic community. With this motivation, this paper presents a comprehensive, flexible and scalable simulation platform for in-depth analysis of multi-tier data centers. Designed as a pluggable threelevel architecture, our simulator captures all the important design specifics of the underlying communication paradigm, kernel level scheduling artifacts, and the application level interactions among the tiers of a three-tier data center. The flexibility of the simulator is attributed to its ability in experimenting with different design alternatives in the three layers, and in analyzing both the performance and power consumption with realistic workloads. The scalability of the simulator is demonstrated with analyses of different data center configurations. In addition, we have designed a prototype three-tier data center on an IBA connected Linux cluster to validate the simulator. Using RUBiS benchmark workload, it is shown that the simulator is quite accurate in estimating the throughput, response time, and power consumption parameters. We then demonstrate the applicability of the simulator in conducting three different types of studies. First, we conduct a comparative analysis of the Infiniband Architecture (IBA) and 10 Gigabit Ethernet (10GigE) under different traffic conditions and with varying size clusters for understanding their relative merits in designing cluster-based servers. Second, measurement and characterization of power consumption across the servers of a three-tier data center is done. Third, we perform a configuration analysis of the Web server (WS), Application Server (AS), and Database Server (DB) for performance optimization. We believe that such a comprehensive simulation infrastructure is critical for providing guidelines in designing efficient and cost-effective multi-tier data centers.

Efficient resource management in data centers and clouds running large distributed data processing frameworks like MapReduce is crucial for enhancing the performance of hosted applications and increasing resource utilization. However, existing resource scheduling schemes in Hadoop MapReduce allocate resources at the granularity of fixed-size, static portions of nodes, called slots. In this work, we show that MapReduce jobs have widely varying demands for multiple resources, making the static and fixed-size slot-level resource allocation a poor choice both from the performance and resource utilization standpoints. Furthermore, lack of coordination in the management of multiple resources across nodes prevents dynamic slot reconfiguration, and leads to resource contention. Motivated by this, we propose MROrchestrator, a MapReduce resource Orchestrator framework, which can dynamically identify resource bottlenecks, and resolve them through fine-grained, coordinated, and on-demand resource allocations. We have implemented MROrchestrator on two 24-node native and virtualized Hadoop clusters. Experimental results with a suite of representative MapReduce benchmarks demonstrate up to 38% reduction in job completion times, and up to 25% increase in resource utilization. We further demonstrate the performance boost in existing resource managers like NGM and Mesos, when augmented with MROrchestrator.

Evaluating the performance of large compute clusters requires benchmarks with representative workloads. At Google, performance benchmarks are used to obtain performance metrics such as task scheduling delays and machine resource utilizations to assess changes in application codes, machine configurations, and scheduling algorithms. Existing approaches to workload characterization for high performance computing and grids focus on task resource requirements for CPU, memory, disk, I/O, network, etc. Such resource requirements address how much resource is consumed by a task. However, in addition to resource requirements, Google workloads commonly include task placement constraints that determine which machine resources are consumed by tasks. Task placement constraints arise because of task dependencies such as those related to hardware architecture and kernel version.

Abstract. Cloud computing has emerged as a popular computing paradigm allowing workloads to automatically scale in response to changes in demand. Clouds use virtualization to enable elasticity by continually reconfiguring the allocation of physical and virtual resources to workloads. Continual changes in clouds may lead to unexpected performance anomalies and traditional problem determination techniques are unable to deal with such cloud-induced anomalies.

Abstract—Performance and power issues are becom-ing increasingly important in the design of large data centers for supporting a multitude of services. There are many perspectives of addressing these issues using various computer science principles. In this survey report, I will discuss the applications of data mining techniques to manage power and performance in data centers.

Abstract—Virtualized environments are attractive because they simplify cluster management, while facilitating cost-effective workload consolidation. As a result, virtual machines, either in public clouds or in private data centers, have become the norm for running many interactive applications such as web servers. On the other hand, batch workloads like MapReduce are typically deployed in a native cluster for avoiding the performance overheads of virtualization.

This paper presents MobiSNA -a mobile video social networking application that supports the exploration, sharing, and creation of video contents through social networks. The MobiSNA project provides the user with an easy to use experience of accessing video content from mobile devices (e.g., mobile phones, PDAs) over wireless broadband networks (e.g., 4G networks). This demo focuses on the key functions of MobiSNA which support social network-based video exploration, real-time video sharing, video blogging, video interest groups, and video story construction. A system architecture of MobiSNA is also proposed.

Collaborative research has been increasingly popular and important in academic circles. However, there is no open platform available for scholars or scientists to effectively discover potential collaborators. This paper discusses Collab-Seer, an open system to recommend potential research collaborators for scholars and scientists. CollabSeer discovers collaborators based on the structure of the coauthor network and a user's research interests. Currently, three different network structure analysis methods that use vertex similarity are supported in CollabSeer: Jaccard similarity, cosine similarity, and our relation strength similarity measure. Users can also request a recommendation by selecting a topic of interest. The topic of interest list is determined by CollabSeer's lexical analysis module, which analyzes the key phrases of previous publications. The CollabSeer system is highly modularized making it easy to add or replace the network analysis module or users' topic of interest analysis module. CollabSeer integrates the results of the two modules to recommend collaborators to users. Initial experimental results over the a subset of the CiteSeerX database shows that CollabSeer can efficiently discover prospective collaborators.

This paper presents MobiSNA -a mobile video social networking application that supports the exploration, sharing, and creation of video contents through social networks. The MobiSNA project provides the user with an easy to use experience of accessing video content from mobile devices (e.g., mobile phones, PDAs) over wireless broadband networks (e.g., 4G networks). This demo focuses on the key functions of MobiSNA which support social network-based video exploration, real-time video sharing, video blogging, video interest groups, and video story construction. A system architecture of MobiSNA is also proposed.

Efficient algorithms are extremely important and can be crucial for certain software projects. Even though many source code search engines have been proposed in the literature to help software developers find source code related to their needs, to our knowledge there has been no effort to develop systems that keep abreast of the latest algorithmic developments. In this paper, we describe our initial effort towards developing such an algorithm search engine. The proposed system extracts and indexes algorithms discussed in academic literature and their associated metadata. Users can search the index through a free text query interface. The source code of proposed system, being developed as a part of a larger open source toolkit, SeerSuite, will be released in due course. We also provide directions for further research and improvements of the current system.

Algorithms are an essential part of computational science. An algorithm search engine, which extracts pseudo-codes and their metadata from documents, and makes it searchable, has recently been developed as part of the Citeseer X suite . However, this algorithm search engine only retrieves and ranks relevant algorithms solely on textual similarity. Here, we propose a method for using the algorithm cocitation network to infer the similarity between algorithms. We apply a graph clustering algorithm on the network for algorithm recommendation and make suggestions on how to improve the current Citeseer X algorithm search engine.

A significant number of scholarly articles in computer science and other disciplines contain algorithms that provide concise descriptions for solving a wide variety of computational problems. For example, Dijkstra's algorithm describes how to find the shortest paths between two nodes in a graph. Automatic identification and extraction of these algorithms from scholarly digital documents would enable automatic algorithm indexing, searching, analysis and discovery. An algorithm search engine, which identifies pseudocodes in scholarly documents and makes them searchable, has been implemented as a part of the CiteSeer X suite. Here, we illustrate the limitations of start-ofthe-art rule based pseudocode detection approach, and present a novel set of machine learning based techniques that extend previous methods.

Social media is emerging as a powerful source of communication, information dissemination and mining. Being colloquial and ubiquitous in nature makes it easier for users to express their opinions and preferences in a seamless, dynamic manner. Epidemic surveillance systems that utilize social media to detect the emergence of diseases have been proposed in the literature. These systems mostly employ traditional document classification techniques that represent a document with a bag of N-grams. However, such techniques are not optimal for social media where sparsity and noise are norms. The authors address the limitations posed by the traditional N-gram based methods and propose to use features that represent different semantic aspects of the data in combination with ensemble machine learning techniques to identify health-related messages in a heterogenous pool of social media data. Furthermore, the results reveal significant improvement in identifying health related social media content which can be critical in the emergence of a novel, unknown disease epidemic.

An innovative consumer (a.k.a. a lead user) is a consumer of a product that faces needs unknown to the public. Innovative consumers play important roles in the product development process as their ideas tend to be innovatively unique and can be potentially useful for development of next generation, innovative products that better satisfy the market needs. Oftentimes, consumers portray their usage experience and opinions about products and product features through social networks such as Twitter and Facebook, making social media a viable, rich in information, and large-scale source for mining product related information. The authors of this work propose a data mining methodology to automatically identify innovative consumers from a heterogeneous pool of social media users. Specifically, a mathematical model is proposed to identify latent features (product features unknown to the public) from social media data. These latent features then serve as the key to discover innovative users from the ever increasing pool of social media users.

The rapid growth of diverse data types and greater volumes available to environmental sciences prompts the scientists to seek knowledge in data from multiple places, times, and scales. To facilitate such need, ONEMercury has recently been implemented as part of the DataONE project to serve as a portal for accessing environmental and observational data across the globe. ONEMercury harvests metadata from the data hosted by multiple repositories and makes it searchable.

The rapid growth of diverse data types and greater volumes available to environmental sciences prompts the scientists to seek knowledge in data from multiple places, times, and scales. To facilitate such need, ONEMercury has recently been implemented as part of the DataONE project to serve as a portal for accessing environmental and observational data across the globe. ONEMercury harvests metadata from the data hosted by multiple repositories and makes it searchable. However, harvested metadata records sometimes are poorly annotated or lacking meaningful keywords, and hence would unlikely be retrieved during the search process. In this paper, we develop an algorithm for automatic metadata annotation. We transform the problem into a tag recommendation problem, and propose a score propagation style algorithm for tag recommendation. Our experiments on four data sets of environmental science metadata records not only show great promises on the performance of our method, but also shed light on the different natures of the data sets.

The increase of the complexity and advancement in ecological and environmental sciences encourages scientists across the world to collect data from multiple places, times, and thematic scales to verify their hypotheses. Accumulated over time, such data not only increases in amount, but also in the diversity of the data sources spread around the world. This poses a huge challenge for scientists who have to manually search for information. To alleviate such problems, ONEMercury has recently been implemented as part of the DataONE project to serve as a portal for accessing environmental and observational data across the globe. ONE-Mercury harvests metadata from the data hosted by multiple repositories and makes it searchable. However, harvested metadata records sometimes are poorly annotated or lacking meaningful keywords, which could affect effective retrieval. Here, we develop algorithms for automatic annotation of metadata. We transform the problem into a tag recommendation problem with a controlled tag library, and propose two variants of an algorithm for recommending tags. Our experiments on four datasets of environmental science metadata records not only show great promises on the performance of our method, but also shed light on the different natures of the datasets.

Academic papers contain multiple figures representing important findings and experimental results; we present a search engine specifically focused on figures in academic documents. This search engine allows users to search on figures in approximately 150,000 chemistry journal articles though the method is easily extendable to other domains. Our system indexes figure caption and mentions extracted from the PDF in documents using a custom built extractor. Recall and precision performance of extracted figures is in the 80 to 90 % range. We give the frame work for the extraction algorithm, architecture and ranking function.

Algorithms are an essential part of computational science. An algorithm search engine, which extracts pseudo-codes and their metadata from documents, and makes it searchable, has recently been developed as part of the Citeseer X suite . However, this algorithm search engine only retrieves and ranks relevant algorithms solely on textual similarity. Here, we propose a method for using the algorithm cocitation network to infer the similarity between algorithms. We apply a graph clustering algorithm on the network for algorithm recommendation and make suggestions on how to improve the current Citeseer X algorithm search engine.