Grid Scheduling

The goal of the Grid Application Development Software (GrADS) Project is to provide programming tools and an execution environment to ease program development for the Grid. This paper presents recent extensions to the GrADS software framework: a new approach to scheduling workflow computations, applied to a 3-D image reconstruction application; a simple stop/migrate/restart approach to rescheduling Grid applications, applied to a QR factorization benchmark; and a process-swapping approach to rescheduling, applied to an N-body simulation. Experiments validating these methods were carried out on both the GrADS MacroGrid (a small but functional Grid) and the MicroGrid (a controlled emulation of the Grid).

Even though middleware support for grid computing has been the subject of extensive research, scheduling policies for the grid context have not been much studied. In addition to processor utilization, it is important to consider the response times of jobs in evaluating the performance of grid scheduling strategies. In this paper we propose distributed scheduling algorithms that use multiple simultaneous requests at different sites. Trace-based simulations show that the use of multiple simultaneous requests provides significant performance benefits. We also show how this scheme can be adapted to provide priority to local jobs, without much loss of performance.

In data-intensive applications data transfer is a primary cause of job execution delay. Data access time depends on bandwidth. The major bottleneck to supporting fast data access in Grids is the high latencies of Wide Area Networks and Internet. Effective scheduling can reduce the amount of data transferred across the internet by dispatching a job to where the needed data are present. Another solution is to use a data replication mechanism. Objective of dynamic replica strategies is reducing file access time which leads to reducing job runtime. In this paper we develop a job scheduling policy and a dynamic data replication strategy, called HRS (Hierarchical Replication Strategy), to improve the data access efficiencies. We study our approach and evaluate it through simulation. The results show that our algorithm has improved 12% over the current strategies.

Large distributed grid systems pose new challenges in job scheduling due to complex workload characteristics and system characteristics. Due to the numerous parameters that must be considered and the complex interactions that can occur between different resource allocation policies, analytical modeling of system behavior appears impractical. Thus, we have developed the GangSim simulator to support studies of scheduling strategies in grid environments, with a particular focus on investigations of the ...

Desktop grids have already been used to perform some of the largest computations in the world and have the potential to grow by several more orders of magnitude. However current approaches to utilizing desktop resources require either centralized servers or extensive knowledge of the underlying system, limiting their scalability.

An automatic Vehicle-to-Grid (V2G) technology can contribute to the utility grid. V2G technology has drawn great interest in the recent years. Success of the sophisticated automatic V2G research depends on efficient scheduling of gridable vehicles in constrained parking lots. Parking lots have constraints of space and current limits for V2G. However, V2G can reduce dependencies on small expensive units in the existing power systems as energy storage that can decrease running costs. It can efficiently manage load fluctuation, peak load; however, it increases spinning reserves and reliability. As number of gridable vehicles in V2G is much higher than small units of existing systems, unit commitment (UC) with V2G is more complex than basic UC for only thermal units. Particle swarm optimization (PSO) is proposed to solve the V2G, as PSO has been demonstrated to reliably and accurately solve complex constrained optimization problems easily and quickly without any dimension limitation and physical computer memory limit. In the proposed model, binary PSO optimizes the on/off states of power generating units easily. Vehicles are presented by signed integer number instead of 0/1 to reduce the dimension of the problem. Typical discrete version of PSO has less balance between local and global searching abilities to optimize the number of charging/discharging gridable vehicles in the constrained system. In the same model, balanced PSO is proposed to optimize the V2G part in the constrained parking lots. Finally, results show a considerable amount of profit for using proper scheduling of gridable vehicles in constrained parking lots.

In this paper, we propose a new algorithm for fair scheduling, and we compare it to other scheduling schemes such as the Earliest Deadline First (EDF) and the First Come First Served (FCFS) schemes. Our algorithm uses a max-min fair sharing approach for providing fair access to users. When there is no shortage of resources, the algorithm assigns to each task enough computational power for it to finish within its deadline. When there is congestion, the main idea is to fairly reduce the CPU rates assigned to the tasks so that the share of resources that each user gets is proportional to the user's weight. The weight of a user may be defined as the user's contribution to the infrastructure or the price he is willing to pay for services or any other socioeconomic consideration. In our algorithms, all tasks whose requirements are lower than their fair share CPU rate are served at their demanded CPU rates. However, the CPU rates of tasks whose requirements are larger than their fair share CPU rate are reduced to fit the total available computational capacity in a fair manner. Three different versions of fair scheduling are adopted in this paper: the Simple Fair Task Order (SFTO), which schedules the tasks according to their respective fair completion times, the Adjusted Fair Task Order (AFTO), which refines the SFTO policy by ordering the tasks using the adjusted fair completion time, and the Max-Min Fair Share (MMFS) scheduling policy, which simultaneously addresses the problem of finding a fair task order and assigning a processor to each task based on a max-min fair sharing policy. Experimental results and comparisons with traditional scheduling schemes such as the EDF and the FCFS are presented using three different error criteria. Validation of the simulations using real experiments of tasks generated from 3D imagerendering processes is also provided. The three proposed scheduling schemes can be integrated into existing Grid computing architectures.

Over the past decade, the grid has emerged as an attractive platform to tackle various large-scale problems, especially in science and engineering. One primary issue associated with the efficient and effective utilization of heterogeneous resources in a grid is scheduling. Grid scheduling involves a number of challenging issues, mainly due to the dynamic nature of the grid. There are only a handful of scheduling schemes for grid environments that realistically deal with this dynamic nature that have been proposed in the literature. In this paper, two novel scheduling algorithms, called the shared-input-data-based listing (SIL) algorithm and the multiple queues with duplication (MQD) algorithm for bag-of-tasks (BoT) applications in grid environments are proposed. The SIL algorithm targets scheduling data-intensive BoT (DBoT) applications, whereas the MQD algorithm deals with scheduling computationally intensive BoT (CBoT) applications. Their common and primary forte is that they make...

Control) has been developed by the CERN LHCb physics experiment to facilitate large scale simulation and user analysis tasks spread across both grid and non-grid computing resources. It consists of a small set of distributed stateless Core Services, which are centrally managed, and Agents which are managed by each computing site. DIRAC utilizes concepts from existing distributed computing models to provide a lightweight, robust, and flexible system. This paper will discuss the architecture, performance, and implementation of the DIRAC system which has recently been used for an intensive physics simulation involving more than forty sites, 90 TB of data, and in excess of one thousand 1 GHz processor-years.

Because of its sheer size, Computational Grids (CGs) require advanced methodologies and strategies to efficiently schedule users tasks and applications to resources. Scheduling becomes even more challenging when energyefficiency, classical makespan criterion and user perceived Quality of Service (QoS) are treated as first-class objectives in CG resource allocation methodologies. In this paper we approach the independent batch scheduling in CG as a biobjective minimization problem with makespan and energy consumption as the scheduling criteria. We use the Dynamic Voltage Scaling (DVS) methodology for reducing the cumulative power energy utilized by the system resources. We develop two Genetic Algorithms (GAs) with elitist and struggle replacement mechanisms as energy-aware schedulers. The proposed algorithms were experimentally evaluated for four CG size scenarios in static and dynamic modes. The simulation results showed that our proposed GA-based schedulers fairly reduce the energy usage to a level that is sufficient to maintain the desired quality level(-s).

This work concentrates on the design of a system intended for study of advanced scheduling techniques for planning various types of jobs in a Grid environment. The solution is able to deal with common problems of the job scheduling in Grids like heterogeneity of jobs and resources, and dynamic runtime changes such as arrivals of new jobs. The Alea is based on the GridSim simulation toolkit which we extended to provide a simulation environment that supports simulation of varying Grid scheduling problems. To demonstrate the features of the GridSim environment, we implemented an experimental centralised Grid scheduler which uses advanced scheduling techniques for schedule generation. By now local search based algorithms and some dispatching rules were tested. The scheduler is capable to handle both static and dynamic situation. In the static case, all jobs are known in advance while the dynamic situation means that jobs appear in the system during simulation. In this case generated schedule is changing through time as some jobs are already finished while the new ones are arriving. Comparison of FCFS, local search and dispatching rules is presented for both cases and we demonstrate that the new local search based algorithm provides the best schedule while keeping the running time acceptable.

In the applied basic model, the Grid system consists of a large number of identical processors that are divided into several machines. Jobs are independent, they have a fixed degree of parallelism, and they are submitted over time. Further, a job can only be executed on the processors belonging to the same machine. It is our goal to minimize the total makespan. We show that the performance of Garey and Graham's list scheduling algorithm is significantly worse in Grids than in multiprocessors. Then we present a Grid scheduling algorithm that guarantees a competitive factor of 5. This algorithm can be implemented using a "job stealing" approach and may be well suited to serve as a starting point for Grid scheduling algorithms in real systems.

Optical networking technologies are expected to play an important role in creating an efficient infrastructure for supporting advanced grid applications. Since both the scheduling methods in grid computing and optical networks are limited to be directly used to achieve optical grid scheduling, we propose a new, to the best of our knowledge, joint scheduling model by extending the classic list scheduling algorithm to achieve communication contention aware task scheduling for the optical grid applications. An effective adaptive routing scheme is also proposed to improve the performance of the extended list scheduling. The impacts of different routing schemes on the extended list scheduling are comparatively investigated by simulations.

Grid computing infrastructures embody a cost-effective computing paradigm that virtualises heterogeneous system resources to meet the dynamic needs of critical business and scientific applications. These applications range from batch processes and long-running tasks to real-time and even transactional applications. Grid computing environments are inherently dynamic and unpredictable environments sharing services amongst many different users. Grid schedulers aim to make the most efficient use of Grid resources (high utilisation) while providing the best possible performance to the Grid applications (reducing makespan) and satisfying the associated performance and Quality of Service (QoS) constraints. Additionally, in commercial Grid settings where economic considerations are an increasingly important part of Grid scheduling, it is necessary to minimise the cost of application execution on the behalf of the Grid users while ensuring that the applications meet their QoS constraints. Furthermore, efficient resource allocation may allow a resource broker to maximise their profit by minimising the quantity of resource procurement. Scheduling in such a large-scale, dynamic and distributed environment is a complex undertaking. In this paper, we propose an approach to Grid scheduling which abstracts over the details of individual applications, focusing instead on the global cost optimisation problem while taking into account the entire workload, dynamically adjusting to the varying service demands. Our model places particular emphasis on the stochastic and unpredictable nature of the Grid, leading to a more accurate reflection of the state of the Grid and hence more efficient and accurate scheduling decisions.

Nowadays, grid computing is increasingly showing a service-oriented tendency and as a result, providing quality of service (QoS) has raised as a relevant issue in such highly dynamic and non-dedicated systems. In this sense, the role of scheduling strategies is critical and new proposals able to deal with the inherent uncertainty of the grid state are needed in a way that QoS can be offered. Fuzzy rule-based schedulers are emerging scheduling schemas in grid computing based on the efficient management of grid resources imprecise state and expert knowledge application to achieve an efficient workload distribution. Given the diverse and usually conflicting nature of the scheduling optimization objectives in grids considering both users and administrators requirements, these strategies can benefit from multi-objective strategies in their knowledge acquisition process greatly. This work suggests the QoS provision in the grid scheduling level with fuzzy rule-based schedulers through multi-objective knowledge acquisition considering multiple optimization criteria. With this aim, a novel learning strategy for the evolution of fuzzy rules based on swarm intelligence, Knowledge Acquisition with a Swarm Intelligence Approach (KASIA) is adapted to the multiobjective evolution of an expert grid meta-scheduler founded on Pareto general optimization theory and its performance with respect to a well-known genetic strategy is analyzed. In addition, the fuzzy scheduler with multi-objective learning results are compared to those of classical scheduling strategies in grid computing.

We address non-preemptive non-clairvoyant online scheduling of parallel jobs on a Grid. We consider a Grid scheduling model with two stages. At the first stage, jobs are allocated to a suitable Grid site, while at the second stage, local scheduling is independently applied to each site. We analyze allocation strategies depending on the type and amount of information they require. We conduct a comprehensive performance evaluation study using simulation and demonstrate that our strategies perform well with respect to several metrics that reflect both user- and system-centric goals. Unfortunately, user run time estimates and information on local schedules does not help to significantly improve the outcome of the allocation strategies. When examining the overall Grid performance based on real data, we determined that an appropriate distribution of job processor requirements over the Grid has a higher performance than an allocation of jobs based on user run time estimates and information on local schedules. In general, our experiments showed that rather simple schedulers with minimal information requirements can provide a good performance.

Grid computing has become one of the most important research topics that appeared in the field of computing in the last years. Simultaneously, we have noticed the growing popularity of new Web-based technologies which allow us to create application-oriented Grid middleware services providing capabilities required for dynamic resource and job management, monitoring, security, etc. Consequently, end users are able to get easier access to geographically distributed resources. In this paper we present the results of our experiments with the Grid(Lab) Resource Management System (GRMS), which acts on behalf of end users and controls their computations efficiently using distributed heterogeneous resources. We show how resource matching techniques used within GRMS can be improved by the use of a job migration based rescheduling policy. The main aim of this policy is to shorten job pending times and reduce machine overloads. The influence of this method on application performance and resource utilization is studied in detail and compared with two other simple policies.

Grid schedulers require individual activity performance predictions to map workflow activities on different Grid sites. The effectiveness of the scheduling systems is hampered by inaccurate predictions due to the inability of existing predictors to effectively model the dynamic and heterogeneous nature of Grid resources, or the wide range of problem sizes and runtime arguments. To address this deficiency, we propose a hybrid Bayesian-neural network approach to dynamically model and predict the execution time of activities in real workflow applications. Bayesian network is used for a high-level representation of activities performance probability distribution against different factors affecting the performance. The important attributes are dynamically selected by the Bayesian network and fed into a radial basis function neural network to make further predictions. Our approach is generic to any type of scientific applications, and flexible to import expert knowledge to further improve accuracies. Experimental results for activities from three realworld workflow applications are presented to show effectiveness of our approach.

In this paper, we propose an efficient non-linear task workload prediction mechanism incorporated with a fair scheduling algorithm for task allocation and resource management in Grid computing. Workload prediction is accomplished in a Grid middleware approach using a non-linear model expressed as a series of finite known functional components using concepts of functional analysis. The coefficient of functional components are obtained using a training set of appropriate samples, the pairs of which are estimated based on a runtime estimation model relied on a least squares approximation scheme. The advantages of the proposed non-linear task workload prediction scheme is that (i) it is not constrained by analysis of source code (analytical methods), which is practically impossible to be implemented in complicated real-life applications or (ii) it does not exploit the variations of the workload statistics as the statistical approaches does. The predicted task workload is then exploited by a novel scheduling algorithm, enabling a fair Quality of Service oriented resource management so that some tasks are not favored against others. The algorithm is based on estimating the adjusted fair completion times of the tasks for task order selection and on an earliest completion time strategy for the grid resource assignment. Experimental results and comparisons with traditional scheduling approaches as implemented in the framework of European Union funded research projects GRIA and GRIDLAB grid infrastructures have revealed the outperformance of the proposed method.

Streaming applications such as video-based surveillance, habitat monitoring, and emergency response are good candidates for executing on high-performance computing (HPC) resources, due to their high computation and communication needs. Such an application can be represented as a coarse-grain dataflow graph, each node corresponding to a stage of the pipeline of transformations that may be applied to the data as it continuously streams through. Mapping such applications to HPC resources has to be sensitive to the computation and communication needs of each stage of the pipeline to ensure QoS criteria such as latency and throughput. Due to the dynamic nature of such applications, they are ideal candidates for using ambient HPC resources made available via the grid. Since grid has evolved out of traditional high-performance computing, the tools available, especially for scheduling, tend to be more appropriate for batch-oriented applications. We have developed a scheduler, called Streamline, that takes into account dynamic nature of the grid and runs periodically to adapt scheduling decisions using application requirements (per-stage computation and communication needs), application constraints (such as co-location of stages on the same node), and current resource availability. The scheduler is designed to be integrated with the existing grid framework using Globus Toolkit. The performance of Streamline is compared with an Optimal placement for small number of resources and approximation algorithms using Simulated Annealing for large resources and dataflow graphs. We have also compared Streamline with a baseline grid scheduler, E-Condor, built on top of Condor for streaming applications. For kernels of such streaming applications, we show that our heuristic performs close to Optimal, and can be nearly an order of magnitude better than the E-Condor under non-uniform load conditions. We considered two Simulated Annealing algorithms with different execution times, and show that neighbor-selection and annealing schedule have a relatively larger impact on the performance of Simulated Annealing for communication-intensive ker-nels than for computation intensive kernels. We have also conducted scalability studies and show that our scheduler is more effective than E-Condor, and performs close to Simulated Annealing algorithms, with smaller scheduling time, in allocating resources for a large streaming application.

Grids consist of both dedicated and non-dedicated clusters. For effective mapping of parallel applications on grid resources, a grid metascheduler has to evaluate different sets of resources in terms of predicted execution times for the applications when executed on the sets of resources. In this work, we have developed a comprehensive set of performance modeling strategies for predicting execution times of parallel applications on both dedicated and non-dedicated environments. Our strategies adapt to changing network and CPU loads on the grid resources. We have evaluated our strategies on 8, 16, 24 and 32-node clusters with random loads and load traces from a grid system. Our strategies give less than 30% average percentage prediction errors in all cases, which, to our knowledge, is the best reported for non-dedicated environments. We also found that grid scheduling using predictions of execution times from our performance modeling techniques will lead to perfect mapping of applications to resources in many cases.

Grid simulation tools provide frameworks for simulating application scheduling in various Grid infrastructures. However, while experimenting with many existing tools, we have encountered two main shortcomings: (i) lack of tools for generating workloads, events and resources; (ii) it is difficult and time consuming to model different Grid levels, i.e. resource brokers, and local level scheduling systems. In this paper we present the Grid Scheduling Simulator (GSSIM), a framework that addresses these shortcomings and provides an easy-to-use Grid scheduling framework for enabling simulations of a wide range of scheduling algorithms in multi-level, heterogeneous Grid infrastructures. In order to foster more collaboration in the community at large, GSSIM is complemented with a portal (http://www.gssim.org) that provides a repository of Grid scheduling algorithms, synthetic workloads and benchmarks for use with GSSIM.

This paper proposes a novel schedule-based approach for scheduling a continuous stream of batch jobs on the machines of a computational Grid. Our new solutions represented by dispatching rule Earliest Gap-Earliest Deadline First (EG-EDF) and Tabu search are based on the idea of filling gaps in the existing schedule. EG-EDF rule is able to build the schedule for all jobs incrementally by applying technique which fills earliest existing gaps in the schedule with newly arriving jobs. If no gap for a coming job is available EG-EDF rule uses Earliest Deadline First (EDF) strategy for including new job into the existing schedule. Such schedule is then optimized using the Tabu search algorithm moving jobs into earliest gaps again. Scheduling choices are taken to meet the Quality of Service (QoS) requested by the submitted jobs, and to optimize the usage of hardware resources. We compared the proposed solution with some of the most common queue-based scheduling algorithms like FCFS, EASY backfilling, and Flexible backfilling. Experiments shows that EG-EDF rule is able to compute good assignments, often with shorter algorithm runtime w.r.t. the other queue-based algorithms. Further Tabu search optimization results in higher QoS and machine usage while keeping the algorithm runtime reasonable.

When deploying Grid infrastructure, the problem of dimensioning arises: how many servers to provide, where to place them, and which network to install for interconnecting server sites and users generating Grid jobs? In contrast to classical optical network design problems, it is typical of optical Grids that the destination of traffic (jobs)

The user-level brokers in grids consider individual application QoS requirements and minimize their cost without considering demands from other users. This results in contention for resources and sub-optimal schedules. Meta-scheduling in grids aims to address this scheduling problem, which is NP hard due to its combinatorial nature. Thus, many heuristic-based solutions using Genetic Algorithm (GA) have been proposed, apart from traditional algorithms such as Greedy and FCFS.

We propose a biologically inspired and fullydecentralized approach to the organization of computation that is based on the autonomous scheduling of strongly mobile agents on a peer-to-peer network. Our approach achieves the following design objectives: near-zero knowledge of network topology, zero knowledge of system status, autonomous scheduling, distributed computation, lack of specialized nodes. Every node is equally responsible for scheduling and computation, both of which are performed with practically no information about the system.

The Service Level Agreement (SLA) based grid superscheduling approach promotes coordinated resource sharing. Superscheduling is facilitated between administratively and topologically distributed grid sites via grid schedulers such as Resource brokers and workflow engines. In this work, we present a market-based SLA coordination mechanism, based on a well known contract net protocol.

In multicluster systems, and more generally in grids, jobs may require co-allocation, that is, the simultaneous or coordinated access of single applications to resources of possibly multiple types in multiple locations managed by different resource managers. Co-allocation presents new challenges to resource management in grids, such as locating sufficient resources in geographically distributed sites, allocating and managing resources in multiple, possibly heterogeneous sites for single applications, and coordinating the execution of single jobs at multiple sites. Moreover, as single jobs now may have to rely on multiple resource managers, co-allocation introduces reliability problems. In this paper, we present the design and implementation of a co-allocating grid scheduler named KOALA that meets these co-allocation challenges. In addition, we report on the results of an analysis of the performance in our multicluster testbed of the co-allocation policies built into KOALA. We also include the results of a performance and reliability test of KOALA while our testbed was unstable.

Scheduling is an important factor for the efficient execution of computational workflows on Grid environments. A large number of static scheduling heuristics has been presented in the literature. These algorithms allocate tasks before job execution starts and assume a precise knowledge of timing information, which may be difficult to obtain in general. To overcome this limitation of static strategies, dynamic scheduling strategies may be needed for a changing environment such as the Grid. While they incur runtime overheads, they may better adapt to timing changes during job execution. In this work, we analyse five well-known heuristics (min-min, max-min, sufferage, HEFT and random) when used as static and dynamic scheduling strategies in a grid environment in which computing resources exhibit congruent performance differences. The analysis shows that nonlist based heuristics are more sensitive than list-based heuristics to inaccuracies in timing information. Static list-based heuristics perform well in the presence of low or moderate inaccuracies. Dynamic versions of these heuristics may be needed only in environments where high inaccuracies are observed. Our analysis also shows that list-based heuristics significantly outperform non-list based heuristics in all cases and, therefore, constitute the most suitable strategies by which to schedule workflows either statically or dynamically.

The Organic Grid is a biologically inspired and fully-decentralized approach to the organization of computation that is based on the autonomous scheduling of strongly mobile agents on a peer-to-peer network. Through the careful design of agent behavior, the emerging organization of the computation can be customized for different classes of applications.

Grids organize resource sharing, a fundamental requirement of large scientific collaborations. Seamless integration of Grids into everyday use requires responsiveness, which can be provided by elastic Clouds, in the Infrastructure as a Service (IaaS) paradigm. This paper proposes a model-free resource provisioning strategy supporting both requirements. Provisioning is modeled as a continuous action-state space, multi-objective reinforcement learning (RL) problem, under realistic hypotheses; simple utility functions capture the high level goals of users, administrators, and shareholders. The model-free approach falls under the general program of autonomic computing, where the incremental learning of the value function associated with the RL model provides the so-called feedback loop. The RL model includes an approximation of the value function through an Echo State Network. Experimental validation on a real data-set from the EGEE Grid shows that introducing a moderate level of elasticity is critical to ensure a high level of user satisfaction.

Scheduling application in grid is a complex task that often fails due to nonavailability of resources and the required execution environment in the resources. The CARE Resource Broker (CRB) proposed in this paper addresses such scheduling scenarios using the concepts of virtualization. The virtualization technology offers effective resource management mechanisms such as isolated, secure job scheduling, and utilization of computing resources to the possible extent. However, lack of protocols and services to support virtualization technology in high level grid architecture does not allow management of virtual machines and virtual clusters in grid environment. CRB proposes and implements necessary protocols and services to support creation and management of virtual resources in the physical hosts. Unlike the conventional grid schedulers, CRB addresses several cases of failure of application scheduling such as nonavailability of enough computing nodes in a cluster, and, nonavailability of software execution environment in any of the grid resources. It deploys the required number of virtual machines in potential computing resources to meet the application requirements, and creates virtual clusters dynamically; configure with the required software execution environment for facilitating application execution. Thus, CRB improves the overall throughput by scheduling more applications than conventional grid schedulers and increases the utilization of underutilized computing resources in grid.

In this paper, we address non-preemptive online scheduling of parallel jobs on a Grid. Our Grid consists of a large number of identical processors that are divided into several machines. We consider a Grid scheduling model with two stages. At the first stage, jobs are allocated to a suitable machine, while at the second stage, local scheduling is independently applied to each machine. We discuss strategies based on various combinations of allocation strategies and local scheduling algorithms. Finally, we propose and analyze a scheme named adaptive admissible allocation. This includes a competitive analysis for different parameters and constraints. We show that the algorithm is beneficial under certain conditions and allows for an efficient implementation in real systems. Furthermore, a dynamic and adaptive approach is presented which can cope with different workloads and Grid properties.

Grid scheduling is essential to Quality of Service provisioning as well as to efficient management of grid resources. Grid scheduling usually considers the state of the grid resources as well application demands. However, such demands are generally unknown for highly demanding applications, since these often generate data which will be transferred during their execution. Without appropriate assessment of these demands, scheduling decisions can lead to poor performance. Thus, it is of paramount importance to consider uncertainties in the formulation of a grid scheduling problem. This paper introduces the IPDT-FUZZY scheduler, a scheduler which considers the demands of grid applications with such uncertainties. The scheduler uses fuzzy optimization, and both computational and communication demands are expressed as fuzzy numbers. Its performance was evaluated, and it was shown to be attractive when communication requirements are uncertain. Its efficacy is compared, via simulation, to that of a deterministic counterpart scheduler and the results reinforce its adequacy for dealing with the lack of accuracy in the estimation of communication demands.

Computational grids are highly complex distributed systems (involving multiple organizations with different goals and policies) which aim at providing computing services without the users need to know the location and features of the required resources. A key issue in managing and scheduling grid resources is the coordination among multiple administrative domains. In this paper, we present a preliminary study which aims at developing auction mechanisms for decentralized scheduling which exhibit minimal communication overhead and an efficient usage of resources.

In this paper we revisit the supernode-shape selection problem, that has been widely discussed in bibliography. In general, the selection of the supernode transformation greatly affects the parallel execution time of the transformed algorithm. Since the minimization of the overall parallel execution time via an appropriate supernode transformation is very difficult to accomplish, researchers have focused on scheduling-aware supernode transformations that maximize parallelism during the execution. In this paper we argue that the communication volume of the transformed algorithm is an important criterion, and its minimization should be given high priority. For this reason we define the metric of the per process communication volume and propose a method to miminize this metric by selecting a communication-aware supernode shape. Our approach is equivalent to defining a proper Cartesian process grid with MPI Cart Create, which means that it can be incorporated in applications in a straightforward manner. Our experimental results illustrate that by selecting the tile shape with the proposed method, the total parallel execution time is significantly reduced due to the minimization of the communication volume, despite the fact that a few more parallel execution steps are required.

Scheduling and resource allocation in large scale distributed environments, such as Computational Grids (CGs), arise new requirements and challenges not considered in traditional distributed computing environments. Among these new requirements, task abortion and security become needful criteria for Grid schedulers. The former arises due to the dynamics of the Grid systems, in which resources are expected to enter and leave the system in an unpredictable way. The latter requirement appears crucial in Grid systems mainly due to a multi-domain nature of CGs. The main aim of this paper is to develop a scheduling model that enables the aggregation of task abortion and security requirements as additional, together with makespan and flowtime, scheduling criteria into a cumulative objective function. We demonstrate the high effectiveness of genetic-based schedulers in finding near-optimal solutions for multi-objective scheduling problem, where all criteria (objectives) are simultaneously optimized. The proposed metaheuristics are experimentally evaluated in static and dynamic Grid scenarios by using a Grid simulator. The obtained results show the fast reduction of the values of basic scheduler performance metrics, especially in the dynamic case, that confirms the usefulness of the proposed approach in real-life scenarios.

We evaluate job scheduling algorithms that integrate both tasks of Grid scheduling: job allocation to Grid sites and local scheduling at the sites. We propose and analyze an adaptive job allocation scheme named admissible allocation. The main idea of this scheme is to set job allocation constraints, and dynamically adapt them to cope with different workloads and Grid properties. We present 3-approximation and 5-competitive algorithms named MLB a + PS and MCT a + PS for the case that all jobs fit to the smallest machine, while we derive an approximation factor of 9 and a competitive factor of 11 for the general case. To show practical applicability of our methods, we perform a comprehensive study of the practical performance of the proposed strategies and their derivatives using simulation. To this end, we use real workload traces and corresponding Grid configurations. We analyze nine scheduling strategies that require a different amount of information on three Grid scenarios. We demonstrate that our strategies perform well across ten metrics that reflect both user-and system-specific goals. is a coordinator of the Parallel Computing Laboratory. He is PI of several national and international research projects. He is active in grid research with a focus on resource optimization. He served as a program committee member of several professional conferences and a general co-chair for International Conferences on Parallel Computing Systems. His main interests include scheduling, multi-objective optimization, resource management in GRID and Cloud computing.

A Grid Resource Broker for a Grid domain, or also known as metascheduler, is a middleware component used for matching works to available Grid resources from one or more IT organizations. A Grid meta-scheduler usually has its own interfaces for the functionalities it provides and its own job scheduling objectives. This situation causes two main problems: the user uniform access to the Grid is lost, and the scheduling decisions are taken separately while they should be done in coordination. These problems have been observed in different efforts such as the HPC-Europa project but they are still open problems. In this paper we discuss the requirements to achieve a more uniform access to the Grids through a new approach to global brokering with new scheduling techniques. As the results of these discussions on brokering requirements, we propose a meta-brokering design, so called meta-metascheduler design, and discuss how it can be realized as a centralized model for the HPC-Europa project, and as a distributed model for the LA Grid project.

Implementation of a commercial application to a Grid infrastructure introduces new challenges in managing the quality-of-service (QoS) requirements; most stem from the fact that negotiation on QoS between the user and the service provider should strictly be satisfied. An interesting commercial application with a wide impact on a variety of fields, which can benefit from the computational Grid technologies, is three-dimensional (3D) rendering. In order to implement, however, 3D rendering to a grid infrastructure, we should develop appropriate scheduling and resource allocation mechanisms so that the negotiated (QoS) requirements are met. Efficient scheduling schemes require modeling and prediction of rendering workload. In this paper workload prediction is addressed based on a combined fuzzy classification and neural network model. Initially, appropriate descriptors are extracted to represent the synthetic world. The descriptors are obtained by parsing RIB formatted files, which provides a general structure for describing computer-generated images. Fuzzy classification is used for organizing rendering descriptor so that a reliable representation is accomplished which increases the prediction accuracy. The neural network performs workload prediction by modeling the nonlinear input-output relationship between rendering descriptors and the respective computational complexity. To increase prediction accuracy, a constructive algorithm is adopted in this paper to train the neural network so that network weights and size are simultaneously estimated. Then, a Grid scheduler scheme is proposed to estimate the queuing order in which the tasks should be executed and the most appropriate processor assignment so that the demanded QoS are satisfied as much as possible. A fair scheduling policy is considered as the most appropriate. Experimental results on a real Grid infrastructure are presented to illustrate the efficiency of the proposed workload prediction-scheduling algorithm compared to other approaches presented in the literature.

Complex applications are describing using workflows. Execution of these workflows in Grid environments require optimized assignment of tasks on available resources according with different constrains. This paper presents a decentralized scheduling algorithm based on genetic algorithms for the problem of DAG scheduling. The genetic algorithm presents a powerful method for optimization and could consider multiple criteria in optimization process. Also, we describe in this paper the integration platform for the proposed algorithm in Grid systems. We make a comparative evaluation with other existing DAG scheduling solution: Cluster ready Children First, Earliest Time First, Highest Level First with Estimated Times, Improved Critical Path with Descendant Prediction) and Hybrid Remapper. We carry out our experiments using a simulation tool with various scheduling scenarios and with heterogeneous input tasks and computation resources. We present several experimental results that offer a support for near-optimal algorithm selection.

Grids consist of the aggregation of numerous dispersed computational, storage and network resources, able to satisfy even the most demanding computing jobs. Due to the data-intensive nature of Grid jobs, there is an increasing interest in Grids using optical transport networks as this technology allows for the timely delivery of large amounts of data. Such Grids are commonly referred to as Lambda Grids.

Grid computing infrastructures are inherently dynamic and unpredictable environments shared by many users. Grid schedulers aim to make efficient use of grid resources while providing the best possible performance to the grid applications and satisfying the associated performance and policy constraints. Additionally, in commercial grid settings, where the grid resource brokering becomes an increasingly important part of grid scheduling, it is necessary to minimise the cost of application execution on the behalf of the grid users, while ensuring that the applications meet their QoS constraints. Efficient resource allocation could in turn also allow the resource broker to maximise its profit by minimising the number of resources procured. Scheduling in such a large-scale, dynamic and distributed environment is a complex undertaking. In this paper, we propose an approach to grid scheduling which abstracts over the details of individual applications, focusing instead on the global cost optimisation problem and the scheduling of the entire grid workload. Our model places particular emphasis on the stochastic and unpredictable nature of the grid, leading to a more accurate reflection of the state of the grid and hence more efficient and accurate scheduling decisions

The workload management task of the DataGrid project was mandated to define and implement a suitable architecture for distributed scheduling and resource management in a Grid environment. The result was the design and implementation of a Grid Workload Management System, a super-scheduler with the distinguishing property of being able to take data access requirements into account when scheduling jobs to the available Grid resources. Many novel issues in various fields were faced such as resource management, resource reservation and co-allocation, Grid accounting. In this paper, the architecture and the functionality provided by the DataGrid Workload Management System are presented.

The United States Environmental Protection Agency forecasts the 2011 national IT electric energy expendi ture will grow toward $7.4 billion [1]. In parallel to economic IT energy concerns, the general public and environmental advocacy groups are demanding proactive steps toward sustain able green processes. Our contribution to the solution of this problem is Environmentally Opportunistic Computing (EOC).

Grid resource management is not just about scheduling jobs on the fastest machines, but rather about scheduling all compute objects and all data objects on machines whose capabilities match the requirements, while preserving site autonomy, recognizing usage policies and respecting conditions for use. In this chapter, we present the Grid resource management of Legion, an object-based Grid infrastructure system. We argue that Grid resource management requires not a one-size-fits-all scheduler but an architectural framework that can accommodate different schedulers for different classes of problems.

The computing-intensive data mining for inherently Internet-wide distributed data, referred to as Distributed Data Mining (DDM), calls for the support of a powerful Grid with an effective scheduling framework. DDM often shares the computing paradigm of local processing and global synthesizing. It involves every phase of Data Mining (DM) processes, which makes the workflow of DDM very complex and can be modelled only by a Directed Acyclic Graph (DAG) with multiple data entries. Motivated by the need for a practical solution of the Grid scheduling problem for the DDM workflow, this paper proposes a novel two-phase scheduling framework, including External Scheduling and Internal Scheduling, on a twolevel Grid architecture (InterGrid, IntraGrid). Currently a DM IntraGrid, named DMGCE (Data Mining Grid Computing Environment), has been developed with a dynamic scheduling framework for competitive DAGs in a heterogeneous computing environment. This system is implemented in an established Multi-Agent System (MAS) environment, in which the reuse of existing DM algorithms is achieved by encapsulating them into agents. Practical classification problems from oil well logging analysis are used to measure the system performance. The detailed experiment procedure and result analysis are also discussed in this paper.

The use of a dynamic reconfigurable optical network is an important requirement for the new advanced resource-intensive, highly distributed Grid applications that begin to emerge on the e-Science field. In this paper, we propose an ACO-based algorithm that is capable of the co-scheduling of both computational and optical network resources. We assess the performance of the proposed algorithm by comparison with traditional publish-and-subscribe grid systems that make use of topological routing of the lightpaths. The proposed algorithm can be used as a viable alternative for grid scheduling in Lambda Grids.