Ian Foster

Followers

163

Following

Co-authors

Public Views

Interests

Uploads

Papers by Ian Foster

A Distributed Resource Management Architecture for Interconnecting Web-of-Things using uBox

by Michael Beigl and Ian Foster

Although there are many smart devices and networked embedded object applications using World Wide... more Although there are many smart devices and networked embedded object applications using World Wide Web technologies , it is still a big step to go towards a true Web of Things. It is e.g. difficult to build ubiquitous WoT applications that work in and accross multiple environments. Approaches which aggregate WoT ressources by centralizing all the resource information, have problems: total dependency on external infrasture, lack of private WoT management, inflexible communication patterns and limited dynamic ressource discovery and mapping. To solve these problems, we propose uBox, a local WoT platform which can be a stand-alone server to make your WoT environment, with interfaces to connect the other local WoT platforms. This way, which we call uBoXing, we can create World Wide WoT platform with a distributed architecture. This paper describes the concept of a distributed resource management architecture, and how we implement the concept into software. Also, we will discuss the platform with the example application in SmartTecO environment.

Download

Wagging the long tail of earth science: Why we need an earth science data web, and how to build it

By adopting, adapting, and applying semantic web and software-as-a-service technologies, we can m... more

Download

Computation Scheduling and Data Replication Algorithms for Data Grids

International Series in Operations Research & Management Science, 2004

Abstract Data Grids seek to harness geographically distributed resources for large-scale data-int... more Abstract Data Grids seek to harness geographically distributed resources for large-scale data-intensive problems such as those encountered in high energy physics, bioinformatics, and other disciplines. These problems typically involve numerous, loosely coupled jobs that both access and generate large data sets. Effective scheduling in such environments is challenging, because of a need to address a variety of metrics and constraints (eg, resource utilization, response time, global and local allocation policies) while dealing with multiple, ...

Scaling TeraGrid Access: A Roadmap for Attribute-based Authorization

We describe motivating use cases and present an implementation approach to enabling attributebase... more We describe motivating use cases and present an implementation approach to enabling attributebased authorization within TeraGrid, a U.S. cyberinfrastructure that provides a large and growing user community with access to computing and storage resources at (currently) nine sites across the U.S. Attribute-based authorization promises to streamline and broaden access control by allowing authorization decisions to be made based not only on user identity (the norm today), but also user attributes and roles. We review the attribute-based authorization capabilities available for the middleware used in TeraGrid's Globus Toolkit-based "Common TeraGrid Software and Services," and propose a path forward for incorporating attribute-based access control with TeraGrid services. We expect this document to be useful not only to TeraGrid architects but also to others interested in the design and deployment of attributed-based authorization.

Download

High-Performance Image Analysis and Visualization for Three-dimensional Light Microscopy

Three-dimensional microscopy imaging is important for understanding complex biological assemblies... more Three-dimensional microscopy imaging is important for understanding complex biological assemblies. Computers and digital image acquisition systems have made possible the three-dimensional reconstruction of images obtained from optical microscopes. Since processing such images requires tremendous CPU, storage, and I/O capacities, a high-performance computing facility is necessary in order to achieve real-time performance. However, it is not practical for many medical centers or biological research institutions to operate their own large computer system. One solution is to provide access to remote high-performance computers via high-speed networks. In this paper, we present such a prototype system that integrates microscopy image acquisition, parallel deblurring, 3D image visualization, and high-speed networks.

Download

Implementing a premium service based on the expedited forwarding per-hop behavior

Draft 1.3- Revised 7/19/2002 OGSA Security Roadmap Global Grid Forum Specification Roadmap towards a Secure OGSA

This document is a roadmap enumerating a set of proposed specifications to be defined in the Glob... more

Download

Security and Certification Issues in Grid Computing

Grid computing is concerned with the sharing and coordinated use of diverse resources in dynamic,... more Grid computing is concerned with the sharing and coordinated use of diverse resources in dynamic, distributed \virtual organizations." The dynamic nature of Grid environments introduces challenging security concerns that demand new technical approaches. In this brief overview, we review key Grid security issues and outline the technologies that are being developed to address those issues. We focus in particular on work being done within the context of the Open Grid Services Architecture, a new initiative aimed at recasting key Grid concepts within a serviceoriented framework. This work involves a tight i n tegration with Web services mechanisms and appears particularly relevant t o t h e c o n c e r n s of e-services.

Download

Predicting the performance of wide area data transfers

Proceedings 16th International Parallel and Distributed Processing Symposium, 2002

As Data Grids become more commonplace, large data sets are being replicated and distributed to mu... more As Data Grids become more commonplace, large data sets are being replicated and distributed to multiple sites, leading to the problem of determining which replica can be accessed most efficiently. The answer to this question can depend on many factors, including physical characteristics of the resources and the load behavior on the CPUs, networks, and storage devices that are part of the end-to-end path linking possible sources and sinks.

Download

Common runtime support for high-performance parallel languages

Proceedings of the 1993 ACM/IEEE conference on Supercomputing - Supercomputing '93, 1993

Download

The Data Grid

Personalized Biomedical Data Integration

Download

Improving parallel data transfer times using predicted variances in shared networks

CCGrid 2005. IEEE International Symposium on Cluster Computing and the Grid, 2005., 2005

The increasingly common practice of using multiple distributed storage systems as a distributed d... more The increasingly common practice of using multiple distributed storage systems as a distributed data store within which large datasets may be replicated has led to the problem of how to access replicated data efficiently. Multiple-source parallel transfers can improve data throughput time by transferring data from several replicas in parallel. However, we then face the problem of deciding how to distribute the data load among different storage resources. We propose a Tuned Conservative scheduling technique that uses predicted mean and variance network information to make data distribution decisions. This stochastic scheduling technique uses a tuning factor to adjust the amount of the data assigned to a link in accordance with the variability of the network performance. We apply our technique to the GridFTP implementation in the Globus Toolkit and demonstrate that the technique can produce data transfer times that are significantly faster and less variable than those of other techniques.

Download

Harnessing grid resources with data-centric task farms

As the size of scientific data sets and the resources required for analysis increase, data locali... more As the size of scientific data sets and the resources required for analysis increase, data locality becomes crucial to the efficient use of large scale distributed systems for scientific and data-intensive applications. In order to support interactive analysis of large quantities of data in many scientific disciplines, we propose a data diffusion approach, in which the resources required for data analysis are acquired dynamically, in response to demand. Acquired resources (compute and storage) can be "cached" for some time, thus allowing more rapid responses to subsequent requests. We define an abstract model for data-centric task farms as a common parallel pattern that drives the independent computational tasks, taking into consideration the data locality in order to optimize the performance of the analysis of large datasets. This approach can provide the benefits of dedicated hardware without the associated high costs. We will validate our abstract model through discrete-event simulations; we expect simulations to show the model is both efficient and scalable given a wide range of simulation parameters. To explore the practical realization of our abstract model, we have developed a Fast and Light-weight tasK executiON framework (Falkon). Falkon provides for dynamic acquisition and release of resources, data management capabilities, and the dispatch of analysis tasks via a data-aware scheduler. We have integrated Falkon into the Swift parallel programming system in order to leverage a large number of applications from various domains (astronomy, astro-physics, medicine, chemistry, economics, etc) which cover a variety of different datasets, workloads, and analysis codes. We believe our data-centric task farm model to generalize to many domains and applications, and could offer application end-to-end performance improvements, higher resource utilization, improved efficiency, and better application scalability.

Download

High Performance Computational Chemistry: (I) Scalable Fock Matrix Construction Algorithms

Abstract Several parallel algorithms for Fock matrix construction are described. The algorithms c... more Abstract Several parallel algorithms for Fock matrix construction are described. The algorithms calculate only the unique integrals, distribute the Fock and density matrices over the processors of a massively parallel computer, use blocking techniques to construct the distributed data structures, and use clustering techniques on each processor to maximize data reuse. Algorithms based on both square and row blocked distributions of the Fock and density matrices are described and evaluated. Variants of the algorithms are discussed ...

The Nimrod computational workbench: a case study in desktop metacomputing

The coordinated use of geographically distributed computers, or metacomputing, can in principle p... more The coordinated use of geographically distributed computers, or metacomputing, can in principle provide more accessible and cost- effective supercomputing than conventional high-performance systems. However, we lack evidence that metacomputing systems can be made easily usable, or that there exist large numbers of applications able to exploit metacomputing resources. In this paper, we present work that addresses both these concerns. The

Download

Materials Design Workbench: An Open-Access Database for Materials and Molecular Design

ABSTRACT

MPI as a Coordination Layer for Communicating HPF Tasks

Data-parallel languages such as High Performance Fortran (HPF) present a simple execution model i... more Data-parallel languages such as High Performance Fortran (HPF) present a simple execution model in which a single thread of control performs high-level operations on distributed arrays. These languages can greatly ease the development of parallel programs. Yet there are large classes of applications for which a mixture of task and data parallelism is most appropriate. Such applications can be structured as collections of data-parallel tasks that communicate by using explicit message passing. Because the Message Passing Interface (MPI) defines standardized, familiar mechanisms for this communication model, we propose that HPF tasks communicate by making calls to a coordination library that provides an HPF binding for MPI. The semantics of a communication interface for sequential languages can be ambiguous when the interface is invoked from a parallel language; we show how these ambiguities can be resolved by describing one possible HPF binding for MPI. We then present the design of a...

Download

A Data Transfer Library for Communicating Data-Parallel Tasks

Many computations can be structured as sets of communicating data-parallel tasks. Individual task... more Many computations can be structured as sets of communicating data-parallel tasks. Individual tasks may be coded in HPF, pC++, etc.; periodically, tasks exchange distributed arrays via channel operations, virtual file operations, message passing, etc. The implementation of these operations is complicated by the fact that the processes engaging in the communication may execute on different numbers of processors and may have different distributions for communicated data structures. In addition, they may be connected by different sorts of networks. In this paper, we describe a communicating data-parallel tasks (CDT) library that we are developing for constructing applications of this sort. We outline the techniques used to implement this library, and we describe a range of data transfer strategies and several algorithms based on these strategies. We also present performance results for several algorithms. The CDT library is being used as a compiler target for an HPF compiler augmented w...

Download

Taxes and the Convergence of Peer-to-Peer and Grid Computing

Peer-to-Peer Systems, 2002

Ian Foster

Uploads

Papers by Ian Foster

Log In