Performing in silico experiments on the Grid: a users perspective

Journal of digital information management, 2004

Abstract: Collaboration by use of common artifacts is at the core of e-science. A recent enabling technology is the Grid, which ties together heterogeneous computation and data resources through the use of middleware, linking the techniques and resources to infer higher-level knowledge. This article presents results from research and development of Grid technology for semantic interoperability between scientific artifacts on the web. The research employs the'industry-as-laboratory'approach to software development. This means ...

Concurrency and Computation: Practice and Experience, 2011

Grid computing facilitates the aggregation and coordination of resources that are distributed across multiple administrative domains for large-scale and complex e-Science experiments. Writing, deploying, and testing grid applications over highly heterogeneous and distributed resources are complex and challenging. The process requires grid-enabled programming tools that can handle the complexity and scale of the infrastructure. However, while a large amount of research has been undertaken into grid middleware, little work has been directed specifically at the area of grid application development tools. This paper presents the design and implementation of ISENGARD, an infrastructure for supporting e-Science and grid application development. ISENGARD provides services, tools, and APIs that simplify grid software development. Copyright © 2010 John Wiley & Sons, Ltd.

2003

my Grid is an e-Science research project developing open source high-level middleware to support in silico experiments in biology. In silico experiments use databases and computational analysis rather than laboratory investigations to test hypothesis. This paper provides an overview of services the my Grid project is developing, and the architecture in which they fit. Registries provide information about available data and computational services, while remote legacy bioinformatics applications are wrapped using a consistent distributed analysis framework Soaplab. As in conventional science, experimental method is as important as final results. my Grid formalises these methods as workflow or query specifications and provides service based middleware components to enact them. e-Science for the individual often has a narrow focus and so personalisation forms a key theme in my Grid service design. Information repositories, service registries and change notification systems are all being developed to provide personalised views of resources. my Grid components make extensive use of metadata to support this need for personalisation and the project is pioneering the use of semantic web technology, to manage annotation, ontologies and semantic discovery. The ultimate goal of my Grid is to supply this collection of services as a toolkit to build end applications. To demonstrate this concept the project is building its own application (the my Grid workBench).

Bioinformatics, 2003

The my Grid project aims to exploit Grid technology, with an emphasis on the Information Grid, and provide middleware layers that make it appropriate for the needs of bioinformatics. my Grid is building high level services for data and application integration such as resource discovery, workflow enactment and distributed query processing. Additional services are provided to support the scientific method and best practice found at the bench but often neglected at the workstation, notably provenance management, change notification and personalisation. Results: We give an overview of these services and their metadata. In particular, semantically rich metadata expressed using ontologies necessary to discover, select and compose services into dynamic workflows. Availability: Software is available on request from the authors and information from http://www.mygrid.org.uk.

IEEE Intelligent Systems, 2004

Services form the key component of the workflows used in the myGrid project to represent bioinformatics experiments. Abstraction allows greater portability, but choosing a service to execute depends on a large amount of additional metadata.

Studies in health technology and informatics, 2007

This paper describes the evolution of the main services of the ProGenGrid (Proteomics & Genomics Grid) system, a distributed and ubiquitous grid environment ("virtual laboratory"), based on Workflow and supporting the design, execution and monitoring of "in silico" experiments in bioinformatics.ProGenGrid is a Grid-based Problem Solving Environment that allows the composition of data sources and bioinformatics programs wrapped as Web Services (WS). The use of WS provides ease of use and fosters re-use. The resulting workflow of WS is then scheduled on the Grid, leveraging Grid-middleware services. In particular, ProGenGrid offers a modular bag of services and currently is focused on the biological simulation of two important bioinformatics problems: prediction of the secondary structure of proteins, and sequence alignment of proteins. Both services are based on an enhanced data access service.

Computer Communications and Networks, 2011

E-Science has been greatly enhanced from the developing capability and usability of cyberinfrastructure. This chapter explains how scientific workflow systems can facilitate e-Science discovery in Grid environments by providing features including resource consolidation, parallelism, provenance tracking, fault tolerance and workflow reuse. We first overview the core services to support e-Science discovery. To demonstrate how these services can be seamlessly assembled, a scientific workflow system, called Kepler, is integrated into the University of California Grid. This architecture is being applied to a computational enzyme design process, which is a formidable and collaborative problem in computational chemistry that challenges our knowledge of protein chemistry. Our implementation and experiments validate how the Kepler workflow system can make the scientific computation process automated, pipelined, efficient, extensible, stable, and easy-to-use.

2004

Abstract The recent advance in information technologies has bought about the borderlessness in every field of both science and business. The borderlessness has increasingly made activities in interdisciplinary field more important. This current situation produces a strong demand that people want to establish a virtual group, organization and society for their business and scientific purposes irrespective of the actual structure formed by organizations.

Concurrency and Computation Practice Experience, 2005

This editorial describes four papers that summarize key Grid technology capabilities to support distributed e-Science applications. These papers discuss the Condor system supporting computing communities, the OGSA-DAI service interfaces for databases, the WS-I+ Grid Service profile and finally WS-GAF (the Web Service Grid Application Framework). We discuss the confluence of mainstream IT industry development and the very latest science and computer science research and urge the communities to reach consensus rapidly. Agreement on a set of core Web Service standards is essential to allow developers to build Grids and distributed business and science applications with some assurance that their investment will not be obviated by the changing Web Service frameworks.

IEEE Software, 2009

Scientific research is increasingly digital. Some activities, like data analysis, search and simulation, can be accelerated by enabling scientists to write workflows and scripts that automate routine activities. These capture pieces of scientific method that can be shared with others. The Taverna Workbench, a widely deployed scientific workflow management system, together with the myExperiment social web site for the sharing of scientific experiments, have been devised according to six principles of designing software for adoption by scientists and six principles of user engagement. Science is becoming increasingly digital, and the scientist's tools are not just the experimental apparatus of the laboratory but also the software apparatus which they use to conduct their researchto analyse data, to search databases, to run simulations and to record their scientific process. New experimental methods-from DNA microarrays to sensor networks in the environment-are generating volumes of data that, without software assistance, just wouldn't get processed. Watch a researcher at work and we see lots of activity at the computer, using applications, services and data that might be local to the laboratory, the enterprise, or out on the Web. These new research tools and methods are evident across a very broad spectrum of disciplines. Some, like bioinformaticians working with protein sequences, might conduct some of their research entirely in silico. Meanwhile, chemists in the laboratory are looking up procedures, designing their experiments, conducting and recording their work, and running simulations, searches and analyses. Climate change researchers are capturing the latest data from radar images and sensor networks, and running and testing their models. Musicologists are extracting features from recorded music as part of musical analysis. Archaeologists are exploring 3D visualisations of their digs. Some of the processing is highly repetitive-repeating processes with new data, varying parameters, and making small changes to the process. It clearly helps speed things up and makes them more repeatable if some of this scientific activity at the computer can be automated. This in turn lets scientists concentrate on the science rather than the repetitive handling of data through multiple tools and applications. So, increasingly, we see these scientific activities at the computer being 'programmed'. Some disciplines have created scripting solutions, perhaps adopting their favourite scripting languages and creating libraries to handle their discipline-specific data, while others adopt workflow systems which automate the processing of data through a series of processing stages. Taverna is one of many scientific workflow management systems (see Workflow Management Systems on page 2) and supports what we might describe as 'application-level workflows', as opposed to some other systems which focus on scheduling tasks across computing resources [1]. Emerging from the UK's e-Science programme, Taverna is used extensively by scientists across a range of Life Science problems: gene and protein annotation; proteomics, phylogeny and phenotypical studies; analysis of microarray data and medical images; high throughput screening of chemical compounds and clinical statistical analysis. Increasingly it is being adopted in other disciplines too.