Grid applications for high energy physics experiments

Proceedings of the sixth …, 2008

The International Lattice Data Grid (ILDG) is an international collaboration that creates standards to enable sharing of data produced by Lattice Quantum ChromoDynamics (QCD) simulations, which are very computationally expensive. In this paper we summarize ...

In this work we present technical details and recent developments for a computing cluster working in a GRID environment, configured for high energy physics experiments at the National Institute of Physics and Nuclear Engineering. Main ideas and concepts behind the GRID technology are described. Two Virtual Organizations (VO) LHCb and ILC using GRID resources for Monte Carlo production, data analysis and data storage are presented together with the recently initiated development of their specific tools.

Future Generation Computer Systems, 2002

2004

The National Fusion Collaboratory focuses on enabling fusion scientists to explore Grid capabilities in support of experimental science. Fusion experiments are structured as a series of plasma pulses initiated roughly every 20 minutes. In the between-pulse intervals scientists perform data analysis and discuss results to reach decisions affecting changes to the next plasma pulse. This interaction can be made more efficient by performing more analysis and engaging more expertise from a geographically distributed team of scientists and resources. In this paper, we will describe the virtual control room experiment which unites collaborative, visualization and Grid technologies to provide such environment and shows how their combined effect can advance experimental science. We will also report on FusionGrid services whose use during the fusion experimental cycle became possible for the first time thanks to this technology. In terms of technology, we will describe the Access Grid, experimental data presentation tools and agreement-based resource management and workflow systems enabling time-bounded end-to-end application execution. The first virtual control room experiment represented a mock-up of a remote interaction with the DIII-D control room and was presented at SC03 and later reviewed at an international ITER Grid Workshop.

Journal of the Korean Physical Society, 2011

Remote Instrumentation Services on the e-Infrastructure, 2010

In this chapter we introduce the concept of the Grid as an Application Service Provider. We discuss the implementation and deployment of a fully operational system in a Synchrotron Radiation Facility. The Grid is traditionally used for HTC computations and not often as a provider for Software on Demand; especially for non-Grid specific GUI based applications. As a computing paradigm, the Grid is successfully utilised by the experimental scientific community. Due to the data-parallelism of the computational problems in the general field of physical sciences and the very high data volumes in terms of storage, the Grid has been closely related to High Throughput and High Performance Computing (HTC/HPC). A substantial number of software applications in the form of scientific computing utilities that do not require HPC are used in Synchrotron labs and other similar establishments. We envisioned a common web portal where the lab user can access a defined set of pre-configured scientific computing applications 1 that rely on Grid resources. The pilot application has been in the SRF Elettra. 2 The model and the intermediate subsystems have been developed and suitably configured. In this chapter we propose (1) a "Software as a Service" model (SaaS) as an alternative deployment for these applications, (2) we introduce an infrastructure where our SaaS model is fully integrated to the Grid-utilising remote computation and storage resources, (3) we outline our technology layout that enables interactivity, transparent access to storage, Web Grid portals, virtual collaboration environments and remote access to SRF instrumentation.

cdsweb.cern.ch

Most of the simulated events for the DZero experiment at Fermilab have been historically produced by the "remote" collaborating institutions. One of the principal challenges reported concerns the maintenance of the local software infrastructure, which is generally different from site to site. As the understanding of the distributed computing community over distributively owned and shared resources progresses, the adoption of grid technologies to address the production of montecarlo events for high energy physics experiments becomes increasingly interesting. The SAM-Grid is a software system developed at Fermilab, which integrates standard grid technologies for job and information management with SAM, the data handling system of the DZero and CDF experiments. During the past few months, this grid system has been tailored for the montecarlo production of DZero. Since the initial phase of deployment, this experience has exposed an interesting series of requirements to the SAM-Grid services, the standard middleware, the resources and their management and to the analysis framework of the experiment. As of today, the inefficiency due to the grid infrastructure has been reduced to as little as 1%. In this paper, we present our statistics and the "lessons learned" in running large high energy physics applications on a grid infrastructure.

2015

Journal of Physics: Conference Series, 2017

The FabrIc for Frontier Experiments (FIFE) project is a major initiative within the Fermilab Scientific Computing Division charged with leading the computing model for Fermilab experiments. Work within the FIFE project creates close collaboration between experimenters and computing professionals to serve high-energy physics experiments of differing size, scope, and physics area. The FIFE project has worked to develop common tools for job submission, certificate management, software and reference data distribution through CVMFS repositories, robust data transfer, job monitoring, and databases for project tracking. Since the projects inception the experiments under the FIFE umbrella have significantly matured, and present an increasingly complex list of requirements to service providers. To meet these requirements, the FIFE project has been involved in transitioning the Fermilab General Purpose Grid cluster to support a partitionable slot model, expanding the resources available to experiments via the Open Science Grid, assisting with commissioning dedicated high-throughput computing resources for individual experiments, supporting the efforts of the HEP Cloud projects to provision a variety of back end resources, including public clouds and high performance computers, and developing rapid onboarding procedures for new experiments and collaborations. The larger demands also require enhanced job monitoring tools, which the project has developed using such tools as ElasticSearch and Grafana. in helping experiments manage their large-scale production workflows. This group in turn requires a structured service to facilitate smooth management of experiment requests, which FIFE provides in the form of the Production Operations Management Service (POMS). POMS is designed to track and manage requests from the FIFE experiments to run particular workflows, and support troubleshooting and triage in case of problems. Recently a new certificate management infrastructure called Distributed Computing Access with Federated Identities (DCAFI) has been put in place that has eliminated our dependence on a Fermilab-specific third-party Certificate Authority service and better accommodates FIFE collaborators without a Fermilab Kerberos account. DCAFI integrates the existing InCommon federated identity infrastructure, CILogon Basic CA, and a MyProxy service using a new general purpose open source tool. We will discuss the general FIFE onboarding strategy, progress in expanding FIFE experiments presence on the Open Science Grid, new tools for job monitoring, the POMS service, and the DCAFI project.