Figure 1: Pegasus WMS Architecture. Figure 3: Hello World DAX (Directed Acyclic graph in XML). Figure 2: Python DAX generator code for Hello World DAX. Figure 4: Translation of a simple Hello World DAX to an Executable Workflow Figure 5: Data flow for a Pegasus Workflows planned with non-shared filesystem mode, running on the Open Science Grid. The main concept in tl he new data management approach is to be able to place the data staging element anywhere in the execution environment. I ample, the data storage e n the previous Open Science Grid ex- ement was an SRM/GridFTP/iRODS server placed near the compute resources. Let’s consider the same workflow, but this time running on an HPC cluster with a shared filesystem (Figure oh. Figure 6: Data management on a shared filesystem resource (only the remote execution environment is shown). 4.3. Task Clustering Figure 9: Example of level-based clustering. Figure 10: Example of label-based clustering. Figure 8: Translation from Abstract to Executable Workflow. Figure 11: Hierarchical Workflows. Expansion ends when a DAX with only compute jobs is encountered. Figure 12: The Galactic Plane workflow, showing the top-level workflow with 1,001 data find tasks and sub-workflows. Figure 14: Workflow Gantt Chart and Job Distribution Pie Charts displayed in pegasus-dashboard. Figure 13: Workflow Statistics displayed in pegasus-dashboard. Pegasus also includes a lightweight web dashboard t hat en- ables easy monitoring and online exploration of workflows based on an embedded board can display all the workflows (running and com: for a particular user. It allows users to browse their flows. For each workflow, users can see a summary workflow run including runtime statistics (snapshot dis in Figure|13 Figure[14p illustratin time and pie charts d web server written in Python. The dash- pleted) work- of the played such as the number of jobs completed, failed, and workflow walltime, and can browse data collected for each job by the system. The dashboard can also generate different types of charts such as workflow gantt charts (snapshot displayed in sis of count and runtime. g the the progress of the workflow through epicting the distribution of jobs on the ba- Table 1: Comparison of CyberShake Studies
