The HUBzero platform for scientific collaboration enables tool developers to build tools that are easily shared with both researchers and educators. This enables users to login and start their analysis without worrying about setup and configuration of the tools. Once, the analysis is done researchers can then analyse the results using various inbuilt capabilities for plotting and visualization. To facilitate handling of more complex workloads, we have integrated Pegasus Workflow Management System with “submit”, the main tool used by tool developers in HUBzero to submit analysis to local and remote compute resources. Pegasus WMS provides a means for representing the application workflow in an abstract form which is independent of the resources available to run it and the location of data and executables. It compiles these abstract workflows into an executable form that can be executed on local or remote distributed resources. Pegasus also captures all the provenance of the workflow lifecycle from the planning stage, through execution, to the final output data. This enables users to easily debug and monitor their computations that occur on remote resources. The advanced data management capabilities of Pegasus allow the tool developers to execute the tightly coupled parts of their workloads on a HPC cluster, while farming out remaining tasks to a distributed HTCondor based computing infrastructure.
The talk will give an introduction to scientific workflows with Pegasus and focus on integration of Pegasus WMS with “submit”, and how it enables tool developers using the Rappture toolkit or “submit” directly, use scientific workflows. Currently the tools benefitting from this integration include BLASTer an online tool to run BLAST on the DiaGrid Hub, CryoEM a tool to reconstruct 3-D structure of macromolecular assemblies, OpenSees workflows through NeesHub and parameter sweep workflows to study ballistic transport in Field Effect Transistors using OSG resources through nanoHUB.
Cite this work
Researchers should cite this work as follows:
HUBzero - HUB Liaison