A general approach to real-time workflow monitoring

Vahi, Karan, Harvey, Ian, Samak, Taghrid, Gunter, Daniel, Evans, Kieran, Rogers, David, Taylor, Ian, Goode, Monte, Silva, Fabio, Al-Shkarchi, Eddie, Mehta, Gaurang, Jones, Andrew Clifford and Deelman, Ewa 2012. A general approach to real-time workflow monitoring. Presented at: 2012 SC Companion: High Performance Computing, Networking, Storage and Analysis (SCC), Salt Lake City, UT, USA, 10-16 November 2012. High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion:. IEEE, pp. 108-118. 10.1109/SC.Companion.2012.26

Full text not available from this repository.

Abstract

Scientific workflow systems support different workflow representations, operational modes and configurations. However, independent of the system used, end users need to track the status of their workflows in real time, be notified of execution anomalies and failures automatically, perform troubleshooting and automate the analysis of the workflow to help categorize and qualify the results. In this paper, we describe how the Stampede monitoring infrastructure, which was previously integrated in the Pegasus Workflow Management System, was employed in Triana in order to add generic real time monitoring and troubleshooting capabilities across both systems. Stampede is an infrastructure that attempts to address interoperable monitoring needs by providing a three-layer model: a common data model to describe workflow and job executions; high-performance tools to load workflow logs conforming to the data model into a data store, and a querying interface for extracting information from the data store in a standard fashion. The resulting integration demonstrates the generic nature of the Stampede monitoring infrastructure that has the potential to provide a common platform for monitoring across scientific workflow engines.

Item Type:	Conference or Workshop Item (Paper)
Status:	Published
Schools:	Computer Science & Informatics
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Publisher:	IEEE
ISBN:	9781467362184
Funders:	EC
Last Modified:	23 Sep 2019 21:21
URI:	https://orca.cardiff.ac.uk/id/eprint/92387

Citation Data

Cited 12 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item