Automating Data-Throttling Analysis for Data-Intensive Workflows

Rodríguez, Ricardo J., Tolosana-Calasanz, Rafael and Rana, Omer Farooq ORCID: https://orcid.org/0000-0003-3597-2646 2012. Automating Data-Throttling Analysis for Data-Intensive Workflows. Presented at: 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 2012, Ottawa, Canada, 13-16 May 2012. Published in: Balaji, P., Buyya, Rajkumar, Majumdar, S. and Pandey, Sunil eds. Proceedings of the 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. Los Alamitos, CA: IEEE, pp. 310-317. 10.1109/CCGrid.2012.27

Full text not available from this repository.

Abstract

Data movement between tasks in scientific workflows has received limited attention compared to task execution. Often the staging of data between tasks is either assumed or the time delay in data transfer is considered to be negligible (compared to task execution). Where data consists of files, such file transfers are accomplished as fast as the network links allow, and once transferred, the files are buffered/stored at their destination. Where a task requires multiple files to execute (from different tasks), it must, however, remain idle until all files are available. Hence, network bandwidth and buffer/storage within a workflow are often not used effectively. We propose an automated workflow structural analysis method for Directed Acyclic Graphs (DAGs) which utilises information from previous workflow executions. The method obtains data-throttling values for the data transfer to enable network bandwidth and buffer/storage capacity to be managed more efficiently. We convert a DAG representation into a Petri net model and analyse the resulting graph using an iterative method to compute data-throttling values. Our approach is demonstrated using the Montage workflow.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics Engineering
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Uncontrolled Keywords:	Performance analysis , Petri nets , Scientific computing , System performance
Publisher:	IEEE
ISBN:	9781467313957
Last Modified:	21 Oct 2022 09:13
URI:	https://orca.cardiff.ac.uk/id/eprint/35614

Citation Data

Cited 3 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item