Why-Diff: Explaining differences amongst similar workflow runs by exploiting scientific metadata

Priyaa Thavasimani; Jacek Cala; Paolo Missier

doi:10.1109/BigData.2017.8258275

Why-Diff: Explaining differences amongst similar workflow runs by exploiting scientific metadata

Priyaa Thavasimani, Jacek Cala, Paolo Missier

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Majority of workflows executed nowadays need to process a massive amount of data. Re-execution of such dataintensive scientific workflows often results in different outputs. Scientific research progresses when discoveries are reproduced and verified. However, simply re-enacting a scientific computation, such as a workflow, does not guarantee the correctness of results because of unintentional changes that may have interfered with the re-enactment process. We investigate the hypothesis that the metadata of a workflow execution can be used to explain why the experimenter observes different results (cause analysis). Similarly, Scientific metadata can be used to determine the impact of intentional variations that the experimenter may have injected into a new version of the workflow. We explore these two complementary cases using a simple algorithm for traversing two metadata traces in lock-step mode, which we illustrate through two human genomics data analysis workflows.

Original language	English
Title of host publication	Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017
Editors	Jian-Yun Nie, Zoran Obradovic, Toyotaro Suzumura, Rumi Ghosh, Raghunath Nambiar, Chonggang Wang, Hui Zang, Ricardo Baeza-Yates, Ricardo Baeza-Yates, Xiaohua Hu, Jeremy Kepner, Alfredo Cuzzocrea, Jian Tang, Masashi Toyoda
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	3031-3041
Number of pages	11
ISBN (Electronic)	9781538627143
DOIs	https://doi.org/10.1109/BigData.2017.8258275
Publication status	Published - 1 Jul 2017
Event	5th IEEE International Conference on Big Data, Big Data 2017 - Boston, United States Duration: 11 Dec 2017 → 14 Dec 2017

Publication series

Name	Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017
Volume	2018-January

Conference

Conference	5th IEEE International Conference on Big Data, Big Data 2017
Country/Territory	United States
City	Boston
Period	11/12/17 → 14/12/17

Bibliographical note

Publisher Copyright:
© 2017 IEEE.

Keywords

Big Data
eScience Central
Metadata
Provenance
Reproducible Research
Why-Diff
Workflow

ASJC Scopus subject areas

Computer Networks and Communications
Hardware and Architecture
Information Systems
Information Systems and Management
Control and Optimization

Access to Document

10.1109/BigData.2017.8258275

Cite this

Thavasimani, P., Cala, J., & Missier, P. (2017). Why-Diff: Explaining differences amongst similar workflow runs by exploiting scientific metadata. In J.-Y. Nie, Z. Obradovic, T. Suzumura, R. Ghosh, R. Nambiar, C. Wang, H. Zang, R. Baeza-Yates, R. Baeza-Yates, X. Hu, J. Kepner, A. Cuzzocrea, J. Tang, & M. Toyoda (Eds.), Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017 (pp. 3031-3041). (Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017; Vol. 2018-January). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/BigData.2017.8258275

Thavasimani, Priyaa ; Cala, Jacek ; Missier, Paolo. / Why-Diff : Explaining differences amongst similar workflow runs by exploiting scientific metadata. Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017. editor / Jian-Yun Nie ; Zoran Obradovic ; Toyotaro Suzumura ; Rumi Ghosh ; Raghunath Nambiar ; Chonggang Wang ; Hui Zang ; Ricardo Baeza-Yates ; Ricardo Baeza-Yates ; Xiaohua Hu ; Jeremy Kepner ; Alfredo Cuzzocrea ; Jian Tang ; Masashi Toyoda. Institute of Electrical and Electronics Engineers (IEEE), 2017. pp. 3031-3041 (Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017).

@inproceedings{ff2a70eb4fcd4c19979e0eae43823587,

title = "Why-Diff: Explaining differences amongst similar workflow runs by exploiting scientific metadata",

abstract = "Majority of workflows executed nowadays need to process a massive amount of data. Re-execution of such dataintensive scientific workflows often results in different outputs. Scientific research progresses when discoveries are reproduced and verified. However, simply re-enacting a scientific computation, such as a workflow, does not guarantee the correctness of results because of unintentional changes that may have interfered with the re-enactment process. We investigate the hypothesis that the metadata of a workflow execution can be used to explain why the experimenter observes different results (cause analysis). Similarly, Scientific metadata can be used to determine the impact of intentional variations that the experimenter may have injected into a new version of the workflow. We explore these two complementary cases using a simple algorithm for traversing two metadata traces in lock-step mode, which we illustrate through two human genomics data analysis workflows.",

keywords = "Big Data, eScience Central, Metadata, Provenance, Reproducible Research, Why-Diff, Workflow",

author = "Priyaa Thavasimani and Jacek Cala and Paolo Missier",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE.; 5th IEEE International Conference on Big Data, Big Data 2017 ; Conference date: 11-12-2017 Through 14-12-2017",

year = "2017",

month = jul,

day = "1",

doi = "10.1109/BigData.2017.8258275",

language = "English",

series = "Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "3031--3041",

editor = "Jian-Yun Nie and Zoran Obradovic and Toyotaro Suzumura and Rumi Ghosh and Raghunath Nambiar and Chonggang Wang and Hui Zang and Ricardo Baeza-Yates and Ricardo Baeza-Yates and Xiaohua Hu and Jeremy Kepner and Alfredo Cuzzocrea and Jian Tang and Masashi Toyoda",

booktitle = "Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017",

}

Thavasimani, P, Cala, J & Missier, P 2017, Why-Diff: Explaining differences amongst similar workflow runs by exploiting scientific metadata. in J-Y Nie, Z Obradovic, T Suzumura, R Ghosh, R Nambiar, C Wang, H Zang, R Baeza-Yates, R Baeza-Yates, X Hu, J Kepner, A Cuzzocrea, J Tang & M Toyoda (eds), Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017. Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017, vol. 2018-January, Institute of Electrical and Electronics Engineers (IEEE), pp. 3031-3041, 5th IEEE International Conference on Big Data, Big Data 2017, Boston, United States, 11/12/17. https://doi.org/10.1109/BigData.2017.8258275

Why-Diff: Explaining differences amongst similar workflow runs by exploiting scientific metadata. / Thavasimani, Priyaa; Cala, Jacek; Missier, Paolo.
Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017. ed. / Jian-Yun Nie; Zoran Obradovic; Toyotaro Suzumura; Rumi Ghosh; Raghunath Nambiar; Chonggang Wang; Hui Zang; Ricardo Baeza-Yates; Ricardo Baeza-Yates; Xiaohua Hu; Jeremy Kepner; Alfredo Cuzzocrea; Jian Tang; Masashi Toyoda. Institute of Electrical and Electronics Engineers (IEEE), 2017. p. 3031-3041 (Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017; Vol. 2018-January).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Why-Diff

T2 - 5th IEEE International Conference on Big Data, Big Data 2017

AU - Thavasimani, Priyaa

AU - Cala, Jacek

AU - Missier, Paolo

PY - 2017/7/1

Y1 - 2017/7/1

N2 - Majority of workflows executed nowadays need to process a massive amount of data. Re-execution of such dataintensive scientific workflows often results in different outputs. Scientific research progresses when discoveries are reproduced and verified. However, simply re-enacting a scientific computation, such as a workflow, does not guarantee the correctness of results because of unintentional changes that may have interfered with the re-enactment process. We investigate the hypothesis that the metadata of a workflow execution can be used to explain why the experimenter observes different results (cause analysis). Similarly, Scientific metadata can be used to determine the impact of intentional variations that the experimenter may have injected into a new version of the workflow. We explore these two complementary cases using a simple algorithm for traversing two metadata traces in lock-step mode, which we illustrate through two human genomics data analysis workflows.

AB - Majority of workflows executed nowadays need to process a massive amount of data. Re-execution of such dataintensive scientific workflows often results in different outputs. Scientific research progresses when discoveries are reproduced and verified. However, simply re-enacting a scientific computation, such as a workflow, does not guarantee the correctness of results because of unintentional changes that may have interfered with the re-enactment process. We investigate the hypothesis that the metadata of a workflow execution can be used to explain why the experimenter observes different results (cause analysis). Similarly, Scientific metadata can be used to determine the impact of intentional variations that the experimenter may have injected into a new version of the workflow. We explore these two complementary cases using a simple algorithm for traversing two metadata traces in lock-step mode, which we illustrate through two human genomics data analysis workflows.

KW - Big Data

KW - eScience Central

KW - Metadata

KW - Provenance

KW - Reproducible Research

KW - Why-Diff

KW - Workflow

UR - http://www.scopus.com/inward/record.url?scp=85047784652&partnerID=8YFLogxK

U2 - 10.1109/BigData.2017.8258275

DO - 10.1109/BigData.2017.8258275

M3 - Conference contribution

AN - SCOPUS:85047784652

T3 - Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017

SP - 3031

EP - 3041

BT - Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017

A2 - Nie, Jian-Yun

A2 - Obradovic, Zoran

A2 - Suzumura, Toyotaro

A2 - Ghosh, Rumi

A2 - Nambiar, Raghunath

A2 - Wang, Chonggang

A2 - Zang, Hui

A2 - Baeza-Yates, Ricardo

A2 - Hu, Xiaohua

A2 - Kepner, Jeremy

A2 - Cuzzocrea, Alfredo

A2 - Tang, Jian

A2 - Toyoda, Masashi

PB - Institute of Electrical and Electronics Engineers (IEEE)

Y2 - 11 December 2017 through 14 December 2017

ER -

Thavasimani P, Cala J, Missier P. Why-Diff: Explaining differences amongst similar workflow runs by exploiting scientific metadata. In Nie JY, Obradovic Z, Suzumura T, Ghosh R, Nambiar R, Wang C, Zang H, Baeza-Yates R, Baeza-Yates R, Hu X, Kepner J, Cuzzocrea A, Tang J, Toyoda M, editors, Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017. Institute of Electrical and Electronics Engineers (IEEE). 2017. p. 3031-3041. (Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017). doi: 10.1109/BigData.2017.8258275

Why-Diff: Explaining differences amongst similar workflow runs by exploiting scientific metadata

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this