The Right (Provenance) Hammer for the Job: A Comparison of Data Provenance Instrumentation

Adriane Chapman; Abhirami Sasikant; Giulia Simonelli; Paolo Missier; Riccardo Torlone

doi:10.1007/978-3-030-67681-0_3

The Right (Provenance) Hammer for the Job: A Comparison of Data Provenance Instrumentation

Adriane Chapman, Abhirami Sasikant, Giulia Simonelli, Paolo Missier, Riccardo Torlone

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Chapter

Abstract

As data science techniques are being applied to solve societal problems, understanding what is happening within the “pipeline” is essential for establishing trust and reproducibility of the results. Provenance captures information about what happened during design and execution in order to support reasoning for trust and reproducibility. However, how and where the information is captured as provenance within the data science pipelines changes how it can be utilized. In this work, we describe three different mechanisms to capture provenance in data science pipelines: human-based, tool-based, and script-based. By using an implementation of all techniques in a standard data science toolkit, we analyze the difference in provenance generated by these methods and how its use changes.

Original language	English
Title of host publication	Provenance in Data Science
Subtitle of host publication	From Data Models to Context-Aware Knowledge Graphs
Editors	Leslie F. Sikos, Oshani W. Seneviratne, Deborah L. McGuinness
Publisher	Springer
Pages	25-45
Number of pages	20
Edition	1
ISBN (Electronic)	9783030676810
ISBN (Print)	9783030676803
DOIs	https://doi.org/10.1007/978-3-030-67681-0_3
Publication status	Published - 27 Apr 2021

Publication series

Name	Advanced Information and Knowledge Processing
Publisher	Springer
ISSN (Print)	1610-3947
ISSN (Electronic)	2197-8441

Access to Document

10.1007/978-3-030-67681-0_3Licence: None: All rights reserved

Cite this

Chapman, A., Sasikant, A., Simonelli, G., Missier, P., & Torlone, R. (2021). The Right (Provenance) Hammer for the Job: A Comparison of Data Provenance Instrumentation. In L. F. Sikos, O. W. Seneviratne, & D. L. McGuinness (Eds.), Provenance in Data Science: From Data Models to Context-Aware Knowledge Graphs (1 ed., pp. 25-45). (Advanced Information and Knowledge Processing). Springer. https://doi.org/10.1007/978-3-030-67681-0_3

Chapman, Adriane ; Sasikant, Abhirami ; Simonelli, Giulia et al. / The Right (Provenance) Hammer for the Job : A Comparison of Data Provenance Instrumentation. Provenance in Data Science: From Data Models to Context-Aware Knowledge Graphs. editor / Leslie F. Sikos ; Oshani W. Seneviratne ; Deborah L. McGuinness. 1. ed. Springer, 2021. pp. 25-45 (Advanced Information and Knowledge Processing).

@inbook{6be2fd7f5a5f45dca1e682c16d6d395f,

title = "The Right (Provenance) Hammer for the Job: A Comparison of Data Provenance Instrumentation",

abstract = "As data science techniques are being applied to solve societal problems, understanding what is happening within the “pipeline” is essential for establishing trust and reproducibility of the results. Provenance captures information about what happened during design and execution in order to support reasoning for trust and reproducibility. However, how and where the information is captured as provenance within the data science pipelines changes how it can be utilized. In this work, we describe three different mechanisms to capture provenance in data science pipelines: human-based, tool-based, and script-based. By using an implementation of all techniques in a standard data science toolkit, we analyze the difference in provenance generated by these methods and how its use changes.",

author = "Adriane Chapman and Abhirami Sasikant and Giulia Simonelli and Paolo Missier and Riccardo Torlone",

year = "2021",

month = apr,

day = "27",

doi = "10.1007/978-3-030-67681-0_3",

language = "English",

isbn = "9783030676803",

series = "Advanced Information and Knowledge Processing",

publisher = "Springer",

pages = "25--45",

editor = "Sikos, {Leslie F.} and Seneviratne, {Oshani W.} and McGuinness, {Deborah L.}",

booktitle = "Provenance in Data Science",

edition = "1",

}

Chapman, A, Sasikant, A, Simonelli, G, Missier, P & Torlone, R 2021, The Right (Provenance) Hammer for the Job: A Comparison of Data Provenance Instrumentation. in LF Sikos, OW Seneviratne & DL McGuinness (eds), Provenance in Data Science: From Data Models to Context-Aware Knowledge Graphs. 1 edn, Advanced Information and Knowledge Processing, Springer, pp. 25-45. https://doi.org/10.1007/978-3-030-67681-0_3

The Right (Provenance) Hammer for the Job: A Comparison of Data Provenance Instrumentation. / Chapman, Adriane; Sasikant, Abhirami; Simonelli, Giulia et al.
Provenance in Data Science: From Data Models to Context-Aware Knowledge Graphs. ed. / Leslie F. Sikos; Oshani W. Seneviratne; Deborah L. McGuinness. 1. ed. Springer, 2021. p. 25-45 (Advanced Information and Knowledge Processing).

Research output: Chapter in Book/Report/Conference proceeding › Chapter

TY - CHAP

T1 - The Right (Provenance) Hammer for the Job

T2 - A Comparison of Data Provenance Instrumentation

AU - Chapman, Adriane

AU - Sasikant, Abhirami

AU - Simonelli, Giulia

AU - Missier, Paolo

AU - Torlone, Riccardo

PY - 2021/4/27

Y1 - 2021/4/27

N2 - As data science techniques are being applied to solve societal problems, understanding what is happening within the “pipeline” is essential for establishing trust and reproducibility of the results. Provenance captures information about what happened during design and execution in order to support reasoning for trust and reproducibility. However, how and where the information is captured as provenance within the data science pipelines changes how it can be utilized. In this work, we describe three different mechanisms to capture provenance in data science pipelines: human-based, tool-based, and script-based. By using an implementation of all techniques in a standard data science toolkit, we analyze the difference in provenance generated by these methods and how its use changes.

AB - As data science techniques are being applied to solve societal problems, understanding what is happening within the “pipeline” is essential for establishing trust and reproducibility of the results. Provenance captures information about what happened during design and execution in order to support reasoning for trust and reproducibility. However, how and where the information is captured as provenance within the data science pipelines changes how it can be utilized. In this work, we describe three different mechanisms to capture provenance in data science pipelines: human-based, tool-based, and script-based. By using an implementation of all techniques in a standard data science toolkit, we analyze the difference in provenance generated by these methods and how its use changes.

UR - https://doi.org/10.1007/978-3-030-67681-0_3

U2 - 10.1007/978-3-030-67681-0_3

DO - 10.1007/978-3-030-67681-0_3

M3 - Chapter

SN - 9783030676803

T3 - Advanced Information and Knowledge Processing

SP - 25

EP - 45

BT - Provenance in Data Science

A2 - Sikos, Leslie F.

A2 - Seneviratne, Oshani W.

A2 - McGuinness, Deborah L.

PB - Springer

ER -

Chapman A, Sasikant A, Simonelli G, Missier P, Torlone R. The Right (Provenance) Hammer for the Job: A Comparison of Data Provenance Instrumentation. In Sikos LF, Seneviratne OW, McGuinness DL, editors, Provenance in Data Science: From Data Models to Context-Aware Knowledge Graphs. 1 ed. Springer. 2021. p. 25-45. (Advanced Information and Knowledge Processing). Epub 2020 Dec 28. doi: 10.1007/978-3-030-67681-0_3

The Right (Provenance) Hammer for the Job: A Comparison of Data Provenance Instrumentation

Abstract

Publication series

Access to Document

Fingerprint

Cite this