The future of sensitivity analysis: an essential discipline for systems modeling and policy support

Saman Razavi; Anthony Jakeman; Andrea Saltelli; Clémentine Prieur; Bertrand Iooss; Emanuele Borgonovo; Elmar Plischke; Samuele Lo Piano; Takuya Iwanaga; William Becker; Stefano Tarantola; Joseph H.A. Guillaume; John Jakeman; Hoshin Gupta; Nicola Melillo; Giovanni Rabitti; Vincent Chabridon; Qingyun Duan; Xifu Sun; Stefán Smith; Razi Sheikholeslami; Nasim Hosseini; Masoud Asadzadeh; Arnald Puy; Sergei Kucherenko; Holger R. Maier

doi:10.1016/j.envsoft.2020.104954

The future of sensitivity analysis: an essential discipline for systems modeling and policy support

Saman Razavi^*, Anthony Jakeman, Andrea Saltelli, Clémentine Prieur, Bertrand Iooss, Emanuele Borgonovo, Elmar Plischke, Samuele Lo Piano, Takuya Iwanaga, William Becker, Stefano Tarantola, Joseph H.A. Guillaume, John Jakeman, Hoshin Gupta, Nicola Melillo, Giovanni Rabitti, Vincent Chabridon, Qingyun Duan, Xifu Sun, Stefán SmithRazi Sheikholeslami, Nasim Hosseini, Masoud Asadzadeh, Arnald Puy, Sergei Kucherenko, Holger R. Maier

^*Corresponding author for this work

Geography, Earth and Environmental Sciences

Research output: Contribution to journal › Article › peer-review

89 Citations (Scopus)

114 Downloads (Pure)

Abstract

Sensitivity analysis (SA) is en route to becoming an integral part of mathematical modeling. The tremendous potential benefits of SA are, however, yet to be fully realized, both for advancing mechanistic and data-driven modeling of human and natural systems, and in support of decision making. In this perspective paper, a multidisciplinary group of researchers and practitioners revisit the current status of SA, and outline research challenges in regard to both theoretical frameworks and their applications to solve real-world problems. Six areas are discussed that warrant further attention, including (1) structuring and standardizing SA as a discipline, (2) realizing the untapped potential of SA for systems modeling, (3) addressing the computational burden of SA, (4) progressing SA in the context of machine learning, (5) clarifying the relationship and role of SA to uncertainty quantification, and (6) evolving the use of SA in support of decision making. An outlook for the future of SA is provided that underlines how SA must underpin a wide variety of activities to better serve science and society.

Original language	English
Article number	104954
Number of pages	22
Journal	Environmental Modelling and Software
Volume	137
Early online date	15 Dec 2020
DOIs	https://doi.org/10.1016/j.envsoft.2020.104954
Publication status	Published - Mar 2021

Bibliographical note

Funding Information:
This perspective paper is the outcome of a one-day workshop called “The Future of Sensitivity Analysis”, which was held as a satellite event to the Ninth International Conference on Sensitivity Analysis of Model Output (SAMO), October 28–31, 2019 in Barcelona, Spain. We are thankful to the sponsors of this event, including the French research association on stochastic methods for the analysis of numerical codes (MASCOT-NUM), Open Evidence Research at Universitat Oberta de Catalunya, the Joint Research Centre of the European Commission, the University of Bergen (Norway), and the French CERFACS, Centre Européen de Recherche et de Formation Avancée en Calcul Scientifique. The financial and logistic support to Saman Razavi, including underwriting open-access publication fees, by the Integrated modeling Program for Canada (IMPC) under the framework of Global Water Futures (GWF) is acknowledged. Furthermore, part of the efforts leading to this paper was supported by The National Socio-Environmental Synthesis Center of the United States under funding received from the National Science Foundation DBI-1639145 . Sandia National Laboratories is a multi-mission laboratory managed and operated by National Technology and Engineering Solutions of Sandia , LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA-0003525. The views expressed in the article do not necessarily represent the views of the U.S. Department of Energy or the United States Government. John Jakeman's work was supported by the U.S. Department of Energy , Office of Science, Office of Advanced Scientific Computing Research, Scientific Discovery through Advanced Computing (SciDAC) program. Joseph Guillaume received funding from an Australian Research Council Discovery Early Career Award (project no. DE190100317). Arnald Puy worked on this paper on a Marie Sklodowska-Curie Global Fellowship, grant number 792178. Takuya Iwanaga is supported through an Australian Government Research Training Program (AGRTP) Scholarship and the ANU Hilda-John Endowment Fund. We would like to thank Dan Ames, Editor-in-Chief, for insightful comments and encouragement.

Publisher Copyright:
© 2021 The Authors

Keywords

Decision making
Machine learning
Mathematical modeling
Model robustness
Model validation and verification
Policy support
Sensitivity analysis
Uncertainty quantification

ASJC Scopus subject areas

Software
Environmental Engineering
Ecological Modelling

Access to Document

10.1016/j.envsoft.2020.104954Licence: Creative Commons: Attribution (CC BY)

RazaviS2020future Final published version, 2.46 MBLicence: Creative Commons: Attribution (CC BY)

Cite this

Razavi, S., Jakeman, A., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Lo Piano, S., Iwanaga, T., Becker, W., Tarantola, S., Guillaume, J. H. A., Jakeman, J., Gupta, H., Melillo, N., Rabitti, G., Chabridon, V., Duan, Q., Sun, X., ... Maier, H. R. (2021). The future of sensitivity analysis: an essential discipline for systems modeling and policy support. Environmental Modelling and Software, 137, Article 104954. Advance online publication. https://doi.org/10.1016/j.envsoft.2020.104954

@article{22656a3ece8b4492ae0e4bbfb4b5ef8d,

title = "The future of sensitivity analysis: an essential discipline for systems modeling and policy support",

abstract = "Sensitivity analysis (SA) is en route to becoming an integral part of mathematical modeling. The tremendous potential benefits of SA are, however, yet to be fully realized, both for advancing mechanistic and data-driven modeling of human and natural systems, and in support of decision making. In this perspective paper, a multidisciplinary group of researchers and practitioners revisit the current status of SA, and outline research challenges in regard to both theoretical frameworks and their applications to solve real-world problems. Six areas are discussed that warrant further attention, including (1) structuring and standardizing SA as a discipline, (2) realizing the untapped potential of SA for systems modeling, (3) addressing the computational burden of SA, (4) progressing SA in the context of machine learning, (5) clarifying the relationship and role of SA to uncertainty quantification, and (6) evolving the use of SA in support of decision making. An outlook for the future of SA is provided that underlines how SA must underpin a wide variety of activities to better serve science and society.",

keywords = "Decision making, Machine learning, Mathematical modeling, Model robustness, Model validation and verification, Policy support, Sensitivity analysis, Uncertainty quantification",

author = "Saman Razavi and Anthony Jakeman and Andrea Saltelli and Cl{\'e}mentine Prieur and Bertrand Iooss and Emanuele Borgonovo and Elmar Plischke and {Lo Piano}, Samuele and Takuya Iwanaga and William Becker and Stefano Tarantola and Guillaume, {Joseph H.A.} and John Jakeman and Hoshin Gupta and Nicola Melillo and Giovanni Rabitti and Vincent Chabridon and Qingyun Duan and Xifu Sun and Stef{\'a}n Smith and Razi Sheikholeslami and Nasim Hosseini and Masoud Asadzadeh and Arnald Puy and Sergei Kucherenko and Maier, {Holger R.}",

note = "Funding Information: This perspective paper is the outcome of a one-day workshop called “The Future of Sensitivity Analysis”, which was held as a satellite event to the Ninth International Conference on Sensitivity Analysis of Model Output (SAMO), October 28–31, 2019 in Barcelona, Spain. We are thankful to the sponsors of this event, including the French research association on stochastic methods for the analysis of numerical codes (MASCOT-NUM), Open Evidence Research at Universitat Oberta de Catalunya, the Joint Research Centre of the European Commission, the University of Bergen (Norway), and the French CERFACS, Centre Europ{\'e}en de Recherche et de Formation Avanc{\'e}e en Calcul Scientifique. The financial and logistic support to Saman Razavi, including underwriting open-access publication fees, by the Integrated modeling Program for Canada (IMPC) under the framework of Global Water Futures (GWF) is acknowledged. Furthermore, part of the efforts leading to this paper was supported by The National Socio-Environmental Synthesis Center of the United States under funding received from the National Science Foundation DBI-1639145 . Sandia National Laboratories is a multi-mission laboratory managed and operated by National Technology and Engineering Solutions of Sandia , LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA-0003525. The views expressed in the article do not necessarily represent the views of the U.S. Department of Energy or the United States Government. John Jakeman's work was supported by the U.S. Department of Energy , Office of Science, Office of Advanced Scientific Computing Research, Scientific Discovery through Advanced Computing (SciDAC) program. Joseph Guillaume received funding from an Australian Research Council Discovery Early Career Award (project no. DE190100317). Arnald Puy worked on this paper on a Marie Sklodowska-Curie Global Fellowship, grant number 792178. Takuya Iwanaga is supported through an Australian Government Research Training Program (AGRTP) Scholarship and the ANU Hilda-John Endowment Fund. We would like to thank Dan Ames, Editor-in-Chief, for insightful comments and encouragement. Publisher Copyright: {\textcopyright} 2021 The Authors",

year = "2021",

month = mar,

doi = "10.1016/j.envsoft.2020.104954",

language = "English",

volume = "137",

journal = "Environmental Modelling and Software",

issn = "1364-8152",

publisher = "Elsevier",

}

Razavi, S, Jakeman, A, Saltelli, A, Prieur, C, Iooss, B, Borgonovo, E, Plischke, E, Lo Piano, S, Iwanaga, T, Becker, W, Tarantola, S, Guillaume, JHA, Jakeman, J, Gupta, H, Melillo, N, Rabitti, G, Chabridon, V, Duan, Q, Sun, X, Smith, S, Sheikholeslami, R, Hosseini, N, Asadzadeh, M, Puy, A, Kucherenko, S & Maier, HR 2021, 'The future of sensitivity analysis: an essential discipline for systems modeling and policy support', Environmental Modelling and Software, vol. 137, 104954. https://doi.org/10.1016/j.envsoft.2020.104954

TY - JOUR

T1 - The future of sensitivity analysis

T2 - an essential discipline for systems modeling and policy support

AU - Razavi, Saman

AU - Jakeman, Anthony

AU - Saltelli, Andrea

AU - Prieur, Clémentine

AU - Iooss, Bertrand

AU - Borgonovo, Emanuele

AU - Plischke, Elmar

AU - Lo Piano, Samuele

AU - Iwanaga, Takuya

AU - Becker, William

AU - Tarantola, Stefano

AU - Guillaume, Joseph H.A.

AU - Jakeman, John

AU - Gupta, Hoshin

AU - Melillo, Nicola

AU - Rabitti, Giovanni

AU - Chabridon, Vincent

AU - Duan, Qingyun

AU - Sun, Xifu

AU - Smith, Stefán

AU - Sheikholeslami, Razi

AU - Hosseini, Nasim

AU - Asadzadeh, Masoud

AU - Puy, Arnald

AU - Kucherenko, Sergei

AU - Maier, Holger R.

N1 - Funding Information: This perspective paper is the outcome of a one-day workshop called “The Future of Sensitivity Analysis”, which was held as a satellite event to the Ninth International Conference on Sensitivity Analysis of Model Output (SAMO), October 28–31, 2019 in Barcelona, Spain. We are thankful to the sponsors of this event, including the French research association on stochastic methods for the analysis of numerical codes (MASCOT-NUM), Open Evidence Research at Universitat Oberta de Catalunya, the Joint Research Centre of the European Commission, the University of Bergen (Norway), and the French CERFACS, Centre Européen de Recherche et de Formation Avancée en Calcul Scientifique. The financial and logistic support to Saman Razavi, including underwriting open-access publication fees, by the Integrated modeling Program for Canada (IMPC) under the framework of Global Water Futures (GWF) is acknowledged. Furthermore, part of the efforts leading to this paper was supported by The National Socio-Environmental Synthesis Center of the United States under funding received from the National Science Foundation DBI-1639145 . Sandia National Laboratories is a multi-mission laboratory managed and operated by National Technology and Engineering Solutions of Sandia , LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA-0003525. The views expressed in the article do not necessarily represent the views of the U.S. Department of Energy or the United States Government. John Jakeman's work was supported by the U.S. Department of Energy , Office of Science, Office of Advanced Scientific Computing Research, Scientific Discovery through Advanced Computing (SciDAC) program. Joseph Guillaume received funding from an Australian Research Council Discovery Early Career Award (project no. DE190100317). Arnald Puy worked on this paper on a Marie Sklodowska-Curie Global Fellowship, grant number 792178. Takuya Iwanaga is supported through an Australian Government Research Training Program (AGRTP) Scholarship and the ANU Hilda-John Endowment Fund. We would like to thank Dan Ames, Editor-in-Chief, for insightful comments and encouragement. Publisher Copyright: © 2021 The Authors

PY - 2021/3

Y1 - 2021/3

N2 - Sensitivity analysis (SA) is en route to becoming an integral part of mathematical modeling. The tremendous potential benefits of SA are, however, yet to be fully realized, both for advancing mechanistic and data-driven modeling of human and natural systems, and in support of decision making. In this perspective paper, a multidisciplinary group of researchers and practitioners revisit the current status of SA, and outline research challenges in regard to both theoretical frameworks and their applications to solve real-world problems. Six areas are discussed that warrant further attention, including (1) structuring and standardizing SA as a discipline, (2) realizing the untapped potential of SA for systems modeling, (3) addressing the computational burden of SA, (4) progressing SA in the context of machine learning, (5) clarifying the relationship and role of SA to uncertainty quantification, and (6) evolving the use of SA in support of decision making. An outlook for the future of SA is provided that underlines how SA must underpin a wide variety of activities to better serve science and society.

AB - Sensitivity analysis (SA) is en route to becoming an integral part of mathematical modeling. The tremendous potential benefits of SA are, however, yet to be fully realized, both for advancing mechanistic and data-driven modeling of human and natural systems, and in support of decision making. In this perspective paper, a multidisciplinary group of researchers and practitioners revisit the current status of SA, and outline research challenges in regard to both theoretical frameworks and their applications to solve real-world problems. Six areas are discussed that warrant further attention, including (1) structuring and standardizing SA as a discipline, (2) realizing the untapped potential of SA for systems modeling, (3) addressing the computational burden of SA, (4) progressing SA in the context of machine learning, (5) clarifying the relationship and role of SA to uncertainty quantification, and (6) evolving the use of SA in support of decision making. An outlook for the future of SA is provided that underlines how SA must underpin a wide variety of activities to better serve science and society.

KW - Decision making

KW - Machine learning

KW - Mathematical modeling

KW - Model robustness

KW - Model validation and verification

KW - Policy support

KW - Sensitivity analysis

KW - Uncertainty quantification

UR - http://www.scopus.com/inward/record.url?scp=85099475423&partnerID=8YFLogxK

U2 - 10.1016/j.envsoft.2020.104954

DO - 10.1016/j.envsoft.2020.104954

M3 - Article

AN - SCOPUS:85099475423

SN - 1364-8152

VL - 137

JO - Environmental Modelling and Software

JF - Environmental Modelling and Software

M1 - 104954

ER -

The future of sensitivity analysis: an essential discipline for systems modeling and policy support

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this