Validation of the Mobile Application Rating Scale (MARS)

Yannik Terhorst; Paula Philippi; Lasse B. Sander; Dana Schultchen; Sarah Paganini; Marco Bardus; Karla Santo; Johannes Knitza; Gustavo C. Machado; Stephanie Schoeppe; Natalie Bauereiß; Alexandra Portenhauser; Matthias Domhardt; Benjamin Walter; Martin Krusche; Harald Baumeister; Eva Maria Messner

doi:10.1371/journal.pone.0241480

Validation of the Mobile Application Rating Scale (MARS)

Yannik Terhorst^*, Paula Philippi, Lasse B. Sander, Dana Schultchen, Sarah Paganini, Marco Bardus, Karla Santo, Johannes Knitza, Gustavo C. Machado, Stephanie Schoeppe, Natalie Bauereiß, Alexandra Portenhauser, Matthias Domhardt, Benjamin Walter, Martin Krusche, Harald Baumeister, Eva Maria Messner

^*Corresponding author for this work

Applied Health Research

Research output: Contribution to journal › Article › peer-review

37 Citations (Scopus)

Abstract

Background Mobile health apps (MHA) have the potential to improve health care. The commercial MHA market is rapidly growing, but the content and quality of available MHA are unknown. Instruments for the assessment of the quality and content of MHA are highly needed. The Mobile Application Rating Scale (MARS) is one of the most widely used tools to evaluate the quality of MHA. Only few validation studies investigated its metric quality. No study has evaluated the construct validity and concurrent validity. Objective This study evaluates the construct validity, concurrent validity, reliability, and objectivity, of the MARS. Methods Data was pooled from 15 international app quality reviews to evaluate the metric properties of the MARS. The MARS measures app quality across four dimensions: Engagement, functionality, aesthetics and information quality. Construct validity was evaluated by assessing related competing confirmatory models by confirmatory factor analysis (CFA). Noncentrality (RMSEA), incremental (CFI, TLI) and residual (SRMR) fit indices were used to evaluate the goodness of fit. As a measure of concurrent validity, the correlations to another quality assessment tool (ENLIGHT) were investigated. Reliability was determined using Omega. Objectivity was assessed by intra-class correlation. Results In total, MARS ratings from 1,299 MHA covering 15 different health domains were included. Confirmatory factor analysis confirmed a bifactor model with a general factor and a factor for each dimension (RMSEA = 0.074, TLI = 0.922, CFI = 0.940, SRMR = 0.059). Reliability was good to excellent (Omega 0.79 to 0.93). Objectivity was high (ICC = 0.82). MARS correlated with ENLIGHT (ps<.05). Conclusion The metric evaluation of the MARS demonstrated its suitability for the quality assessment. As such, the MARS could be used to make the quality of MHA transparent to health care stakeholders and patients. Future studies could extend the present findings by investigating the re-test reliability and predictive validity of the MARS.

Original language	English
Article number	e0241480
Journal	PLoS ONE
Volume	15
Issue number	11 November
DOIs	https://doi.org/10.1371/journal.pone.0241480
Publication status	Published - Nov 2020

Bibliographical note

Publisher Copyright:
© 2020 Terhorst et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

ASJC Scopus subject areas

General

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1371/journal.pone.0241480

37 Citations
1 Article

The Persian version of the Mobile Application Rating Scale (MARS-Fa): translation and validation study
Barzegari, S., Sharifi Kia, A., Bardus, M., Stoyanov, S. R., GhaziSaeedi, M. & Rafizadeh, M., 5 Dec 2022, In: JMIR Formative Research. 6, 12, 12 p., e42225.
Research output: Contribution to journal › Article › peer-review

Open Access
File
49 Downloads (Pure)

University Research Board (URB) Grant
Bardus, Marco (Recipient), Ghandour, Lilian A. (Recipient), Fares, Elie Jacques (Recipient) & Gherbal, Tarek (Recipient), Mar 2018
Prize: Fellowship awarded competitively

Cite this

Terhorst, Y., Philippi, P., Sander, L. B., Schultchen, D., Paganini, S., Bardus, M., Santo, K., Knitza, J., Machado, G. C., Schoeppe, S., Bauereiß, N., Portenhauser, A., Domhardt, M., Walter, B., Krusche, M., Baumeister, H., & Messner, E. M. (2020). Validation of the Mobile Application Rating Scale (MARS). PLoS ONE, 15(11 November), Article e0241480. https://doi.org/10.1371/journal.pone.0241480

@article{642eb2c4ce404a52b2db6aa742a51572,

title = "Validation of the Mobile Application Rating Scale (MARS)",

abstract = "Background Mobile health apps (MHA) have the potential to improve health care. The commercial MHA market is rapidly growing, but the content and quality of available MHA are unknown. Instruments for the assessment of the quality and content of MHA are highly needed. The Mobile Application Rating Scale (MARS) is one of the most widely used tools to evaluate the quality of MHA. Only few validation studies investigated its metric quality. No study has evaluated the construct validity and concurrent validity. Objective This study evaluates the construct validity, concurrent validity, reliability, and objectivity, of the MARS. Methods Data was pooled from 15 international app quality reviews to evaluate the metric properties of the MARS. The MARS measures app quality across four dimensions: Engagement, functionality, aesthetics and information quality. Construct validity was evaluated by assessing related competing confirmatory models by confirmatory factor analysis (CFA). Noncentrality (RMSEA), incremental (CFI, TLI) and residual (SRMR) fit indices were used to evaluate the goodness of fit. As a measure of concurrent validity, the correlations to another quality assessment tool (ENLIGHT) were investigated. Reliability was determined using Omega. Objectivity was assessed by intra-class correlation. Results In total, MARS ratings from 1,299 MHA covering 15 different health domains were included. Confirmatory factor analysis confirmed a bifactor model with a general factor and a factor for each dimension (RMSEA = 0.074, TLI = 0.922, CFI = 0.940, SRMR = 0.059). Reliability was good to excellent (Omega 0.79 to 0.93). Objectivity was high (ICC = 0.82). MARS correlated with ENLIGHT (ps<.05). Conclusion The metric evaluation of the MARS demonstrated its suitability for the quality assessment. As such, the MARS could be used to make the quality of MHA transparent to health care stakeholders and patients. Future studies could extend the present findings by investigating the re-test reliability and predictive validity of the MARS.",

author = "Yannik Terhorst and Paula Philippi and Sander, {Lasse B.} and Dana Schultchen and Sarah Paganini and Marco Bardus and Karla Santo and Johannes Knitza and Machado, {Gustavo C.} and Stephanie Schoeppe and Natalie Bauerei{\ss} and Alexandra Portenhauser and Matthias Domhardt and Benjamin Walter and Martin Krusche and Harald Baumeister and Messner, {Eva Maria}",

note = "Publisher Copyright: {\textcopyright} 2020 Terhorst et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.",

year = "2020",

month = nov,

doi = "10.1371/journal.pone.0241480",

language = "English",

volume = "15",

journal = "PLoS ONE",

issn = "1932-6203",

publisher = "Public Library of Science (PLOS)",

number = "11 November",

}

Terhorst, Y, Philippi, P, Sander, LB, Schultchen, D, Paganini, S, Bardus, M, Santo, K, Knitza, J, Machado, GC, Schoeppe, S, Bauereiß, N, Portenhauser, A, Domhardt, M, Walter, B, Krusche, M, Baumeister, H & Messner, EM 2020, 'Validation of the Mobile Application Rating Scale (MARS)', PLoS ONE, vol. 15, no. 11 November, e0241480. https://doi.org/10.1371/journal.pone.0241480

TY - JOUR

T1 - Validation of the Mobile Application Rating Scale (MARS)

AU - Terhorst, Yannik

AU - Philippi, Paula

AU - Sander, Lasse B.

AU - Schultchen, Dana

AU - Paganini, Sarah

AU - Bardus, Marco

AU - Santo, Karla

AU - Knitza, Johannes

AU - Machado, Gustavo C.

AU - Schoeppe, Stephanie

AU - Bauereiß, Natalie

AU - Portenhauser, Alexandra

AU - Domhardt, Matthias

AU - Walter, Benjamin

AU - Krusche, Martin

AU - Baumeister, Harald

AU - Messner, Eva Maria

N1 - Publisher Copyright: © 2020 Terhorst et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

PY - 2020/11

Y1 - 2020/11

N2 - Background Mobile health apps (MHA) have the potential to improve health care. The commercial MHA market is rapidly growing, but the content and quality of available MHA are unknown. Instruments for the assessment of the quality and content of MHA are highly needed. The Mobile Application Rating Scale (MARS) is one of the most widely used tools to evaluate the quality of MHA. Only few validation studies investigated its metric quality. No study has evaluated the construct validity and concurrent validity. Objective This study evaluates the construct validity, concurrent validity, reliability, and objectivity, of the MARS. Methods Data was pooled from 15 international app quality reviews to evaluate the metric properties of the MARS. The MARS measures app quality across four dimensions: Engagement, functionality, aesthetics and information quality. Construct validity was evaluated by assessing related competing confirmatory models by confirmatory factor analysis (CFA). Noncentrality (RMSEA), incremental (CFI, TLI) and residual (SRMR) fit indices were used to evaluate the goodness of fit. As a measure of concurrent validity, the correlations to another quality assessment tool (ENLIGHT) were investigated. Reliability was determined using Omega. Objectivity was assessed by intra-class correlation. Results In total, MARS ratings from 1,299 MHA covering 15 different health domains were included. Confirmatory factor analysis confirmed a bifactor model with a general factor and a factor for each dimension (RMSEA = 0.074, TLI = 0.922, CFI = 0.940, SRMR = 0.059). Reliability was good to excellent (Omega 0.79 to 0.93). Objectivity was high (ICC = 0.82). MARS correlated with ENLIGHT (ps<.05). Conclusion The metric evaluation of the MARS demonstrated its suitability for the quality assessment. As such, the MARS could be used to make the quality of MHA transparent to health care stakeholders and patients. Future studies could extend the present findings by investigating the re-test reliability and predictive validity of the MARS.

AB - Background Mobile health apps (MHA) have the potential to improve health care. The commercial MHA market is rapidly growing, but the content and quality of available MHA are unknown. Instruments for the assessment of the quality and content of MHA are highly needed. The Mobile Application Rating Scale (MARS) is one of the most widely used tools to evaluate the quality of MHA. Only few validation studies investigated its metric quality. No study has evaluated the construct validity and concurrent validity. Objective This study evaluates the construct validity, concurrent validity, reliability, and objectivity, of the MARS. Methods Data was pooled from 15 international app quality reviews to evaluate the metric properties of the MARS. The MARS measures app quality across four dimensions: Engagement, functionality, aesthetics and information quality. Construct validity was evaluated by assessing related competing confirmatory models by confirmatory factor analysis (CFA). Noncentrality (RMSEA), incremental (CFI, TLI) and residual (SRMR) fit indices were used to evaluate the goodness of fit. As a measure of concurrent validity, the correlations to another quality assessment tool (ENLIGHT) were investigated. Reliability was determined using Omega. Objectivity was assessed by intra-class correlation. Results In total, MARS ratings from 1,299 MHA covering 15 different health domains were included. Confirmatory factor analysis confirmed a bifactor model with a general factor and a factor for each dimension (RMSEA = 0.074, TLI = 0.922, CFI = 0.940, SRMR = 0.059). Reliability was good to excellent (Omega 0.79 to 0.93). Objectivity was high (ICC = 0.82). MARS correlated with ENLIGHT (ps<.05). Conclusion The metric evaluation of the MARS demonstrated its suitability for the quality assessment. As such, the MARS could be used to make the quality of MHA transparent to health care stakeholders and patients. Future studies could extend the present findings by investigating the re-test reliability and predictive validity of the MARS.

UR - http://www.scopus.com/inward/record.url?scp=85095405860&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0241480

DO - 10.1371/journal.pone.0241480

M3 - Article

C2 - 33137123

AN - SCOPUS:85095405860

SN - 1932-6203

VL - 15

JO - PLoS ONE

JF - PLoS ONE

IS - 11 November

M1 - e0241480

ER -

Validation of the Mobile Application Rating Scale (MARS)

Abstract

Bibliographical note

ASJC Scopus subject areas

UN SDGs

Access to Document

Fingerprint

Research output

The Persian version of the Mobile Application Rating Scale (MARS-Fa): translation and validation study

Prizes

University Research Board (URB) Grant

Cite this