A foundation model for generalizable disease detection from retinal images

Yukun Zhou; Mark A. Chia; Siegfried K. Wagner; Murat S. Ayhan; Dominic J. Williamson; Robbert R. Struyven; Timing Liu; Moucheng Xu; Mateo G. Lozano; Peter Woodward-Court; Yuka Kihara; Andre Altmann; Aaron Y. Lee; Eric J. Topol; Alastair K. Denniston; Daniel C. Alexander; Pearse A. Keane

doi:10.1038/s41586-023-06555-x

A foundation model for generalizable disease detection from retinal images

Yukun Zhou^*, Mark A. Chia, Siegfried K. Wagner, Murat S. Ayhan, Dominic J. Williamson, Robbert R. Struyven, Timing Liu, Moucheng Xu, Mateo G. Lozano, Peter Woodward-Court, Yuka Kihara, Andre Altmann, Aaron Y. Lee, Eric J. Topol, Alastair K. Denniston, Daniel C. Alexander, Pearse A. Keane^*

^*Corresponding author for this work

Inflammation and Ageing

Research output: Contribution to journal › Article › peer-review

11 Downloads (Pure)

Abstract

Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders¹. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications². Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.

Original language	English
Pages (from-to)	156-163
Number of pages	8
Journal	Nature
Volume	622
Issue number	7981
Early online date	13 Sept 2023
DOIs	https://doi.org/10.1038/s41586-023-06555-x
Publication status	Published - 5 Oct 2023

Bibliographical note

Acknowledgements:
We thank P. Rawlinson for project management, C. Green and L. Wickham for information governance expertise, and A. Wenban, S. St John-Green and M. Barnfield for information technology support. This work is supported by Engineering and Physical Sciences Research Council grant nos. EP/M020533/1, EP/R014019/1 and EP/V034537/1, as well as the NIHR UCLH Biomedical Research Centre. S.K.W. is supported by a Medical Research Council Clinical Research Training Fellowship (grant no. MR/TR000953/1). P.A.K. is supported by a Moorfields Eye Charity Career Development Award (grant no. R190028A) and a UK Research & Innovation Future Leaders Fellowship (grant no. MR/T019050/1). For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising.

Access to Document

10.1038/s41586-023-06555-xLicence: Creative Commons: Attribution (CC BY)

ZhouY2023foundationFinal published version, 21.1 MBLicence: Creative Commons: Attribution (CC BY)

Cite this

Zhou, Y., Chia, M. A., Wagner, S. K., Ayhan, M. S., Williamson, D. J., Struyven, R. R., Liu, T., Xu, M., Lozano, M. G., Woodward-Court, P., Kihara, Y., Altmann, A., Lee, A. Y., Topol, E. J., Denniston, A. K., Alexander, D. C., & Keane, P. A. (2023). A foundation model for generalizable disease detection from retinal images. Nature, 622(7981), 156-163. https://doi.org/10.1038/s41586-023-06555-x

@article{5fbde9c217414226b6b2d1665c9466d4,

title = "A foundation model for generalizable disease detection from retinal images",

abstract = "Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders1. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications2. Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.",

author = "Yukun Zhou and Chia, {Mark A.} and Wagner, {Siegfried K.} and Ayhan, {Murat S.} and Williamson, {Dominic J.} and Struyven, {Robbert R.} and Timing Liu and Moucheng Xu and Lozano, {Mateo G.} and Peter Woodward-Court and Yuka Kihara and Andre Altmann and Lee, {Aaron Y.} and Topol, {Eric J.} and Denniston, {Alastair K.} and Alexander, {Daniel C.} and Keane, {Pearse A.}",

note = "Acknowledgements: We thank P. Rawlinson for project management, C. Green and L. Wickham for information governance expertise, and A. Wenban, S. St John-Green and M. Barnfield for information technology support. This work is supported by Engineering and Physical Sciences Research Council grant nos. EP/M020533/1, EP/R014019/1 and EP/V034537/1, as well as the NIHR UCLH Biomedical Research Centre. S.K.W. is supported by a Medical Research Council Clinical Research Training Fellowship (grant no. MR/TR000953/1). P.A.K. is supported by a Moorfields Eye Charity Career Development Award (grant no. R190028A) and a UK Research & Innovation Future Leaders Fellowship (grant no. MR/T019050/1). For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising.",

year = "2023",

month = oct,

day = "5",

doi = "10.1038/s41586-023-06555-x",

language = "English",

volume = "622",

pages = "156--163",

journal = "Nature",

issn = "0028-0836",

publisher = "Nature Publishing Group",

number = "7981",

}

TY - JOUR

T1 - A foundation model for generalizable disease detection from retinal images

AU - Zhou, Yukun

AU - Chia, Mark A.

AU - Wagner, Siegfried K.

AU - Ayhan, Murat S.

AU - Williamson, Dominic J.

AU - Struyven, Robbert R.

AU - Liu, Timing

AU - Xu, Moucheng

AU - Lozano, Mateo G.

AU - Woodward-Court, Peter

AU - Kihara, Yuka

AU - Altmann, Andre

AU - Lee, Aaron Y.

AU - Topol, Eric J.

AU - Denniston, Alastair K.

AU - Alexander, Daniel C.

AU - Keane, Pearse A.

N1 - Acknowledgements: We thank P. Rawlinson for project management, C. Green and L. Wickham for information governance expertise, and A. Wenban, S. St John-Green and M. Barnfield for information technology support. This work is supported by Engineering and Physical Sciences Research Council grant nos. EP/M020533/1, EP/R014019/1 and EP/V034537/1, as well as the NIHR UCLH Biomedical Research Centre. S.K.W. is supported by a Medical Research Council Clinical Research Training Fellowship (grant no. MR/TR000953/1). P.A.K. is supported by a Moorfields Eye Charity Career Development Award (grant no. R190028A) and a UK Research & Innovation Future Leaders Fellowship (grant no. MR/T019050/1). For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising.

PY - 2023/10/5

Y1 - 2023/10/5

N2 - Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders1. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications2. Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.

AB - Medical artificial intelligence (AI) offers great potential for recognizing signs of health conditions in retinal images and expediting the diagnosis of eye diseases and systemic disorders1. However, the development of AI models requires substantial annotation and models are usually task-specific with limited generalizability to different clinical applications2. Here, we present RETFound, a foundation model for retinal images that learns generalizable representations from unlabelled retinal images and provides a basis for label-efficient model adaptation in several applications. Specifically, RETFound is trained on 1.6 million unlabelled retinal images by means of self-supervised learning and then adapted to disease detection tasks with explicit labels. We show that adapted RETFound consistently outperforms several comparison models in the diagnosis and prognosis of sight-threatening eye diseases, as well as incident prediction of complex systemic disorders such as heart failure and myocardial infarction with fewer labelled data. RETFound provides a generalizable solution to improve model performance and alleviate the annotation workload of experts to enable broad clinical AI applications from retinal imaging.

U2 - 10.1038/s41586-023-06555-x

DO - 10.1038/s41586-023-06555-x

M3 - Article

SN - 0028-0836

VL - 622

SP - 156

EP - 163

JO - Nature

JF - Nature

IS - 7981

ER -

A foundation model for generalizable disease detection from retinal images

Abstract

Bibliographical note

Access to Document

Fingerprint

Cite this