FreMAE: Fourier Transform Meets Masked Autoencoders for Medical Image Segmentation

Wenxuan Wang, Jing Wang, Chen Chen, Jianbo Jiao, Lichao Sun, Yuanxiu Cai, Shanshan Song, Jiangyun Li

Research output: Working paper/PreprintPreprint

315 Downloads (Pure)

Abstract

The research community has witnessed the powerful potential of self-supervised Masked Image Modeling (MIM), which enables the models capable of learning visual representation from unlabeled data. In this paper, to incorporate both the crucial global structural information and local details for dense prediction tasks, we alter the perspective to the frequency domain and present a new MIM-based framework named FreMAE for self-supervised pre-training for medical image segmentation. Based on the observations that the detailed structural information mainly lies in the high-frequency components and the high-level semantics are abundant in the low-frequency counterparts, we further incorporate multi-stage supervision to guide the representation learning during the pre-training phase. Extensive experiments on three benchmark datasets show the superior advantage of our proposed FreMAE over previous state-of-the-art MIM methods. Compared with various baselines trained from scratch, our FreMAE could consistently bring considerable improvements to the model performance. To the best our knowledge, this is the first attempt towards MIM with Fourier Transform in medical image segmentation.
Original languageEnglish
PublisherarXiv
DOIs
Publication statusPublished - 21 Apr 2023

Fingerprint

Dive into the research topics of 'FreMAE: Fourier Transform Meets Masked Autoencoders for Medical Image Segmentation'. Together they form a unique fingerprint.

Cite this