Abstract
The evaluation of 3D face reconstruction results typically relies on a rigid shape alignment between the estimated 3D model and the ground-truth scan. We observe that aligning two shapes with different reference points can largely affect the evaluation results. This poses difficulties for precisely diagnosing and improving a 3D face reconstruction method. In this paper, we propose a novel evaluation approach with a new benchmark REALY, consists of 100 globally aligned face scans with accurate facial keypoints, high-quality region masks, and topology-consistent meshes. Our approach performs region-wise shape alignment and leads to more accurate, bidirectional correspondences during computing the shape errors. The fine-grained, region-wise evaluation results provide us detailed understandings about the performance of state-of-the-art 3D face reconstruction methods. For example, our experiments on single-image based reconstruction methods reveal that DECA performs the best on nose regions, while GANFit performs better on cheek regions. Besides, a new and high-quality 3DMM basis, HIFI3D ++, is further derived using the same procedure as we construct REALY to align and retopologize several 3D face datasets. We will release REALY, HIFI3D ++, and our new evaluation pipeline at https://realy3dface.com.
Original language | English |
---|---|
Title of host publication | Computer Vision – ECCV 2022 |
Subtitle of host publication | 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part VIII |
Editors | Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, Tal Hassner |
Publisher | Springer |
Pages | 74-92 |
Number of pages | 19 |
ISBN (Electronic) | 9783031200748 |
ISBN (Print) | 9783031200731 |
DOIs | |
Publication status | Published - 12 Nov 2022 |
Event | 17th European Conference on Computer Vision, ECCV 2022 - Tel Aviv, Israel Duration: 23 Oct 2022 → 27 Oct 2022 |
Publication series
Name | Lecture Notes in Computer Science |
---|---|
Volume | 13668 |
ISSN (Print) | 0302-9743 |
ISSN (Electronic) | 1611-3349 |
Conference
Conference | 17th European Conference on Computer Vision, ECCV 2022 |
---|---|
Abbreviated title | ECCV 2022 |
Country/Territory | Israel |
City | Tel Aviv |
Period | 23/10/22 → 27/10/22 |
Bibliographical note
Funding Information:Acknowledgment. This work was supported by SZSTC Grant No. JCYJ20190 809172201639 and WDZC20200820200655001, Shenzhen Key Laboratory ZDSY S20210623092001004.
Publisher Copyright:
© 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
Keywords
- 3D Face Reconstruction
- 3DMM
- Benchmark
- Evaluation
ASJC Scopus subject areas
- Theoretical Computer Science
- General Computer Science