Look deeper into depth: monocular depth estimation with semantic booster and attention-driven loss

Jianbo Jiao*, Ying Cao, Yibing Song, Rynson Lau

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

129 Citations (Scopus)


Monocular depth estimation benefits greatly from learning based techniques. By studying the training data, we observe that the per-pixel depth values in existing datasets typically exhibit a long-tailed distribution. However, most previous approaches treat all the regions in the training data equally regardless of the imbalanced depth distribution, which restricts the model performance particularly on distant depth regions. In this paper, we investigate the long tail property and delve deeper into the distant depth regions (i.e. the tail part) to propose an attention-driven loss for the network supervision. In addition, to better leverage the semantic information for monocular depth estimation, we propose a synergy network to automatically learn the information sharing strategies between the two tasks. With the proposed attention-driven loss and synergy network, the depth estimation and semantic labeling tasks can be mutually improved. Experiments on the challenging indoor dataset show that the proposed approach achieves state-of-the-art performance on both monocular depth estimation and semantic labeling tasks.

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2018
Subtitle of host publication15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XV
EditorsYair Weiss, Vittorio Ferrari, Cristian Sminchisescu, Martial Hebert
Number of pages17
ISBN (Electronic)9783030012670
ISBN (Print)9783030012663
Publication statusPublished - 7 Oct 2018
Event15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany
Duration: 8 Sept 201814 Sept 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference15th European Conference on Computer Vision, ECCV 2018

Bibliographical note

Funding Information:
Acknowledgments. This work is partially supported by the Hong Kong PhD Fellowship Scheme (HKPFS) from the RGC of Hong Kong.

Publisher Copyright:
© Springer Nature Switzerland AG 2018.


  • Attention loss
  • Monocular depth
  • Semantic labeling

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Look deeper into depth: monocular depth estimation with semantic booster and attention-driven loss'. Together they form a unique fingerprint.

Cite this