Connecting Image Denoising and High-Level Vision Tasks via Deep Learning

Ding Liu*, Bihan Wen, Jianbo Jiao, Xianming Liu, Zhangyang Wang, Thomas S. Huang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

69 Citations (Scopus)


Image denoising and high-level vision tasks are usually handled independently in the conventional practice of computer vision, and their connection is fragile. In this paper, we cope with the two jointly and explore the mutual influence between them with the focus on two questions, namely (1) how image denoising can help improving high-level vision tasks, and (2) how the semantic information from high-level vision tasks can be used to guide image denoising. First for image denoising we propose a convolutional neural network in which convolutions are conducted in various spatial resolutions via downsampling and upsampling operations in order to fuse and exploit contextual information on different scales. Second we propose a deep neural network solution that cascades two modules for image denoising and various high-level tasks, respectively, and use the joint loss for updating only the denoising network via back-propagation. We experimentally show that on one hand, the proposed denoiser has the generality to overcome the performance degradation of different high-level vision tasks. On the other hand, with the guidance of high-level vision information, the denoising network produces more visually appealing results. Extensive experiments demonstrate the benefit of exploiting image semantics simultaneously for image denoising and high-level vision tasks via deep learning. The code is available online:

Original languageEnglish
Pages (from-to)3695-3706
Number of pages12
JournalIEEE Transactions on Image Processing
Publication statusPublished - 15 Jan 2020

Bibliographical note

Funding Information:
Manuscript received August 31, 2018; revised April 15, 2019; accepted December 17, 2019. Date of publication January 15, 2020; date of current version January 30, 2020. This work was supported by the U.S. Army Research Office under Grant W911NF-15-1-0317. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Jana Ehmann. (Corresponding author: Ding Liu.) Ding Liu is with Bytedance Inc., Mountain View, CA 61801 USA (e-mail:

Publisher Copyright:
© 1992-2012 IEEE.


  • Deep learning
  • high-level vision
  • image denoising
  • neural network

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Connecting Image Denoising and High-Level Vision Tasks via Deep Learning'. Together they form a unique fingerprint.

Cite this