RT-GENE: Real-Time Eye Gaze Estimation in Natural Environments

Tobias Fischer, Hyung Jin Chang, Yiannis Demiris

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Citations (Scopus)
604 Downloads (Pure)


In this work, we consider the problem of robust gaze estimation in natural environments. Large camera-to-subject distances and high variations in head pose and eye gaze angles are common in such environments. This leads to two main shortfalls in state-of-the-art methods for gaze estimation: hindered ground truth gaze annotation and diminished gaze estimation accuracy as image resolution decreases with distance. We first record a novel dataset of varied gaze and head pose images in a natural environment, addressing the issue of ground truth annotation by measuring head pose using a motion capture system and eye gaze using mobile eyetracking glasses. We apply semantic image inpainting to the area covered by the glasses to bridge the gap between training and testing images by removing the obtrusiveness of the glasses. We also present a new real-time algorithm involving appearance-based deep convolutional neural networks with increased capacity to cope with the diverse images in the new dataset. Experiments with this network architecture are conducted on a number of diverse eye-gaze datasets including our own, and in cross dataset evaluations. We demonstrate state-of-the-art performance in terms of estimation accuracy in all experiments, and the architecture performs well even on lower resolution images.
Original languageEnglish
Title of host publicationComputer Vision – ECCV 2018
Subtitle of host publication15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XV
ISBN (Electronic)978-3-030-01267-0
ISBN (Print)978-3-030-01266-3
Publication statusPublished - 8 Nov 2018
EventThe European Conference on Computer Vision (ECCV), 2018 - Munich, Germany
Duration: 8 Sept 201814 Sept 2018

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743


ConferenceThe European Conference on Computer Vision (ECCV), 2018


  • Gaze estimation
  • Gaze dataset
  • Convolutional neural network
  • Semantic inpainting
  • Eyetracking glasses


Dive into the research topics of 'RT-GENE: Real-Time Eye Gaze Estimation in Natural Environments'. Together they form a unique fingerprint.

Cite this