Quantised transforming auto-encoders: achieving equivariance to arbitrary transformations in deep networks

Jianbo Jiao, João F. Henriques

Research output: Chapter in Book/Report/Conference proceedingConference contribution

72 Downloads (Pure)

Abstract

In this work we investigate how to achieve equivariance to input transformations in deep networks, purely from data, without being given a model of those transformations. Convolutional Neural Networks (CNNs), for example, are equivariant to image translation, a transformation that can be easily modelled (by shifting the pixels vertically or horizontally). Other transformations, such as out-of-plane rotations, do not admit a simple analytic model. We propose an auto-encoder architecture whose embedding obeys an arbitrary set of equivariance relations simultaneously, such as translation, rotation, colour changes, and many others. This means that it can take an input image, and produce versions transformed by a given amount that were not observed before (e.g. a different point of view of the same object, or a colour variation). Despite extending to many (even non-geometric) transformations, our model reduces exactly to a CNN in the special case of translation-equivariance. Equivariances are important for the interpretability and robustness of deep networks, and we demonstrate results of successful re-rendering of transformed versions of input images on several synthetic and real datasets, as well as results on object pose estimation.
Original languageEnglish
Title of host publicationThe 32nd British Machine Vision Conference proceedings
Publication statusPublished - 25 Nov 2021
EventThe 32nd British Machine Vision Conference -
Duration: 22 Nov 202125 Nov 2021

Conference

ConferenceThe 32nd British Machine Vision Conference
Abbreviated titleBMVC 2021
Period22/11/2125/11/21

Bibliographical note

Funding Information:
We are grateful for the support of the EPSRC Programme Grant Visual AI (EP/T028572/1), and the Royal Academy of Engineering (RF\201819\18\163).

Publisher Copyright:
© 2021. The copyright of this document resides with its authors.

Keywords

  • cs.CV
  • cs.AI
  • cs.LG

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition

Fingerprint

Dive into the research topics of 'Quantised transforming auto-encoders: achieving equivariance to arbitrary transformations in deep networks'. Together they form a unique fingerprint.

Cite this