Unraveling neural coding of dynamic natural visual scenes via convolutional recurrent neural networks

Yajing Zheng; Shanshan Jia; Zhaofei Yu; Jian K Liu; Tiejun Huang

doi:10.1016/j.patter.2021.100350

Unraveling neural coding of dynamic natural visual scenes via convolutional recurrent neural networks

Yajing Zheng
, Shanshan Jia
, Zhaofei Yu^*
, Jian K Liu
, Tiejun Huang^*

^*Corresponding author for this work

Computer Science

Research output: Contribution to journal › Article › peer-review

51 Downloads (Pure)

Abstract

Traditional models of retinal system identification analyze the neural response to artificial stimuli using models consisting of predefined components. The model design is limited to prior knowledge, and the artificial stimuli are too simple to be compared with stimuli processed by the retina. To fill in this gap with an explainable model that reveals how a population of neurons work together to encode the larger field of natural scenes, here we used a deep-learning model for identifying the computational elements of the retinal circuit that contribute to learning the dynamics of natural scenes. Experimental results verify that the recurrent connection plays a key role in encoding complex dynamic visual scenes while learning biological computational underpinnings of the retinal circuit. In addition, the proposed models reveal both the shapes and the locations of the spatiotemporal receptive fields of ganglion cells.

Original language	English
Article number	100350
Number of pages	16
Journal	Patterns
Volume	2
Issue number	10
Early online date	17 Sept 2021
DOIs	https://doi.org/10.1016/j.patter.2021.100350
Publication status	Published - 8 Oct 2021

Bibliographical note

Acknowledgments:
This work was supported by the National Natural Science Foundation of China (62088102, 61961130392); the International Youth Talent Fund of Zhejiang Lab (ZJ2020JS017), Zhejiang Lab, China (2019KC0AB03 and 2019KC0AD02); and the Royal Society Newton Advanced Fellowship, UK (NAF-R1-191082).

Copyright:
© 2021 The Authors.

Keywords

convolutional neural network
recurrent neural network
neural coding
visual coding
retina
video analysis

Access to Document

10.1016/j.patter.2021.100350Licence: Creative Commons: Attribution (CC BY)

ZhengY2021UnravelingFinal published version, 4.81 MBLicence: Creative Commons: Attribution (CC BY)

https://linkinghub.elsevier.com/retrieve/pii/S2666389921002051Licence: Creative Commons: Attribution (CC BY)

Cite this

@article{506c8a26f07c47e9912245219aa8fa2d,

title = "Unraveling neural coding of dynamic natural visual scenes via convolutional recurrent neural networks",

abstract = "Traditional models of retinal system identification analyze the neural response to artificial stimuli using models consisting of predefined components. The model design is limited to prior knowledge, and the artificial stimuli are too simple to be compared with stimuli processed by the retina. To fill in this gap with an explainable model that reveals how a population of neurons work together to encode the larger field of natural scenes, here we used a deep-learning model for identifying the computational elements of the retinal circuit that contribute to learning the dynamics of natural scenes. Experimental results verify that the recurrent connection plays a key role in encoding complex dynamic visual scenes while learning biological computational underpinnings of the retinal circuit. In addition, the proposed models reveal both the shapes and the locations of the spatiotemporal receptive fields of ganglion cells.",

keywords = "convolutional neural network, recurrent neural network, neural coding, visual coding, retina, video analysis",

author = "Yajing Zheng and Shanshan Jia and Zhaofei Yu and Liu, \{Jian K\} and Tiejun Huang",

note = "Acknowledgments: This work was supported by the National Natural Science Foundation of China (62088102, 61961130392); the International Youth Talent Fund of Zhejiang Lab (ZJ2020JS017), Zhejiang Lab, China (2019KC0AB03 and 2019KC0AD02); and the Royal Society Newton Advanced Fellowship, UK (NAF-R1-191082). Copyright: {\textcopyright} 2021 The Authors.",

year = "2021",

month = oct,

day = "8",

doi = "10.1016/j.patter.2021.100350",

language = "English",

volume = "2",

journal = "Patterns",

issn = "2666-3899",

publisher = "Cell Press",

number = "10",

}

TY - JOUR

T1 - Unraveling neural coding of dynamic natural visual scenes via convolutional recurrent neural networks

AU - Zheng, Yajing

AU - Jia, Shanshan

AU - Yu, Zhaofei

AU - Liu, Jian K

AU - Huang, Tiejun

N1 - Acknowledgments: This work was supported by the National Natural Science Foundation of China (62088102, 61961130392); the International Youth Talent Fund of Zhejiang Lab (ZJ2020JS017), Zhejiang Lab, China (2019KC0AB03 and 2019KC0AD02); and the Royal Society Newton Advanced Fellowship, UK (NAF-R1-191082). Copyright: © 2021 The Authors.

PY - 2021/10/8

Y1 - 2021/10/8

N2 - Traditional models of retinal system identification analyze the neural response to artificial stimuli using models consisting of predefined components. The model design is limited to prior knowledge, and the artificial stimuli are too simple to be compared with stimuli processed by the retina. To fill in this gap with an explainable model that reveals how a population of neurons work together to encode the larger field of natural scenes, here we used a deep-learning model for identifying the computational elements of the retinal circuit that contribute to learning the dynamics of natural scenes. Experimental results verify that the recurrent connection plays a key role in encoding complex dynamic visual scenes while learning biological computational underpinnings of the retinal circuit. In addition, the proposed models reveal both the shapes and the locations of the spatiotemporal receptive fields of ganglion cells.

AB - Traditional models of retinal system identification analyze the neural response to artificial stimuli using models consisting of predefined components. The model design is limited to prior knowledge, and the artificial stimuli are too simple to be compared with stimuli processed by the retina. To fill in this gap with an explainable model that reveals how a population of neurons work together to encode the larger field of natural scenes, here we used a deep-learning model for identifying the computational elements of the retinal circuit that contribute to learning the dynamics of natural scenes. Experimental results verify that the recurrent connection plays a key role in encoding complex dynamic visual scenes while learning biological computational underpinnings of the retinal circuit. In addition, the proposed models reveal both the shapes and the locations of the spatiotemporal receptive fields of ganglion cells.

KW - convolutional neural network

KW - recurrent neural network

KW - neural coding

KW - visual coding

KW - retina

KW - video analysis

U2 - 10.1016/j.patter.2021.100350

DO - 10.1016/j.patter.2021.100350

M3 - Article

C2 - 34693375

SN - 2666-3899

VL - 2

JO - Patterns

JF - Patterns

IS - 10

M1 - 100350

ER -

Unraveling neural coding of dynamic natural visual scenes via convolutional recurrent neural networks

Abstract

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this