Enhancing Visual Coding Through Collaborative Perception

A central challenge facing the nature human-computer interaction involves understanding how neural circuits process visual perceptual information to improve the user’s operation ability under complex tasks. Visual coding models aim to explore the biological characteristics of retinal ganglion cells to provide quantitative predictions of responses to a range of visual stimuli. The existing visual coding models lack adaptability in natural and complex scenes. Therefore this paper proposes an enhanced visual coding model through collaborative perception. Our model first extracts the multi-modal spatiotemporal features of the input video to simulate the retinal response characteristics adaptively. Secondly, it uses the basis function to compile the input stimulus into a multi-modal stimulus matrix. Afterward, the upstream and downstream filters reform the stimulus matrix to generate the spike sequence. Experiments show that the proposed model reproduces the physiological characteristics of ganglion cells in the biological retina, leading to the high accuracy, good adaptability, and biological interpretability in comparison with its rivals.