Multi-source vision perception for human-robot collaboration in manufacturing

Sida Yang, Wenjun Xu, Zhihao Liu, Zude Zhou, Duc Pham

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)


Nowadays, human-robot collaboration is playing a more important role in manufacturing. However, in order to collaborate efficiently and safely with human, it is quiet essential for the industrial robots used in production process to percept and acquire the condition information of human workers and environment surroundings, and the vision perception is definitely one of the most effective approaches to realize such functions since it is contactless and economical. In this paper, a multi-source heterogeneous vision perception framework is proposed to acquire the various condition information of the human workers and the working environment surroundings during the processes of human-robot collaboration in manufacturing. The multi-source data are captured from multiple heterogeneous vision sensors deployed in different physical positions, e.g. RGB-D cameras (cameras with color and depth information output) around working area which can produce 3D point cloud data, the binocular cameras on the workbench which can track worker's hands, etc. Moreover, a novel data fusion method is developed to process the multi-source vision data to facilitate the perception function of the industrial robots. The experimental results demonstrate the proposed approaches can get an expansive and shadeless perception of environment and workers including workers' hands status at a high average framerate.

Original languageEnglish
Title of host publicationICNSC 2018 - 15th IEEE International Conference on Networking, Sensing and Control
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages6
ISBN (Electronic)9781538650530
ISBN (Print)978-1-5386-5054-7
Publication statusPublished - 21 May 2018
Event15th IEEE International Conference on Networking, Sensing and Control, ICNSC 2018 - Zhuhai, China
Duration: 27 Mar 201829 Mar 2018


Conference15th IEEE International Conference on Networking, Sensing and Control, ICNSC 2018


  • data fusion
  • human-robot collaboration
  • manufacturing
  • multi-source vision perception

ASJC Scopus subject areas

  • Instrumentation
  • Artificial Intelligence
  • Computer Networks and Communications
  • Control and Optimization
  • Modelling and Simulation


Dive into the research topics of 'Multi-source vision perception for human-robot collaboration in manufacturing'. Together they form a unique fingerprint.

Cite this