Multi-source vision perception for human-robot collaboration in manufacturing

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Authors

  • Sida Yang
  • Wenjun Xu
  • Zhihao Liu
  • Zude Zhou
  • Duc Pham

Colleges, School and Institutes

External organisations

  • Wuhan University of Technology

Abstract

Nowadays, human-robot collaboration is playing a more important role in manufacturing. However, in order to collaborate efficiently and safely with human, it is quiet essential for the industrial robots used in production process to percept and acquire the condition information of human workers and environment surroundings, and the vision perception is definitely one of the most effective approaches to realize such functions since it is contactless and economical. In this paper, a multi-source heterogeneous vision perception framework is proposed to acquire the various condition information of the human workers and the working environment surroundings during the processes of human-robot collaboration in manufacturing. The multi-source data are captured from multiple heterogeneous vision sensors deployed in different physical positions, e.g. RGB-D cameras (cameras with color and depth information output) around working area which can produce 3D point cloud data, the binocular cameras on the workbench which can track worker's hands, etc. Moreover, a novel data fusion method is developed to process the multi-source vision data to facilitate the perception function of the industrial robots. The experimental results demonstrate the proposed approaches can get an expansive and shadeless perception of environment and workers including workers' hands status at a high average framerate.

Details

Original languageEnglish
Title of host publicationICNSC 2018 - 15th IEEE International Conference on Networking, Sensing and Control
Publication statusPublished - 21 May 2018
Event15th IEEE International Conference on Networking, Sensing and Control, ICNSC 2018 - Zhuhai, China
Duration: 27 Mar 201829 Mar 2018

Conference

Conference15th IEEE International Conference on Networking, Sensing and Control, ICNSC 2018
CountryChina
CityZhuhai
Period27/03/1829/03/18

Keywords

  • data fusion, human-robot collaboration, manufacturing, multi-source vision perception