Autonomous vision-guided bi-manual grasping and manipulation

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Colleges, School and Institutes

External organisations

  • University of Birmingham
  • KUKA Robotics UK Ltd


This paper describes the implementation, demonstration and evaluation of a variety of autonomous, vision-guided manipulation capabilities, using a dual-arm Baxter robot. Initially, symmetric coordinated bi-manual manipulation based on kinematic tracking algorithm was implemented on the robot to enable a master-slave manipulation system. We demonstrate the efficacy of this approach with a human-robot collaboration experiment, where a human operator moves the master arm along arbitrary trajectories and the slave arm automatically follows the master arm while maintaining a constant relative pose between the two end-effectors. Next, this concept was extended to perform dual-arm manipulation without human intervention. To this extent, an image-based visual servoing scheme has been developed to control the motion of arms for positioning them at a desired grasp locations. Next we combine this with a dynamic position controller to move the grasped object using both arms in a prescribed trajectory. The presented approach has been validated by performing numerous symmetric and asymmetric bi-manual manipulations at different conditions. Our experiments demonstrated 80% success rate in performing the symmetric dual-arm manipulation tasks; and 73% success rate in performing asymmetric dualarm manipulation tasks.


Original languageEnglish
Title of host publication2017 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO)
Publication statusPublished - 4 Sep 2017
Event2017 IEEE Workshop on Advanced Robotics and its Social Impacts, ARSO 2017 - Austin, United States
Duration: 8 Mar 201710 Mar 2017

Publication series

NameIEEE Workshop on Advanced Robotics and its Social Impacts. Conference Proceedings
PublisherInstitute of Electrical and Electronics Engineers
ISSN (Print)2162-7568
ISSN (Electronic)2162-7576


Conference2017 IEEE Workshop on Advanced Robotics and its Social Impacts, ARSO 2017
CountryUnited States


  • Robot kinematics, Tracking, Aerospace electronics, Manipulators, Visualization, Trajectory