Autonomous vision-guided bi-manual grasping and manipulation

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)
366 Downloads (Pure)


This paper describes the implementation, demonstration and evaluation of a variety of autonomous, vision-guided manipulation capabilities, using a dual-arm Baxter robot. Initially, symmetric coordinated bi-manual manipulation based on kinematic tracking algorithm was implemented on the robot to enable a master-slave manipulation system. We demonstrate the efficacy of this approach with a human-robot collaboration experiment, where a human operator moves the master arm along arbitrary trajectories and the slave arm automatically follows the master arm while maintaining a constant relative pose between the two end-effectors. Next, this concept was extended to perform dual-arm manipulation without human intervention. To this extent, an image-based visual servoing scheme has been developed to control the motion of arms for positioning them at a desired grasp locations. Next we combine this with a dynamic position controller to move the grasped object using both arms in a prescribed trajectory. The presented approach has been validated by performing numerous symmetric and asymmetric bi-manual manipulations at different conditions. Our experiments demonstrated 80% success rate in performing the symmetric dual-arm manipulation tasks; and 73% success rate in performing asymmetric dualarm manipulation tasks.

Original languageEnglish
Title of host publication2017 IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO)
PublisherIEEE Computer Society
Number of pages7
ISBN (Electronic)9781509004751
ISBN (Print)978-1-5090-0476-8 (PoD)
Publication statusPublished - 4 Sept 2017
Event2017 IEEE Workshop on Advanced Robotics and its Social Impacts, ARSO 2017 - Austin, United States
Duration: 8 Mar 201710 Mar 2017

Publication series

NameIEEE Workshop on Advanced Robotics and its Social Impacts. Conference Proceedings
PublisherInstitute of Electrical and Electronics Engineers
ISSN (Print)2162-7568
ISSN (Electronic)2162-7576


Conference2017 IEEE Workshop on Advanced Robotics and its Social Impacts, ARSO 2017
Country/TerritoryUnited States


  • Robot kinematics
  • Tracking
  • Aerospace electronics
  • Manipulators
  • Visualization
  • Trajectory

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering


Dive into the research topics of 'Autonomous vision-guided bi-manual grasping and manipulation'. Together they form a unique fingerprint.

Cite this