Active vision for dexterous grasping of novel objects

Ermano Arruda, Marek Kopicki, Jeremy Wyatt

Research output: Chapter in Book/Report/Conference proceedingConference contribution

18 Citations (Scopus)
775 Downloads (Pure)


How should a robot direct active vision so as to ensure reliable grasping? We answer this question for the case of dexterous grasping of unfamiliar objects. When an object is unfamiliar, much of its shape is by definition unknown. An initial view will recover only some surfaces, leaving most of the object’s surface unmodelled, and also leaving shadow regions which may or may not contain obstacles. These two features make it difficult both to select reliable grasps, and
to plan safe reach-to-grasp trajectories. Grasps typically fail in one of two ways, either unmodelled objects in the scene cause collisions, or object reconstruction is insufficient to ensure that the grasp points provide a stable force closure. These problems can be solved more easily if active sensing is guided by the anticipated actions. Our approach has three stages. First, we take a single view and generate candidate grasps from the resulting partial object reconstruction. Second, we drive active vision to maximise surface reconstruction quality
around the planned contact points. During this phase the anticipated grasp is continually refined. Third, we direct gaze to unmodelled regions that will affect the planned reach to grasp trajectory, so as to confirm that this trajectory is safe. We show, on a dexterous manipulator with camera on wrist, that our approach (85.7% success rate) outperforms a randomised algorithm (64.2% success rate). Our approach also matches the grasp success of our original method, but with fewer views to pick the grasp.
Original languageEnglish
Title of host publication2016 IEEE/RSJ International Conference on Intelligent Robots and Systems
PublisherIEEE Computer Society Press
Number of pages7
ISBN (Print)978-1-5090-3762-9
Publication statusPublished - 1 Dec 2016
Event2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2016) - Daejeon, Korea, Republic of
Duration: 9 Oct 201614 Oct 2016


Conference2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2016)
Country/TerritoryKorea, Republic of


Dive into the research topics of 'Active vision for dexterous grasping of novel objects'. Together they form a unique fingerprint.

Cite this