Vision-Based Pseudo-Tactile Information Extraction and Localization for Dexterous Grasping
Keywords: Pseudo-Tactile Information, Dexterous Grasping, Vision-Based Perception, Robotic Localization
TL;DR: This study extracts pseudo-tactile information from everyday objects using vision, enabling real-time localization of fingertip contact points and their corresponding pseudo-tactile 3D point cloud information in robotic dexterous hand grasping.
Abstract: This study addresses the challenges of tactile perception in robotic dexterous hand grasping by focusing on two main tasks: 1) Acquiring tactile information from everyday objects using vision, termed "pseudo-tactile" information, and 2) Building a Dexterous Hand (RH8D) model in Isaac Sim for real-time fingertip contact localization. Utilizing Isaac Sim enables safe, cost-effective experimentation and high-precision simulations that facilitate data collection for model validation. The research establishes a scientific connection between simulated 3D coordinates, actual 3D coordinates, and pseudo-tactile information derived from point clouds, quantified through normal vectors and grayscale variance analysis. Results demonstrate the ability to extract clear object surface textures, accurately locate fingertip contact points in real-time (with precision up to $0.001 m$), and provide tactile information at contact points. This framework enhances robotic grasping capabilities and offers low-cost sensory data. The source code and dataset are publicly available now.
Supplementary Material: zip
Primary Area: applications to robotics, autonomy, planning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 14012
Loading