Two examples of input to our approach for a successful grasp. Each row of three images, shows the output of the TACTO simulator, for both left (left image) and right gripper fingers (middle image), with the image on the right showing a camera view of the scene. Our network can successfully predict grasp success using this data. The intensity of the pixel roughly represents the depth of displacement for the gel inside the DIGIT sensor, where the darker the pixel, the greater the depth.
Publication: 2022 19th Conference on Robots and Vision (CRV)
Abstract: We investigate how high-resolution tactile sensors can be utilized in combination with vision and depth sensing, to improve grasp stability prediction. Recent advances in simulating high-resolution tactile sensing, in particular the TACTO simulator, enabled us to evaluate how neural networks can be trained with a combination of sensing modalities. With the large amounts of data needed to train large neural networks, robotic simulators provide a fast way to automate the data collection process. We expand on the existing work through an ablation study and an increased set of objects taken from the YCB benchmark set. Our results indicate that while the combination of vision, depth, and tactile sensing provides the best prediction results on known objects, the network fails to generalize to unknown objects. Our work also addresses existing issues with robotic grasping in tactile simulation and how to overcome them.
Bibtex:
@inproceedings{Chumbley2022Integrating,
author = {Chumbley, Lachlan and Gu, Morris and Newbury, Rhys and Leitner, Jürgen and Cosgun, Akansel},
booktitle = {2022 19th Conference on Robots and Vision (CRV)},
title = {Integrating High-Resolution Tactile Sensing into Grasp Stability Prediction},
year = {2022},
volume = {},
number = {},
pages = {98-105},
doi = {10.1109/CRV55824.2022.00021}
}