PneuGelSight: Soft Robotic Vision-Based Proprioception and Tactile Sensing

Ruohan Zhang, Uksang Yoo, Yichen Li, Arpit Agarwal, Wenzhen Yuan

Project Page arXiv:2508.18443 PDF

Abstract

Soft pneumatic robot manipulators are popular in industrial and human-interactive applications due to their compliance and flexibility. However, deploying them in real-world scenarios requires advanced sensing for tactile feedback and proprioception. Our work presents a novel vision-based approach for sensorizing soft robots. We demonstrate our approach on PneuGelSight, a pioneering pneumatic manipulator featuring high-resolution proprioception and tactile sensing via an embedded camera. To optimize the sensor’s performance, we introduce a comprehensive pipeline that accurately simulates its optical and dynamic properties, facilitating a zero-shot knowledge transition from simulation to real-world applications. PneuGelSight and our sim-to-real pipeline provide a novel, easily implementable, and robust sensing methodology for soft robots, paving the way for the development of more advanced soft robots with enhanced sensory capabilities.

Citation

@article{zhang2025pneugelsight,
  title         = {PneuGelSight: Soft Robotic Vision-Based Proprioception and Tactile Sensing},
  author        = {Zhang, Ruohan and Yoo, Uksang and Li, Yichen and Agarwal, Arpit and Yuan, Wenzhen},
  journal       = {The International Journal of Robotics Research},
  year          = {2025},
  eprint        = {2508.18443},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2508.18443}
}

Recommended citation: Ruohan Zhang, Uksang Yoo, Yichen Li, Arpit Agarwal, Wenzhen Yuan. (2025). "PneuGelSight: Soft Robotic Vision-Based Proprioception and Tactile Sensing." The International Journal of Robotics Research (IJRR).
Download Paper