More Works from Our Lab

Toward Zero-Shot Sim-to-Real Transfer Learning for Pneumatic Soft Robot 3D Proprioceptive Sensing

The authors present a simulation-to-real pipeline that trains vision-based proprioceptive shape reconstruction for a pneumatic soft robot entirely from synthetic point-cloud data and deploys it on real embedded camera images with promising accuracy.

IEEE RA-L, arXiv 2024
GelBelt: A Vision-based Tactile Sensor for Continuous Sensing of Large Surfaces

A novel tactile sensor using an elastomeric belt and two wheels that can continuously scan large surfaces with high accuracy; shows robust shape reconstruction at speeds up to ~45 mm/s. :contentReference[oaicite:1]{index=1}

IEEE RA-L, arXiv 2025
NormalFlow: Fast, Robust, and Accurate Contact-based Object 6DoF Pose Tracking with Vision-based Tactile Sensors

An algorithm for real-time 6-DoF pose tracking using vision-based tactile sensors via precise normal estimation; shows strong accuracy even on low-texture objects. Based on RA-L 2024.

IEEE RA-L, arXiv 2024

PneuGelSight: Soft Robotic Vision-Based Proprioception and Tactile Sensing

Ruohan Zhang, Uksang Yoo, Yichen Li, Arpit Argawal, Wenzhen Yuan
University of Illinois, Urbana-Champaign
The International Journal of Robotics Research, 2025
Paper Design and Simulation Files arXiv

PneuGelSight in action: a soft pneumatic manipulator equipped with high-resolution vision-based tactile sensing. The embedded camera reconstructs contact geometry in real time, enabling delicate and intelligent interaction with the physical world.

Abstract

Soft pneumatic robot manipulators are popular in industrial and human-interactive applications due to their compliance and flexibility. However, deploying them in real-world scenarios requires advanced sensing for tactile feedback and proprioception. Our work presents a novel vision-based approach for sensorizing soft robots. We demonstrate our approach on PneuGelSight, a pioneering pneumatic manipulator featuring high-resolution proprioception and tactile sensing via an embedded camera. To optimize the sensor's performance, we introduce a comprehensive pipeline that accurately simulates its optical and dynamic properties, facilitating a zero-shot knowledge transition from simulation to real-world applications. PneuGelSight and our sim-to-real pipeline provide a novel, easily implementable, and robust sensing methodology for soft robots, paving the way for the development of more advanced soft robots with enhanced sensory capabilities.

2025 ICRA Keynote Talk

Paper

BibTeX

@misc{zhang2025pneugelsightsoftroboticvisionbased,
      title={PneuGelSight: Soft Robotic Vision-Based Proprioception and Tactile Sensing}, 
      author={Ruohan Zhang and Uksang Yoo and Yichen Li and Arpit Argawal and Wenzhen Yuan},
      year={2025},
      eprint={2508.18443},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2508.18443}, 
}