TacImag

Imagining the Sense of Touch: Touch-Informed Manipulation via Imagined Tactile Representations

Zhiyuan Zhang*, Adeesh Mahesh Desai*, Jyun-Chi Hu, Yosuke Saka, Quan Khanh Luu, Jiuzhou Lei,
Davood Soleymanzadeh, Bihao Zhang, Minghui Zheng, Yu She†

Purdue University   |   Texas A&M University
* Equal Contribution   † Corresponding Author

arXiv Code (Coming Soon) Video
Learning from Touch | Deploying without Tactile Sensors | Vision-to-Tactile Imagination
TacImag teaser

TacImag predicts task-relevant tactile representations from vision, including TacFF (force-field tactile representation) for contact-sensitive tasks and TacRGB (tactile image representation) for texture-sensitive tasks. These imagined tactile signals enable touch-informed manipulation without physical tactile sensors at deployment.

Simulation Rollouts

TacImag improves contact-sensitive and texture-sensitive manipulation by generating imagined tactile representations according to the task requirement.

Contact-Sensitive Rollout

Texture-Sensitive Rollout

Method

TacImag uses a two-stage pipeline: first, a tactile imagination model learns to generate tactile observations from visual and proprioceptive inputs; second, a manipulation policy uses the imagined tactile representations as auxiliary observations during policy execution.

TacImag architecture

Tactile Imagination Process

The tactile imagination model progressively denoises latent tactile observations and produces task-relevant tactile representations online.

Real-World Validation

Representative real-world deployments. Click a task button to switch videos.

Whiteboard Wiping

Mechanism: Touch without Tactile Sensors

TacImag does not recover missing physical measurements directly; instead, imagined tactile observations provide contact-aware supervision that transforms subtle visual interaction cues into policy-friendly representations.

BibTeX

@article{zhang2026tacimag,
  title={TacImag: Touch-Informed Manipulation through Imagined Tactile Representations},
  author={Zhang, Zhiyuan and Desai, Adeesh Mahesh and Hu, Jyun-Chi and Saka, Yosuke and Luu, Quan Khanh and Lei, Jiuzhou and Soleymanzadeh, Davood and Zhang, Bihao and Zheng, Minghui and She, Yu},
  journal={arXiv preprint},
  year={2026}
}