Improve model card and add metadata

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +42 -3
README.md CHANGED
@@ -1,4 +1,43 @@
1
- Official checkpoints and evaluation masks for <br>
2
- *Seeing Through Touch: Tactile-Driven Visual Localization of Material Regions (CVPR 2026)*.
 
3
 
4
- Please refer to [project page](https://mm.kaist.ac.kr/projects/SeeingThroughTouch/) for details.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: image-segmentation
3
+ ---
4
 
5
+ # Seeing Through Touch: Tactile-Driven Visual Localization of Material Regions
6
+
7
+ Official checkpoints and evaluation masks for the paper **Seeing Through Touch: Tactile-Driven Visual Localization of Material Regions**, presented at CVPR 2026.
8
+
9
+ [**Project Page**](https://mm.kaist.ac.kr/projects/SeeingThroughTouch/) | [**Paper**](https://huggingface.co/papers/2604.11579) | [**GitHub**](https://github.com/kaistmm/SeeingThroughTouch)
10
+
11
+ ## Introduction
12
+
13
+ Tactile localization aims to identify image regions that share the same material properties as a tactile input. This model learns local visuo-tactile alignment via dense cross-modal feature interactions, producing tactile saliency maps for touch-conditioned material segmentation.
14
+
15
+ To overcome dataset constraints, the authors introduced in-the-wild multi-material scene images and a material-diversity pairing strategy to improve contextual localization and robustness.
16
+
17
+ ## Evaluation and Training
18
+
19
+ For detailed instructions on environment setup, data preparation, evaluation, and training, please refer to the [official GitHub repository](https://github.com/kaistmm/SeeingThroughTouch).
20
+
21
+ ### Available Checkpoints
22
+
23
+ The repository contains the following checkpoints:
24
+ - `STT.pth`: Main SeeingThroughTouch (STT) model.
25
+ - `STT-Local.pth`: Local-only variant.
26
+ - `STT-Indomain.pth`: In-domain trained variant.
27
+
28
+ ## Citation
29
+
30
+ If you find this work useful, please cite it as:
31
+
32
+ ```bibtex
33
+ @inproceedings{kim2026seeingthroughtouch,
34
+ author = {Seongyu Kim and Seungwoo Lee and Hyeonggon Ryu and Joon Son Chung and Arda Senocak},
35
+ title = {Seeing Through Touch: Tactile-Driven Visual Localization of Material Regions},
36
+ booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
37
+ year = {2026},
38
+ }
39
+ ```
40
+
41
+ ## Acknowledgments
42
+
43
+ This codebase is built upon [TVL (ICML2024)](https://github.com/Max-Fu/tvl).