File size: 4,068 Bytes
6c03241 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ---
pipeline_tag: image-segmentation
library_name: pytorch
license: apache-2.0
tags:
- transformer
- transparent-object-segmentation
- semantic-segmentation
- navigation-assistance
- iccvw-2021
- t-its-2022
- trans10k
- cocostuff
datasets:
- cocostuff
---
# Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation
Official model checkpoints for **Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World** (ICCVW 2021) and **Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance** (IEEE T-ITS 2022).
<div align="center">
[](https://arxiv.org/abs/2107.03172)
[](https://arxiv.org/abs/2108.09174)
[](https://github.com/InSAI-Lab/Trans4trans)
[](https://huggingface.co/collections/InSAI-Lab)
</div>
---
## 📌 Introduction
Trans4Trans is an efficient Transformer-based system designed for transparent object and semantic scene segmentation, targeting real-world wearable navigation assistance for visually impaired people. The architecture couples a lightweight hierarchical transformer backbone with a Frequency-aware Parsing Transformer (FPT) decoder and an assistive system integrating Intel RealSense depth sensing and acoustic feedback.
---
## 📂 Model Checkpoints
The repository contains pretrained weights organized by dataset benchmark:
```text
InSAI-Lab/Trans4trans
├── cocostuff/
│ └── model_cocostuff.pth # General scene context (COCO-Stuff, 162.6 MB)
└── trans10kv2/
└── model_trans.pth # Transparent object segmentation (Trans10K-v2, 150.3 MB)
```
| Checkpoint | Target Task / Dataset | Size | Direct Download |
| :--- | :--- | :---: | :--- |
| `cocostuff/model_cocostuff.pth` | COCO-Stuff General Context | 162.6 MB | [Download](https://huggingface.co/InSAI-Lab/Trans4trans/resolve/main/cocostuff/model_cocostuff.pth) |
| `trans10kv2/model_trans.pth` | Trans10K-v2 Transparent Objects | 150.3 MB | [Download](https://huggingface.co/InSAI-Lab/Trans4trans/resolve/main/trans10kv2/model_trans.pth) |
---
## 🚀 Quick Start
### Download via Hugging Face CLI
```bash
# Download all checkpoints into workdirs/
hf download InSAI-Lab/Trans4trans --local-dir workdirs
```
### Download via Python (`huggingface_hub`)
```python
from huggingface_hub import hf_hub_download
# Download transparent object segmentation model
ckpt_path = hf_hub_download(
repo_id="InSAI-Lab/Trans4trans",
filename="trans10kv2/model_trans.pth",
local_dir="workdirs"
)
print(f"Downloaded model to {ckpt_path}")
```
### Assistive System Demo
```bash
# Save weights under workdirs/cocostuff and workdirs/trans10kv2
bash demo.sh
```
---
## 📑 Citation
```bibtex
@inproceedings{zhang2021trans4trans,
title={Trans4Trans: Efficient transformer for transparent object segmentation to help visually impaired people navigate in the real world},
author={Zhang, Jiaming and Yang, Kailun and Constantinescu, Angela and Peng, Kunyu and M{"u}ller, Karin and Stiefelhagen, Rainer},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops},
pages={1760--1770},
year={2021}
}
@article{zhang2022trans4trans,
title={Trans4Trans: Efficient transformer for transparent object and semantic scene segmentation in real-world navigation assistance},
author={Zhang, Jiaming and Yang, Kailun and Constantinescu, Angela and Peng, Kunyu and M{"u}ller, Karin and Stiefelhagen, Rainer},
journal={IEEE Transactions on Intelligent Transportation Systems},
year={2022},
publisher={IEEE}
}
```
|