| --- |
| title: ViT-Up Feature Upsampler |
| emoji: 🔼 |
| colorFrom: blue |
| colorTo: gray |
| sdk: gradio |
| sdk_version: "5.50.0" |
| app_file: app.py |
| short_description: DINOv3 feature upsampling with ViT-Up |
| python_version: "3.12" |
| startup_duration_timeout: "15m" |
| --- |
| |
| # ViT-Up: Faithful Feature Upsampling for Vision Transformers |
|
|
| This Space demonstrates **ViT-Up**, an implicit feature upsampler for Vision |
| Transformers that predicts backbone-aligned features at arbitrary continuous |
| image coordinates. |
|
|
| ## How it works |
|
|
| 1. **Input**: An image is padded to square, resized to 448×448, and normalised |
| with ImageNet statistics. |
| 2. **Backbone**: A DINOv3-S+ ViT backbone (loaded from the non-gated |
| `timm/vit_small_plus_patch16_dinov3.lvd1689m` mirror) extracts multi-layer |
| hidden states. LoRA adapters from the ViT-Up checkpoint are applied. |
| 3. **Upsampling**: ViT-Up queries features at a dense grid of user-selected |
| resolution (e.g. 112×112), producing high-resolution feature maps aligned |
| with the backbone. |
| 4. **Visualization**: The 3 principal components of the upsampled features are |
| projected to RGB via PCA, showing the semantic structure learned by ViT-Up. |
|
|
| ## Model |
|
|
| - **Paper**: [ViT-Up: Faithful Feature Upsampling for Vision Transformers](https://huggingface.co/papers/2606.14024) |
| - **Weights**: [Krispin/vit-up](https://huggingface.co/Krispin/vit-up) |
| - **Code**: [GitHub](https://github.com/krispinwandel/vit-up) |
| - **License**: CC-BY-NC-SA-4.0 |
