File size: 1,453 Bytes
72b879b b06cf47 72b879b b06cf47 72b879b b06cf47 ef1a09f 72b879b b06cf47 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | ---
title: ViT-Up Feature Upsampler
emoji: 🔼
colorFrom: blue
colorTo: gray
sdk: gradio
sdk_version: "5.50.0"
app_file: app.py
short_description: DINOv3 feature upsampling with ViT-Up
python_version: "3.12"
startup_duration_timeout: "15m"
---
# ViT-Up: Faithful Feature Upsampling for Vision Transformers
This Space demonstrates **ViT-Up**, an implicit feature upsampler for Vision
Transformers that predicts backbone-aligned features at arbitrary continuous
image coordinates.
## How it works
1. **Input**: An image is padded to square, resized to 448×448, and normalised
with ImageNet statistics.
2. **Backbone**: A DINOv3-S+ ViT backbone (loaded from the non-gated
`timm/vit_small_plus_patch16_dinov3.lvd1689m` mirror) extracts multi-layer
hidden states. LoRA adapters from the ViT-Up checkpoint are applied.
3. **Upsampling**: ViT-Up queries features at a dense grid of user-selected
resolution (e.g. 112×112), producing high-resolution feature maps aligned
with the backbone.
4. **Visualization**: The 3 principal components of the upsampled features are
projected to RGB via PCA, showing the semantic structure learned by ViT-Up.
## Model
- **Paper**: [ViT-Up: Faithful Feature Upsampling for Vision Transformers](https://huggingface.co/papers/2606.14024)
- **Weights**: [Krispin/vit-up](https://huggingface.co/Krispin/vit-up)
- **Code**: [GitHub](https://github.com/krispinwandel/vit-up)
- **License**: CC-BY-NC-SA-4.0 |