README.md

metadata

title: ViT-Up Feature Upsampler
emoji: 🔼
colorFrom: blue
colorTo: gray
sdk: gradio
sdk_version: 5.50.0
app_file: app.py
short_description: DINOv3 feature upsampling with ViT-Up
python_version: '3.12'
startup_duration_timeout: 15m

ViT-Up: Faithful Feature Upsampling for Vision Transformers

This Space demonstrates ViT-Up, an implicit feature upsampler for Vision Transformers that predicts backbone-aligned features at arbitrary continuous image coordinates.

How it works

1. Input: An image is padded to square, resized to 448×448, and normalised with ImageNet statistics.
2. Backbone: A DINOv3-S+ ViT backbone (loaded from the non-gated timm/vit_small_plus_patch16_dinov3.lvd1689m mirror) extracts multi-layer hidden states. LoRA adapters from the ViT-Up checkpoint are applied.
3. Upsampling: ViT-Up queries features at a dense grid of user-selected resolution (e.g. 112×112), producing high-resolution feature maps aligned with the backbone.
4. Visualization: The 3 principal components of the upsampled features are projected to RGB via PCA, showing the semantic structure learned by ViT-Up.

Model

• Paper: ViT-Up: Faithful Feature Upsampling for Vision Transformers
• Weights: Krispin/vit-up
• Code: GitHub
• License: CC-BY-NC-SA-4.0