README.md

---
license: mit
library_name: transformers
tags:
  - gaze-estimation
  - gaze-target-estimation
  - dinov3
pipeline_tag: image-feature-extraction
---

# PaGE ViT-H+

Huge+ flagship teacher; finetuned end-to-end. Part of the [PaGE](https://huggingface.co/Octopus1/PaGE) gaze target estimation family.

- **Backbone:** DINOv3 ViT-H+ (gated MLP) (derivative of DINOv3, full-parameter trained)
- **Params:** ~1.7B
- **Scene input:** 512×512, **Head input:** 256×256, **Heatmap output:** 64×64
- **Source checkpoint:** `vithplus_ft.pt`

## Self-contained weights

This checkpoint includes the full DINOv3 backbone weights in its `safetensors` files. **No external
DINOv3 weights are downloaded** — the DINOv3 model *structure* is provided by `transformers==5.6.2`
(built-in `dinov3_vit`), and the backbone weights here are derivative weights from full-parameter
training of DINOv3. The model code (`modeling_page.py`) is loaded automatically from
[`Octopus1/PaGE`](https://huggingface.co/Octopus1/PaGE) via `auto_map` when you pass
`trust_remote_code=True`.

## Installation

```bash
pip install torch torchvision timm "transformers==5.6.2" safetensors pillow
```

Tested with `transformers` 5.6.2.

## Usage

```python
from transformers import AutoModel, AutoImageProcessor
from PIL import Image
import torch

repo = "Octopus1/page-vithplus"
model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval()
processor = AutoImageProcessor.from_pretrained(repo, trust_remote_code=True)

scene = Image.open("scene.jpg").convert("RGB")
head = Image.open("head.jpg").convert("RGB")

inputs = processor(scene, head_crops=[head], bboxes=[[(0.10, 0.10, 0.30, 0.40)]])

with torch.no_grad():
    out = model(inputs)

heatmap = out["heatmap"][0]   # [Np, 64, 64]
inout   = out["inout"][0]     # [Np]
```

## Inputs / Outputs

See the family [README](https://huggingface.co/Octopus1/PaGE) for the full spec.

- Input dict: `images` (list of `[B,3,512,512]`), `head_images` (list of `[sum(Np),3,256,256]`),
  `bboxes` (per-image list of `(xmin,ymin,xmax,ymax)` in `[0,1]`).
- Output dict: `heatmap` (list of `[Np,64,64]`, sigmoid), `inout` (list of `[Np]`, sigmoid).

## License

- The PaGE decoder and gaze heads are released under the **MIT License** (see `LICENSE`).
- The **DINOv3 backbone is a derivative work of DINOv3** ([facebook/dinov3](https://huggingface.co/facebook/dinov3)).
  The backbone was initialized from the public DINOv3 self-supervised weights and then **trained in
  full (all parameters updated)** as part of PaGE training — i.e. the backbone weights here are
  **derivative weights produced by full-parameter training of DINOv3**, not the original DINOv3
  weights verbatim.
- DINOv3 is released by Meta AI under the **Meta DINO License** (see `DINOv3_LICENSE.md`). Under its
  Section 1.b.i, derivative works of DINOv3 (including these backbone weights) are distributed under
  the DINO License terms, and `DINOv3_LICENSE.md` must accompany any redistribution.
- By using or redistributing this model you agree to the DINO License for the DINOv3-derived portions.

See the family [README](https://huggingface.co/Octopus1/PaGE) for full license details.