README.md

---
license: mit
library_name: transformers
tags:
  - gaze-estimation
  - gaze-target-estimation
  - computer-vision
  - dinov3
language:
  - en
---

# PaGE: Towards Practical Human-Level Gaze Target Estimation

**Code:** [github.com/OctopusWen/PaGE](https://github.com/OctopusWen/PaGE)  • 
**Project page:** [page-26.github.io](https://page-26.github.io/)  • 
**Paper:** [arXiv:2607.04860](https://arxiv.org/abs/2607.04860) • 
**Demo:** [page-crossgaze-page.hf.space](https://page-crossgaze-page.hf.space/)

PaGE (Practical Gaze Estimator) is a gaze target estimation model that predicts where a person
is looking in a scene. Gaze target estimation combines high-level understanding of global scene
semantics with precise spatial reasoning from human appearance (pose, eye orientation). PaGE
explicitly models the complex interaction between scene and head features, and achieves
state-of-the-art performance, **outperforming humans in 7 out of 9 metrics** on GazeFollow,
VideoAttentionTarget (VAT) and ChildPlay while reducing the human–AI gap by ≥60% on the remaining 2.

This repository holds the **model code** (`modeling_page.py`) referenced by all PaGE weight
repositories via `auto_map`. The weight checkpoints live in their own repos (see Model Zoo).

## Model Zoo

All checkpoints contain the **full backbone weights** in their `safetensors` files — no external
DINOv3 weights are downloaded. The DINOv3 model *structure* is provided by `transformers==5.6.2`
(built-in `dinov3_vit`).

| Model | Backbone | GFLOPs  | Weight repo |
|-------|----------|--------|-------------|
| PaGE ViT-H+ | DINOv3 ViT-H+ | 2373.6  | [`Octopus1/page-vithplus`](https://huggingface.co/Octopus1/page-vithplus) |
| PaGE ViT-B Distill | DINOv3 ViT-B | 283.1  | [`Octopus1/page-vitb`](https://huggingface.co/Octopus1/page-vitb) |
| PaGE ViT-B Distill (Screen) | DINOv3 ViT-B | 283.1  | [`Octopus1/page-vitb-screen`](https://huggingface.co/Octopus1/page-vitb-screen) |
| PaGE ViT-S+ Distill | DINOv3 ViT-S+ | 115.2  | [`Octopus1/page-vitsplus`](https://huggingface.co/Octopus1/page-vitsplus) |
| PaGE ViT-S Distill | DINOv3 ViT-S | 96.9  | [`Octopus1/page-vits`](https://huggingface.co/Octopus1/page-vits) |

The ViT-H+ teacher is finetuned end-to-end; the student models are distilled from the teacher via
token-level feature distillation on 1.17M unlabeled head crops, then finetuned on the labeled set.

## Method

PaGE builds upon DINOv3 with a **Scene-head Interaction Module (SIM)** that uses cross-attention
between scene and head branches to model inter-branch feature interaction in a ViT-native manner.
Training follows a two-stage recipe: decoder-only training with a frozen backbone, followed by
supervised finetuning (SFT) of the full model. Lightweight student models are trained via
token-level feature distillation from a PaGE ViT-H+ teacher.

Architecture: two DINOv3 ViT backbones (scene @ 512², head @ 256²) → 1× self-attn each → 5×
scene/head cross-attention interaction layers (axial 2D RoPE) → heatmap head (deconv + 1×1 conv)
+ in/out head (MLP on pooled scene+head inout tokens). Decoder dim 256, 8 heads, GEGLU FFN,
4 register tokens + 1 inout token per stream.

## Results

| Model | GazeFollow AUC↑ | GF Avg L2↓ | GF Min L2↓ | VAT AUC↑ | VAT L2↓ | VAT AP↑ | ChildPlay AUC↑ | ChildPlay L2↓ | ChildPlay AP↑ |
|-------|------|------|------|------|------|------|------|------|------|
| PaGE ViT-S Distill | 0.964 | 0.086 | 0.033 | 0.964 | 0.074 | 0.937 | 0.970 | 0.075 | 0.997 |
| PaGE ViT-S+ Distill | 0.965 | 0.086 | 0.033 | 0.965 | 0.074 | 0.939 | 0.970 | 0.075 | 0.997 |
| PaGE ViT-B Distill | 0.966 | 0.081 | 0.029 | 0.969 | 0.068 | 0.945 | 0.973 | 0.070 | 0.997 |
| PaGE ViT-H+ | 0.966 | 0.080 | 0.029 | 0.972 | 0.064 | 0.951 | 0.975 | 0.069 | 0.995 |
| Human | 0.924 | 0.096 | 0.040 | 0.921 | 0.051 | 0.925 | 0.911 | 0.048 | 0.993 |

All four PaGE models far outperform the previous SotA, with PaGE ViT-H+ and PaGE ViT-B Distill
achieving human-level performance.

## Installation

```bash
pip install torch torchvision timm "transformers==5.6.2" safetensors pillow
```

Tested with `transformers` 5.6.2. The DINOv3 model structure ships built-in from transformers
4.56 onward; pinning to 5.6.2 is recommended for reproducibility.

## Quick start

```python
from transformers import AutoModel, AutoImageProcessor
from PIL import Image
import torch

repo = "Octopus1/page-vitb"
model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval()
processor = AutoImageProcessor.from_pretrained(repo, trust_remote_code=True)

scene = Image.open("scene.jpg").convert("RGB")
head_crop = Image.open("head.jpg").convert("RGB")  # cropped head of the person

# bboxes: list (one per scene image) of bbox lists; bbox = (xmin, ymin, xmax, ymax) in [0,1]
inputs = processor(scene, head_crops=[head_crop], bboxes=[[(0.10, 0.10, 0.30, 0.40)]])

with torch.no_grad():
    out = model(inputs)

heatmap = out["heatmap"][0]   # [Np, 64, 64] gaze heatmap per person
inout   = out["inout"][0]     # [Np] in/out score per person
```

## Inputs

The model's `forward` takes a dict:

- `images`: list of scene tensors `[B, 3, 512, 512]`
- `head_images`: list of head-crop tensors `[sum(Np), 3, 256, 256]` (one entry per backbone stream)
- `bboxes`: list (len `B`) of lists of `Np` bboxes; each bbox is `(xmin, ymin, xmax, ymax)` in `[0, 1]` image coordinates

The `PaGEImageProcessor` (via `AutoImageProcessor`) builds this dict from a PIL scene image,
per-person head crops, and bboxes.

## Outputs

- `heatmap`: list (len `B`) of `[Np, 64, 64]` tensors (sigmoid applied)
- `inout`: list (len `B`) of `[Np]` tensors (sigmoid applied)

## BibTeX

```bibtex
@misc{ye2026pagepracticalhumanlevelgaze,
      title={PAGE: Towards Practical Human-level Gaze Target Estimation}, 
      author={Zhoutong Ye and Chengwen Zhang and Zhaibin Cui and Mingze Sun and Jiaqi Liu and Xiangwu Li and Qingyang Wan and Chang Liu and Xutong Wang and Huan-ang Gao and Yu Mei and Chun Yu and Yuanchun Shi},
      year={2026},
      eprint={2607.04860},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.04860}, 
}
```

## License

- The PaGE model code (`modeling_page.py`) and the PaGE-specific gaze decoder / heads are released
  under the **MIT License** (see `LICENSE`).
- The **DINOv3 backbones** are **derivative works of DINOv3**
  ([facebook/dinov3](https://huggingface.co/facebook/dinov3)). The DINOv3 ViT backbones were
  initialized from the publicly released DINOv3 self-supervised weights and then **trained in full
  (all parameters updated)** as part of PaGE training — i.e. the backbone weights shipped here are
  **derivative weights produced by full-parameter training of DINOv3**, not the original DINOv3
  weights verbatim.

### DINOv3 License

DINOv3 is released by Meta AI under the **Meta DINO License** (a custom, non-Apache license — see
`DINOv3_LICENSE.md`). Under Section 1.b.i of that license, distribution of DINOv3 Materials and
**any derivative works thereof** (which includes the DINOv3-derived backbone weights in these
checkpoints) is subject to the DINO License terms, and **a copy of the DINO License must be
provided with any such distribution**. Accordingly, `DINOv3_LICENSE.md` is included in every PaGE
weight repository and in this code repository.

In summary:

- The DINOv3-derived backbone portions of the checkpoints are governed by the **Meta DINO License**
  (`DINOv3_LICENSE.md`).
- The PaGE decoder, gaze heads, and model code are additionally governed by the **MIT License**.
- By using or redistributing these models you agree to be bound by the DINO License for the
  DINOv3-derived portions, and you must retain and provide `DINOv3_LICENSE.md` with any
  redistribution.

If you use these models, please also cite the DINOv3 work.

Project page: [page-26.github.io](https://page-26.github.io/) · Code: [github.com/OctopusWen/PaGE](https://github.com/OctopusWen/PaGE)