Hugging Face's logo

Octopus1
/
PaGE

Transformers
English
gaze-estimation
gaze-target-estimation
computer-vision
dinov3
Model card Files
xet
 Community
PaGE / README.md
Octopus1's picture
Octopus1
add arxiv link
7147c67 verified
preview code
|
Raw
History Blame Contribute Delete
8.02 kB
metadata 
license: mit
library_name: transformers
tags:
  - gaze-estimation
  - gaze-target-estimation
  - computer-vision
  - dinov3
language:
  - en

PaGE: Towards Practical Human-Level Gaze Target Estimation

Code: github.com/OctopusWen/PaGE  •  Project page: page-26.github.io  •  Paper: arXiv:2607.04860 •  Demo: page-crossgaze-page.hf.space

PaGE (Practical Gaze Estimator) is a gaze target estimation model that predicts where a person is looking in a scene. Gaze target estimation combines high-level understanding of global scene semantics with precise spatial reasoning from human appearance (pose, eye orientation). PaGE explicitly models the complex interaction between scene and head features, and achieves state-of-the-art performance, outperforming humans in 7 out of 9 metrics on GazeFollow, VideoAttentionTarget (VAT) and ChildPlay while reducing the human–AI gap by ≥60% on the remaining 2.

This repository holds the model code (modeling_page.py) referenced by all PaGE weight repositories via auto_map. The weight checkpoints live in their own repos (see Model Zoo).

Model Zoo

All checkpoints contain the full backbone weights in their safetensors files — no external DINOv3 weights are downloaded. The DINOv3 model structure is provided by transformers==5.6.2 (built-in dinov3_vit).

Model Backbone GFLOPs Weight repo
PaGE ViT-H+ DINOv3 ViT-H+ 2373.6 Octopus1/page-vithplus
PaGE ViT-B Distill DINOv3 ViT-B 283.1 Octopus1/page-vitb
PaGE ViT-B Distill (Screen) DINOv3 ViT-B 283.1 Octopus1/page-vitb-screen
PaGE ViT-S+ Distill DINOv3 ViT-S+ 115.2 Octopus1/page-vitsplus
PaGE ViT-S Distill DINOv3 ViT-S 96.9 Octopus1/page-vits

The ViT-H+ teacher is finetuned end-to-end; the student models are distilled from the teacher via token-level feature distillation on 1.17M unlabeled head crops, then finetuned on the labeled set.

Method

PaGE builds upon DINOv3 with a Scene-head Interaction Module (SIM) that uses cross-attention between scene and head branches to model inter-branch feature interaction in a ViT-native manner. Training follows a two-stage recipe: decoder-only training with a frozen backbone, followed by supervised finetuning (SFT) of the full model. Lightweight student models are trained via token-level feature distillation from a PaGE ViT-H+ teacher.

Architecture: two DINOv3 ViT backbones (scene @ 512², head @ 256²) → 1× self-attn each → 5× scene/head cross-attention interaction layers (axial 2D RoPE) → heatmap head (deconv + 1×1 conv)

  • in/out head (MLP on pooled scene+head inout tokens). Decoder dim 256, 8 heads, GEGLU FFN, 4 register tokens + 1 inout token per stream.

Results

Model GazeFollow AUC↑ GF Avg L2↓ GF Min L2↓ VAT AUC↑ VAT L2↓ VAT AP↑ ChildPlay AUC↑ ChildPlay L2↓ ChildPlay AP↑
PaGE ViT-S Distill 0.964 0.086 0.033 0.964 0.074 0.937 0.970 0.075 0.997
PaGE ViT-S+ Distill 0.965 0.086 0.033 0.965 0.074 0.939 0.970 0.075 0.997
PaGE ViT-B Distill 0.966 0.081 0.029 0.969 0.068 0.945 0.973 0.070 0.997
PaGE ViT-H+ 0.966 0.080 0.029 0.972 0.064 0.951 0.975 0.069 0.995
Human 0.924 0.096 0.040 0.921 0.051 0.925 0.911 0.048 0.993

All four PaGE models far outperform the previous SotA, with PaGE ViT-H+ and PaGE ViT-B Distill achieving human-level performance.

Installation 

pip install torch torchvision timm "transformers==5.6.2" safetensors pillow

Tested with transformers 5.6.2. The DINOv3 model structure ships built-in from transformers 4.56 onward; pinning to 5.6.2 is recommended for reproducibility.

Quick start 

from transformers import AutoModel, AutoImageProcessor
from PIL import Image
import torch

repo = "Octopus1/page-vitb"
model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval()
processor = AutoImageProcessor.from_pretrained(repo, trust_remote_code=True)

scene = Image.open("scene.jpg").convert("RGB")
head_crop = Image.open("head.jpg").convert("RGB")  # cropped head of the person

# bboxes: list (one per scene image) of bbox lists; bbox = (xmin, ymin, xmax, ymax) in [0,1]
inputs = processor(scene, head_crops=[head_crop], bboxes=[[(0.10, 0.10, 0.30, 0.40)]])

with torch.no_grad():
    out = model(inputs)

heatmap = out["heatmap"][0]   # [Np, 64, 64] gaze heatmap per person
inout   = out["inout"][0]     # [Np] in/out score per person

Inputs

The model's forward takes a dict:

  • images: list of scene tensors [B, 3, 512, 512]
  • head_images: list of head-crop tensors [sum(Np), 3, 256, 256] (one entry per backbone stream)
  • bboxes: list (len B) of lists of Np bboxes; each bbox is (xmin, ymin, xmax, ymax) in [0, 1] image coordinates

The PaGEImageProcessor (via AutoImageProcessor) builds this dict from a PIL scene image, per-person head crops, and bboxes.

Outputs

  • heatmap: list (len B) of [Np, 64, 64] tensors (sigmoid applied)
  • inout: list (len B) of [Np] tensors (sigmoid applied)

BibTeX 

@misc{ye2026pagepracticalhumanlevelgaze,
      title={PAGE: Towards Practical Human-level Gaze Target Estimation}, 
      author={Zhoutong Ye and Chengwen Zhang and Zhaibin Cui and Mingze Sun and Jiaqi Liu and Xiangwu Li and Qingyang Wan and Chang Liu and Xutong Wang and Huan-ang Gao and Yu Mei and Chun Yu and Yuanchun Shi},
      year={2026},
      eprint={2607.04860},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.04860}, 
}

License

  • The PaGE model code (modeling_page.py) and the PaGE-specific gaze decoder / heads are released under the MIT License (see LICENSE).
  • The DINOv3 backbones are derivative works of DINOv3 (facebook/dinov3). The DINOv3 ViT backbones were initialized from the publicly released DINOv3 self-supervised weights and then trained in full (all parameters updated) as part of PaGE training — i.e. the backbone weights shipped here are derivative weights produced by full-parameter training of DINOv3, not the original DINOv3 weights verbatim.

DINOv3 License

DINOv3 is released by Meta AI under the Meta DINO License (a custom, non-Apache license — see DINOv3_LICENSE.md). Under Section 1.b.i of that license, distribution of DINOv3 Materials and any derivative works thereof (which includes the DINOv3-derived backbone weights in these checkpoints) is subject to the DINO License terms, and a copy of the DINO License must be provided with any such distribution. Accordingly, DINOv3_LICENSE.md is included in every PaGE weight repository and in this code repository.

In summary:

  • The DINOv3-derived backbone portions of the checkpoints are governed by the Meta DINO License (DINOv3_LICENSE.md).
  • The PaGE decoder, gaze heads, and model code are additionally governed by the MIT License.
  • By using or redistributing these models you agree to be bound by the DINO License for the DINOv3-derived portions, and you must retain and provide DINOv3_LICENSE.md with any redistribution.

If you use these models, please also cite the DINOv3 work.

Project page: page-26.github.io · Code: github.com/OctopusWen/PaGE