--- license: mit library_name: transformers tags: - gaze-estimation - gaze-target-estimation - computer-vision - dinov3 language: - en --- # PaGE: Towards Practical Human-Level Gaze Target Estimation **Code:** [github.com/OctopusWen/PaGE](https://github.com/OctopusWen/PaGE)  •  **Project page:** [page-26.github.io](https://page-26.github.io/)  •  **Paper:** [arXiv:2607.04860](https://arxiv.org/abs/2607.04860) •  **Demo:** [page-crossgaze-page.hf.space](https://page-crossgaze-page.hf.space/) PaGE (Practical Gaze Estimator) is a gaze target estimation model that predicts where a person is looking in a scene. Gaze target estimation combines high-level understanding of global scene semantics with precise spatial reasoning from human appearance (pose, eye orientation). PaGE explicitly models the complex interaction between scene and head features, and achieves state-of-the-art performance, **outperforming humans in 7 out of 9 metrics** on GazeFollow, VideoAttentionTarget (VAT) and ChildPlay while reducing the human–AI gap by ≥60% on the remaining 2. This repository holds the **model code** (`modeling_page.py`) referenced by all PaGE weight repositories via `auto_map`. The weight checkpoints live in their own repos (see Model Zoo). ## Model Zoo All checkpoints contain the **full backbone weights** in their `safetensors` files — no external DINOv3 weights are downloaded. The DINOv3 model *structure* is provided by `transformers==5.6.2` (built-in `dinov3_vit`). | Model | Backbone | GFLOPs | Weight repo | |-------|----------|--------|-------------| | PaGE ViT-H+ | DINOv3 ViT-H+ | 2373.6 | [`Octopus1/page-vithplus`](https://huggingface.co/Octopus1/page-vithplus) | | PaGE ViT-B Distill | DINOv3 ViT-B | 283.1 | [`Octopus1/page-vitb`](https://huggingface.co/Octopus1/page-vitb) | | PaGE ViT-B Distill (Screen) | DINOv3 ViT-B | 283.1 | [`Octopus1/page-vitb-screen`](https://huggingface.co/Octopus1/page-vitb-screen) | | PaGE ViT-S+ Distill | DINOv3 ViT-S+ | 115.2 | [`Octopus1/page-vitsplus`](https://huggingface.co/Octopus1/page-vitsplus) | | PaGE ViT-S Distill | DINOv3 ViT-S | 96.9 | [`Octopus1/page-vits`](https://huggingface.co/Octopus1/page-vits) | The ViT-H+ teacher is finetuned end-to-end; the student models are distilled from the teacher via token-level feature distillation on 1.17M unlabeled head crops, then finetuned on the labeled set. ## Method PaGE builds upon DINOv3 with a **Scene-head Interaction Module (SIM)** that uses cross-attention between scene and head branches to model inter-branch feature interaction in a ViT-native manner. Training follows a two-stage recipe: decoder-only training with a frozen backbone, followed by supervised finetuning (SFT) of the full model. Lightweight student models are trained via token-level feature distillation from a PaGE ViT-H+ teacher. Architecture: two DINOv3 ViT backbones (scene @ 512², head @ 256²) → 1× self-attn each → 5× scene/head cross-attention interaction layers (axial 2D RoPE) → heatmap head (deconv + 1×1 conv) + in/out head (MLP on pooled scene+head inout tokens). Decoder dim 256, 8 heads, GEGLU FFN, 4 register tokens + 1 inout token per stream. ## Results | Model | GazeFollow AUC↑ | GF Avg L2↓ | GF Min L2↓ | VAT AUC↑ | VAT L2↓ | VAT AP↑ | ChildPlay AUC↑ | ChildPlay L2↓ | ChildPlay AP↑ | |-------|------|------|------|------|------|------|------|------|------| | PaGE ViT-S Distill | 0.964 | 0.086 | 0.033 | 0.964 | 0.074 | 0.937 | 0.970 | 0.075 | 0.997 | | PaGE ViT-S+ Distill | 0.965 | 0.086 | 0.033 | 0.965 | 0.074 | 0.939 | 0.970 | 0.075 | 0.997 | | PaGE ViT-B Distill | 0.966 | 0.081 | 0.029 | 0.969 | 0.068 | 0.945 | 0.973 | 0.070 | 0.997 | | PaGE ViT-H+ | 0.966 | 0.080 | 0.029 | 0.972 | 0.064 | 0.951 | 0.975 | 0.069 | 0.995 | | Human | 0.924 | 0.096 | 0.040 | 0.921 | 0.051 | 0.925 | 0.911 | 0.048 | 0.993 | All four PaGE models far outperform the previous SotA, with PaGE ViT-H+ and PaGE ViT-B Distill achieving human-level performance. ## Installation ```bash pip install torch torchvision timm "transformers==5.6.2" safetensors pillow ``` Tested with `transformers` 5.6.2. The DINOv3 model structure ships built-in from transformers 4.56 onward; pinning to 5.6.2 is recommended for reproducibility. ## Quick start ```python from transformers import AutoModel, AutoImageProcessor from PIL import Image import torch repo = "Octopus1/page-vitb" model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval() processor = AutoImageProcessor.from_pretrained(repo, trust_remote_code=True) scene = Image.open("scene.jpg").convert("RGB") head_crop = Image.open("head.jpg").convert("RGB") # cropped head of the person # bboxes: list (one per scene image) of bbox lists; bbox = (xmin, ymin, xmax, ymax) in [0,1] inputs = processor(scene, head_crops=[head_crop], bboxes=[[(0.10, 0.10, 0.30, 0.40)]]) with torch.no_grad(): out = model(inputs) heatmap = out["heatmap"][0] # [Np, 64, 64] gaze heatmap per person inout = out["inout"][0] # [Np] in/out score per person ``` ## Inputs The model's `forward` takes a dict: - `images`: list of scene tensors `[B, 3, 512, 512]` - `head_images`: list of head-crop tensors `[sum(Np), 3, 256, 256]` (one entry per backbone stream) - `bboxes`: list (len `B`) of lists of `Np` bboxes; each bbox is `(xmin, ymin, xmax, ymax)` in `[0, 1]` image coordinates The `PaGEImageProcessor` (via `AutoImageProcessor`) builds this dict from a PIL scene image, per-person head crops, and bboxes. ## Outputs - `heatmap`: list (len `B`) of `[Np, 64, 64]` tensors (sigmoid applied) - `inout`: list (len `B`) of `[Np]` tensors (sigmoid applied) ## BibTeX ```bibtex @misc{ye2026pagepracticalhumanlevelgaze, title={PAGE: Towards Practical Human-level Gaze Target Estimation}, author={Zhoutong Ye and Chengwen Zhang and Zhaibin Cui and Mingze Sun and Jiaqi Liu and Xiangwu Li and Qingyang Wan and Chang Liu and Xutong Wang and Huan-ang Gao and Yu Mei and Chun Yu and Yuanchun Shi}, year={2026}, eprint={2607.04860}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2607.04860}, } ``` ## License - The PaGE model code (`modeling_page.py`) and the PaGE-specific gaze decoder / heads are released under the **MIT License** (see `LICENSE`). - The **DINOv3 backbones** are **derivative works of DINOv3** ([facebook/dinov3](https://huggingface.co/facebook/dinov3)). The DINOv3 ViT backbones were initialized from the publicly released DINOv3 self-supervised weights and then **trained in full (all parameters updated)** as part of PaGE training — i.e. the backbone weights shipped here are **derivative weights produced by full-parameter training of DINOv3**, not the original DINOv3 weights verbatim. ### DINOv3 License DINOv3 is released by Meta AI under the **Meta DINO License** (a custom, non-Apache license — see `DINOv3_LICENSE.md`). Under Section 1.b.i of that license, distribution of DINOv3 Materials and **any derivative works thereof** (which includes the DINOv3-derived backbone weights in these checkpoints) is subject to the DINO License terms, and **a copy of the DINO License must be provided with any such distribution**. Accordingly, `DINOv3_LICENSE.md` is included in every PaGE weight repository and in this code repository. In summary: - The DINOv3-derived backbone portions of the checkpoints are governed by the **Meta DINO License** (`DINOv3_LICENSE.md`). - The PaGE decoder, gaze heads, and model code are additionally governed by the **MIT License**. - By using or redistributing these models you agree to be bound by the DINO License for the DINOv3-derived portions, and you must retain and provide `DINOv3_LICENSE.md` with any redistribution. If you use these models, please also cite the DINOv3 work. Project page: [page-26.github.io](https://page-26.github.io/) · Code: [github.com/OctopusWen/PaGE](https://github.com/OctopusWen/PaGE)