SuperGlance-9B

Paper: arXiv:2611.19138 ยท PDF ยท Project page: guobapei.github.io/SuperGlance ยท Code: GuobaPei/SuperGlance

Other checkpoints: SuperGlance-9B-Stage1 ยท SuperGlance-9B-Stage2

Waymo Open Dataset notice. This model was made using the Waymo Open Dataset, provided by Waymo LLC under the Waymo Dataset License Agreement for Non-Commercial Use, available at waymo.com/open/terms, and your access and use of such work are governed by the terms and conditions therein. This model is released under that Agreement. Any further downstream use or modification of it is subject to the Terms of that Agreement, including its non-commercial restrictions. It may not be used in the operation of a vehicle, in production systems or for primarily commercial purposes.

The release terms of this repository are not decided yet; see Licence below.

SuperGlance is an end-to-end driving model. It reads the current front-camera frame and, when all three are available, the frames 1.5, 1.0 and 0.5 s earlier. The prompt gives the ego speed, acceleration, past 1.5 s of positions, the navigation command and any instruction. The model then scores a fixed library of 5,273 candidate trajectories. Each candidate is 8 future poses (x, y, heading). This repository holds the final model after the reinforcement-learning stage (RL), with its LoRA merged into the stage-2 model, SuperGlance-9B-Stage2. That model is Qwen3.5-9B after SuperGlance's two question-answering stages.

Files

file content
model.safetensors-*-of-00004.safetensors, model.safetensors.index.json, config.json, tokenizer and processor files the merged model (bf16), loads with AutoModelForImageTextToText
choice_head.safetensors the Policy Head: weight [5273, 4096], bias [5273]
candidates_k5273.npy the candidate library, [5273, 8, 3]: x, y (m) and heading (rad) of 8 future poses
SuperGlance.pdf the paper (the arXiv version)
extend_head.json how the head was extended from 5,121 to 5,273 candidates before RL
adapter/ RL's LoRA and Policy Head before the merge (the path the paper's numbers were read with). Its config names Qwen/Qwen3.5-9B as the architecture, but it applies to SuperGlance-9B-Stage2: load that model first and use it as the base
MERGE_INFO.json merge arithmetic: Wโ€ฒ = round_bf16(W + 2.0 ยท BยทA) for 200 matrices
LICENSE-Qwen3.5-9B the base model's licence (Apache-2.0), kept as its terms require

MERGE_INFO.json and extend_head.json use internal names: S1 is the last phase, RL' is RL, and the choice head is the Policy Head.

Readout

  1. h = the final hidden state at the last prompt position.
  2. Logits = hยทWแต€ + b over the 5,273 candidates.
  3. Plan = the softmax (ฯ„ = 0.2) over the 64 highest logits, as weights on those candidates' poses.

The code release has the prompt builder and the readout.

The merged weights and the adapter path are not bit-identical. On 16 navtest scenes:

  • the head is identical;
  • the top candidate agrees on 15 of 16 scenes;
  • the largest plan difference is 0.16 m in position and 0.009 rad in heading.

These differences come from rounding the merged weights to bf16.

Results (paper)

benchmark metric value
NAVSIM v1.1 navtest PDMS 91.51
WOD-E2E test RFS 7.987

Training

The base model is Qwen/Qwen3.5-9B (Apache-2.0). The training ran in this order:

  1. Stage 1. Multiple-choice QA, 1,158,150 questions.
  2. Stage 2. The stage-1 questions except text_nota and openimages_vertical (1,115,662), plus 124,378 DriveLM-nuScenes questions, giving SuperGlance-9B-Stage2.
  3. SFT. Supervised driving on NAVSIM navtrain (OpenScene / nuPlan), PhysicalAI-AV and WOD-E2E rows, with the evaluation splits removed.
  4. Last phase. A continuation on the same pool, with the WOD-E2E rows replaced by train-split frames 135โ€“165.
  5. RL. Reinforcement learning on the candidate library.

Licence

The release terms are pending. These are the terms of the training data, taken from each source's own page:

data terms
Qwen3.5-9B (base) Apache-2.0
WOD-E2E Waymo non-commercial licence; trained models may be shared under the conditions above
NAVSIM navtrain (OpenScene / nuPlan) CC BY-NC-SA 4.0 (OpenScene) plus the nuPlan / nuScenes terms of use
DriveLM-nuScenes CC BY-NC-SA 4.0, plus the nuScenes terms of use
PhysicalAI-AV NVIDIA Autonomous Vehicle Dataset License Agreement (internal development use; no derivative works or redistribution)
stage-1 QA sources see the SuperGlance-QA dataset card
Downloads last month
79
Safetensors
Model size
10B params
Tensor type
BF16
ยท
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for pei2333/SuperGlance-9B

Finetuned
Qwen/Qwen3.5-9B
Finetuned
(1)
this model

Paper for pei2333/SuperGlance-9B