SeeQ-3B

SeeQ: Training Generalist Value Functions for Long-Horizon Robotic Manipulation

Saksham Singh, Zheyuan Hu, Max Sobol Mark, Jeffrey Yu, Zackory Erickson, Aviral Kumar (Carnegie Mellon University)

Project website (with videos): https://saksham002.github.io/seeq/

Model description

SeeQ (Subtask-elicited Q-functions) is a generalist Q-value function for language-conditioned robot manipulation. Rather than modeling sparse success over the full task, SeeQ estimates the expected return for the currently active subtask, which shortens the value-prediction horizon and makes temporal-difference (TD) learning effective on long-horizon tasks. At inference time the model first autoregressively decodes the active subtask in natural language, then scores candidate action chunks conditioned on it. The resulting Q-values are used for best-of-N steering of a base policy (pi-0.5 in the paper).

This repository contains the RoboCOIN-pretrained generalist checkpoint (230k training steps). In the paper this checkpoint is fine-tuned on each downstream task before deployment.

Architecture

• Backbone: PaliGemma 3B (SigLIP image encoder + Gemma language model), initialized from the base PaliGemma weights.
• Inputs: 3 camera images (224x224), the task instruction, the active subtask text, and a 50-step, 14-dim bimanual action chunk (linearly projected into action tokens). Proprioceptive state is not an input.
• Image and task tokens attend bidirectionally. Subtask tokens are causal (trained with a next-token loss) and attend to the images, the task and preceding subtask text. Action tokens attend bidirectionally within the chunk and to all preceding image and language tokens.
• A learned value token at the end of the sequence attends to all valid preceding tokens; a linear readout maps it to a scalar Q-value.

Training

Setting	Value
Data	RoboCOIN bimanual subset: 131 tasks, ~40k episodes, ~31.3M steps, 3 embodiments (Agilex Cobot Magic, Agilex Split ALOHA, Galaxea R1 Lite)
Objective	Subtask-level TD-BoN (no bootstrapping across subtask boundaries) + subtask next-token loss (weight 0.1)
Backup candidates	N = 8 action chunks from pi-0.5
Discount	0.999
Action horizon	50 (14-dim, end-effector, chunk-wise delta, quantile-normalized)
Target network	Polyak averaging, tau = 0.005
Optimizer	AdamW, weight decay 1e-6
LR schedule	Cosine, 1000 warmup steps, peak 1e-5, decayed to 1e-6
Batch size / steps	256 / 230,000
Precision	float32

Training config: robocoin_bimanual_paligemma_cql_rlds_subtask_ar.

Files

This is an openpi-style Orbax checkpoint (JAX / Flax NNX):

• params/ — model parameters (Orbax PyTree checkpoint). Optimizer state is not included.
• assets/embodiment_wise/norm_stats.json — normalization statistics used during training; inputs must be normalized with these at inference time.
• _CHECKPOINT_METADATA — Orbax checkpoint metadata.

Intended use and limitations

• Intended as an initialization for fine-tuning a Q-function on a downstream bimanual manipulation task, and for research on value learning and policy steering.
• Values are subtask-level returns under the RoboCOIN data distribution; they are not calibrated task-success probabilities. Out of the box, results on new embodiments, camera setups or action spaces are not guaranteed; the paper fine-tunes on target-task data before deployment.
• This model scores actions; it does not generate them. It must be paired with a base policy that proposes candidate action chunks.

License

The weights are derived from PaliGemma and are subject to the Gemma Terms of Use.