REKI-2B

REKI is a 2.4B vision-language-action model (VLA) with memory, pretrained on public data only.

• Memory at negligible cost. The memory gives the policy access to its recent past: 32 register tokens in the action expert summarize the output of the VLM at each frame and persist in the expert's cache. Reading them adds less than 1% to the time of predicting a chunk. Memory raises the success rate by 1.5 points on LIBERO-Plus, 5.5 on RoboTwin clean scenes and 5.4 on RoboCasa365, and makes oscillations between poses about four times rarer.
• Fast inference. With compilation, REKI predicts an action chunk in 21.8 ms on one H100 from two 256×256 camera images.
• Strong out-of-distribution generalization. Among models pretrained on public data, REKI has the highest scores on RoboTwin Clean2Rand (65.8) and LIBERO-Plus (90.4).
• Public data only. REKI is pretrained on public real-robot, simulation, human-hand and vision-language data only.

Left: REKI. The register keys and values of past frames are kept in a cache and read by self-attention. Right: one expert block. Action tokens read the current and past registers and themselves; registers never read them.

Model Details

• Developed by: Kyutai, Inria, École nationale des ponts et chaussées
• Model type: Vision-language-action model (flow matching)
• Parameters: 2.4B (2.2B VLM, 0.2B action expert)
• Base model: Qwen/Qwen3.5-2B
• License: CC BY-NC-SA 4.0 (weights), MIT (code)

Model Sources

• Code: github.com/kyutai-labs/reki
• Paper: REKI: Registering Knowledge in VLAs
• Project page: kyutai.org/reki

Architecture

Part	Value
VLM	Qwen3.5-2B, images at 448 px
Action expert	0.2B flow-matching transformer after the VLM, reads the final VLM layer by cross-attention
State	per-embodiment MLP into the expert's adaptive RMSNorm
Memory	32 register tokens per frame, cached across the last 3 frames (1 s apart)

Files

• model.safetensors: raw weights.
• ema.safetensors: fp32 EMA of the trainable parameters. RekiPolicy.from_pretrained overlays it (use_ema=False loads the raw weights). Finetuning starts from the raw model.safetensors.

Released models

Model	Robot	Benchmark
kyutai/reki-2b	pretraining mixture	—
kyutai/reki-2b-libero	Franka Panda	LIBERO, LIBERO-Plus
kyutai/reki-2b-robotwin-clean	AgileX ALOHA (bimanual)	RoboTwin Clean2Rand
kyutai/reki-2b-robotwin-cleanrand	AgileX ALOHA (bimanual)	RoboTwin 2.0, clean + randomized scenes
kyutai/reki-2b-robocasa365	PandaOmron (mobile)	RoboCasa365

Uses

• Finetuning on a benchmark or a new robot, with the finetuning recipe.
• Real-robot deployment of a finetuned model, over the LeRobot async-inference gRPC protocol.

How to use

See the README for installation. To finetune on a benchmark:

pixi run python launchers/sft.py table            # benchmarks and their settings
pixi run python launchers/sft.py select 0 submit  # e.g. LIBERO

Training

Data

The mixture holds 1.0B frames of real-robot teleoperation, simulation and egocentric human video, plus 12.2M vision-language samples.

Category	Datasets	Frames	Share
Real robots (teleoperation)	Open X-Embodiment (subset), BridgeData V2, DROID 1.0.1, AgiBot World, Galaxea Open-World, RoboMIND 1.2, rdt-ft-data, RoboCOIN (subset), ABC-130K, RoboSet-Teleoperation, Berkeley RPT	639.8M	51.0%
Simulation	GR00T X-Embodiment Sim (11 tasks), InternData-A1 (subset)	193.9M	10.3%
Human hands (egocentric video)	EgoDex, EgoVerse (Aria), Xperience-10M	170.9M	28.8%
Vision-language	FineVision (subset), LLaVA-OneVision-1.5 (subset), RoboPoint, RefSpatial, PixMo-Points (subset)	12.2M rows	10.0%

The robot sources are filtered before training: channel selection, degenerate episodes, sudden jumps, annotation review and convention correction. The robot data covers 14 state layouts, from single and dual arms to mobile manipulators and humanoids, each with its own state MLP. The text prompt gives the embodiment and the control mode. The data docs give the full mixture, the filters and the state layouts.

Training procedure

configs/cpt.yaml holds the hyperparameters.

Setting	Value
Initialization	VLM from Qwen/Qwen3.5-2B; action expert and state MLPs from scratch
Steps	200,000
Loss	flow matching, plus a cross-entropy term (weight 0.1) on subtask and vision-language text
GPUs	32 H100

Evaluation

Success rate of the finetuned models. The card of each finetuned model gives the protocol, the per-axis results and the evaluation commands.

Success rate (%)	Model	REKI
LIBERO (4 suites)	reki-2b-libero	97.6
LIBERO-Plus (7 perturbation axes)	reki-2b-libero	90.4
RoboTwin 2.0, Easy / Hard	reki-2b-robotwin-cleanrand	85.9 / 84.6
RoboTwin Clean2Rand (mean of 6 conditions)	reki-2b-robotwin-clean	65.8
RoboCasa365 (50 target tasks)	reki-2b-robocasa365	38.4

Limitations

• A new state layout needs a new state input layer, trained from scratch.
• Match the replan interval at inference to the training stride (history_stride_s, 1 s for all released models), so the past frames stay in distribution.
• The policy reflects its demonstration data, which is mostly tabletop manipulation with parallel-jaw grippers. Expect weaker behavior on other tasks or hands without finetuning.

Citation

@article{marrie2026reki,
  title   = {{REKI}: Registering Knowledge in {VLAs}},
  author  = {Marrie, Juliette and Pacaud, Paul and Ramanana Rahary, Adrien and Chen, Shizhe and B{\"o}hle, Moritz},
  journal = {arXiv:2610.22178},
  year    = {2026}
}