shrg7/molmobot-pi0-toys-reasoninit

MolmoBot-Pi0 — reason-init action policies on real-DROID toys

MolmoBot-Pi0 (π0 / PaliGemma-backbone) action policies fine-tuned on a real-DROID tabletop pick-and-place set. Each fine-tuned variant is initialized from a different reasoning-only LoRA baked into the backbone before action training.

subfolder	what it is
full/	reasoning init: all annotation modules
vqa_only/	reasoning init: free-text rationale only
trace_only/	reasoning init: end-effector trace only
merged_alpha0.5/	full/ interpolated halfway back toward the pretrained base
merged_alpha0.5_no_reasoning/	the same interpolation applied to a policy trained without any reasoning stage — the baseline for the merge

Training

Three sequential stages; reasoning is not present at inference:

1. Reasoning-only LoRA on the backbone, no action loss.
2. Merge the LoRA into the MolmoBot-Pi0-DROID checkpoint.
3. Action fine-tune with a flow-matching loss, vision tower frozen.

Action targets are absolute future joint positions (7 arm joints + gripper), horizon 16. Observations are two cameras, letterboxed to 224×224.

Weight interpolation (merged_alpha0.5/)

RETAIN / WiSE-FT style: the language-model weights are interpolated between the pretrained base and the fine-tuned policy, while the vision tower and action expert stay fully fine-tuned.

theta_llm = (1 - alpha) * base + alpha * finetuned      # alpha = 0.5

Interpolation traded terminal precision for smoother, better-scaled trajectories in our evaluation. It is a property of the merge itself and not of the reasoning initialization: the same effect appears when merging a policy trained without any reasoning stage, which is provided as merged_alpha0.5_no_reasoning/ so the two can be compared directly at matched alpha.

Contents

Each subfolder contains model.safetensors (flat π0 checkpoint), metadata.pt, and assets/ with the normalization statistics. Optimizer state is not included. The normalizer comes from the training set's statistics.

Limitations

Research checkpoints from an ablation study, released for reproducibility. They are trained on a single scene with one robot and are not tuned or benchmarked as a general-purpose policy; the three reasoning-init arms are not claimed to differ in downstream performance.