README.md

metadata

license: apache-2.0
tags:
  - molmo2
  - codec
  - adacodec
  - video
  - p-tokenizer

molmo2-codec Stage-1 (P-tokenizer)

Stage-1 pretrained P-tokenizer for the AdaCodec-on-molmo2 video pipeline (codec-aligned video input: I-frames -> 81 tokens after 3x3 pool, P-frames -> ~5 tokens via this P-tokenizer, ~4.2x token compression).

• ptokenizer_step3999.pt — final checkpoint (used by Stage-2 SFT via CODEC_STAGE1_PTOK).
• ptokenizer_step{1000,2000,3000}.pt — intermediate checkpoints.

Code: https://github.com/weikaih04/molmo2-codec (branch adacodec). Loaded into codec_ptok.proj (the trainable connector) in launch_scripts/train_codec_sft.py.

V2 (2026-07-10) — v2/

• v2/ptokenizer_step3999.pt — use this one. N_P=8 (was 5), temporal embedding e_t (paper z_t^P = E_P(u_t) + e_t), trained on paper-style CHAIN samples (GOP's I + P_1..P_n, target = frame of P_n), motion search widened to ±16px (hierarchical). Probe (nextqa->mlvu quick protocol): codec 37.5% vs dense 21.0% at 26.4% token budget.
• v2/ptokenizer_step3999_bigbatch.pt — same recipe at 4x batch (14 epochs): LOWER Stage-1 loss (0.154 vs 0.25) but WORSE downstream probe (23.5%) — kept as an overfitting datapoint.
• V1 ckpts (repo root) are incompatible with the V2 config (query shape 5 vs 8).