| --- |
| license: apache-2.0 |
| tags: [molmo2, codec, adacodec, video, p-tokenizer] |
| --- |
| |
| # molmo2-codec Stage-1 (P-tokenizer) |
|
|
| Stage-1 pretrained **P-tokenizer** for the AdaCodec-on-molmo2 video pipeline |
| (codec-aligned video input: I-frames -> 81 tokens after 3x3 pool, P-frames -> ~5 tokens via |
| this P-tokenizer, ~4.2x token compression). |
|
|
| - `ptokenizer_step3999.pt` — final checkpoint (used by Stage-2 SFT via `CODEC_STAGE1_PTOK`). |
| - `ptokenizer_step{1000,2000,3000}.pt` — intermediate checkpoints. |
|
|
| Code: https://github.com/weikaih04/molmo2-codec (branch `adacodec`). Loaded into |
| `codec_ptok.proj` (the trainable connector) in `launch_scripts/train_codec_sft.py`. |
|
|
|
|
| ## V2 (2026-07-10) — `v2/` |
|
|
| - **`v2/ptokenizer_step3999.pt`** — **use this one.** N_P=8 (was 5), temporal embedding `e_t` |
| (paper `z_t^P = E_P(u_t) + e_t`), trained on paper-style CHAIN samples (GOP's I + P_1..P_n, |
| target = frame of P_n), motion search widened to ±16px (hierarchical). Probe (nextqa->mlvu |
| quick protocol): codec 37.5% vs dense 21.0% at 26.4% token budget. |
| - `v2/ptokenizer_step3999_bigbatch.pt` — same recipe at 4x batch (14 epochs): LOWER Stage-1 |
| loss (0.154 vs 0.25) but WORSE downstream probe (23.5%) — kept as an overfitting datapoint. |
| - V1 ckpts (repo root) are incompatible with the V2 config (query shape 5 vs 8). |
| |
