The Generator Is the Tracker: LoRA checkpoints
In-context LoRA adapters for LTX-2.3 that translate an RGB clip into an ID-map clip: every person painted a flat, distinct colour that stays the same over time (same colour, same person). Chained over a whole video, the generated colours are the tracks: there is no separately trained detector, motion model or association step.
Paper: arXiv:2607.17120 · Code: github.com/Sonam525/Generator-as-the-tracker · Project page: sonam525.github.io/Generator-as-the-tracker
Files
| File | Dataset | Training phase | Used in the paper for |
|---|---|---|---|
dancetrack_phaseB_continuation_step2500.safetensors |
DanceTrack | B: continuation (prefix-trained), step 2,500 | The chained tracker: test-server entry (40.3 HOTA), factorial cells A and D, re-acquisition |
dancetrack_phaseA_translation_step1500.safetensors |
DanceTrack | A: translation only, step 1,500 | Factorial cells B and C (the controls) |
sportsmot_phaseB_continuation_step1500.safetensors |
SportsMOT | B, step 1,500 | SportsMOT validation results |
sportsmot_phaseA_translation_step750.safetensors |
SportsMOT | A, step 750 | SportsMOT controls |
Each file is a rank-32 LoRA (alpha 32) on the video-stream attention (QKV, output) and
feed-forward projections of LTX-2.3 22B dev (ltx-2.3-22b-dev.safetensors); the audio stream
is untouched. Checkpoints were selected by validation foreground IoU every 500 steps (DanceTrack)
or 750 steps (SportsMOT). SHA-256 checksums are in SHA256SUMS.
Use
The adapters are in the format of the LTX-2 trainer
(IC-LoRA). Generation settings, the fixed prompt, the decode rules and the full chaining
procedure (49-frame windows at stride 40, 9-frame prefix from the cleaned tail of the previous
window) are in the paper's Appendix A and in video_generator/ of the code repository
(ltx_run_chain.py with ltx_chain.py). Decode the generated ID-maps with the repository's
codec (core/gentrack); the palette index is the track id.
Limits
About 0.4 frames per second end to end on one A100; detection is the weak side (test DetA 37.6), worst in crowds. See the paper's limitations section.
Licence and data
These adapters are derivatives of LTX-2.3 and are subject to the LTX-2 Community License Agreement. They were trained on DanceTrack and SportsMOT, which are licensed for non-commercial research; use these checkpoints for non-commercial research only. Silhouette training labels were made with SAM 3.
Citation
@article{yang2026generator,
title = {The generator is the tracker: Multi-object tracking by painting persistent
identity colours},
author = {Yang, Haiyu and Hostens, Miel},
journal = {arXiv preprint arXiv:2607.17120},
year = {2026},
doi = {10.48550/arXiv.2607.17120}
}
Model tree for Sonam5/Generator-as-the-tracker
Base model
Lightricks/LTX-2.3