The Generator Is the Tracker: LoRA checkpoints

In-context LoRA adapters for LTX-2.3 that translate an RGB clip into an ID-map clip: every person painted a flat, distinct colour that stays the same over time (same colour, same person). Chained over a whole video, the generated colours are the tracks: there is no separately trained detector, motion model or association step.

Paper: arXiv:2607.17120 · Code: github.com/Sonam525/Generator-as-the-tracker · Project page: sonam525.github.io/Generator-as-the-tracker

Files

File Dataset Training phase Used in the paper for
dancetrack_phaseB_continuation_step2500.safetensors DanceTrack B: continuation (prefix-trained), step 2,500 The chained tracker: test-server entry (40.3 HOTA), factorial cells A and D, re-acquisition
dancetrack_phaseA_translation_step1500.safetensors DanceTrack A: translation only, step 1,500 Factorial cells B and C (the controls)
sportsmot_phaseB_continuation_step1500.safetensors SportsMOT B, step 1,500 SportsMOT validation results
sportsmot_phaseA_translation_step750.safetensors SportsMOT A, step 750 SportsMOT controls

Each file is a rank-32 LoRA (alpha 32) on the video-stream attention (QKV, output) and feed-forward projections of LTX-2.3 22B dev (ltx-2.3-22b-dev.safetensors); the audio stream is untouched. Checkpoints were selected by validation foreground IoU every 500 steps (DanceTrack) or 750 steps (SportsMOT). SHA-256 checksums are in SHA256SUMS.

Use

The adapters are in the format of the LTX-2 trainer (IC-LoRA). Generation settings, the fixed prompt, the decode rules and the full chaining procedure (49-frame windows at stride 40, 9-frame prefix from the cleaned tail of the previous window) are in the paper's Appendix A and in video_generator/ of the code repository (ltx_run_chain.py with ltx_chain.py). Decode the generated ID-maps with the repository's codec (core/gentrack); the palette index is the track id.

Limits

About 0.4 frames per second end to end on one A100; detection is the weak side (test DetA 37.6), worst in crowds. See the paper's limitations section.

Licence and data

These adapters are derivatives of LTX-2.3 and are subject to the LTX-2 Community License Agreement. They were trained on DanceTrack and SportsMOT, which are licensed for non-commercial research; use these checkpoints for non-commercial research only. Silhouette training labels were made with SAM 3.

Citation

@article{yang2026generator,
  title   = {The generator is the tracker: Multi-object tracking by painting persistent
             identity colours},
  author  = {Yang, Haiyu and Hostens, Miel},
  journal = {arXiv preprint arXiv:2607.17120},
  year    = {2026},
  doi     = {10.48550/arXiv.2607.17120}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Sonam5/Generator-as-the-tracker

Adapter
(122)
this model

Paper for Sonam5/Generator-as-the-tracker