mseritan/LabelFormer-AV2-smoke

LabelFormer — AV2 smoke checkpoint

⚠️ This is a small-scale smoke checkpoint, not a paper reproduction. It is a 1.16M-parameter model trained locally on a laptop (Apple Silicon, MPS) on 12 of the 700 ArgoVerse 2 sensor train logs for 30 epochs (~11 minutes). It exists so that the reference implementations below can be run end-to-end without training. Do not use it to benchmark against the paper.

Reference checkpoint for two open-source implementations of LabelFormer (Object Trajectory Refinement for Offboard Perception from LiDAR Point Clouds, Yang et al., CoRL 2023, arXiv:2311.01444):

• labelrefinery/LabelFormer.py — PyTorch implementation + ArgoVerse 2 training pipeline (this checkpoint was trained with it).
• labelrefinery/LabelFormer.mojo — pure-Mojo inferencer, verified to match PyTorch within 1.5e-6 on the refined boxes using the mojo/ files here.

What the model does

LabelFormer refines noisy BEV object trajectories (auto-labels) from LiDAR point clouds: per-frame boxes and object points are encoded independently (box MLP + PointPillars-style CNN), a transformer with ALiBi relative position biases reasons over the whole trajectory, and the model decodes per-frame pose residuals plus one trajectory-level object size.

Training setup (differs from the paper)

	this checkpoint	paper
params	1.16M (d=128, 3 layers)	~6M (d=256, 6 layers)
data	12 AV2 train logs (739 vehicle tracks)	700 logs
initial noisy tracks	perturbed ground truth (±0.25 m, ±10°, size jitter)	detector + tracker outputs
training	30 epochs, ~11 min, Apple M-series (MPS)	40 epochs, GPU
pillar grid	0.2 m, 19.2 m × 6.4 m	0.1 m, 24 m × 8 m

Results (4 AV2 val logs, 261 tracks, refined vs. perturbed input)

metric	initial	refined
mean IoU	0.794	0.939
recall@0.7	0.966	0.999
recall@0.8	0.450	0.992

Full numbers in eval_val.json; per-epoch curves in history.json.

Files

• best.pt — PyTorch checkpoint (model state dict + config), load with LabelFormer.py's evaluate.py or torch.load.
• config.yaml — the exact training config (configs/smoke.yaml).
• mojo/weights.lft — weights with BatchNorms folded into convs, in the LFT1 container consumed by LabelFormer.mojo.
• mojo/sample_*.lft — three real AV2 val trajectories with expected per-stage outputs, for parity testing the Mojo inferencer.

License and data provenance

Trained on the ArgoVerse 2 sensor dataset, which is released under CC BY-NC-SA 4.0; these weights are published under the same license and are intended for non-commercial research use.