| --- |
| license: cc-by-nc-4.0 |
| tags: |
| - autonomous-driving |
| - depth-estimation |
| - segmentation |
| - motion-prediction |
| - self-supervised |
| library_name: pytorch |
| --- |
| |
| # LFG |
|
|
| LFG checkpoint that jointly predicts **dense depth / 3D points, camera pose, per-point confidence, object segmentation (7 classes), and per-pixel motion** from monocular driving video — no human labels. |
|
|
| LFG: Self-Supervised 4D Learning from In-the-Wild Driving Videos — trained on 58M+ unlabeled dashcam driving frames with frozen teachers generating pseudo ground truth on the fly. See the [project page](https://lfg-ai.github.io/) and the [inference repo](https://github.com/Applied-Intuition-Open-Source/LFG). |
|
|
| ## Model details |
|
|
| | | | |
| |---|---| |
| | Architecture | LFG (autoregressive transformer with future-frame prediction) | |
| | Parameters | 1.22B (fp32) | |
| | Input | monocular video, 3 history frames | |
| | Encoder | DINOv2 | |
| | Heads | point/depth, camera/pose, confidence, segmentation (7 classes), motion | |
|
|
| ## Files |
|
|
| - `lfg_seg_motion_1.3b.pt` — minimal inference checkpoint: `model_state_dict` (inference weights only), `config` (architecture settings), `global_step`. Loads with `torch.load(..., weights_only=True)`. |
|
|
| ## Usage |
|
|
| Use the [official inference repo](https://github.com/Applied-Intuition-Open-Source/LFG): |
|
|
| ```bash |
| python infer.py <video-or-image-dir> \ |
| --checkpoint lfg_seg_motion_1.3b.pt \ |
| --output-dir outputs/ \ |
| --save-visualizations |
| ``` |
|
|
| Outputs per window: `points`, `local_points`, `camera_poses`, `conf`, `segmentation`, `motion`. |
|
|
