File size: 3,124 Bytes
f0826bb a5403b3 f0826bb a5403b3 f0826bb a5403b3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | ---
license: other
library_name: pytorch
base_model: facebook/VGGT-1B
tags:
- vggt
- depth-estimation
- 3d-vision
- camera-pose
- test-time-training
- lact
pipeline_tag: depth-estimation
---
# VGGT LaCT (stage 1) — slim adapter weights
These files are **LaCT-block weights only** (~200 MB), not a full VGGT checkpoint. They plug into the public **[facebook/VGGT-1B](https://huggingface.co/facebook/VGGT-1B)** backbone: DINOv2 patch embed, frame-wise attention, and prediction heads stay at Meta’s pretrained VGGT-1B; only the **global-attention layers are replaced** by LaCT-style fast-weight GLU blocks trained with stage-1 distillation against the frozen teacher.
**Code:** [github.com/Akrao9/vggt_ttt](https://github.com/Akrao9/vggt_ttt) (install `vggt` from [facebookresearch/vggt](https://github.com/facebookresearch/vggt) as in that README).
## Files
| File | Description |
|------|-------------|
| `vggt_ttt_lact_stage1.pt` | Stage 1 distilled LaCT state dict (`lact_state_dict()` format). Keys are prefixed with `aggregator.lact_blocks.`. |
## Load (Python)
```python
import torch
from huggingface_hub import hf_hub_download
# From the vggt_ttt repo (with `vggt` installed per upstream README):
from model.vggt_ttt import VGGT_TTT
from model.io_utils import torch_load_checkpoint
ckpt_path = hf_hub_download("akrao9/VGGT-LACT", "vggt_ttt_lact_stage1.pt")
device = "cuda"
model = VGGT_TTT.from_pretrained("facebook/VGGT-1B", chunk_size=16).to(device).eval()
state = torch_load_checkpoint(ckpt_path, map_location=device)
model.load_lact_state_dict(state, strict=True)
```
Use a local path instead of `hf_hub_download` if you already downloaded the `.pt` file.
## Inference CLI
From the [vggt_ttt](https://github.com/Akrao9/vggt_ttt) repo, after downloading this checkpoint locally:
```bash
python scripts/run_inference.py \
--input path/to/video.mp4 --fps 2 \
--checkpoint ./vggt_ttt_lact_stage1.pt \
--out ./out
```
(`--checkpoint` accepts this LaCT-only dict; see `scripts/run_inference.py`.)
## Training summary
- **Stage 1:** distillation from frozen `facebook/VGGT-1B` (pose / depth / world points), trainable parameters confined to the 24 LaCT blocks; `c_proj` zero-init for a near-identity start.
- **Checkpoints:** saved with `torch.save(model.lact_state_dict(), path)` — same tensor layout as this Hub file.
## Hardware / scaling
LaCT path is aimed at **longer frame sequences** with more favorable VRAM scaling than full global attention; see the GitHub README for benchmark tables (DL3DV-style eval).
## License and attribution
- This **adapter** repository and the training code release are under **Apache 2.0** (see project `LICENSE` / `NOTICE` on GitHub).
- **VGGT-1B** is subject to Meta’s license and terms on its model card; you must comply with those when using the backbone.
- Method builds on **VGGT** and **LaCT**-style components as described in the upstream README.
## Citation
If you use these weights or the [vggt_ttt](https://github.com/Akrao9/vggt_ttt) codebase, cite the original **VGGT** paper/repo and credit this adapter as appropriate for your venue. |