README.md

# Compresser Encoder (Perceiver Resampler)

Phase 0 pretrained Perceiver for the Mamba-3 Semantic Video Compressor.

## Architecture
- **Type**: Perceiver Resampler (cross-attention compressor)
- **Input**: [B, 576, 1664] — V-JEPA 2.1 ViT-Gigantic patch latents
- **Output**: [B, 64, 512] — compressed tokens
- **Params**: ~20.6M
- **Details**: 64 learned queries, 6 cross-attention layers, 16 heads, FFN 512→2048

## Training
- **Dataset**: [Vjepa_mamba_dataset_v2](https://huggingface.co/datasets/rookierufus/Vjepa_mamba_dataset_v2) (50 hours video, 384×384, 8fps)
- **V-JEPA**: Frozen [vjepa2_1_vit_gigantic_384](https://github.com/facebookresearch/vjepa2) (2.2B params)
- **Loss**: MSE reconstruction via autoencoder (Perceiver → Decoder → V-JEPA latent)
- **Optimizer**: AdamW, lr=1e-4, cosine to 1e-6
- **Hardware**: RTX 4090 (48 GB), bf16

## Usage
```python
from model.models.perceiver import PerceiverResampler

model = PerceiverResampler(input_dim=1664, output_dim=512, num_queries=64)
model.load_state_dict(torch.load("perceiver_stepX_hrsY.pt"))
# Input: [B, 576, 1664] V-JEPA latents → Output: [B, 64, 512]
```

Part of the Mamba-3 Semantic Video Compressor pipeline.