rookierufus commited on
Commit
5dc972d
·
verified ·
1 Parent(s): e22baf8

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +29 -0
README.md ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Compresser Decoder (Inverse Perceiver)
2
+
3
+ Phase 0 pretrained Decoder for the Mamba-3 Semantic Video Compressor.
4
+
5
+ ## Architecture
6
+ - **Type**: Inverse Perceiver (cross-attention expansion)
7
+ - **Input**: [B, 64, 512] — Perceiver compressed tokens
8
+ - **Output**: [B, 576, 1664] — reconstructed V-JEPA latents
9
+ - **Params**: ~11.1M
10
+ - **Details**: 576 learned queries, 3 cross-attention layers, 16 heads, FFN 512→2048→512
11
+
12
+ ## Training
13
+ - **Dataset**: [Vjepa_mamba_dataset_v2](https://huggingface.co/datasets/rookierufus/Vjepa_mamba_dataset_v2) (50 hours video, 384×384, 8fps)
14
+ - **V-JEPA**: Frozen [vjepa2_1_vit_gigantic_384](https://github.com/facebookresearch/vjepa2) (2.2B params)
15
+ - **Loss**: MSE reconstruction (autoencoder target = V-JEPA latent)
16
+ - **Optimizer**: AdamW, lr=1e-4, cosine to 1e-6
17
+ - **Hardware**: RTX 4090 (48 GB), bf16
18
+
19
+ ## Usage
20
+ ```python
21
+ from compressor.decoder import PerceiverDecoder
22
+
23
+ model = PerceiverDecoder(input_dim=512, output_dim=1664, num_queries=576)
24
+ model.load_state_dict(torch.load("decoder_stepX_hrsY.pt"))
25
+ # Input: [B, 64, 512] Perceiver output → Output: [B, 576, 1664] V-JEPA latents
26
+ ```
27
+
28
+ ## Note
29
+ Disposable after Phase 0 — only the Encoder (Perceiver) carries forward to the main pipeline.