Add V-JEPA and V-JEPA 2 citations
Browse files
README.md
CHANGED
|
@@ -37,7 +37,7 @@ import torch
|
|
| 37 |
from transformers import AutoModel
|
| 38 |
|
| 39 |
model = AutoModel.from_pretrained(
|
| 40 |
-
"epfl-neuroai/vjepa2-
|
| 41 |
trust_remote_code=True,
|
| 42 |
)
|
| 43 |
model.eval()
|
|
@@ -56,7 +56,7 @@ For decoder-only debugging, the model can also run from precomputed V-JEPA2 laye
|
|
| 56 |
|
| 57 |
```python
|
| 58 |
model = AutoModel.from_pretrained(
|
| 59 |
-
"epfl-neuroai/vjepa2-
|
| 60 |
trust_remote_code=True,
|
| 61 |
load_vjepa=False,
|
| 62 |
)
|
|
@@ -85,11 +85,35 @@ This repository does not include the underlying fMRI datasets or stimulus videos
|
|
| 85 |
- `config.json`, `configuration_vjepa2_fmri_encoder.py`, `modeling_vjepa2_fmri_encoder.py`: custom Transformers files for `AutoModel` loading.
|
| 86 |
- `requirements.txt`: minimal Python dependencies.
|
| 87 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
## Citations
|
| 89 |
|
| 90 |
-
If you use this checkpoint, please cite the source datasets:
|
| 91 |
|
| 92 |
```bibtex
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
@article{tang2025diverse,
|
| 94 |
title={Diverse perceptual representations across visual pathways emerge from a single objective},
|
| 95 |
author={Tang, Yingtian and Gokce, Abdulkadir and Al-Karkari, Khaled Jedoui and Yamins, Daniel and Schrimpf, Martin},
|
|
|
|
| 37 |
from transformers import AutoModel
|
| 38 |
|
| 39 |
model = AutoModel.from_pretrained(
|
| 40 |
+
"epfl-neuroai/vjepa2-encoder-basic",
|
| 41 |
trust_remote_code=True,
|
| 42 |
)
|
| 43 |
model.eval()
|
|
|
|
| 56 |
|
| 57 |
```python
|
| 58 |
model = AutoModel.from_pretrained(
|
| 59 |
+
"epfl-neuroai/vjepa2-encoder-basic",
|
| 60 |
trust_remote_code=True,
|
| 61 |
load_vjepa=False,
|
| 62 |
)
|
|
|
|
| 85 |
- `config.json`, `configuration_vjepa2_fmri_encoder.py`, `modeling_vjepa2_fmri_encoder.py`: custom Transformers files for `AutoModel` loading.
|
| 86 |
- `requirements.txt`: minimal Python dependencies.
|
| 87 |
|
| 88 |
+
## Backbone Source
|
| 89 |
+
|
| 90 |
+
The V-JEPA2 backbone weights are shipped in this repository as:
|
| 91 |
+
|
| 92 |
+
```text
|
| 93 |
+
vitl.pt
|
| 94 |
+
```
|
| 95 |
+
|
| 96 |
+
The loader uses the V-JEPA2 Torch Hub architecture with `pretrained=False`, then loads the local `vitl.pt` weights directly. This avoids relying on the moving `facebookresearch/vjepa2` Torch Hub checkpoint URL while preserving compatibility with the original decoder features. The decoder checkpoint uses canonical `extractor_config["layer_names"]` metadata.
|
| 97 |
+
|
| 98 |
## Citations
|
| 99 |
|
| 100 |
+
If you use this checkpoint, please cite the V-JEPA/V-JEPA 2 backbone papers and source datasets:
|
| 101 |
|
| 102 |
```bibtex
|
| 103 |
+
@article{bardes2024revisiting,
|
| 104 |
+
title={Revisiting Feature Prediction for Learning Visual Representations from Video},
|
| 105 |
+
author={Bardes, Adrien and Garrido, Quentin and Ponce, Jean and Chen, Xinlei and Rabbat, Michael and LeCun, Yann and Assran, Mahmoud and Ballas, Nicolas},
|
| 106 |
+
journal={arXiv preprint arXiv:2404.08471},
|
| 107 |
+
year={2024}
|
| 108 |
+
}
|
| 109 |
+
|
| 110 |
+
@article{assran2025vjepa2,
|
| 111 |
+
title={V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning},
|
| 112 |
+
author={Assran, Mido and Bardes, Adrien and Fan, David and Garrido, Quentin and Howes, Russell and Komeili, Mojtaba and Muckley, Matthew and Rizvi, Ammar and Roberts, Claire and Sinha, Koustuv and others},
|
| 113 |
+
journal={arXiv preprint arXiv:2506.09985},
|
| 114 |
+
year={2025}
|
| 115 |
+
}
|
| 116 |
+
|
| 117 |
@article{tang2025diverse,
|
| 118 |
title={Diverse perceptual representations across visual pathways emerge from a single objective},
|
| 119 |
author={Tang, Yingtian and Gokce, Abdulkadir and Al-Karkari, Khaled Jedoui and Yamins, Daniel and Schrimpf, Martin},
|