dawidtang commited on
Commit
d04b429
·
verified ·
1 Parent(s): 30bf2d9

Add V-JEPA and V-JEPA 2 citations

Browse files
Files changed (1) hide show
  1. README.md +27 -3
README.md CHANGED
@@ -37,7 +37,7 @@ import torch
37
  from transformers import AutoModel
38
 
39
  model = AutoModel.from_pretrained(
40
- "epfl-neuroai/vjepa2-enoder-basic",
41
  trust_remote_code=True,
42
  )
43
  model.eval()
@@ -56,7 +56,7 @@ For decoder-only debugging, the model can also run from precomputed V-JEPA2 laye
56
 
57
  ```python
58
  model = AutoModel.from_pretrained(
59
- "epfl-neuroai/vjepa2-enoder-basic",
60
  trust_remote_code=True,
61
  load_vjepa=False,
62
  )
@@ -85,11 +85,35 @@ This repository does not include the underlying fMRI datasets or stimulus videos
85
  - `config.json`, `configuration_vjepa2_fmri_encoder.py`, `modeling_vjepa2_fmri_encoder.py`: custom Transformers files for `AutoModel` loading.
86
  - `requirements.txt`: minimal Python dependencies.
87
 
 
 
 
 
 
 
 
 
 
 
88
  ## Citations
89
 
90
- If you use this checkpoint, please cite the source datasets:
91
 
92
  ```bibtex
 
 
 
 
 
 
 
 
 
 
 
 
 
 
93
  @article{tang2025diverse,
94
  title={Diverse perceptual representations across visual pathways emerge from a single objective},
95
  author={Tang, Yingtian and Gokce, Abdulkadir and Al-Karkari, Khaled Jedoui and Yamins, Daniel and Schrimpf, Martin},
 
37
  from transformers import AutoModel
38
 
39
  model = AutoModel.from_pretrained(
40
+ "epfl-neuroai/vjepa2-encoder-basic",
41
  trust_remote_code=True,
42
  )
43
  model.eval()
 
56
 
57
  ```python
58
  model = AutoModel.from_pretrained(
59
+ "epfl-neuroai/vjepa2-encoder-basic",
60
  trust_remote_code=True,
61
  load_vjepa=False,
62
  )
 
85
  - `config.json`, `configuration_vjepa2_fmri_encoder.py`, `modeling_vjepa2_fmri_encoder.py`: custom Transformers files for `AutoModel` loading.
86
  - `requirements.txt`: minimal Python dependencies.
87
 
88
+ ## Backbone Source
89
+
90
+ The V-JEPA2 backbone weights are shipped in this repository as:
91
+
92
+ ```text
93
+ vitl.pt
94
+ ```
95
+
96
+ The loader uses the V-JEPA2 Torch Hub architecture with `pretrained=False`, then loads the local `vitl.pt` weights directly. This avoids relying on the moving `facebookresearch/vjepa2` Torch Hub checkpoint URL while preserving compatibility with the original decoder features. The decoder checkpoint uses canonical `extractor_config["layer_names"]` metadata.
97
+
98
  ## Citations
99
 
100
+ If you use this checkpoint, please cite the V-JEPA/V-JEPA 2 backbone papers and source datasets:
101
 
102
  ```bibtex
103
+ @article{bardes2024revisiting,
104
+ title={Revisiting Feature Prediction for Learning Visual Representations from Video},
105
+ author={Bardes, Adrien and Garrido, Quentin and Ponce, Jean and Chen, Xinlei and Rabbat, Michael and LeCun, Yann and Assran, Mahmoud and Ballas, Nicolas},
106
+ journal={arXiv preprint arXiv:2404.08471},
107
+ year={2024}
108
+ }
109
+
110
+ @article{assran2025vjepa2,
111
+ title={V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning},
112
+ author={Assran, Mido and Bardes, Adrien and Fan, David and Garrido, Quentin and Howes, Russell and Komeili, Mojtaba and Muckley, Matthew and Rizvi, Ammar and Roberts, Claire and Sinha, Koustuv and others},
113
+ journal={arXiv preprint arXiv:2506.09985},
114
+ year={2025}
115
+ }
116
+
117
  @article{tang2025diverse,
118
  title={Diverse perceptual representations across visual pathways emerge from a single objective},
119
  author={Tang, Yingtian and Gokce, Abdulkadir and Al-Karkari, Khaled Jedoui and Yamins, Daniel and Schrimpf, Martin},