litcoderr commited on
Commit
c4f25e3
verified
1 Parent(s): a85f4eb

Reframe the lead around language compositionality

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -22,13 +22,13 @@ Representation Learning. EMNLP 2026, main conference.
22
 
23
  [Paper][paper] 路 [Project page][project] 路 [Code][code]
24
 
25
- PRISM decomposes each video clip into a view-invariant latent `z_vi` and a view-variant
26
- latent `z_vv`, then recomposes the two under language supervision: `z_vi` of clip A paired
27
- with `z_vv` of clip B is trained to match the embedding of the correspondingly recombined
28
- caption. That composition never occurs in the training data, so semantics that leaked
29
- between the streams cannot survive it. The clip embedding this model returns is the
30
- view-invariant stream, which makes cosine similarity a measure of what is happening rather
31
- than of where the camera is.
32
 
33
  This checkpoint is trained on EgoExo4D, using both ego and exo views and captions generated
34
  by Gemini 3.
 
22
 
23
  [Paper][paper] 路 [Project page][project] 路 [Code][code]
24
 
25
+ PRISM separates what an action is from the viewpoint and scene it happens to be seen in.
26
+ Language is what makes that separation trainable: descriptions recombine freely, so the
27
+ action of one clip can be put together with the context of another into a composition that
28
+ never occurred in the training data, and the model is asked to predict the meaning of that
29
+ composition. Semantics that leaked between the two factors cannot survive the swap. What
30
+ this model returns is the view-invariant side of the split, so cosine similarity measures
31
+ what is happening rather than where the camera is.
32
 
33
  This checkpoint is trained on EgoExo4D, using both ego and exo views and captions generated
34
  by Gemini 3.