bpiyush commited on
Commit
0bdafe3
·
verified ·
1 Parent(s): ddf25ae

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +33 -0
README.md ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # TARA Model
2
+
3
+ TARA (Tarsier-based Audio-Visual Representation) is a multimodal model for video and text understanding.
4
+
5
+ ## Installation
6
+
7
+ See `INSTALL.md` for detailed installation instructions.
8
+
9
+ ## Quick Start
10
+
11
+ ```python
12
+ import torch
13
+ from modeling_tara import TARA
14
+
15
+ # Load the model
16
+ model = TARA.from_pretrained(
17
+ "bpiyush/TARA",
18
+ device_map='auto',
19
+ torch_dtype=torch.bfloat16,
20
+ )
21
+
22
+ # Encode a video
23
+ from modeling_tara import read_frames_decord
24
+ video_tensor = read_frames_decord("path/to/video.mp4", num_frames=16)
25
+ video_tensor = video_tensor.unsqueeze(0).to(model.model.device)
26
+ with torch.no_grad():
27
+ video_emb = model.encode_vision(video_tensor)
28
+
29
+ # Encode text
30
+ text = "someone is folding a paper"
31
+ with torch.no_grad():
32
+ text_emb = model.encode_text(text)
33
+ ```