Add paper link, pipeline tag, and repository info

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +16 -11
README.md CHANGED
@@ -1,7 +1,8 @@
1
  ---
 
2
  library_name: transformers
 
3
  license: other
4
- base_model: Qwen/Qwen2.5-VL-3B-Instruct
5
  tags:
6
  - llama-factory
7
  - full
@@ -13,19 +14,27 @@ model-index:
13
 
14
  # CaMo-3B
15
 
16
- This model is a fine-tuned version of [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) on the camerabench_10K and the spld datasets.
 
 
 
 
17
 
18
  ## Model description
19
 
20
- More information needed
 
 
21
 
22
  ## Intended uses & limitations
23
 
24
- More information needed
25
 
26
  ## Training and evaluation data
27
 
28
- More information needed
 
 
29
 
30
  ## Training procedure
31
 
@@ -41,18 +50,14 @@ The following hyperparameters were used during training:
41
  - gradient_accumulation_steps: 2
42
  - total_train_batch_size: 128
43
  - total_eval_batch_size: 64
44
- - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
45
  - lr_scheduler_type: cosine
46
  - lr_scheduler_warmup_ratio: 0.1
47
  - num_epochs: 1
48
 
49
- ### Training results
50
-
51
-
52
-
53
  ### Framework versions
54
 
55
  - Transformers 4.57.1
56
  - Pytorch 2.5.1+cu124
57
  - Datasets 4.0.0
58
- - Tokenizers 0.22.1
 
1
  ---
2
+ base_model: Qwen/Qwen2.5-VL-3B-Instruct
3
  library_name: transformers
4
+ pipeline_tag: video-text-to-text
5
  license: other
 
6
  tags:
7
  - llama-factory
8
  - full
 
14
 
15
  # CaMo-3B
16
 
17
+ This model is a fine-tuned version of [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) on the **CameraBench** and the **SpatialLadder** datasets.
18
+
19
+ It was introduced in the paper [CaMo: Camera Motion Grounded Evaluation and Training for Vision-Language Models](https://arxiv.org/abs/2605.20165).
20
+
21
+ - **Code:** [GitHub Repository](https://github.com/hsiangwei0903/CaMo)
22
 
23
  ## Model description
24
 
25
+ CaMo is a camera motion grounded Vision-Language Model (VLM) designed to address the gap in spatial cognition among existing VLMs. While state-of-the-art models often perform well on static spatial QA benchmarks, they frequently lack an understanding of camera motion—a key component of spatial intelligence.
26
+
27
+ CaMo is trained to generate explicit spatial narratives that capture both scene semantics and camera motion. It achieves consistent performance across the proposed Spatial Narrative Score (SNS) evaluation and direct spatial question answering tasks.
28
 
29
  ## Intended uses & limitations
30
 
31
+ The model is intended for research in 3D spatial understanding, video analysis, and robotics, where understanding camera trajectories (e.g., panning, tilting, zooming) is crucial.
32
 
33
  ## Training and evaluation data
34
 
35
+ The model was fine-tuned on:
36
+ - **SpatialLadder-26k**: A dataset for fine-grained spatial understanding.
37
+ - **CameraBench**: A benchmark and training set specifically targeting camera motion understanding.
38
 
39
  ## Training procedure
40
 
 
50
  - gradient_accumulation_steps: 2
51
  - total_train_batch_size: 128
52
  - total_eval_batch_size: 64
53
+ - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08
54
  - lr_scheduler_type: cosine
55
  - lr_scheduler_warmup_ratio: 0.1
56
  - num_epochs: 1
57
 
 
 
 
 
58
  ### Framework versions
59
 
60
  - Transformers 4.57.1
61
  - Pytorch 2.5.1+cu124
62
  - Datasets 4.0.0
63
+ - Tokenizers 0.22.1