Hugging Face's logo

reubk
/
Molmo_7B_Human_Pointing_LoRA

Text Generation
PEFT
Safetensors
English
lora
finetune
agent
Model card Files
xet
Community
reubk commited on
Commit
c6b0ab8
·
verified ·
1 Parent(s): 856cfe3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -0
README.md CHANGED
@@ -10,6 +10,7 @@ tags:
10
  - lora
11
  - finetune
12
  - agent
 
13
  ---
14
 
15
  Testing a QLoRA adaptor for [allenai/Molmo-7B-D-0924](https://huggingface.co/allenai/Molmo-7B-D-0924),
@@ -20,6 +21,14 @@ Trained on 47 screenshots of a low-poly video game with ragdoll casualties
20
 
21
  Evaluated on 44 screenshots of aforementioned video game
22
 
 
 
 
 
 
 
 
 
23
  Comparison of Model performance with and without QLora on Eval dataset
24
  |Model| Molmo-7B-D | Molmo-7B-D w/ QLora |
25
  |----------|------|------|
 
10
  - lora
11
  - finetune
12
  - agent
13
+
14
  ---
15
 
16
  Testing a QLoRA adaptor for [allenai/Molmo-7B-D-0924](https://huggingface.co/allenai/Molmo-7B-D-0924),
 
21
 
22
  Evaluated on 44 screenshots of aforementioned video game
23
 
24
+ Molmo has an edge case where it declares there are no humans in an image:
25
+ ![img1 (2)](https://cdn-uploads.huggingface.co/production/uploads/6367f8dd46919b9619bc7bf2/8zsuqnz-QCTamBDOgWGM-.png)
26
+
27
+ This custom QLoRA successfully reduces the occurance of these cases
28
+ ![img1 (1)](https://cdn-uploads.huggingface.co/production/uploads/6367f8dd46919b9619bc7bf2/-HENqZx5SiLYX35tx3ADs.png)
29
+
30
+ However, pointing to non-human objects is observed to increase.
31
+
32
  Comparison of Model performance with and without QLora on Eval dataset
33
  |Model| Molmo-7B-D | Molmo-7B-D w/ QLora |
34
  |----------|------|------|