Add pipeline tag, library name and link to paper

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +7 -101
README.md CHANGED
@@ -1,8 +1,12 @@
1
  ---
 
2
  license: apache-2.0
3
- library_name: pytorch
 
4
  tags:
5
  - pytorch
 
 
6
  ---
7
 
8
  <a id="top"></a>
@@ -26,9 +30,9 @@ tags:
26
  </p>
27
  </div>
28
 
29
- These are the official implementation resources, model weights, and prediction files for **ChartLens**, our champion solution for **DataMFM Challenge Track 2: Chart Understanding** at CVPR 2026.
30
 
31
- πŸ”— **Paper:** [Arxiv](https://arxiv.org/pdf/2606.10640)
32
  πŸ”— **GitHub Repository:** [iLearnLab/CVPRW26-ChartLens](https://github.com/iLearnLab/CVPRW26-ChartLens)
33
  πŸ”— **Challenge Page:** [DataMFM Challenge](https://datamfm.github.io/challenge.html)
34
 
@@ -114,104 +118,6 @@ python code/infer_granite_with_lora.py \
114
 
115
  Use `code/infer_chartnet_granite.py` for base Granite Vision inference without a LoRA adapter.
116
 
117
- ### Step 4: SAVC CSV Correction
118
-
119
- ```bash
120
- export OPENAI_API_KEY="..."
121
-
122
- python code/calibrate_baseline_with_ai.py \
123
- --split all \
124
- --baseline_root /path/to/baseline_predictions \
125
- --image_root /path/to/data \
126
- --output_root /path/to/savc_output \
127
- --base_url "https://your-openai-compatible-endpoint" \
128
- --model gemini-3.5-flash \
129
- --threshold 85
130
- ```
131
-
132
- `--baseline_root` should contain split directories such as `real/` and `synthetic/`, each with `chart2csv_predictions.jsonl` and `chart2summary_predictions.jsonl`.
133
-
134
- ### Step 5: TRSR Summary Refinement
135
-
136
- ```bash
137
- python code/ocr.py \
138
- --real_images /path/to/data/real/images \
139
- --synthetic_images /path/to/data/synthetic/images \
140
- --real_summary /path/to/baseline/real/chart2summary_predictions.jsonl \
141
- --synthetic_summary /path/to/baseline/synthetic/chart2summary_predictions.jsonl \
142
- --output_dir /path/to/ocr_text_copy_coverage \
143
- --threshold 0.8
144
-
145
- export AIGCBEST_API_KEY="..."
146
-
147
- python code/repair_summary.py \
148
- --split all \
149
- --workers 20 \
150
- --ocr_eval_root /path/to/ocr_text_copy_coverage \
151
- --output_root /path/to/trsr_output
152
- ```
153
-
154
- ### Step 6: Training (Optional)
155
-
156
- Train the LoRA adapter on the prepared ChartNet SFT data:
157
-
158
- ```bash
159
- python code/train_lora_chartnet.py \
160
- --model_path /path/to/granite-vision-4.1-4b \
161
- --train_jsonl Fine-tuning/Dataset/sft/train.jsonl \
162
- --val_jsonl Fine-tuning/Dataset/sft/val.jsonl \
163
- --output_dir Fine-tuning/FT/model/granite_chartnet_lora_bs2 \
164
- --gpu_id 0 \
165
- --epochs 2 \
166
- --batch_size 1 \
167
- --grad_accum 8
168
- ```
169
-
170
- ---
171
-
172
- ## πŸ“¦ Submission Format
173
-
174
- For DataMFM Track 2, organize the final predictions as:
175
-
176
- ```bash
177
- submission.zip
178
- β”œβ”€β”€ real/
179
- β”‚ β”œβ”€β”€ chart2csv_predictions.jsonl
180
- β”‚ └── chart2summary_predictions.jsonl
181
- └── synthetic/
182
- β”œβ”€β”€ chart2csv_predictions.jsonl
183
- └── chart2summary_predictions.jsonl
184
- ```
185
-
186
- Each CSV prediction line:
187
-
188
- ```json
189
- {"imagename": "example.png", "predicted_csv": "Header A,Header B\nA,1\nB,2"}
190
- ```
191
-
192
- Each summary prediction line:
193
-
194
- ```json
195
- {"imagename": "example.png", "predicted_summary": "One paragraph summary grounded in the chart."}
196
- ```
197
-
198
- ---
199
-
200
- ## ⚠️ Limitations & Notes
201
-
202
- **Disclaimer:** This framework and its model weights are intended for **academic research purposes only**.
203
-
204
- - Chart-to-CSV extraction may still struggle with dense layouts, asymmetric legends, or adjacent semantic-column misalignment.
205
- - Summary refinement depends on OCR quality; OCR errors can affect text-retention scoring and repair decisions.
206
- - GPU execution is expected for Granite Vision inference and LoRA training.
207
- - API-backed correction scripts require valid credentials and an OpenAI-compatible endpoint.
208
-
209
- ---
210
-
211
- ## 🀝 Acknowledgements & Contact
212
-
213
- - **Contact:** If you have any questions or encounter issues, feel free to contact Hao Liu at liuh90210@gmail.com or Ruping Cao at caoruping657@gmail.com.
214
-
215
  ---
216
 
217
  ## πŸ“β­οΈ Citation
 
1
  ---
2
+ library_name: peft
3
  license: apache-2.0
4
+ base_model: ibm-granite/granite-vision-4.1-4b
5
+ pipeline_tag: image-text-to-text
6
  tags:
7
  - pytorch
8
+ - lora
9
+ - chart-understanding
10
  ---
11
 
12
  <a id="top"></a>
 
30
  </p>
31
  </div>
32
 
33
+ These are the official implementation resources, model weights, and prediction files for **ChartLens**, the champion solution for **DataMFM Challenge Track 2: Chart Understanding** at CVPR 2026.
34
 
35
+ πŸ”— **Paper:** [ChartLens: A Dual-Branch Framework for Chart Data Correction and Factual Summary Refinement](https://huggingface.co/papers/2606.10640)
36
  πŸ”— **GitHub Repository:** [iLearnLab/CVPRW26-ChartLens](https://github.com/iLearnLab/CVPRW26-ChartLens)
37
  πŸ”— **Challenge Page:** [DataMFM Challenge](https://datamfm.github.io/challenge.html)
38
 
 
118
 
119
  Use `code/infer_chartnet_granite.py` for base Granite Vision inference without a LoRA adapter.
120
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
121
  ---
122
 
123
  ## πŸ“β­οΈ Citation