Instructions to use Junteng/Chart_CLIP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Junteng/Chart_CLIP with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-feature-extraction", model="Junteng/Chart_CLIP")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Junteng/Chart_CLIP", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Improve model card: Add metadata and GitHub link
Browse filesThis PR enhances the model card by:
- Adding `pipeline_tag: image-feature-extraction` for better discoverability, reflecting the model's core function.
- Adding `library_name: transformers` as evidence from `config.json` and `tokenizer_config.json` confirms compatibility with the Hugging Face Transformers library.
- Adding the `datasets` tag `Junteng/Vision4Chart` as the model uses this dataset for training, as explicitly mentioned in the model card content.
- Including a direct link to the GitHub repository for easy access to the project's code.
These changes will improve the model's visibility and provide users with clearer information on its usage and resources.
README.md
CHANGED
|
@@ -1,7 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# CLIP Model for Chart Understanding
|
| 2 |
|
| 3 |
This repository contains the CLIP model implementation from our paper "[On the Perception Bottleneck of VLMs for Chart Understanding](https://arxiv.org/abs/2503.18435)".
|
| 4 |
|
|
|
|
|
|
|
| 5 |
## Overview
|
| 6 |
|
| 7 |
This CLIP model is specifically trained to address the perception bottleneck in Vision Language Models (VLMs) when processing and understanding charts and visualizations. Our work explores and aims to improve how CLIP effect its LVLMs.
|
|
@@ -29,5 +38,4 @@ If you find this model useful in your research, please consider citing our paper
|
|
| 29 |
primaryClass={cs.CV},
|
| 30 |
url={https://arxiv.org/abs/2503.18435},
|
| 31 |
}
|
| 32 |
-
```
|
| 33 |
-
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: image-feature-extraction
|
| 3 |
+
library_name: transformers
|
| 4 |
+
datasets:
|
| 5 |
+
- Junteng/Vision4Chart
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
# CLIP Model for Chart Understanding
|
| 9 |
|
| 10 |
This repository contains the CLIP model implementation from our paper "[On the Perception Bottleneck of VLMs for Chart Understanding](https://arxiv.org/abs/2503.18435)".
|
| 11 |
|
| 12 |
+
**Code**: https://github.com/hkust-nlp/Vision4Chart
|
| 13 |
+
|
| 14 |
## Overview
|
| 15 |
|
| 16 |
This CLIP model is specifically trained to address the perception bottleneck in Vision Language Models (VLMs) when processing and understanding charts and visualizations. Our work explores and aims to improve how CLIP effect its LVLMs.
|
|
|
|
| 38 |
primaryClass={cs.CV},
|
| 39 |
url={https://arxiv.org/abs/2503.18435},
|
| 40 |
}
|
| 41 |
+
```
|
|
|