Image-to-Text
Transformers
PyTorch
ONNX
Safetensors
vision-encoder-decoder
image-text-to-text
ocr
image to latex
Instructions to use OleehyO/TexTeller with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OleehyO/TexTeller with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("image-to-text", model="OleehyO/TexTeller")# Load model directly from transformers import AutoTokenizer, AutoModelForMultimodalLM tokenizer = AutoTokenizer.from_pretrained("OleehyO/TexTeller") model = AutoModelForMultimodalLM.from_pretrained("OleehyO/TexTeller", device_map="auto") - Notebooks
- Google Colab
- Kaggle
How to finetune this model?
#1
by catastropiyush - opened
I wanted to start finetuning VLMs but I am not able to find a good resource for this. Can we finetune this on Google Colab's T4 GPU? How to manage different image sizes in the dataset? Thank you.
Yes, you can use T4 to train TexTeller, please refer to TexTeller’s GitHub repository for more details.
For images of different sizes in the dataset, they will be resized to 448 * 448 while maintaining the aspect ratio (the extra parts will be padded).