Instructions to use google/pix2struct-textcaps-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use google/pix2struct-textcaps-base with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("image-to-text", model="google/pix2struct-textcaps-base")# Load model directly from transformers import AutoProcessor, AutoModelForImageTextToText processor = AutoProcessor.from_pretrained("google/pix2struct-textcaps-base") model = AutoModelForImageTextToText.from_pretrained("google/pix2struct-textcaps-base") - Notebooks
- Google Colab
- Kaggle
ImportError: cannot import name 'Pix2StructForConditionalGeneration'
Getting this error after running:
import requests
from PIL import Image
from transformers import Pix2StructForConditionalGeneration, Pix2StructProcessor
Full error message:
ImportError Traceback (most recent call last)
<ipython-input-3-faf76521f7e8> in <module>
1 import requests
2 from PIL import Image
----> 3 from transformers import Pix2StructForConditionalGeneration, Pix2StructProcessor
4
ImportError: cannot import name 'Pix2StructForConditionalGeneration'
-----------
Also this link is broken: https://github.com/huggingface/transformers/blob/main/src/transformers/models/pix2struct/convert_pix2struct_checkpoint_to_pytorch.py
thank you!
Hello, @jalbarracin
thanks for trying out the model so quickly!
in fact you need to install transformers from this branch: https://github.com/huggingface/transformers/pull/21400
Hi! great work. I am a newbie in this. Couldn't figure out how to install transformers from that branch :(
Thanks! No worries! You can just run:
pip install git+https://github.com/younesbelkada/transformers.git@add-pix2struct-clean
let me know if any of the example script does not work! The plan is also to add all the checkpoints once the PR: https://github.com/huggingface/transformers/pull/21400 gets approved
Works GREAT!
I see the PR got approved, but I'm still seeing the same error
@nigelhenry
Can you try to install transformers from source?
pip install git+https://github.com/huggingface/transformers.git