Instructions to use nlpconnect/vit-gpt2-image-captioning with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nlpconnect/vit-gpt2-image-captioning with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("image-to-text", model="nlpconnect/vit-gpt2-image-captioning")# Load model directly from transformers import AutoTokenizer, AutoModelForImageTextToText tokenizer = AutoTokenizer.from_pretrained("nlpconnect/vit-gpt2-image-captioning") model = AutoModelForImageTextToText.from_pretrained("nlpconnect/vit-gpt2-image-captioning") - Notebooks
- Google Colab
- Kaggle
Cannot run
#20
by maroc - opened
i follow the doc after run i get this error:
Traceback (most recent call last):
File "/Users/.../AI/hugging-face/learn/app.py", line 14, in <module>
img2text("2.jpg")
File "/Users/.../AI/hugging-face/learn/app.py", line 7, in img2text
image_to_text = pipeline("image-to-text", model="nlpconnect/vit-gpt2-image-captioning")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/transformers/pipelines/__init__.py", line 788, in pipeline
framework, model = infer_framework_load_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/transformers/pipelines/base.py", line 278, in infer_framework_load_model
raise ValueError(f"Could not load model {model} with any of the following classes: {class_tuple}.")
ValueError: Could not load model nlpconnect/vit-gpt2-image-captioning with any of the following classes: (<class 'transformers.models.auto.
modeling_tf_auto.TFAutoModelForVision2Seq'>, <class 'transformers.models.vision_encoder_decoder.modeling_tf_vision_encoder_decoder.TFVision
EncoderDecoderModel'>).
same problem hit