Instructions to use adept/fuyu-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use adept/fuyu-8b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="adept/fuyu-8b")# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("adept/fuyu-8b") model = AutoModelForMultimodalLM.from_pretrained("adept/fuyu-8b") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use adept/fuyu-8b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "adept/fuyu-8b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "adept/fuyu-8b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/adept/fuyu-8b
- SGLang
How to use adept/fuyu-8b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "adept/fuyu-8b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "adept/fuyu-8b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "adept/fuyu-8b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "adept/fuyu-8b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use adept/fuyu-8b with Docker Model Runner:
docker model run hf.co/adept/fuyu-8b
ImportError for FuyuProcessor in Transformers v4.34.1
Hello HuggingFace Community,
I've been trying to utilize the FuyuProcessor and FuyuForCausalLM from the transformers library, specifically for the Fuyu-8B model released by Adept AI. However, I've run into an import issue that I can't seem to resolve. I'm reaching out in hopes that someone here might have encountered a similar issue or could provide some insights.
Here's the problem I faced:
When running my inference script, I encountered the following error:
(adept) cwicharz@im-wicharz:~/project/Testing-Multimodal-LLMs$ /home/users/cwicharz/project/Testing-Multimodal-LLMs/venvs/adept/bin/python /home/users/cwicharz/project/Testing-Multimodal-LLMs/models/adept/inference.py
Traceback (most recent call last):
File "/home/users/cwicharz/project/Testing-Multimodal-LLMs/models/adept/inference.py", line 1, in
from transformers import FuyuProcessor, FuyuForCausalLM
ImportError: cannot import name 'FuyuProcessor' from 'transformers' (/home/users/cwicharz/project/Testing-Multimodal-LLMs/venvs/adept/lib/python3.8/site-packages/transformers/init.py)
I then checked the version of my transformers library:
(adept) cwicharz@im-wicharz:~/project/Testing-Multimodal-LLMs$ pip show transformers
Name: transformers
Version: 4.34.1
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
...
Based on the model's documentation, I expected these imports to be available in the library. Has anyone else faced a similar issue? Any advice or solutions would be greatly appreciated.
Thanks in advance for your help!
I had the same issue.
if you need the transformers use:
pip install git+https://github.com/huggingface/transformers.git
but i think I just updated the folder using "git pull". I am not sure if I added the "https://huggingface.co/adept/fuyu-8b" at the end but I can check when i get home from school because I can't do much on a school chromebook.
after that try closing vscode then opening the folder again.
I m trying to run it via google colab, got the same message, any solution: ImportError: cannot import name 'FuyuProcessor' from 'transformers' (/usr/local/lib/python3.10/dist-packages/transformers/init.py)
Hi @saeedasadibagloee ! Same answer as above by @FantasticMrCat42 , until a new version of transformers is released in PyPi, you need to install from the main branch. In Google Colab, you should run this cell first:
!pip install git+https://github.com/huggingface/transformers.git