Instructions to use Intel/llava-gemma-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Intel/llava-gemma-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Intel/llava-gemma-7b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Intel/llava-gemma-7b", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Intel/llava-gemma-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Intel/llava-gemma-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Intel/llava-gemma-7b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Intel/llava-gemma-7b

SGLang

How to use Intel/llava-gemma-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Intel/llava-gemma-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Intel/llava-gemma-7b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Intel/llava-gemma-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Intel/llava-gemma-7b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Intel/llava-gemma-7b with Docker Model Runner:
```
docker model run hf.co/Intel/llava-gemma-7b
```

bconsolvo commited on Jun 4, 2024

Commit

5d5baaf

verified ·

1 Parent(s): 2e61487

Update LICENSE.md

Browse files

Files changed (1) hide show

LICENSE.md +2 -9

LICENSE.md CHANGED Viewed

@@ -19,16 +19,9 @@ non-exclusive, worldwide, royalty-free, license under Intel’s copyrights in th
 Material, to use, reproduce, display, and distribute the Material solely for
 research purposes.
-    2.1 Restrictions.
-Except as authorized above, You will not: (a) use Material in any other way; (b)
-reverse engineer, decompile, or disassemble the Material, or (c) use Material to
-violate or aid in the violation of any international human right.
-    2.2 No Implied License.
-Except for the express license in Section 2, Intel does not grant You any
-express or implied license under any legal theory. Any other licenses from Intel
-require additional consideration. Nothing in this Agreement requires Intel to
-grant any additional license.
 3. Third party programs.
 Your use of certain third-party software with or within the Material is subject

 Material, to use, reproduce, display, and distribute the Material solely for
 research purposes.
+    2.1 Restrictions. Except as authorized above, You will not: (a) use Material in any other way; (b) reverse engineer, decompile, or disassemble the Material, or (c) use Material to violate or aid in the violation of any international human right.
+    2.2 No Implied License. Except for the express license in Section 2, Intel does not grant You any express or implied license under any legal theory. Any other licenses from Intel require additional consideration. Nothing in this Agreement requires Intel to grant any additional license.
 3. Third party programs.
 Your use of certain third-party software with or within the Material is subject