Instructions to use Infi-MM/infimm-hd with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Infi-MM/infimm-hd with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Infi-MM/infimm-hd", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Infi-MM/infimm-hd", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Infi-MM/infimm-hd with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Infi-MM/infimm-hd"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Infi-MM/infimm-hd",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Infi-MM/infimm-hd

SGLang

How to use Infi-MM/infimm-hd with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Infi-MM/infimm-hd" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Infi-MM/infimm-hd",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Infi-MM/infimm-hd" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Infi-MM/infimm-hd",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Infi-MM/infimm-hd with Docker Model Runner:
```
docker model run hf.co/Infi-MM/infimm-hd
```

infimm-hd

Commit History

Update README.md

c20153b
verified

lllliuhhhhggg commited on Mar 6, 2024

Update README.md

9bac684
verified

xiaotianhan commited on Mar 6, 2024

Create LICENSE

650d069
verified

lllliuhhhhggg commited on Mar 6, 2024

Update README.md

613511c
verified

lllliuhhhhggg commited on Mar 6, 2024

Update README.md

a08aeba
verified

lllliuhhhhggg commited on Mar 5, 2024

Update README.md

0836073
verified

lllliuhhhhggg commited on Mar 5, 2024

Update README.md

806b358
verified

lllliuhhhhggg commited on Mar 5, 2024

Update eva_vit_model.py

af3d978
verified

lllliuhhhhggg commited on Mar 4, 2024

Update README.md

aa89ed1
verified

lllliuhhhhggg commited on Mar 4, 2024

Update README.md

bbc3253
verified

lllliuhhhhggg commited on Mar 3, 2024

Update README.md

a5bd512
verified

lllliuhhhhggg commited on Mar 3, 2024

first commit

b0b3b00

liuhaogeng commited on Mar 3, 2024

Update README.md

6906069
verified

lllliuhhhhggg commited on Mar 3, 2024

Upload README.md

8d6cfcb
verified

lllliuhhhhggg commited on Mar 3, 2024

initial commit

470878c
verified

lllliuhhhhggg commited on Mar 3, 2024

Commit History

Update README.md c20153b verified

Update README.md 9bac684 verified

Create LICENSE 650d069 verified

Update README.md 613511c verified

Update README.md a08aeba verified

Update README.md 0836073 verified

Update README.md 806b358 verified

Update eva_vit_model.py af3d978 verified

Update README.md aa89ed1 verified

Update README.md bbc3253 verified

Update README.md a5bd512 verified

first commit b0b3b00

Update README.md 6906069 verified

Upload README.md 8d6cfcb verified

initial commit 470878c verified

Update README.md

c20153b
verified

Update README.md

9bac684
verified

Create LICENSE

650d069
verified

Update README.md

613511c
verified

Update README.md

a08aeba
verified

Update README.md

0836073
verified

Update README.md

806b358
verified

Update eva_vit_model.py

af3d978
verified

Update README.md

aa89ed1
verified

Update README.md

bbc3253
verified

Update README.md

a5bd512
verified

first commit

b0b3b00

Update README.md

6906069
verified

Upload README.md

8d6cfcb
verified

initial commit

470878c
verified