Instructions to use naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B

SGLang

How to use naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B with Docker Model Runner:
```
docker model run hf.co/naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B
```

Key Error and Value Error in AutoModelForCausalLM.from_pretrained()

by aiyaworld - opened Apr 24, 2025

Discussion

aiyaworld

Apr 24, 2025

I got error when I tried to execute AutoModelForCausalLM.from_pretrained().
I reinstalled transformers from github.
But code does not working.

code

model = AutoModelForCausalLM.from_pretrained(
model_path,
low_cpu_mem_usage=True,
device_map="sequential",
max_memory=max_memory,
offload_folder=os.path.join("./tmp/", f"{uuid4()}"),
offload_state_dict=True,
torch_dtype=DTYPE,
)

Error Message

KeyError Traceback (most recent call last)
/usr/local/lib/python3.11/dist-packages/transformers/models/auto/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
1154 try:
-> 1155 config_class = CONFIG_MAPPING[config_dict["model_type"]]
1156 except KeyError:

3 frames
KeyError: 'hyperclovax_vlm'

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call last)
/usr/local/lib/python3.11/dist-packages/transformers/models/auto/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
1155 config_class = CONFIG_MAPPING[config_dict["model_type"]]
1156 except KeyError:
-> 1157 raise ValueError(
1158 f"The checkpoint you are trying to load has model type {config_dict['model_type']} "
1159 "but Transformers does not recognize this architecture. This could be because of an "

ValueError: The checkpoint you are trying to load has model type hyperclovax_vlm but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

BDLEE

Apr 24, 2025

This comment has been hidden

DongHyunKim

Apr 24, 2025

Hello. May I ask if you have registered the model with AutoConfig/AutoModel ?
Could you please try running the following code?

AutoConfig.register("hyperclovax_vlm", HCXVisionConfig)
AutoModelForCausalLM.register(HCXVisionConfig, HCXVisionForCausalLM)

Thank you.

DongHyunKim

Apr 24, 2025

We have updated the examples to make it easier for users to use the model.

Could you please take a look at the updated code and try again?

Thank you.

DongHyunKim

May 13, 2025

Since the issue seems to be resolved, I will close this discussion.
If you have any additional issues, please feel free to reopen this discussion.

Thank you.

DongHyunKim changed discussion status to closed May 13, 2025

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment