Instructions to use naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B

SGLang

How to use naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B with Docker Model Runner:
```
docker model run hf.co/naver-hyperclovax/HyperCLOVAX-SEED-Vision-Instruct-3B
```

vLLM serve error :: AttributeError: 'HCXVisionConfig' object has no attribute 'num_attention_heads'

#18

by horang22 - opened May 7, 2025

Discussion

horang22

May 7, 2025

ERROR 05-07 14:12:09 [core.py:396] EngineCore failed to start.
ERROR 05-07 14:12:09 [core.py:396] Traceback (most recent call last):
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 387, in run_engine_core
ERROR 05-07 14:12:09 [core.py:396] engine_core = EngineCoreProc(*args, **kwargs)
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 329, in init
ERROR 05-07 14:12:09 [core.py:396] super().init(vllm_config, executor_class, log_stats,
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 64, in init
ERROR 05-07 14:12:09 [core.py:396] self.model_executor = executor_class(vllm_config)
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/executor/executor_base.py", line 52, in init
ERROR 05-07 14:12:09 [core.py:396] self._init_executor()
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/executor/uniproc_executor.py", line 46, in _init_executor
ERROR 05-07 14:12:09 [core.py:396] self.collective_rpc("init_device")
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/executor/uniproc_executor.py", line 56, in collective_rpc
ERROR 05-07 14:12:09 [core.py:396] answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/utils.py", line 2456, in run_method
ERROR 05-07 14:12:09 [core.py:396] return func(*args, **kwargs)
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/worker/worker_base.py", line 604, in init_device
ERROR 05-07 14:12:09 [core.py:396] self.worker.init_device() # type: ignore
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/v1/worker/gpu_worker.py", line 142, in init_device
ERROR 05-07 14:12:09 [core.py:396] self.model_runner: GPUModelRunner = GPUModelRunner(
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 118, in init
ERROR 05-07 14:12:09 [core.py:396] self.num_kv_heads = model_config.get_num_kv_heads(parallel_config)
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/config.py", line 1081, in get_num_kv_heads
ERROR 05-07 14:12:09 [core.py:396] total_num_kv_heads = self.get_total_num_kv_heads()
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/vllm/config.py", line 1073, in get_total_num_kv_heads
ERROR 05-07 14:12:09 [core.py:396] return self.hf_text_config.num_attention_heads
ERROR 05-07 14:12:09 [core.py:396] File "/home/horang/venv/llama/lib/python3.10/site-packages/transformers/configuration_utils.py", line 210, in getattribute
ERROR 05-07 14:12:09 [core.py:396] return super().getattribute(key)
ERROR 05-07 14:12:09 [core.py:396] AttributeError: 'HCXVisionConfig' object has no attribute 'num_attention_heads'

Env : docker container
transformers version : 4.51.3
vllm version : 0.8.5.post1
huggingface-hub version : 0.30.2

horang22 changed discussion title from vLLM serve error to vLLM serve error :: AttributeError: 'HCXVisionConfig' object has no attribute 'num_attention_heads' May 7, 2025

BDLEE

May 19, 2025

Hi, thank you for your interest in our model.
We are currently working on supporting vLLM (including the VLM module), and we expect to have support ready by the end of June.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment