Instructions to use OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov")
model = AutoModelForCausalLM.from_pretrained("OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov

SGLang

How to use OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov with Docker Model Runner:
```
docker model run hf.co/OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov
```

Check 'm_element_type.is_static()' failed at src\inference\src\dev\make_tensor.cpp:58

by lopatnov - opened 7 days ago

Discussion

lopatnov

7 days ago

Hi, can't start ovms with this model on Acer SFG14-73T with Intel Arc (32Gb RAM). Could you help?
In Intel Graphics Software I changed Graphics -> Shared GPU/NPU Memory Override -> Memory Limit 76% (24.1Gb)

I changed graph.pbtxt variables:
max_num_seqs:1,
device: "MULTI:GPU,CPU",

OpenVINO Model Server as service on Windows
My logs:
C:\models>ovms --config_path qwen-coder3-config.json --port 9000 --rest_port 8000
[2026-07-12 12:00:24.838][6680][serving][info][server.cpp:117] OpenVINO Model Server 2026.2.1.1122f03bf
[2026-07-12 12:00:24.838][6680][serving][info][server.cpp:118] OpenVINO backend 2026.2.1-21919-ede283a88e3-releases/2026/2
[2026-07-12 12:00:24.838][6680][serving][info][server.cpp:121] OpenVINO GenAI backend 2026.2.1.0-3123-7dea0459b2a
[2026-07-12 12:00:24.838][6680][serving][info][pythoninterpretermodule.cpp:37] PythonInterpreterModule starting
[2026-07-12 12:00:24.838][6680][serving][info][pythoninterpretermodule.cpp:40] Initializing python interpreter
Python version:
3.12.10 (tags/v3.12.10:0cc8128, Apr 8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
Python sys.path output:
['', 'c:\ovms\python\python312', 'c:\ovms\python', 'c:\ovms\python\Scripts', 'c:\ovms\python\Lib\site-packages']
[2026-07-12 12:00:24.854][6680][serving][info][pythoninterpretermodule.cpp:59] PythonInterpreterModule started
[2026-07-12 12:00:25.005][6680][modelmanager][info][modelmanager.cpp:180] Available devices for Open VINO: CPU, GPU, NPU
[2026-07-12 12:00:25.007][6680][serving][info][capimodule.cpp:40] C-APIModule starting
[2026-07-12 12:00:25.008][6680][serving][info][capimodule.cpp:42] C-APIModule started
[2026-07-12 12:00:25.008][6680][serving][info][grpcservermodule.cpp:110] GRPCServerModule starting
[2026-07-12 12:00:25.008][6680][serving][info][grpcservermodule.cpp:137] Binding gRPC server to address: 0.0.0.0:9000
[2026-07-12 12:00:25.009][6680][serving][info][grpcservermodule.cpp:192] GRPCServerModule started
[2026-07-12 12:00:25.009][6680][serving][info][grpcservermodule.cpp:193] Started gRPC server on port 9000
[2026-07-12 12:00:25.009][6680][serving][info][httpservermodule.cpp:35] HTTPServerModule starting
[2026-07-12 12:00:25.009][6680][serving][info][httpservermodule.cpp:39] Will start 22 REST workers
[2026-07-12 12:00:25.010][13280][serving][info][drogon_http_server.cpp:157] Binding REST server to address: 0.0.0.0:8000
[2026-07-12 12:00:25.069][6680][serving][info][drogon_http_server.cpp:187] REST server listening on port 8000 with 22 unary threads and 22 streaming threads
[2026-07-12 12:00:25.069][6680][serving][info][http_server.cpp:249] API key not provided via --api_key_file or API_KEY environment variable. Authentication will be disabled.
[2026-07-12 12:00:25.070][6680][serving][info][httpservermodule.cpp:52] HTTPServerModule started
[2026-07-12 12:00:25.070][6680][serving][info][httpservermodule.cpp:53] Started REST server at 0.0.0.0:8000
[2026-07-12 12:00:25.070][6680][serving][info][servablemanagermodule.cpp:51] ServableManagerModule starting
[2026-07-12 12:00:25.075][6680][serving][info][mediapipegraphdefinition.cpp:101] Graph queue globally disabled via OVMS_GRAPH_QUEUE_OFF=1 for mediapipe: qwen3-coder
[2026-07-12 12:00:25.078][6680][serving][info][mediapipegraphdefinition.cpp:479] MediapipeGraphDefinition initializing graph nodes
[2026-07-12 12:00:25.079][6680][modelmanager][info][servable_initializer.cpp:439] Initializing Language Model Continuous Batching servable
[2026-07-12 12:06:39.775][6680][serving][error][servable_initializer.cpp:240] Error during llm node initialization for models_path: c:/models/OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov./ exception: Exception from src\inference\src\cpp\compiled_model.cpp:128:
Check 'm_element_type.is_static()' failed at src\inference\src\dev\make_tensor.cpp:58

[2026-07-12 12:06:39.778][6680][modelmanager][error][servable_initializer.cpp:444] Error during LLM node resources initialization: The LLM Node resource initialization failed
[2026-07-12 12:06:39.779][6680][serving][error][llm_node_initializer.cpp:60] Failed to process LLM node graph qwen3-coder
[2026-07-12 12:06:39.780][6680][modelmanager][info][pipelinedefinitionstatus.hpp:60] Mediapipe: qwen3-coder state changed to: LOADING_PRECONDITION_FAILED after handling: ValidationFailedEvent:
[2026-07-12 12:06:39.781][6680][serving][info][servablemanagermodule.cpp:55] ServableManagerModule started
[2026-07-12 12:06:39.782][14908][modelmanager][info][modelmanager.cpp:1060] Started model manager thread
[2026-07-12 12:06:39.782][1280][modelmanager][info][modelmanager.cpp:1079] Started cleaner thread

lopatnov

7 days ago

If I'll use cmd from model card, I'll have another error:
C:\models>start-qwen-coder3.bat

C:\models>ovms.exe --rest_port 8000 --source_model OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov --model_repository_path models --tool_parser qwen3coder --target_device GPU --cache_size 2 --task text_generation
[2026-07-12 13:47:23.663][16560][serving][info][server.cpp:117] OpenVINO Model Server 2026.2.1.1122f03bf
[2026-07-12 13:47:23.663][16560][serving][info][server.cpp:118] OpenVINO backend 2026.2.1-21919-ede283a88e3-releases/2026/2
[2026-07-12 13:47:23.663][16560][serving][info][server.cpp:121] OpenVINO GenAI backend 2026.2.1.0-3123-7dea0459b2a
[2026-07-12 13:47:23.664][16560][serving][info][pythoninterpretermodule.cpp:37] PythonInterpreterModule starting
[2026-07-12 13:47:23.664][16560][serving][info][pythoninterpretermodule.cpp:40] Initializing python interpreter
Python version:
3.12.10 (tags/v3.12.10:0cc8128, Apr 8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
Python sys.path output:
['', 'c:\ovms\python\python312', 'c:\ovms\python', 'c:\ovms\python\Scripts', 'c:\ovms\python\Lib\site-packages']
[2026-07-12 13:47:23.679][16560][serving][info][pythoninterpretermodule.cpp:59] PythonInterpreterModule started
[2026-07-12 13:47:23.826][16560][modelmanager][info][modelmanager.cpp:180] Available devices for Open VINO: CPU, GPU, NPU
[2026-07-12 13:47:23.829][16560][serving][info][capimodule.cpp:40] C-APIModule starting
[2026-07-12 13:47:23.829][16560][serving][info][capimodule.cpp:42] C-APIModule started
[2026-07-12 13:47:23.829][16560][serving][info][grpcservermodule.cpp:110] GRPCServerModule starting
[2026-07-12 13:47:23.829][16560][serving][info][grpcservermodule.cpp:114] GRPCServerModule started
[2026-07-12 13:47:23.829][16560][serving][info][grpcservermodule.cpp:115] Port was not set. GRPC server will not be started.
[2026-07-12 13:47:23.829][16560][serving][info][httpservermodule.cpp:35] HTTPServerModule starting
[2026-07-12 13:47:23.829][16560][serving][info][httpservermodule.cpp:39] Will start 22 REST workers
[2026-07-12 13:47:23.830][9600][serving][info][drogon_http_server.cpp:157] Binding REST server to address: 0.0.0.0:8000
[2026-07-12 13:47:23.892][16560][serving][info][drogon_http_server.cpp:187] REST server listening on port 8000 with 22 unary threads and 22 streaming threads
[2026-07-12 13:47:23.892][16560][serving][info][http_server.cpp:249] API key not provided via --api_key_file or API_KEY environment variable. Authentication will be disabled.
[2026-07-12 13:47:23.893][16560][serving][info][httpservermodule.cpp:52] HTTPServerModule started
[2026-07-12 13:47:23.893][16560][serving][info][httpservermodule.cpp:53] Started REST server at 0.0.0.0:8000
[INFO] LFS resume: attempts=5 interval=10 s
remote: Enumerating objects: 31, done.
remote: Total 31 (delta 0), reused 0 (delta 0), pack-reused 31 (from 1)
Receiving objects: 100% (31/31), 64.04 KiB | 0.00 B/s, done.
Resolving deltas: 100% (6/6), done.
Checking out files: 0% (0/19)
Downloading lfs size: 2.19 MB file: C:/models/models/OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov/openvino_detokenizer.bin
Progress: [##################################################] 100.00% of 2.19 MB [2.19 MB/s]
Checking out files: 47% (9/19)
Downloading lfs size: 16.31 GB file: C:/models/models/OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov/openvino_model.bin
Progress: [##### ] 11.01% of 16.31 GB [7.71 MB/s] [ERROR] Error downloading object: openvino_model.bin (03936cfcf2b9edd8d5a492e67eeb90862b7aeaef724b748154ebc2b55fe90d31)
[ERROR] LFS object download failed: Operation was aborted by an application callback
[ERROR] cURL detail: Callback aborted
[ERROR] URL: https://us.aws.cdn.hf.co/xet-bridge-us/69a053ef4f4adf6424b801ec/31651009ded69165c6bd1ce536d337312d75e6aedc33a0a035e5abd58a17bcf1?response-content-disposition=inline;+filename*=UTF-8''openvino_model.bin;+filename="openvino_model.bin";&user_id=public&X-Xet-Cas-Uid=public&response-content-type=application/octet-stream&Expires=1783856845&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly91cy5hd3MuY2RuLmhmLmNvL3hldC1icmlkZ2UtdXMvNjlhMDUzZWY0ZjRhZGY2NDI0YjgwMWVjLzMxNjUxMDA5ZGVkNjkxNjVjNmJkMWNlNTM2ZDMzNzMxMmQ3NWU2YWVkYzMzYTBhMDM1ZTVhYmQ1OGExN2JjZjFcXD9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPWlubGluZSUzQitmaWxlbmFtZSUyQSUzRFVURi04JTI3JTI3b3BlbnZpbm9fbW9kZWwuYmluJTNCK2ZpbGVuYW1lJTNEJTIyb3BlbnZpbm9fbW9kZWwuYmluJTIyJTNCJnVzZXJfaWQ9cHVibGljJlgtWGV0LUNhcy1VaWQ9cHVibGljJnJlc3BvbnNlLWNvbnRlbnQtdHlwZT1hcHBsaWNhdGlvbiUyRm9jdGV0LXN0cmVhbSIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiRXBvY2hUaW1lIjoxNzgzODU2ODQ1fX19XX0_&Signature=MEUCIHZo3EGJtjs0zIHsTIY2Cm5SXDbbTI2lW5Oma3D56VzuAiEA31ZL7h-P47HbgUrETUrqjWOgokESAPWOU~NR6cAHvW4_&Key-Pair-Id=01KAYHXK2CBJSW0YZTMNXK9W1M
[ERROR] HTTP status: 200
[ERROR] Remote endpoint: 35.181.67.163:443 (local: 192.168.0.3:1756)
[ERROR] LFS download failed for C:/models/models/OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov/openvino_model.bin
[ERROR] LFS download failed for C:/models/models/OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov/openvino_tokenizer.bin
[ERROR] LFS download failed for C:/models/models/OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov/tokenizer.json
Checking out files: 100% (19/19), done.
[2026-07-12 13:51:20.790][16560][serving][error][libgit2.cpp:886] [ERROR] LFS download failed for C:/models/models/OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov/tokenizer.json

[2026-07-12 13:51:20.790][16560][serving][error][libgit2.cpp:1361] Model download failed: LFS errors recorded above. Re-run the same command to resume, or use --overwrite_models to restart from scratch.
[2026-07-12 13:51:20.790][16560][serving][info][grpcservermodule.cpp:201] GRPCServerModule shutting down
[2026-07-12 13:51:20.790][16560][serving][info][grpcservermodule.cpp:211] GRPCServerModule shutdown
[2026-07-12 13:51:20.791][16560][serving][info][httpservermodule.cpp:59] HTTPServerModule shutting down
[2026-07-12 13:51:20.794][16560][serving][info][httpservermodule.cpp:64] Shutdown HTTP server
[2026-07-12 13:51:20.794][16560][serving][info][servablemanagermodule.cpp:65] ServableManagerModule shutting down
[2026-07-12 13:51:20.823][16560][serving][info][servablemanagermodule.cpp:71] ServableManagerModule shutdown
[2026-07-12 13:51:20.823][16560][serving][info][pythoninterpretermodule.cpp:70] PythonInterpreterModule shutting down
[2026-07-12 13:51:20.823][16560][serving][info][pythoninterpretermodule.cpp:74] PythonInterpreterModule shutdown
[2026-07-12 13:51:20.827][16560][serving][info][capimodule.cpp:50] C-APIModule shutting down
[2026-07-12 13:51:20.827][16560][serving][info][capimodule.cpp:52] C-APIModule shutdown
Terminate batch job (Y/N)? Н
Terminate batch job (Y/N)? Y

C:\models>start-qwen-coder3.bat

C:\models>ovms.exe --rest_port 8000 --source_model OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov --model_repository_path . --tool_parser qwen3coder --target_device GPU --cache_size 2 --task text_generation
[2026-07-12 13:51:38.779][15888][serving][info][server.cpp:117] OpenVINO Model Server 2026.2.1.1122f03bf
[2026-07-12 13:51:38.779][15888][serving][info][server.cpp:118] OpenVINO backend 2026.2.1-21919-ede283a88e3-releases/2026/2
[2026-07-12 13:51:38.779][15888][serving][info][server.cpp:121] OpenVINO GenAI backend 2026.2.1.0-3123-7dea0459b2a
[2026-07-12 13:51:38.780][15888][serving][info][pythoninterpretermodule.cpp:37] PythonInterpreterModule starting
[2026-07-12 13:51:38.780][15888][serving][info][pythoninterpretermodule.cpp:40] Initializing python interpreter
Python version:
3.12.10 (tags/v3.12.10:0cc8128, Apr 8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
Python sys.path output:
['', 'c:\ovms\python\python312', 'c:\ovms\python', 'c:\ovms\python\Scripts', 'c:\ovms\python\Lib\site-packages']
[2026-07-12 13:51:38.796][15888][serving][info][pythoninterpretermodule.cpp:59] PythonInterpreterModule started
[2026-07-12 13:51:38.949][15888][modelmanager][info][modelmanager.cpp:180] Available devices for Open VINO: CPU, GPU, NPU
[2026-07-12 13:51:38.952][15888][serving][info][capimodule.cpp:40] C-APIModule starting
[2026-07-12 13:51:38.952][15888][serving][info][capimodule.cpp:42] C-APIModule started
[2026-07-12 13:51:38.952][15888][serving][info][grpcservermodule.cpp:110] GRPCServerModule starting
[2026-07-12 13:51:38.952][15888][serving][info][grpcservermodule.cpp:114] GRPCServerModule started
[2026-07-12 13:51:38.952][15888][serving][info][grpcservermodule.cpp:115] Port was not set. GRPC server will not be started.
[2026-07-12 13:51:38.952][15888][serving][info][httpservermodule.cpp:35] HTTPServerModule starting
[2026-07-12 13:51:38.952][15888][serving][info][httpservermodule.cpp:39] Will start 22 REST workers
[2026-07-12 13:51:38.953][9516][serving][info][drogon_http_server.cpp:157] Binding REST server to address: 0.0.0.0:8000
[2026-07-12 13:51:39.014][15888][serving][info][drogon_http_server.cpp:187] REST server listening on port 8000 with 22 unary threads and 22 streaming threads
[2026-07-12 13:51:39.014][15888][serving][info][http_server.cpp:249] API key not provided via --api_key_file or API_KEY environment variable. Authentication will be disabled.
[2026-07-12 13:51:39.014][15888][serving][info][httpservermodule.cpp:52] HTTPServerModule started
[2026-07-12 13:51:39.014][15888][serving][info][httpservermodule.cpp:53] Started REST server at 0.0.0.0:8000
[INFO] LFS resume: attempts=5 interval=10 s
[2026-07-12 13:51:39.163][15888][serving][info][libgit2.cpp:1282] Path already exists on local filesystem. Skipping download to path: .\OpenVINO\Qwen3-Coder-30B-A3B-Instruct-int4-ov
Model: OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov downloaded to: .\OpenVINO\Qwen3-Coder-30B-A3B-Instruct-int4-ov
Graph: graph.pbtxt created in: .\OpenVINO\Qwen3-Coder-30B-A3B-Instruct-int4-ov
[2026-07-12 13:51:39.165][15888][serving][info][servablemanagermodule.cpp:51] ServableManagerModule starting
[2026-07-12 13:51:39.194][15888][serving][info][mediapipegraphdefinition.cpp:101] Graph queue globally disabled via OVMS_GRAPH_QUEUE_OFF=1 for mediapipe: OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov
[2026-07-12 13:51:39.195][15888][serving][info][mediapipegraphdefinition.cpp:479] MediapipeGraphDefinition initializing graph nodes
[2026-07-12 13:51:39.195][15888][modelmanager][info][servable_initializer.cpp:439] Initializing Language Model Continuous Batching servable
[2026-07-12 13:56:23.818][15888][serving][error][servable_initializer.cpp:240] Error during llm node initialization for models_path: C:\models.\OpenVINO\Qwen3-Coder-30B-A3B-Instruct-int4-ov./ exception: Exception from src\inference\src\cpp\core.cpp:120:
Exception from src\inference\src\dev\plugin.cpp:54:
Check 'false' failed at src\plugins\intel_gpu\src\plugin\program_builder.cpp:168:
[GPU] ProgramBuilder build failed!
[CL ext] Can not allocate 402653184 bytes for USM Device. ptr: 0000000000000000, error: 0

[2026-07-12 13:56:23.822][15888][modelmanager][error][servable_initializer.cpp:444] Error during LLM node resources initialization: The LLM Node resource initialization failed
[2026-07-12 13:56:23.823][15888][serving][error][llm_node_initializer.cpp:60] Failed to process LLM node graph OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov
[2026-07-12 13:56:23.825][15888][modelmanager][info][pipelinedefinitionstatus.hpp:60] Mediapipe: OpenVINO/Qwen3-Coder-30B-A3B-Instruct-int4-ov state changed to: LOADING_PRECONDITION_FAILED after handling: ValidationFailedEvent:
[2026-07-12 13:56:23.827][15888][modelmanager][error][modelmanager.cpp:208] Couldn't start model manager
[2026-07-12 13:56:23.827][15888][serving][error][servablemanagermodule.cpp:58] ovms::ModelManager::Start() Error: The LLM Node resource initialization failed
[2026-07-12 13:56:23.828][15888][serving][info][grpcservermodule.cpp:201] GRPCServerModule shutting down
[2026-07-12 13:56:23.828][15888][serving][info][grpcservermodule.cpp:211] GRPCServerModule shutdown
[2026-07-12 13:56:23.828][15888][serving][info][httpservermodule.cpp:59] HTTPServerModule shutting down
[2026-07-12 13:56:23.847][15888][serving][info][httpservermodule.cpp:64] Shutdown HTTP server
[2026-07-12 13:56:23.847][15888][serving][info][servablemanagermodule.cpp:65] ServableManagerModule shutting down
[2026-07-12 13:56:27.953][15888][serving][info][servablemanagermodule.cpp:71] ServableManagerModule shutdown
[2026-07-12 13:56:27.954][15888][serving][info][pythoninterpretermodule.cpp:70] PythonInterpreterModule shutting down
[2026-07-12 13:56:27.955][15888][serving][info][pythoninterpretermodule.cpp:74] PythonInterpreterModule shutdown
[2026-07-12 13:56:27.970][15888][serving][info][capimodule.cpp:50] C-APIModule shutting down
[2026-07-12 13:56:27.970][15888][serving][info][capimodule.cpp:52] C-APIModule shutdown

C:\models>

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment