Instructions to use HCHs/RivetCoder-9B-A4B-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HCHs/RivetCoder-9B-A4B-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HCHs/RivetCoder-9B-A4B-FP8", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("HCHs/RivetCoder-9B-A4B-FP8", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use HCHs/RivetCoder-9B-A4B-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HCHs/RivetCoder-9B-A4B-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HCHs/RivetCoder-9B-A4B-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/HCHs/RivetCoder-9B-A4B-FP8

SGLang

How to use HCHs/RivetCoder-9B-A4B-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HCHs/RivetCoder-9B-A4B-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HCHs/RivetCoder-9B-A4B-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HCHs/RivetCoder-9B-A4B-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HCHs/RivetCoder-9B-A4B-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use HCHs/RivetCoder-9B-A4B-FP8 with Docker Model Runner:
```
docker model run hf.co/HCHs/RivetCoder-9B-A4B-FP8
```

RivetCoder-9B-A4B-FP8 / provenance /assembly.json

HCHs

Upload RivetCoder-9B-A4B-FP8 TorchAO checkpoint

cd55fe8 verified about 15 hours ago

Raw

History Blame Contribute Delete

8.89 kB

	{
	"created_at": "2026-08-27T13:15:51.908322+00:00",
	"output": {
	"config_sha256": "8582a503ded278e710a929f31848356b8ca95ea76a957b0842d6d5871ad16e1a",
	"configuration_code_sha256": "922dfb636b84d09d2f92ba0c5bc226d00d7b41999c7e9aef1939ccb022192427",
	"directory": "<workspace>\\RivetCoder-9B-A4B",
	"index_sha256": "939b9f2bf0c48523d6a1a2ef93d987876a3e20613b33fe3715bf6b78bbc2a1bb",
	"linked_weight_files": [
	{
	"method": "hardlink",
	"name": "host-model-00001-of-00002.safetensors",
	"role": "host_weight",
	"size": 5329406264,
	"source": "<workspace>\\models\\LiquidAI\\LFM2.5-2.6B\\model-00001-of-00002.safetensors"
	},
	{
	"method": "hardlink",
	"name": "host-model-00002-of-00002.safetensors",
	"role": "host_weight",
	"size": 65021192,
	"source": "<workspace>\\models\\LiquidAI\\LFM2.5-2.6B\\model-00002-of-00002.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00001-of-00012.safetensors",
	"role": "expert_weight",
	"size": 1065764144,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00001.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00002-of-00012.safetensors",
	"role": "expert_weight",
	"size": 1065764160,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00002.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00003-of-00012.safetensors",
	"role": "expert_weight",
	"size": 1065632784,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00003.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00004-of-00012.safetensors",
	"role": "expert_weight",
	"size": 1065764176,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00004.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00005-of-00012.safetensors",
	"role": "expert_weight",
	"size": 1065764296,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00005.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00006-of-00012.safetensors",
	"role": "expert_weight",
	"size": 1065632912,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00006.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00007-of-00012.safetensors",
	"role": "expert_weight",
	"size": 1065764280,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00007.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00008-of-00012.safetensors",
	"role": "expert_weight",
	"size": 1065764296,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00008.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00009-of-00012.safetensors",
	"role": "expert_weight",
	"size": 1065632912,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00009.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00010-of-00012.safetensors",
	"role": "expert_weight",
	"size": 1065764288,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00010.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00011-of-00012.safetensors",
	"role": "expert_weight",
	"size": 1065764296,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00011.safetensors"
	},
	{
	"method": "hardlink",
	"name": "expert-model-00012-of-00012.safetensors",
	"role": "expert_weight",
	"size": 360716032,
	"source": "<workspace>\\artifacts\\folded-glm-experts-fixed16-top4-identity-hadamard\\folded-experts-00012.safetensors"
	},
	{
	"method": "hardlink",
	"name": "routing-controls.safetensors",
	"role": "routing_controls",
	"size": 4191720,
	"source": "<workspace>\\artifacts\\router-training-qwen-all30-fixed16-top4\\checkpoint-00000060\\trainable.safetensors"
	}
	],
	"max_shard_bytes": 2139130024,
	"model_card_sha256": "ec7558e18405570c051065b40d0811fe5635fca0b057825b0c9be3490846f676",
	"modeling_code_sha256": "2fe578e98265fa3704aa6971d1234784c4e1ef6ebf9a6df6661c831e340d3f33",
	"physical_duplicate_keys": 0,
	"physical_tensor_count": 1856,
	"removed_precedence_sources": [
	"expert-model-00001-of-00012.safetensors",
	"expert-model-00002-of-00012.safetensors",
	"expert-model-00003-of-00012.safetensors",
	"expert-model-00004-of-00012.safetensors",
	"expert-model-00005-of-00012.safetensors",
	"expert-model-00006-of-00012.safetensors",
	"expert-model-00007-of-00012.safetensors",
	"expert-model-00008-of-00012.safetensors",
	"expert-model-00009-of-00012.safetensors",
	"expert-model-00010-of-00012.safetensors",
	"expert-model-00011-of-00012.safetensors",
	"expert-model-00012-of-00012.safetensors",
	"host-model-00001-of-00002.safetensors",
	"host-model-00002-of-00002.safetensors",
	"routing-controls.safetensors"
	],
	"shards": [
	{
	"name": "model-00001-of-00009.safetensors",
	"sha256": "65eb65a6001739bee7959aebe2d6ff564065db69658b768e05ee085f2dbceac4",
	"size": 2139129752,
	"tensor_count": 261
	},
	{
	"name": "model-00002-of-00009.safetensors",
	"sha256": "b689619b9816a94218227afd89ce1c29580b6ec27307c971fabf450e27c6586f",
	"size": 2139129608,
	"tensor_count": 260
	},
	{
	"name": "model-00003-of-00009.safetensors",
	"sha256": "8603121c0ae3e5062526deea09a67f6f17964d8d4018772050aeb368e6767e3d",
	"size": 2139129848,
	"tensor_count": 260
	},
	{
	"name": "model-00004-of-00009.safetensors",
	"sha256": "427ed53be708fbc114263b296a1800cde40ae590b47ac5c65a3afbda5f449fe3",
	"size": 2139130024,
	"tensor_count": 261
	},
	{
	"name": "model-00005-of-00009.safetensors",
	"sha256": "e6d934e3c3709e322cf0b9cba7f5962c31aa50f83fab768b49d2677c08c0dacd",
	"size": 2139129840,
	"tensor_count": 260
	},
	{
	"name": "model-00006-of-00009.safetensors",
	"sha256": "3c2bfeae45226c395ee5a80fc5bb40dcdab0564901746e1e204b2ed1f986097c",
	"size": 2107698648,
	"tensor_count": 181
	},
	{
	"name": "model-00007-of-00009.safetensors",
	"sha256": "1fd608240effcf5db7819f0ec066d938b7f6bbd8a13d48f9bdd8639c217d7069",
	"size": 2126742608,
	"tensor_count": 111
	},
	{
	"name": "model-00008-of-00009.safetensors",
	"sha256": "1b15e4727b4b81c5e9d99ba2537d7cf72c6954c6cb235a5821723b48e4fa0701",
	"size": 2137241792,
	"tensor_count": 119
	},
	{
	"name": "model-00009-of-00009.safetensors",
	"sha256": "dbc94cfe4bca880602ca8bc58283d2e2147501e731cb69ae639bb557d3f315a7",
	"size": 411078784,
	"tensor_count": 143
	}
	]
	},
	"repo_id": "HCHs/RivetCoder-9B-A4B",
	"schema": "rivetcoder-hf-package",
	"schema_version": 1,
	"sources": {
	"expert_index_sha256": "c156f43793b0cee11290e32619c767d86eaaf4ca1a6ec268ece9eb8d62b6880c",
	"host_index_sha256": "1be35d0d99dc56f68e6c5b306f7d4d99b6da97d1ab41ee753832bf1495655c4a",
	"routing_controls_sha256": "740f15e9bcf68efcb4d9c0f4b2bcfebff66ba199aead9cd2666156dffc60b1ad",
	"training_summary_sha256": "6cef5c9b40ce3f184f8ab32a58cc2a795dedb220b6dc324c6eb5a1c429613c68"
	},
	"weights": {
	"combined_tensor_count": 1856,
	"combined_total_size": 17478172784,
	"expert_shards": 12,
	"final_shards": 9,
	"folded_total_size": 12083529600,
	"host_shards": 2,
	"host_total_size": 5394397184,
	"physical_duplicate_keys": 0,
	"repacked": true,
	"replaced_warmstart_router_bytes": 3932160,
	"routing_control_files": 1,
	"trained_control_bytes": 4178160
	}
	}