Instructions to use cbert33/Raon-1-mini-preview-Calibrated with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cbert33/Raon-1-mini-preview-Calibrated with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cbert33/Raon-1-mini-preview-Calibrated", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("cbert33/Raon-1-mini-preview-Calibrated", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use cbert33/Raon-1-mini-preview-Calibrated with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cbert33/Raon-1-mini-preview-Calibrated"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cbert33/Raon-1-mini-preview-Calibrated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/cbert33/Raon-1-mini-preview-Calibrated

SGLang

How to use cbert33/Raon-1-mini-preview-Calibrated with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cbert33/Raon-1-mini-preview-Calibrated" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cbert33/Raon-1-mini-preview-Calibrated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cbert33/Raon-1-mini-preview-Calibrated" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cbert33/Raon-1-mini-preview-Calibrated",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use cbert33/Raon-1-mini-preview-Calibrated with Docker Model Runner:
```
docker model run hf.co/cbert33/Raon-1-mini-preview-Calibrated
```

Raon-1-mini-preview-Calibrated / required-native-runtime-patch /README.md

cbert33

Upload calibrated FP8 KV model card and metadata

ff7679d verified 8 days ago

preview code

Raw History Blame Contribute Delete

1.28 kB

# Required native Raon FP8 KV patch

Based on `krafton-ai/vllm` revision `9886842e20102e330389a505debe80d4c834940a`.

Copy `mla_attention.py` and `kv_cache_quant.py` into `vllm/models/raon1_mini/`
in that pinned source tree before building the runtime.
The relative import requires the helper beside the attention file.
Keep the remaining Raon implementation at the same revision.

The adapter is scoped to Raon attention with the original ModelOpt MXFP8 configuration
and explicit `kv_cache_quant_algo="FP8"`. Projection and expert quantization are unchanged.
It creates scalar KV parameters through BaseKVCacheMethod and applies native scale
finalization before Raon absorbs weights and computes reciprocal scales.
The cached-prefix/chunked-prefill helper multiplies gathered plain-FP8 bytes by the
K scale before latent projection or use of the positional tail.

Calibration targets E4M3FN with a maximum of 448. It does not qualify E5M2,
`fp8_ds_mla`, or NVFP4 cache execution. Existing BF16 gather paths remain unchanged.

CPU contracts verify scalar loading, finalization, reciprocal scales, and nonunit
cached-prefix restoration. Full native GPU serving and cached-prefix/chunked-prefill
GPU execution remain untested. Do not infer deployment acceptance from these checks.

	# Required native Raon FP8 KV patch

	Based on `krafton-ai/vllm` revision `9886842e20102e330389a505debe80d4c834940a`.

	Copy `mla_attention.py` and `kv_cache_quant.py` into `vllm/models/raon1_mini/`
	in that pinned source tree before building the runtime.
	The relative import requires the helper beside the attention file.
	Keep the remaining Raon implementation at the same revision.

	The adapter is scoped to Raon attention with the original ModelOpt MXFP8 configuration
	and explicit `kv_cache_quant_algo="FP8"`. Projection and expert quantization are unchanged.
	It creates scalar KV parameters through BaseKVCacheMethod and applies native scale
	finalization before Raon absorbs weights and computes reciprocal scales.
	The cached-prefix/chunked-prefill helper multiplies gathered plain-FP8 bytes by the
	K scale before latent projection or use of the positional tail.

	Calibration targets E4M3FN with a maximum of 448. It does not qualify E5M2,
	`fp8_ds_mla`, or NVFP4 cache execution. Existing BF16 gather paths remain unchanged.

	CPU contracts verify scalar loading, finalization, reciprocal scales, and nonunit
	cached-prefix restoration. Full native GPU serving and cached-prefix/chunked-prefill
	GPU execution remain untested. Do not infer deployment acceptance from these checks.