Instructions to use spidyun/kmoshi with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use spidyun/kmoshi with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="spidyun/kmoshi")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForSeq2SeqLM
model = AutoModelForSeq2SeqLM.from_pretrained("spidyun/kmoshi", device_map="auto")

Moshi

How to use spidyun/kmoshi with Moshi:

# pip install moshi
# Run the interactive web server
python -m moshi.server --hf-repo "spidyun/kmoshi"
# Then open https://localhost:8998 in your browser

# pip install moshi
import torch
from moshi.models import loaders

# Load checkpoint info from HuggingFace
checkpoint = loaders.CheckpointInfo.from_hf_repo("spidyun/kmoshi")

# Load the Mimi audio codec
mimi = checkpoint.get_mimi(device="cuda")
mimi.set_num_codebooks(8)

# Encode audio (24kHz, mono)
wav = torch.randn(1, 1, 24000 * 10)  # [batch, channels, samples]
with torch.no_grad():
    codes = mimi.encode(wav.cuda())
    decoded = mimi.decode(codes)

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use spidyun/kmoshi with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "spidyun/kmoshi"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spidyun/kmoshi",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/spidyun/kmoshi

SGLang

How to use spidyun/kmoshi with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "spidyun/kmoshi" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spidyun/kmoshi",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "spidyun/kmoshi" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spidyun/kmoshi",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use spidyun/kmoshi with Docker Model Runner:
```
docker model run hf.co/spidyun/kmoshi
```

kmoshi / README.md

spidyun

Upload folder using huggingface_hub

0f48a55 verified 25 days ago

preview code

Raw

History Blame Contribute Delete

1.8 kB

	---
	library_name: transformers
	license: other
	license_name: nvidia-open-model-license
	license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
	base_model:
	- Qwen/Qwen3-8B
	- nvidia/personaplex-7b-v1
	base_model_relation: merge
	tags:
	- kmoshi
	- moshi
	- full-duplex
	- speech
	- korean
	---

	# KMoshi (init)

	An assembled checkpoint that couples the Qwen3-8B temporal backbone (Korean-capable
	text LLM) with the PersonaPlex audio stack (Mimi codec, depth decoder with dep_q=16,
	and 16 audio input embeddings), toward a Korean full-duplex spoken dialogue model.

	> Status: before coupling training. The backbone was swapped from Helium to Qwen3-8B,
	> so the audio stack is not yet aligned with it — audio output is expected to be noise.
	> The text stream is already fluent (evidence that the backbone transplant is numerically
	> correct). Background and assembly code: https://github.com/un1876/k-moshi

	## Usage

	# pip install "git+https://github.com/un1876/k-moshi.git@main" (not in upstream transformers)
	from transformers.models.kmoshi.modeling_kmoshi import KmoshiForConditionalGeneration
	model = KmoshiForConditionalGeneration.from_pretrained("spidyun/kmoshi")

	## Attribution / Licenses

	- Backbone weights: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) — Apache-2.0
	- Audio stack (Mimi codec, depth decoder, audio input embeddings):
	[nvidia/personaplex-7b-v1](https://huggingface.co/nvidia/personaplex-7b-v1) —
	NVIDIA Open Model License (+ CC-BY-4.0). See `LICENSE` / `Notice` in this repository.
	- The Mimi codec is byte-identical to stock Moshi (kyutai, CC-BY-4.0).
	- `depth_decoder.text_embed_tokens` is re-initialized for the new tokenizer vocab
	(151936) and belongs to the coupling-training stage.