Instructions to use spidyun/kmoshi with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use spidyun/kmoshi with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="spidyun/kmoshi")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForSeq2SeqLM
model = AutoModelForSeq2SeqLM.from_pretrained("spidyun/kmoshi", device_map="auto")

Moshi

How to use spidyun/kmoshi with Moshi:

# pip install moshi
# Run the interactive web server
python -m moshi.server --hf-repo "spidyun/kmoshi"
# Then open https://localhost:8998 in your browser

# pip install moshi
import torch
from moshi.models import loaders

# Load checkpoint info from HuggingFace
checkpoint = loaders.CheckpointInfo.from_hf_repo("spidyun/kmoshi")

# Load the Mimi audio codec
mimi = checkpoint.get_mimi(device="cuda")
mimi.set_num_codebooks(8)

# Encode audio (24kHz, mono)
wav = torch.randn(1, 1, 24000 * 10)  # [batch, channels, samples]
with torch.no_grad():
    codes = mimi.encode(wav.cuda())
    decoded = mimi.decode(codes)

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use spidyun/kmoshi with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "spidyun/kmoshi"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spidyun/kmoshi",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/spidyun/kmoshi

SGLang

How to use spidyun/kmoshi with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "spidyun/kmoshi" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spidyun/kmoshi",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "spidyun/kmoshi" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spidyun/kmoshi",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use spidyun/kmoshi with Docker Model Runner:
```
docker model run hf.co/spidyun/kmoshi
```

kmoshi

File size: 1,796 Bytes

221dead
0f48a55
 
 
 
 
 
 
 
 
 
 
 
 
 
221dead
0f48a55

---
library_name: transformers
license: other
license_name: nvidia-open-model-license
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
base_model:
- Qwen/Qwen3-8B
- nvidia/personaplex-7b-v1
base_model_relation: merge
tags:
- kmoshi
- moshi
- full-duplex
- speech
- korean
---

# KMoshi (init)

An assembled checkpoint that couples the **Qwen3-8B** temporal backbone (Korean-capable
text LLM) with the **PersonaPlex** audio stack (Mimi codec, depth decoder with dep_q=16,
and 16 audio input embeddings), toward a Korean full-duplex spoken dialogue model.

> Status: **before coupling training.** The backbone was swapped from Helium to Qwen3-8B,
> so the audio stack is not yet aligned with it — **audio output is expected to be noise**.
> The text stream is already fluent (evidence that the backbone transplant is numerically
> correct). Background and assembly code: https://github.com/un1876/k-moshi

## Usage

    # pip install "git+https://github.com/un1876/k-moshi.git@main"  (not in upstream transformers)
    from transformers.models.kmoshi.modeling_kmoshi import KmoshiForConditionalGeneration
    model = KmoshiForConditionalGeneration.from_pretrained("spidyun/kmoshi")

## Attribution / Licenses

- Backbone weights: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) — Apache-2.0
- Audio stack (Mimi codec, depth decoder, audio input embeddings):
  [nvidia/personaplex-7b-v1](https://huggingface.co/nvidia/personaplex-7b-v1) —
  NVIDIA Open Model License (+ CC-BY-4.0). See `LICENSE` / `Notice` in this repository.
- The Mimi codec is byte-identical to stock Moshi (kyutai, CC-BY-4.0).
- `depth_decoder.text_embed_tokens` is re-initialized for the new tokenizer vocab
  (151936) and belongs to the coupling-training stage.