Instructions to use spidyun/kmoshi with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use spidyun/kmoshi with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="spidyun/kmoshi")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForSeq2SeqLM
model = AutoModelForSeq2SeqLM.from_pretrained("spidyun/kmoshi", device_map="auto")

Moshi

How to use spidyun/kmoshi with Moshi:

# pip install moshi
# Run the interactive web server
python -m moshi.server --hf-repo "spidyun/kmoshi"
# Then open https://localhost:8998 in your browser

# pip install moshi
import torch
from moshi.models import loaders

# Load checkpoint info from HuggingFace
checkpoint = loaders.CheckpointInfo.from_hf_repo("spidyun/kmoshi")

# Load the Mimi audio codec
mimi = checkpoint.get_mimi(device="cuda")
mimi.set_num_codebooks(8)

# Encode audio (24kHz, mono)
wav = torch.randn(1, 1, 24000 * 10)  # [batch, channels, samples]
with torch.no_grad():
    codes = mimi.encode(wav.cuda())
    decoded = mimi.decode(codes)

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use spidyun/kmoshi with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "spidyun/kmoshi"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spidyun/kmoshi",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/spidyun/kmoshi

SGLang

How to use spidyun/kmoshi with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "spidyun/kmoshi" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spidyun/kmoshi",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "spidyun/kmoshi" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "spidyun/kmoshi",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use spidyun/kmoshi with Docker Model Runner:
```
docker model run hf.co/spidyun/kmoshi
```

kmoshi / README.md

spidyun

Upload folder using huggingface_hub

0f48a55 verified 25 days ago

preview code

Raw

History Blame Contribute Delete

1.8 kB

metadata

library_name: transformers
license: other
license_name: nvidia-open-model-license
license_link: >-
  https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
base_model:
  - Qwen/Qwen3-8B
  - nvidia/personaplex-7b-v1
base_model_relation: merge
tags:
  - kmoshi
  - moshi
  - full-duplex
  - speech
  - korean

KMoshi (init)

An assembled checkpoint that couples the Qwen3-8B temporal backbone (Korean-capable text LLM) with the PersonaPlex audio stack (Mimi codec, depth decoder with dep_q=16, and 16 audio input embeddings), toward a Korean full-duplex spoken dialogue model.

Status: before coupling training. The backbone was swapped from Helium to Qwen3-8B, so the audio stack is not yet aligned with it — audio output is expected to be noise. The text stream is already fluent (evidence that the backbone transplant is numerically correct). Background and assembly code: https://github.com/un1876/k-moshi

Usage

# pip install "git+https://github.com/un1876/k-moshi.git@main"  (not in upstream transformers)
from transformers.models.kmoshi.modeling_kmoshi import KmoshiForConditionalGeneration
model = KmoshiForConditionalGeneration.from_pretrained("spidyun/kmoshi")

Attribution / Licenses

Backbone weights: Qwen/Qwen3-8B — Apache-2.0
Audio stack (Mimi codec, depth decoder, audio input embeddings): nvidia/personaplex-7b-v1 — NVIDIA Open Model License (+ CC-BY-4.0). See LICENSE / Notice in this repository.
The Mimi codec is byte-identical to stock Moshi (kyutai, CC-BY-4.0).
depth_decoder.text_embed_tokens is re-initialized for the new tokenizer vocab (151936) and belongs to the coupling-training stage.