| --- |
| license: mit |
| library_name: sglang |
| pipeline_tag: image-text-to-text |
| tags: |
| - rmdw |
| - jah |
| - private-ai |
| - fp8 |
| - mixture-of-experts |
| - vision |
| --- |
| |
| # Jah 5.0 |
|
|
| Jah 5.0 is the model behind [Kiyomi](https://kiyomibot.ai), RMDW's private AI platform. These are the exact weights serving Kiyomi customers right now. Open, downloadable, verifiable. Don't take our word for what powers your work: inspect it. |
|
|
| Same promise as every Jah release: open weights, no secrets. |
|
|
| ## What's new in 5.0 |
|
|
| - **A 12x jump in knowledge capacity over Jah 4.0.** 321B parameters with 18B active per token. Only a fraction of the model wakes for each token, so it streams fast for its size. |
| - **Thinking on demand.** Ask for deep reasoning and it thinks before it answers, in a cleanly separated reasoning channel. Ask plain and it answers instantly. |
| - **Perfect run on our hardest internal suite** with the orchestrated engine, and it solves problems single-shot that Jah 4.0 could not solve at any setting. |
| - **Vision.** Send a screenshot, an invoice, a whiteboard photo. Jah reads it like text. |
| - **Native 1M token context.** 1,048,576 positions, no scaling tricks. |
|
|
| ## Specs |
|
|
| | | | |
| |---|---| |
| | Parameters | 321B total, 18B active per token | |
| | Attention | Hybrid: linear attention + sparse attention, 45 layers | |
| | Context | 1,048,576 tokens native | |
| | Precision | FP8 (block quantized), BF16 sensitive paths | |
| | Modalities | Text + vision in, text out | |
| | Tool calling | Native | |
| | License | MIT | |
|
|
| ## Running it |
|
|
| Full precision wants roughly 330GB of GPU memory plus KV cache. A 4x RTX PRO 6000 class rig (384GB) serves it comfortably; that is exactly what Kiyomi runs on. Community 4-bit quantizations bring it to a 512GB Mac Studio. |
|
|
| Serve with SGLang (a current build with this architecture's day-0 support): |
|
|
| ```bash |
| python3 -m sglang.launch_server \ |
| --model-path RMDWLLC/Jah-5.0 \ |
| --tp 4 --mem-fraction-static 0.85 \ |
| --context-length 1048576 \ |
| --kv-cache-dtype bfloat16 \ |
| --reasoning-parser glm45 --tool-call-parser glm47 \ |
| --enable-multimodal --trust-remote-code |
| ``` |
|
|
| On workstation Blackwell GPUs (RTX PRO 6000 class, SM120) add `--attention-backend triton` and disable DeepGEMM (`SGLANG_ENABLE_JIT_DEEPGEMM=0 SGLANG_ENABLE_DEEP_GEMM=0 SGLANG_OPT_DEEPGEMM_HC_PRENORM=false`); datacenter kernels for this architecture have not landed for SM120 yet. |
|
|
| Recommended sampling: `temperature=1.0, top_p=0.95` with thinking on; `temperature=0.6` for instant mode. |
|
|
| **Thinking off** (instant mode): pre-close the reasoning block by prefilling the assistant turn with `<think></think>` and continuing the final message. **Thinking on** is the default. |
|
|
| ## No rack at home? |
|
|
| That's the API: the same Jah 5.0, on our hardware, one line away. Keys at [kiyomibot.ai](https://kiyomibot.ai). |
|
|
| --- |
|
|
| Built and served by [RMDW](https://rmdw.ai), Atlanta. Private by physics: your data never leaves the machine. |
|
|
