| --- |
| license: mit |
| library_name: transformers |
| pipeline_tag: text-generation |
| tags: |
| - private-ai |
| - sovereign-ai |
| - rmdw |
| - kiyomi |
| - jah |
| - fp8 |
| - blackwell |
| - vllm |
| - thinking |
| - tool-calling |
| language: |
| - en |
| --- |
| |
| # Jah 4.0 |
|
|
| **Jah is the private AI that powers [Kiyomi](https://kiyomibot.ai), RMDW's private alternative to ChatGPT and Claude.** It runs entirely on hardware RMDW owns and controls. Nothing you type leaves to a third-party cloud, nothing is stored externally, and nothing is ever used to train another company's model. What you bring to Jah stays yours. |
|
|
| This is not a chatbot demo. Jah is the brain of a full private-AI product that people pay for and use every day, and this repo holds the exact weights serving them right now. Open, downloadable, verifiable. |
|
|
| ## What's new in 4.0 |
|
|
| Jah 4.0 is the first engine in the family with **native thinking** and a **new architecture** built for long-horizon agentic work. |
|
|
| - **Built-in reasoning.** Jah 4.0 thinks before it answers. A dedicated reasoning pass runs on every request — planning, self-checking, and verifying — so the model catches its own mistakes before they reach you. No external reasoning wrapper. No prompt tricks. The thinking is in the weights. |
| - **Hybrid attention architecture.** A new attention design that interleaves linear and full attention layers, giving Jah 4.0 efficient long-context processing without the quadratic cost of standard transformers. |
| - **1M token context.** A 1,048,576 token window, served with YaRN scaling and FP8 KV cache. A whole codebase with its history, or a full document set, held in one conversation. |
| - **Speculative decoding.** Multi-token prediction (MTP) with 3 speculative tokens per step. Jah 4.0 generates multiple tokens per forward pass, cutting wall-clock latency without changing output quality. |
| - **Native tool calling.** Files, terminal, browser, macOS control, connectors, web search, and sub-agents all run through the model's own function-calling format. The harness holds the run open until the work is proven. |
| - **FP8 precision.** Fine-grained FP8 quantization (block size 128) with performance metrics nearly identical to the original BF16 model. Smaller footprint, same quality. |
|
|
| ## What Jah does in production |
|
|
| - **Private chat** with artifacts, persistent memory, cited web search, and code execution. |
| - **Builds real apps and sites.** Describe it; Jah writes the project, pushes it to your own GitHub, and deploys it to a live URL you own. |
| - **Runs the developer API.** Point Claude Code, Codex, or any OpenAI/Anthropic-compatible tool at kiyomibot.ai and Jah writes and refactors code on private hardware. |
| - **Runs on your desk.** The Kiyomi macOS app drives files, terminal, and the browser as a full agentic coder, and an iPhone remote controls it from anywhere. |
|
|
| All of it private. Try it at **[kiyomibot.ai](https://kiyomibot.ai)**. |
|
|
| ## Serving |
|
|
| Jah 4.0 is served via vLLM with tensor parallelism across 4x NVIDIA RTX PRO 6000 Blackwell (96GB each). Key serving parameters: |
|
|
| - `--tensor-parallel-size 4` |
| - `--max-model-len 1000000` |
| - `--kv-cache-dtype fp8` |
| - `--speculative-config {"method":"mtp","num_speculative_tokens":3}` |
| - `--enable-auto-tool-choice` |
| - `--enable-chunked-prefill --enable-prefix-caching` |
|
|
| Generation defaults: `temperature=0.6`, `top_p=0.95`, `top_k=20`. |
|
|
| --- |
|
|
| *Built by [RMDW AI Consulting](https://rmdw.ai). Private AI on hardware you can point at.* |
|
|
