File size: 3,430 Bytes
a5ae5a3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | ---
license: mit
library_name: transformers
pipeline_tag: text-generation
tags:
- private-ai
- sovereign-ai
- rmdw
- kiyomi
- jah
- fp8
- blackwell
- vllm
- thinking
- tool-calling
language:
- en
---
# Jah 4.0
**Jah is the private AI that powers [Kiyomi](https://kiyomibot.ai), RMDW's private alternative to ChatGPT and Claude.** It runs entirely on hardware RMDW owns and controls. Nothing you type leaves to a third-party cloud, nothing is stored externally, and nothing is ever used to train another company's model. What you bring to Jah stays yours.
This is not a chatbot demo. Jah is the brain of a full private-AI product that people pay for and use every day, and this repo holds the exact weights serving them right now. Open, downloadable, verifiable.
## What's new in 4.0
Jah 4.0 is the first engine in the family with **native thinking** and a **new architecture** built for long-horizon agentic work.
- **Built-in reasoning.** Jah 4.0 thinks before it answers. A dedicated reasoning pass runs on every request — planning, self-checking, and verifying — so the model catches its own mistakes before they reach you. No external reasoning wrapper. No prompt tricks. The thinking is in the weights.
- **Hybrid attention architecture.** A new attention design that interleaves linear and full attention layers, giving Jah 4.0 efficient long-context processing without the quadratic cost of standard transformers.
- **1M token context.** A 1,048,576 token window, served with YaRN scaling and FP8 KV cache. A whole codebase with its history, or a full document set, held in one conversation.
- **Speculative decoding.** Multi-token prediction (MTP) with 3 speculative tokens per step. Jah 4.0 generates multiple tokens per forward pass, cutting wall-clock latency without changing output quality.
- **Native tool calling.** Files, terminal, browser, macOS control, connectors, web search, and sub-agents all run through the model's own function-calling format. The harness holds the run open until the work is proven.
- **FP8 precision.** Fine-grained FP8 quantization (block size 128) with performance metrics nearly identical to the original BF16 model. Smaller footprint, same quality.
## What Jah does in production
- **Private chat** with artifacts, persistent memory, cited web search, and code execution.
- **Builds real apps and sites.** Describe it; Jah writes the project, pushes it to your own GitHub, and deploys it to a live URL you own.
- **Runs the developer API.** Point Claude Code, Codex, or any OpenAI/Anthropic-compatible tool at kiyomibot.ai and Jah writes and refactors code on private hardware.
- **Runs on your desk.** The Kiyomi macOS app drives files, terminal, and the browser as a full agentic coder, and an iPhone remote controls it from anywhere.
All of it private. Try it at **[kiyomibot.ai](https://kiyomibot.ai)**.
## Serving
Jah 4.0 is served via vLLM with tensor parallelism across 4x NVIDIA RTX PRO 6000 Blackwell (96GB each). Key serving parameters:
- `--tensor-parallel-size 4`
- `--max-model-len 1000000`
- `--kv-cache-dtype fp8`
- `--speculative-config {"method":"mtp","num_speculative_tokens":3}`
- `--enable-auto-tool-choice`
- `--enable-chunked-prefill --enable-prefix-caching`
Generation defaults: `temperature=0.6`, `top_p=0.95`, `top_k=20`.
---
*Built by [RMDW AI Consulting](https://rmdw.ai). Private AI on hardware you can point at.*
|