---
license: mit
library_name: transformers
pipeline_tag: text-generation
tags:
  - private-ai
  - sovereign-ai
  - rmdw
  - kiyomi
  - jah
  - fp8
  - blackwell
  - vllm
  - thinking
  - tool-calling
language:
  - en
---

# Jah 4.0

**Jah is the private AI that powers [Kiyomi](https://kiyomibot.ai), RMDW's private alternative to ChatGPT and Claude.** It runs entirely on hardware RMDW owns and controls. Nothing you type leaves to a third-party cloud, nothing is stored externally, and nothing is ever used to train another company's model. What you bring to Jah stays yours.

This is not a chatbot demo. Jah is the brain of a full private-AI product that people pay for and use every day, and this repo holds the exact weights serving them right now. Open, downloadable, verifiable.

## What's new in 4.0

Jah 4.0 is the first engine in the family with **native thinking** and a **new architecture** built for long-horizon agentic work.

- **Built-in reasoning.** Jah 4.0 thinks before it answers. A dedicated reasoning pass runs on every request — planning, self-checking, and verifying — so the model catches its own mistakes before they reach you. No external reasoning wrapper. No prompt tricks. The thinking is in the weights.
- **Hybrid attention architecture.** A new attention design that interleaves linear and full attention layers, giving Jah 4.0 efficient long-context processing without the quadratic cost of standard transformers.
- **1M token context.** A 1,048,576 token window, served with YaRN scaling and FP8 KV cache. A whole codebase with its history, or a full document set, held in one conversation.
- **Speculative decoding.** Multi-token prediction (MTP) with 3 speculative tokens per step. Jah 4.0 generates multiple tokens per forward pass, cutting wall-clock latency without changing output quality.
- **Native tool calling.** Files, terminal, browser, macOS control, connectors, web search, and sub-agents all run through the model's own function-calling format. The harness holds the run open until the work is proven.
- **FP8 precision.** Fine-grained FP8 quantization (block size 128) with performance metrics nearly identical to the original BF16 model. Smaller footprint, same quality.

## What Jah does in production

- **Private chat** with artifacts, persistent memory, cited web search, and code execution.
- **Builds real apps and sites.** Describe it; Jah writes the project, pushes it to your own GitHub, and deploys it to a live URL you own.
- **Runs the developer API.** Point Claude Code, Codex, or any OpenAI/Anthropic-compatible tool at kiyomibot.ai and Jah writes and refactors code on private hardware.
- **Runs on your desk.** The Kiyomi macOS app drives files, terminal, and the browser as a full agentic coder, and an iPhone remote controls it from anywhere.

All of it private. Try it at **[kiyomibot.ai](https://kiyomibot.ai)**.

## Serving

Jah 4.0 is served via vLLM with tensor parallelism across 4x NVIDIA RTX PRO 6000 Blackwell (96GB each). Key serving parameters:

- `--tensor-parallel-size 4`
- `--max-model-len 1000000`
- `--kv-cache-dtype fp8`
- `--speculative-config {"method":"mtp","num_speculative_tokens":3}`
- `--enable-auto-tool-choice`
- `--enable-chunked-prefill --enable-prefix-caching`

Generation defaults: `temperature=0.6`, `top_p=0.95`, `top_k=20`.

---

*Built by [RMDW AI Consulting](https://rmdw.ai). Private AI on hardware you can point at.*