Instructions to use RMDWLLC/Jah-2.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RMDWLLC/Jah-2.0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RMDWLLC/Jah-2.0")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("RMDWLLC/Jah-2.0")
model = AutoModelForCausalLM.from_pretrained("RMDWLLC/Jah-2.0", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use RMDWLLC/Jah-2.0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RMDWLLC/Jah-2.0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RMDWLLC/Jah-2.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/RMDWLLC/Jah-2.0

SGLang

How to use RMDWLLC/Jah-2.0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RMDWLLC/Jah-2.0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RMDWLLC/Jah-2.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RMDWLLC/Jah-2.0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RMDWLLC/Jah-2.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use RMDWLLC/Jah-2.0 with Docker Model Runner:
```
docker model run hf.co/RMDWLLC/Jah-2.0
```

Jah-2.0 / README.md

restokes92

Upload README.md with huggingface_hub

b366314 verified 16 days ago

preview code

Raw

History Blame Contribute Delete

3.56 kB

	---
	license: mit
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- private-ai
	- sovereign-ai
	- rmdw
	- echols
	- jah
	- nvfp4
	- vllm
	language:
	- en
	---

	# Jah 2.0

	Jah is the private AI that powers [Echols](https://echols.ai), RMDW's private alternative to ChatGPT and Claude. It runs entirely on hardware RMDW owns and controls. Nothing you type leaves to a third-party cloud, nothing is stored externally, and nothing is ever used to train another company's model. What you bring to Jah stays yours.

	This is not a chatbot demo. Jah is the brain of a full private-AI product that people pay for and use every day, and this repo holds the exact weights serving them right now. Open, downloadable, verifiable.

	## What's new in 2.0

	Jah 2.0 is the biggest leap in the family's history. It replaces two models at once: Jah 1.0 (chat) and Jeshua 0.5 (the build engine), one brain now running the entire Echols suite.

	- Triple the speed. Around 65 tokens per second on Echols hardware, up from ~20.
	- Eight times the context. A 250,000 token window, up from 32k. Whole codebases, whole document sets, one conversation.
	- Reasoning on. Jah 2.0 thinks before it answers, with the reasoning kept separate from the reply.
	- Agentic for real. Native tool calling drives the Echols connector layer: live email, calendar, files, and payment data answered from a business's own accounts, read-only.
	- A far bigger brain. 2.0 keeps substantially more of its full capability than the 1.0 generation did, while still fitting in 384 GB of VRAM.

	## What Jah does in production

	- Private chat with artifacts, persistent memory, cited web search, and code execution.
	- Builds real apps. Describe it; Jah writes the project, pushes it to your own GitHub, and deploys it to a live URL you own.
	- Runs the developer API. Point Claude Code or any OpenAI/Anthropic-compatible tool at echols.ai and Jah writes and refactors code on private hardware.
	- Powers Echols for Business. Companies get their own subdomain, their own knowledge base with cited answers, live connections to their tools, and a weekly money review, all served by this model.

	All of it private, for $25/mo. Try it at [echols.ai](https://echols.ai).

	## Run it yourself

	The weights are open because owning your AI end to end should be possible. Jah 2.0 is served with vLLM in NVFP4 on 4x NVIDIA RTX PRO 6000 Blackwell (SM120), 4-way tensor parallel, with MTP speculative decoding.

	Two settings matter if you serve these weights:

	```bash
	--quantization modelopt_fp4 \
	--hf-overrides '{"index_topk_pattern":"FFFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSS"}'
	```

	The per-layer sparse-attention pattern (78 characters, 21 F / 57 S) is required for long-context coherence. Thinking is controlled per request with `"chat_template_kwargs": {"enable_thinking": true\|false}`.

	The previous generations remain available: [Jah 1.0](https://huggingface.co/RMDWLLC/Jah-1.0) and [Jeshua 0.5](https://huggingface.co/RMDWLLC/Jeshua-0.5).

	## The bigger idea

	Most AI today is rented from a handful of companies. Jah is owned: a private model running a real product on hardware RMDW controls, with open weights so anyone can verify exactly what answers them. That is the future RMDW is building: AI you own, not AI you rent. → [rmdw.ai](https://rmdw.ai)

	---

	Jah 2.0, RMDW LLC. Private AI, run on your terms. Weights redistributed under the MIT license; see LICENSE in this repository.