Instructions to use CMSManhattan/JiRackPrecision_72b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use CMSManhattan/JiRackPrecision_72b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M # Run inference directly in the terminal: llama cli -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M # Run inference directly in the terminal: llama cli -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Use Docker
docker model run hf.co/CMSManhattan/JiRackPrecision_72b:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use CMSManhattan/JiRackPrecision_72b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "CMSManhattan/JiRackPrecision_72b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CMSManhattan/JiRackPrecision_72b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/CMSManhattan/JiRackPrecision_72b:Q4_K_M
- Ollama
How to use CMSManhattan/JiRackPrecision_72b with Ollama:
ollama run hf.co/CMSManhattan/JiRackPrecision_72b:Q4_K_M
- Unsloth Desktop
- Pi
How to use CMSManhattan/JiRackPrecision_72b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "CMSManhattan/JiRackPrecision_72b:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use CMSManhattan/JiRackPrecision_72b with Docker Model Runner:
docker model run hf.co/CMSManhattan/JiRackPrecision_72b:Q4_K_M
- Lemonade
How to use CMSManhattan/JiRackPrecision_72b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull CMSManhattan/JiRackPrecision_72b:Q4_K_M
Run and chat with the model
lemonade run user.JiRackPrecision_72b-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use CMSManhattan/JiRackPrecision_72b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default CMSManhattan/JiRackPrecision_72b:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use CMSManhattan/JiRackPrecision_72b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "CMSManhattan/JiRackPrecision_72b:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
JiRack Precision 72B
JiRack Precision 72B is a 72.1B-parameter language model by CMSManhattan (Konstantin Grabko), built in the JiRack architecture from a DeepSeek model. It answers directly and briefly, without a separate reasoning phase, which makes it the fastest model of the JiRack 70B line. It writes code, solves math problems and calls tools.
Model
| Parameters | 72.1B |
| Layers | 132 |
| Hidden size | 5,120 |
| Attention | 40 query heads, 8 key/value heads, head size 128 |
| Context | 16,384 tokens |
| Vocabulary | 152,064 tokens |
| Answers | direct, no separate reasoning phase |
| Tool calling | single and parallel calls |
| Files | gguf/JiRack-Precision-72B-Q8_0.gguf (76.7 GB), gguf/JiRack-Precision-72B-Q4_K_M.gguf (43.6 GB) |
Benchmarks
GGUF Q4_K_M, llama.cpp, greedy decoding, 0-shot, 8,192-token limit per answer.
| Benchmark | Result |
|---|---|
| GSM8K (1,319 problems) | 93.5% |
| HumanEval pass@1 (164 problems) | 87.2% |
Average answer length: 545 tokens on GSM8K, 171 tokens on HumanEval.
DEMO chat
https://huggingface.co/spaces/CMSManhattan/JiRackDeltaNet-180B-demo
Test results
llama.cpp, greedy decoding, one 96 GB GPU.
| Test | Q8_0 | Q4_K_M |
|---|---|---|
| 46 prompts (code, math, general), up to 4,096 tokens | 46/46 complete answers | 46/46 complete answers |
| The same 46 prompts through the chat API | not run | 46/46 complete answers |
| Tool calling: one call, two parallel calls | correct | correct |
| Tool calling: no tool fits the question | no call made | no call made |
| Numeric stability | no NaN outputs | no NaN outputs |
Activation range: every matrix input of all 132 layers was measured on these answers. The largest values stay more than 1,000 times (Q8_0) and 170 times (Q4_K_M) below the float16 limits of llama.cpp.
Usage
llama-server -m JiRack-Precision-72B-Q4_K_M.gguf -c 16384 --jinja -ngl 999 --port 8080
The server speaks the OpenAI chat API, including tool calls. Recommended sampling: temperature 0.6, top-p 0.95 (stored in the GGUF files).
Memory: the weights take 76.7 GB (Q8_0) or 43.6 GB (Q4_K_M); the KV cache adds 0.54 MB per token of context (about 8.9 GB for 16,384 tokens). Q8_0 runs on one 96 GB GPU.
Limitations
- The model can make mistakes; check its output in automated pipelines.
- When a required tool argument is missing, the model may fill in a placeholder value instead of asking for it.
- When no tool fits a question, the model may say so instead of answering the question itself.
- With greedy decoding, long math solutions can start repeating themselves; the recommended sampling settings reduce this.
- Text only.
- Not for medical, legal or financial advice.
License
JiRack Precision 72B is a modified version of a DeepSeek model by DeepSeek, released under the MIT License (copy in LICENSE-MIT), which is itself derived from a Qwen model by the Qwen team, Alibaba Cloud, released under the Apache License 2.0 (copy in LICENSE-APACHE). CMSManhattan modified the architecture and weights and fine-tuned the model.
License of JiRack Precision 72B: proprietary JiRack license, see LICENSE.
📧 Contact
For joint venture opportunities, hardware integration, or licensing inquiries:
- Email: grabko@cmsmanhattan.com
- Phone: +1 (516) 777-0945
- Location: New York, USA
- Downloads last month
- 82