How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Use Docker
docker model run hf.co/CMSManhattan/JiRackPrecision_72b:Q4_K_M
Quick Links

JiRack Precision 72B

JiRack Precision 72B is a 72.1B-parameter language model by CMSManhattan (Konstantin Grabko), built in the JiRack architecture from a DeepSeek model. It answers directly and briefly, without a separate reasoning phase, which makes it the fastest model of the JiRack 70B line. It writes code, solves math problems and calls tools.

Model

Parameters 72.1B
Layers 132
Hidden size 5,120
Attention 40 query heads, 8 key/value heads, head size 128
Context 16,384 tokens
Vocabulary 152,064 tokens
Answers direct, no separate reasoning phase
Tool calling single and parallel calls
Files gguf/JiRack-Precision-72B-Q8_0.gguf (76.7 GB), gguf/JiRack-Precision-72B-Q4_K_M.gguf (43.6 GB)

Benchmarks

GGUF Q4_K_M, llama.cpp, greedy decoding, 0-shot, 8,192-token limit per answer.

Benchmark Result
GSM8K (1,319 problems) 93.5%
HumanEval pass@1 (164 problems) 87.2%

Average answer length: 545 tokens on GSM8K, 171 tokens on HumanEval.

DEMO chat

https://huggingface.co/spaces/CMSManhattan/JiRackDeltaNet-180B-demo

Test results

llama.cpp, greedy decoding, one 96 GB GPU.

Test Q8_0 Q4_K_M
46 prompts (code, math, general), up to 4,096 tokens 46/46 complete answers 46/46 complete answers
The same 46 prompts through the chat API not run 46/46 complete answers
Tool calling: one call, two parallel calls correct correct
Tool calling: no tool fits the question no call made no call made
Numeric stability no NaN outputs no NaN outputs

Activation range: every matrix input of all 132 layers was measured on these answers. The largest values stay more than 1,000 times (Q8_0) and 170 times (Q4_K_M) below the float16 limits of llama.cpp.

Usage

llama-server -m JiRack-Precision-72B-Q4_K_M.gguf -c 16384 --jinja -ngl 999 --port 8080

The server speaks the OpenAI chat API, including tool calls. Recommended sampling: temperature 0.6, top-p 0.95 (stored in the GGUF files).

Memory: the weights take 76.7 GB (Q8_0) or 43.6 GB (Q4_K_M); the KV cache adds 0.54 MB per token of context (about 8.9 GB for 16,384 tokens). Q8_0 runs on one 96 GB GPU.

Limitations

  • The model can make mistakes; check its output in automated pipelines.
  • When a required tool argument is missing, the model may fill in a placeholder value instead of asking for it.
  • When no tool fits a question, the model may say so instead of answering the question itself.
  • With greedy decoding, long math solutions can start repeating themselves; the recommended sampling settings reduce this.
  • Text only.
  • Not for medical, legal or financial advice.

License

JiRack Precision 72B is a modified version of a DeepSeek model by DeepSeek, released under the MIT License (copy in LICENSE-MIT), which is itself derived from a Qwen model by the Qwen team, Alibaba Cloud, released under the Apache License 2.0 (copy in LICENSE-APACHE). CMSManhattan modified the architecture and weights and fine-tuned the model.

License of JiRack Precision 72B: proprietary JiRack license, see LICENSE.

📧 Contact

For joint venture opportunities, hardware integration, or licensing inquiries:

Downloads last month
82
Safetensors
Model size
72B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Space using CMSManhattan/JiRackPrecision_72b 1