--- license: other license_name: jirack-proprietary license_link: LICENSE pipeline_tag: text-generation tags: - jirack - gguf - tool-calling - code - math --- # JiRack Precision 72B JiRack Precision 72B is a 72.1B-parameter language model by CMSManhattan (Konstantin Grabko), built in the JiRack architecture from a DeepSeek model. It answers directly and briefly, without a separate reasoning phase, which makes it the fastest model of the JiRack 70B line. It writes code, solves math problems and calls tools. ## Model | | | | --- | --- | | Parameters | 72.1B | | Layers | 132 | | Hidden size | 5,120 | | Attention | 40 query heads, 8 key/value heads, head size 128 | | Context | 16,384 tokens | | Vocabulary | 152,064 tokens | | Answers | direct, no separate reasoning phase | | Tool calling | single and parallel calls | | Files | `gguf/JiRack-Precision-72B-Q8_0.gguf` (76.7 GB), `gguf/JiRack-Precision-72B-Q4_K_M.gguf` (43.6 GB) | ## Benchmarks GGUF Q4_K_M, llama.cpp, greedy decoding, 0-shot, 8,192-token limit per answer. | Benchmark | Result | | --- | --- | | GSM8K (1,319 problems) | 93.5% | | HumanEval pass@1 (164 problems) | 87.2% | Average answer length: 545 tokens on GSM8K, 171 tokens on HumanEval. ## DEMO chat https://huggingface.co/spaces/CMSManhattan/JiRackDeltaNet-180B-demo ## Test results llama.cpp, greedy decoding, one 96 GB GPU. | Test | Q8_0 | Q4_K_M | | --- | --- | --- | | 46 prompts (code, math, general), up to 4,096 tokens | 46/46 complete answers | 46/46 complete answers | | The same 46 prompts through the chat API | not run | 46/46 complete answers | | Tool calling: one call, two parallel calls | correct | correct | | Tool calling: no tool fits the question | no call made | no call made | | Numeric stability | no NaN outputs | no NaN outputs | Activation range: every matrix input of all 132 layers was measured on these answers. The largest values stay more than 1,000 times (Q8_0) and 170 times (Q4_K_M) below the float16 limits of llama.cpp. ## Usage ```bash llama-server -m JiRack-Precision-72B-Q4_K_M.gguf -c 16384 --jinja -ngl 999 --port 8080 ``` The server speaks the OpenAI chat API, including tool calls. Recommended sampling: temperature 0.6, top-p 0.95 (stored in the GGUF files). Memory: the weights take 76.7 GB (Q8_0) or 43.6 GB (Q4_K_M); the KV cache adds 0.54 MB per token of context (about 8.9 GB for 16,384 tokens). Q8_0 runs on one 96 GB GPU. ## Limitations - The model can make mistakes; check its output in automated pipelines. - When a required tool argument is missing, the model may fill in a placeholder value instead of asking for it. - When no tool fits a question, the model may say so instead of answering the question itself. - With greedy decoding, long math solutions can start repeating themselves; the recommended sampling settings reduce this. - Text only. - Not for medical, legal or financial advice. ## License JiRack Precision 72B is a modified version of a DeepSeek model by DeepSeek, released under the MIT License (copy in `LICENSE-MIT`), which is itself derived from a Qwen model by the Qwen team, Alibaba Cloud, released under the Apache License 2.0 (copy in `LICENSE-APACHE`). CMSManhattan modified the architecture and weights and fine-tuned the model. License of JiRack Precision 72B: proprietary JiRack license, see `LICENSE`. ## 📧 Contact For joint venture opportunities, hardware integration, or licensing inquiries: - **Email:** grabko@cmsmanhattan.com - **Phone:** +1 (516) 777-0945 - **Location:** New York, USA