JiRackPrecision_72b / README.md
kgrabko's picture
JiRack Precision 72B
07c4223
|
Raw History Blame Contribute Delete
3.53 kB
---
license: other
license_name: jirack-proprietary
license_link: LICENSE
pipeline_tag: text-generation
tags:
- jirack
- gguf
- tool-calling
- code
- math
---
# JiRack Precision 72B
JiRack Precision 72B is a 72.1B-parameter language model by CMSManhattan (Konstantin Grabko), built in the JiRack architecture from a DeepSeek model. It answers directly and briefly, without a separate reasoning phase, which makes it the fastest model of the JiRack 70B line. It writes code, solves math problems and calls tools.
## Model
| | |
| --- | --- |
| Parameters | 72.1B |
| Layers | 132 |
| Hidden size | 5,120 |
| Attention | 40 query heads, 8 key/value heads, head size 128 |
| Context | 16,384 tokens |
| Vocabulary | 152,064 tokens |
| Answers | direct, no separate reasoning phase |
| Tool calling | single and parallel calls |
| Files | `gguf/JiRack-Precision-72B-Q8_0.gguf` (76.7 GB), `gguf/JiRack-Precision-72B-Q4_K_M.gguf` (43.6 GB) |
## Benchmarks
GGUF Q4_K_M, llama.cpp, greedy decoding, 0-shot, 8,192-token limit per answer.
| Benchmark | Result |
| --- | --- |
| GSM8K (1,319 problems) | 93.5% |
| HumanEval pass@1 (164 problems) | 87.2% |
Average answer length: 545 tokens on GSM8K, 171 tokens on HumanEval.
## DEMO chat
https://huggingface.co/spaces/CMSManhattan/JiRackDeltaNet-180B-demo
## Test results
llama.cpp, greedy decoding, one 96 GB GPU.
| Test | Q8_0 | Q4_K_M |
| --- | --- | --- |
| 46 prompts (code, math, general), up to 4,096 tokens | 46/46 complete answers | 46/46 complete answers |
| The same 46 prompts through the chat API | not run | 46/46 complete answers |
| Tool calling: one call, two parallel calls | correct | correct |
| Tool calling: no tool fits the question | no call made | no call made |
| Numeric stability | no NaN outputs | no NaN outputs |
Activation range: every matrix input of all 132 layers was measured on these answers. The largest values stay more than 1,000 times (Q8_0) and 170 times (Q4_K_M) below the float16 limits of llama.cpp.
## Usage
```bash
llama-server -m JiRack-Precision-72B-Q4_K_M.gguf -c 16384 --jinja -ngl 999 --port 8080
```
The server speaks the OpenAI chat API, including tool calls. Recommended sampling: temperature 0.6, top-p 0.95 (stored in the GGUF files).
Memory: the weights take 76.7 GB (Q8_0) or 43.6 GB (Q4_K_M); the KV cache adds 0.54 MB per token of context (about 8.9 GB for 16,384 tokens). Q8_0 runs on one 96 GB GPU.
## Limitations
- The model can make mistakes; check its output in automated pipelines.
- When a required tool argument is missing, the model may fill in a placeholder value instead of asking for it.
- When no tool fits a question, the model may say so instead of answering the question itself.
- With greedy decoding, long math solutions can start repeating themselves; the recommended sampling settings reduce this.
- Text only.
- Not for medical, legal or financial advice.
## License
JiRack Precision 72B is a modified version of a DeepSeek model by DeepSeek, released under the MIT License (copy in `LICENSE-MIT`), which is itself derived from a Qwen model by the Qwen team, Alibaba Cloud, released under the Apache License 2.0 (copy in `LICENSE-APACHE`). CMSManhattan modified the architecture and weights and fine-tuned the model.
License of JiRack Precision 72B: proprietary JiRack license, see `LICENSE`.
## 📧 Contact
For joint venture opportunities, hardware integration, or licensing inquiries:
- **Email:** grabko@cmsmanhattan.com
- **Phone:** +1 (516) 777-0945
- **Location:** New York, USA