Instructions to use CMSManhattan/JiRackPrecision_72b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use CMSManhattan/JiRackPrecision_72b with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M # Run inference directly in the terminal: llama cli -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M # Run inference directly in the terminal: llama cli -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Use Docker
docker model run hf.co/CMSManhattan/JiRackPrecision_72b:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use CMSManhattan/JiRackPrecision_72b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "CMSManhattan/JiRackPrecision_72b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "CMSManhattan/JiRackPrecision_72b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/CMSManhattan/JiRackPrecision_72b:Q4_K_M
- Ollama
How to use CMSManhattan/JiRackPrecision_72b with Ollama:
ollama run hf.co/CMSManhattan/JiRackPrecision_72b:Q4_K_M
- Unsloth Desktop
- Pi
How to use CMSManhattan/JiRackPrecision_72b with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "CMSManhattan/JiRackPrecision_72b:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use CMSManhattan/JiRackPrecision_72b with Docker Model Runner:
docker model run hf.co/CMSManhattan/JiRackPrecision_72b:Q4_K_M
- Lemonade
How to use CMSManhattan/JiRackPrecision_72b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull CMSManhattan/JiRackPrecision_72b:Q4_K_M
Run and chat with the model
lemonade run user.JiRackPrecision_72b-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use CMSManhattan/JiRackPrecision_72b with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default CMSManhattan/JiRackPrecision_72b:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use CMSManhattan/JiRackPrecision_72b with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf CMSManhattan/JiRackPrecision_72b:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "CMSManhattan/JiRackPrecision_72b:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from CMSManhattan/JiRackPrecision_72b: direct link, hf CLI and curl.
- Browser
- Download file 3.53 kB
-
https://huggingface.co/CMSManhattan/JiRackPrecision_72b/resolve/main/README.md
- Command line
-
hf download hf://CMSManhattan/JiRackPrecision_72b/README.md
-
curl -L -o README.md https://huggingface.co/CMSManhattan/JiRackPrecision_72b/resolve/main/README.md
3.53 kB
| license: other | |
| license_name: jirack-proprietary | |
| license_link: LICENSE | |
| pipeline_tag: text-generation | |
| tags: | |
| - jirack | |
| - gguf | |
| - tool-calling | |
| - code | |
| - math | |
| # JiRack Precision 72B | |
| JiRack Precision 72B is a 72.1B-parameter language model by CMSManhattan (Konstantin Grabko), built in the JiRack architecture from a DeepSeek model. It answers directly and briefly, without a separate reasoning phase, which makes it the fastest model of the JiRack 70B line. It writes code, solves math problems and calls tools. | |
| ## Model | |
| | | | | |
| | --- | --- | | |
| | Parameters | 72.1B | | |
| | Layers | 132 | | |
| | Hidden size | 5,120 | | |
| | Attention | 40 query heads, 8 key/value heads, head size 128 | | |
| | Context | 16,384 tokens | | |
| | Vocabulary | 152,064 tokens | | |
| | Answers | direct, no separate reasoning phase | | |
| | Tool calling | single and parallel calls | | |
| | Files | `gguf/JiRack-Precision-72B-Q8_0.gguf` (76.7 GB), `gguf/JiRack-Precision-72B-Q4_K_M.gguf` (43.6 GB) | | |
| ## Benchmarks | |
| GGUF Q4_K_M, llama.cpp, greedy decoding, 0-shot, 8,192-token limit per answer. | |
| | Benchmark | Result | | |
| | --- | --- | | |
| | GSM8K (1,319 problems) | 93.5% | | |
| | HumanEval pass@1 (164 problems) | 87.2% | | |
| Average answer length: 545 tokens on GSM8K, 171 tokens on HumanEval. | |
| ## DEMO chat | |
| https://huggingface.co/spaces/CMSManhattan/JiRackDeltaNet-180B-demo | |
| ## Test results | |
| llama.cpp, greedy decoding, one 96 GB GPU. | |
| | Test | Q8_0 | Q4_K_M | | |
| | --- | --- | --- | | |
| | 46 prompts (code, math, general), up to 4,096 tokens | 46/46 complete answers | 46/46 complete answers | | |
| | The same 46 prompts through the chat API | not run | 46/46 complete answers | | |
| | Tool calling: one call, two parallel calls | correct | correct | | |
| | Tool calling: no tool fits the question | no call made | no call made | | |
| | Numeric stability | no NaN outputs | no NaN outputs | | |
| Activation range: every matrix input of all 132 layers was measured on these answers. The largest values stay more than 1,000 times (Q8_0) and 170 times (Q4_K_M) below the float16 limits of llama.cpp. | |
| ## Usage | |
| ```bash | |
| llama-server -m JiRack-Precision-72B-Q4_K_M.gguf -c 16384 --jinja -ngl 999 --port 8080 | |
| ``` | |
| The server speaks the OpenAI chat API, including tool calls. Recommended sampling: temperature 0.6, top-p 0.95 (stored in the GGUF files). | |
| Memory: the weights take 76.7 GB (Q8_0) or 43.6 GB (Q4_K_M); the KV cache adds 0.54 MB per token of context (about 8.9 GB for 16,384 tokens). Q8_0 runs on one 96 GB GPU. | |
| ## Limitations | |
| - The model can make mistakes; check its output in automated pipelines. | |
| - When a required tool argument is missing, the model may fill in a placeholder value instead of asking for it. | |
| - When no tool fits a question, the model may say so instead of answering the question itself. | |
| - With greedy decoding, long math solutions can start repeating themselves; the recommended sampling settings reduce this. | |
| - Text only. | |
| - Not for medical, legal or financial advice. | |
| ## License | |
| JiRack Precision 72B is a modified version of a DeepSeek model by DeepSeek, released under the MIT License (copy in `LICENSE-MIT`), which is itself derived from a Qwen model by the Qwen team, Alibaba Cloud, released under the Apache License 2.0 (copy in `LICENSE-APACHE`). CMSManhattan modified the architecture and weights and fine-tuned the model. | |
| License of JiRack Precision 72B: proprietary JiRack license, see `LICENSE`. | |
| ## 📧 Contact | |
| For joint venture opportunities, hardware integration, or licensing inquiries: | |
| - **Email:** grabko@cmsmanhattan.com | |
| - **Phone:** +1 (516) 777-0945 | |
| - **Location:** New York, USA | |