| --- |
| language: |
| - en |
| - zh |
| - ja |
| - ko |
| - fr |
| - es |
| - pt |
| - de |
| - it |
| - ru |
| - ar |
| - vi |
| - th |
| tags: |
| - code |
| - coding |
| - qwen3.0 |
| - onnx |
| - int8 |
| - web-ui |
| license: unknown |
| --- |
| |
| # JiRack Coder Reasoing 8B INT4 |
|
|
| A fast and efficient coding assistant with a clean built-in web UI, powered by Qwen3.0-Coder-8B-Instruct base and optimized using Microsoft ONNX Runtime. |
|
|
| - JiRack is cloud model and save money on cloud and can be used as expert model in RAG on cloud with ONNX JiRack java server as alternative. |
| - Subscription 1$ per month per user in updated license if not company |
|
|
| ## Quick Start |
| Watch the JiRack Coder 8B in action: |
| **DEMO**: [JiRack Coder Reasoing 8B Web UI](https://youtu.be/mq1DxIov7Bw) |
|
|
|
|
| ### Run with Docker |
|
|
| --- |
| --Default CPU-- |
|
|
| - docker run -d \ |
| --name jirack_coder_reasoing_8b \ |
| -p 7869:7869 \ |
| --restart unless-stopped \ |
| cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest |
| |
| --Multi CPU-- |
| |
| - docker run -d \ |
| --name jirack_coder_reasoing_8b \ |
| -p 7869:7869 \ |
| --restart unless-stopped \ |
| --memory=20g \ |
| --cpus=12 \ |
| cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest |
|
|
| ---GPU-- |
| -- comming soon |
|
|
| - docker run -d \ |
| --name jirack_coder_reasoing_8b \ |
| -p 7869:7869 \ |
| --gpus all \ |
| --restart unless-stopped \ |
| cmsmanhattan/jirack_coder_8b_int4_gpu_qwenbase:latest |
|
|
| --- |
|
|
| services: |
| |
| |
| image: cmsmanhattan/jirack_coder_8b_int4_qwenbase:latest |
| container_name: jirack_onnx_service |
| ports: |
| - "7869:7869" |
| volumes: |
| - .:/app |
| - ./web:/app/web |
| environment: |
| - MAX_TOKENS=1024 |
| - TEMPERATURE=0.7 |
| - TOP_P=0.9 |
| - DEFAULT_STREAM=False |
| - INTRA_THREADS=4 |
| - USE_ENV_ALLOCATOR=1 |
| deploy: |
| resources: |
| limits: |
| memory: 16g |
| |
| ## Access the UI |
|
|
| Once the container is running, open your browser and navigate to: |
|
|
| **`http://localhost:7869`** |
|
|
| This opens the **JiRack Coder UI** — a clean web interface designed for coding. |
|
|
| ## Changing the Port |
|
|
| The listening port can be easily modified directly from the **Settings** panel within the JiRack Coder UI. |
|
|
| ## Licensing |
|
|
| - The **JiRack Coder 8B model** is provided under a commercial license. It ia about 12$ for year per user . |
| - All **JiRack UI clients** are provided under a commercial license. |
| - However, the UI clients can be used for free when running together with the official JiRack Docker containers, as long as they are not redistributed separately. |
|
|
|
|
| **JiRack Coder 32B** is available exclusively under a commercial enterprise license. |
|
|
| For commercial licensing, cluster deployment, or enterprise use of the JiRack Coder 32B and JiRack Coder 14B , please contact us. |
| - JiRack MS Windows 11 Desktop chat client with ollama API setup : https://huggingface.co/kgrabko/JiRackTernary_1b/resolve/main/jirack-chat.zip |
| - Live email chat with model via support@cmsmanhattan.com |
| |
| |
| ## Hardware Recommendations for AMD Systems |
| It is more heavy then JiRack Coder 7B INT8 |
| ### Recommended Hardware for JiRack Coder Reasoing 8B INT8 . It is one dcoker container |
| |
| | Use Case | CPU | GPU (ROCm) | VRAM / RAM | Expected Speed | Recommendation | |
| |-----------------------|----------------------------------|-----------------------------------|----------------|---------------------|--------------------| |
| | **Recommended** | Ryzen 7 7700 / 9700X | RX 7900 XTX / 7900 XT | 24GB VRAM | 50-75 tokens/s | Best choice | |
| | **High Performance** | Ryzen 9 7950X / 9950X | RX 7900 XTX | 24GB+ VRAM | 65-90 tokens/s | Excellent | |
| | **Enterprise** | EPYC 7003/9004 series | MI300X or 2x RX 7900 XTX | 48GB+ VRAM | 90-140 tokens/s | For 32B model | |
| | **Budget Option** | Ryzen 5 7600 / 9600X | RX 7800 XT (16GB) | 16GB VRAM | 35-50 tokens/s | Acceptable | |
| |
| ### Important Memory Notes |
| |
| Even though the 8B INT4 model itself takes approximately **5–6 GB**, we recommend **at least 24GB VRAM** for the following reasons: |
| |
| - KV-cache consumption during generation (especially with long context) |
| - ONNX Runtime overhead and temporary buffers |
| - System stability and to avoid Out of Memory errors |
| - Room for larger context windows |
| |
| **Minimum recommended:** 24GB VRAM (RX 7900 series) |
| **Ideal:** 24–32GB VRAM |
| |
| For pure CPU inference (no GPU), we recommend at least **64GB system RAM** (Ryzen 9 7950X/9950X). |
| |
| --- |
| I will the default model in full FP32 precision for quantization, allowing us to find the optimal balance between model size and performance. |
| |
| |
| ## 📧 Contact & Licensing |
| For joint venture opportunities, hardware integration, or licensing inquiries: |
| - **Email:** [grabko@cmsmanhattan.com](mailto:grabko@cmsmanhattan.com) |
| - **Phone:** +1 (516) 777-0945 |
| - **Location:** New York, USA |
| |
| |
| |
