Shiftedx's picture
Document tested strict JSON profiles, vision results and repeated speed comparison
d002e2f verified
|
Raw History Blame Contribute Delete
3.32 kB
---
license: apache-2.0
library_name: mlx
pipeline_tag: text-generation
base_model: Jackrong/Qwopus3.8-27B-Coder-Flash
base_model_relation: quantized
language:
- en
- zh
tags:
- mlx
- mixed-precision
- qwen3_5
- coding
- tool-use
- experimental
inference: false
---
# Qwopus3.8-27B Coder Flash · MLX Fast
**16.59 GB weights · 5/8 executable coding smoke · 64K/128K retrieval, tools and cached coding passed.** Select Balanced for the higher local coding score, Fast for lower memory and faster decode.
3-bit/4-bit group-64 body. Full/recurrent attention at 4-bit; embeddings/head at 8-bit. Sensitivity-ranked MLP promotions. Both retain 96 BF16 recurrent gating projections, publisher tokenizer/template, BF16 vision and native MTP. [Exact precision recipe](quantization_recipe.json).
| Local replay | 64K | 128K |
|---|---:|---:|
| Prompt tokens | 61,441 | 126,976 |
| Cold turn | 194.00 s | 580.07 s |
| Cached coding decode | 82.4 tok/s | 63.7 tok/s |
M4 Max 40-core GPU / 64 GiB; Kapitsa 0.2.27 + DFlash2; one slot; disk cache off; greedy; thinking off. One run per context cell. The eight-task coding smoke missed CSV trailing empty fields, weighted interval scheduling and TTL-cache semantics. [Evaluation contract](evaluation_summary.json) · [Weight hashes](weights_manifest.json).
```bash
hf download Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Fast --local-dir ./qwopus-fast
hf download incoai/Qwen3.8-27B-DFlash2 --local-dir ./dflash2
kapitsa serve --private --local --model ./qwopus-fast \
--model-id qwopus-coder --context 131072 --slots 1 \
--dflash ./dflash2 --disk-cache 0
```
Use --context 65536 for 64K. Send temperature=0, top_p=1, top_k=0 and chat_template_kwargs={"enable_thinking":false} to /v1/chat/completions. Prompt and output share the limit. DFlash2 is separate.
**Structured JSON / vision.** With the request schemas, both models passed **8/8 original cases twice, 12/12 held-out cases and 8/8 with native MTP3**, graded as exact bare JSON. Apply the matching response_format from [structured_output_profiles.json](structured_output_profiles.json) for OCR, chart, spatial-array or two-image replies. The schemas constrain structure without supplying answers. This request profile fixes the previously observed fences and nested arrays. [Validation](structured_output_validation.json).
| Kapitsa 0.2.31 + DFlash2 | Balanced | Fast |
|---|---:|---:|
| Structured JSON decode | 128.1 tok/s | 165.6 tok/s |
| Code/prose decode | 90.2 tok/s | 108.0 tok/s |
Median of three repetitions on M4 Max / 64 GiB. Native MTP depths 1–3 and DFlash2 each matched plain greedy text on 4/4 short probes. Use --dflash none --spec 3 for native MTP or --spec 0 for plain decoding. [Earlier capability checks](capability_validation.json).
Experimental refusal-direction edit, strength 0.5 on attention outputs in layers 10–17. Refusal reduction and BF16 fidelity unverified. Text, still-image and short MTP checks only; long-context MTP untested.
Converted from pinned Apache-2.0 BF16 using streamed weights. [Jackrong source](https://huggingface.co/Jackrong/Qwopus3.8-27B-Coder-Flash/tree/c7d46e5dadf43ab838f2d594839805b2c2f6ccd3) · [Kapitsa v0.2.27](https://github.com/nace-ai/kapitsa/tree/v0.2.27) · [Other variant](https://huggingface.co/Shiftedx/Qwopus3.8-27B-Coder-Flash-MLX-Balanced).