| --- |
| license: apache-2.0 |
| tags: |
| - gguf |
| - qwen3.5 |
| - openmythos |
| - build-small-hackathon |
| datasets: |
| - build-small-hackathon/CVE_Vulnerailities_Detailed |
| - himanshu17HF/ArvixImport-Filtered-Final |
| base_model: |
| - build-small-hackathon/OpenMythos |
| - Qwen/Qwen3.6-27B |
| --- |
| |
| # OpenMythos 27B - GGUF |
|
|
| GGUF quantisation of [build-small-hackathon/OpenMythos](https://huggingface.co/build-small-hackathon/OpenMythos), |
| a fine-tune of [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B). |
|
|
| Converted with `convert_hf_to_gguf.py --no-mtp` from llama.cpp build 9658. |
| The fine-tune does not include MTP head weights (dropped during training), so MTP |
| is not available in this GGUF. |
|
|
| ## Available Quantisations |
|
|
| | File | Size | Type | |
| |------|------|------| |
| | OpenMythos-27B-F16.gguf | 53.8 GB | F16 | |
| | OpenMythos-27B-Q5_K.gguf | 18.3 GB | Q5_K_M | |
| | OpenMythos-27B-Q4_K.gguf | 15.4 GB | Q4_K_M | |
| | OpenMythos-27B-Q6_K.gguf | 21.2 GB | Q6_K | |
|
|
| ## Benchmark |
|
|
| Evaluated with [SecEval](https://github.com/XuanwuAI/SecEval) (commit 7aef317) on 2189 |
| multiple-choice security questions. Backend: llama.cpp OpenAI-compatible server, fully |
| offloaded to GPU. No chain-of-thought / reasoning enabled (`enable_thinking=false`). |
|
|
| Prompt formatted with a system prompt requesting letter-only answers (no explanation). |
|
|
| | Set | Model | Score | |
| |-----|-------|-------| |
| | A | OpenMythos-27B-Q5_K | 1703 / 2189 (77.8%) | |
| | B | VulnLLM-R-7B | 1315 / 2189 (60.1%) | |
| |
| ### OpenMythos-27B-Q5_K test parameters |
|
|
| - model: `OpenMythos-27B-Q5_K.gguf` |
| - inference: `temp=0.2`, `top_p=0.8`, `top_k=20`, `min_p=0.05`, `repeat_penalty=1.02` |
| - benchmark script: `/mnt/storage/SecEval-tmp/run_bench.py` |
| - output: `seceval-1781809723.json` |
| - prompt speed: 282 tok/s | generation speed: 68 tok/s |
|
|
| #### Per-topic scores |
|
|
| | Topic | Score | |
| |-------|-------| |
| | PenTest | 84.2% | |
| | MemorySafety | 83.3% | |
| | WebSecurity | 82.7% | |
| | Vulnerability | 77.8% | |
| | NetworkSecurity | 77.4% | |
| | SoftwareSecurity | 75.0% | |
| | ApplicationSecurity | 74.8% | |
| | SystemSecurity | 73.6% | |
| | Cryptography | 71.4% | |
|
|
| ### VulnLLM-R-7B test parameters |
|
|
| - model: `VulnLLM-R-7B.Q6_K.gguf` |
| - inference: same settings as above |
| - output: `seceval-1781811525.json` |
| - prompt speed: 148 tok/s | generation speed: 39 tok/s |
|
|
| #### Per-topic scores |
|
|
| | Topic | Score | |
| |-------|-------| |
| | PenTest | 70.9% | |
| | WebSecurity | 66.4% | |
| | Vulnerability | 58.7% | |
| | NetworkSecurity | 58.3% | |
| | SystemSecurity | 56.4% | |
| | SoftwareSecurity | 54.7% | |
| | ApplicationSecurity | 54.7% | |
| | MemorySafety | 54.2% | |
| | Cryptography | 28.6% | |
|
|
| Full detailed results are included in this repo: `seceval-1781809723.json` and |
| `seceval-1781811525.json`. |
|
|
| ## Usage |
|
|
| ### llama-server (recommended) |
|
|
| ```ini |
| [OpenMythos-27B] |
| model = /mnt/storage/models/OpenMythos/OpenMythos-27B-Q5_K.gguf |
| chat-template-file = /mnt/storage/llama-server/chat_template-v15.jinja |
| ctx-size = 65536 |
| cache-type-k = q8_0 |
| cache-type-v = q8_0 |
| cache-prompt = on |
| cache-reuse = 2048 |
| batch-size = 4096 |
| ubatch-size = 4096 |
| kv-unified = on |
| parallel = 1 |
| gpu-layers = all |
| temp = 0.2 |
| top-p = 0.8 |
| top-k = 20 |
| min-p = 0.05 |
| presence-penalty = 0.2 |
| repeat-penalty = 1.02 |
| spec-type = ngram-mod |
| spec-draft-n-max = 5 |
| reasoning-format = deepseek |
| swa-checkpoints = 5 |
| ``` |
|
|
| ### llama-cli |
|
|
| ```bash |
| /mnt/storage/llama.cpp/build/bin/llama-cli \ |
| -m /mnt/storage/models/OpenMythos/OpenMythos-27B-Q5_K.gguf \ |
| --chat-template-file /mnt/storage/llama-server/chat_template-v15.jinja \ |
| -c 65536 -b 4096 --ubatch-size 4096 \ |
| --cache-type-k q8_0 --cache-type-v q8_0 \ |
| --kv-unified -t 8 -fa \ |
| --temp 0.2 --top-p 0.8 --top-k 20 --min-p 0.05 \ |
| --presence-penalty 0.2 --repeat-penalty 1.02 \ |
| -ngl all \ |
| -p "Your prompt here" |
| ``` |
|
|
