glyphsoftware/sentinel-r3-gguf

Sentinel-R3 — GGUF

GGUF quantizations of glyphsoftware/sentinel-r3, an offensive-security agent for authorized penetration testing, for use with llama.cpp and compatible runtimes (Ollama, LM Studio, llama-server, …).

Authorized security work only. This model is trained to enumerate targets, obtain a foothold, and escalate privileges. Use it only against systems you own or are explicitly authorized to test. You are responsible for complying with all applicable law and with the LICENSE in this repository.

No capability or safety evaluation has been run on the quantized weights. The parent repository publishes multiple-choice cyber-knowledge benchmarks and held-out loss only — no end-to-end agentic task-success benchmark and no safety benchmark. Quantization was not separately evaluated, so any additional quality loss from the quants below is unmeasured.

⚡ Run it with Sentinel

Glyph-Software/sentinel is the reference harness for this model line — "a red-teaming agent for the terminal — and every command it runs happens inside a container it cannot escape."

curl -fsSL https://raw.githubusercontent.com/Glyph-Software/sentinel/main/install.sh | sh

It exposes the same 19 native function-calling tools this model was trained on, so the action space at inference matches the one it saw during fine-tuning. Point it at any OpenAI-compatible endpoint — including llama-server running one of the files below.

Files

File	Quant	Size	Notes
Sentinel-R3-Q4_K_M.gguf	Q4_K_M	~16 GB	Best size/quality tradeoff; the usual default
Sentinel-R3-Q5_K_M.gguf	Q5_K_M	~19 GB	Higher fidelity, still comfortable on 24 GB
Sentinel-R3-Q8_0.gguf	Q8_0	~29 GB	Near-lossless; use when you have the memory

All are converted from the bf16 merged weights with convert_hf_to_gguf.py and quantized with llama-quantize.

Requirements

You need a llama.cpp build with qwen35 support. This is a Qwen3.5 hybrid model — 48 Gated-DeltaNet linear-attention layers interleaved with 16 full-attention layers. Support is LLM_ARCH_QWEN35 in llama-arch.cpp. Older builds will fail to load these files with an unknown-architecture error. Build from a recent master if your package-manager llama.cpp is behind.

Usage

llama-cli -m Sentinel-R3-Q4_K_M.gguf \
  -p "Assess the authorized lab host at 10.129.0.10. Enumerate it, get a foothold, and escalate." \
  -c 16384 -n 1024 --temp 0.3 --top-p 0.9

As an OpenAI-compatible server:

llama-server -m Sentinel-R3-Q4_K_M.gguf -c 32768 --host 0.0.0.0 --port 8080

The system prompt is built in

The chat template embedded in these files always emits the operator system prompt — the pentester persona and its tool surface — so you get the training-matched prompt with no setup. A system message you supply is appended to that default rather than replacing it, so use it for engagement-specific additions:

Scope: 10.129.0.0/24 only. Do not touch the domain controller.

Recommended generation settings

Parameter	Value
--temp	0.2 – 0.4
--top-p	0.9
-n	1024+ (reasoning and tool calls consume tokens)
-c	16384+ (engagements accumulate long tool output)

Notes and limitations

• Text-only. The parent model is a vision-language checkpoint, but these GGUFs contain the language model only — no mmproj, so no image input. The training data was text-only, so security behavior is unaffected.
• Prose tool surface. The model was trained with its tools described in prose in the system message, not passed as a structured tools= list. Harnesses that inject tool schemas structurally present a different prompt than training used.
• ~9% of training tool calls are synthesized and the task-list, cron, and multi-agent tools have no precedent in the source corpus — they teach syntax and plausible placement, not verified strategy.
• Sparse tail tools — Glob, Edit, TaskOutput, TaskStop have fewer than 10 training examples each.
• Not a substitute for a skilled operator. Outputs may be incorrect, incomplete, or unsafe to run. Review every command before executing it.
• Harness responsibility. Command execution, scoping, network isolation, and guardrails are the operator's responsibility, not the model's.

Full training details, benchmarks, and limitations are in the parent model card: glyphsoftware/sentinel-r3.

License

Proprietary — All Rights Reserved. Governed by the Glyph Proprietary License v1.0 in the bundled LICENSE file. The base model Qwen/Qwen3.8-27B is distributed by its authors under Apache-2.0; that license governs the base weights, and the terms in LICENSE govern this fine-tuned derivative.

© 2026 Glyph Software LLP. All rights reserved.