---
base_model: Altworld/Astrea-R8-Chat-9B
license: apache-2.0
library_name: gguf
pipeline_tag: text-generation
tags:
- gguf
- llama.cpp
- qwen3.5
- chat
- creative-writing
- non-reasoning
---

# Astrea R8 Chat 9B — Q8_0 GGUF

This is an unofficial community Q8_0 GGUF conversion of
[Altworld/Astrea-R8-Chat-9B](https://huggingface.co/Altworld/Astrea-R8-Chat-9B)
for llama.cpp, with an explicit non-reasoning chat template.

## File

| File | Quantization | Size |
|---|---:|---:|
| `Astrea-R8-Chat-9B-Q8_0.gguf` | Q8_0 | 9,527,501,280 bytes (8.87 GiB) |

## Embedded non-reasoning template

The GGUF contains a hard non-reasoning Jinja template in
`tokenizer.chat_template`; no external template file is required.

Astrea's optional thinking mode was not reliable in local llama.cpp testing:
simple prompts could consume hundreds of tokens before emitting `</think>`, and often did not end reasoning at all, and simply responded as if reasoning was not enabled.
The bundled template therefore always places a closed, empty thinking block in
the prompt and does not expose an `enable_thinking` template variable. It also
omits hidden reasoning when replaying assistant messages into conversation
history. A standalone copy is included as `chat_template.jinja` for inspection.

## llama.cpp

```powershell
llama-server.exe `
  --model Astrea-R8-Chat-9B-Q8_0.gguf `
  --jinja `
  --reasoning off `
  --reasoning-format none `
  --ctx-size 32768 `
  --n-gpu-layers all `
  --temp 0.8 `
  --top-p 1.0 `
  --top-k 0 `
  --min-p 0.025 `
  --repeat-penalty 1.08
```

The model metadata advertises a 262,144-token context window. Choose a context
size appropriate for your available VRAM/RAM. The command above starts at a
more conservative 32,768 tokens. I was able to easily run a much more ambitous setup with `-ngl all --fit off -c 147456 -np 4 --kv-unified` on a 16GB VRAM card (5070 Ti).

## Conversion notes

1. The original safetensors were converted to BF16 GGUF with llama.cpp's
   `convert_hf_to_gguf.py` using `--no-mtp`. The downloaded checkpoint did not
   contain the extra MTP-layer tensors declared by its configuration.
2. BF16 was quantized with `llama-quantize` using `Q8_0`.
3. llama.cpp's `gguf_new_metadata.py` embedded the hard non-reasoning template;
   this metadata-only copy did not requantize tensors.

The tensor-only SHA-256 reported by `llama-gguf-hash` was identical before and
after the metadata rewrite:

```text
20d213a0c5ee663ef6d02ffcff8d0b28cbff18b559c67aca5250cd5e6a22d624
```

The final whole-file checksums are in `SHA256SUMS`.

## Validation

The final GGUF was loaded directly by `llama-server` without
`--chat-template-file`. Its exposed template matched the bundled standalone
Jinja, and a request that explicitly supplied `enable_thinking=true` still
returned normal content with no `reasoning_content`.

## License and attribution

The source model is released under Apache-2.0. See `LICENSE` and `NOTICE`, and
refer to the [source model card](https://huggingface.co/Altworld/Astrea-R8-Chat-9B)
for its intended use, evaluation results, and limitations.