README.md

---
license: apache-2.0
base_model: Altworld/Astrea-R8-Chat-9B
pipeline_tag: text-generation
language:
- en
tags:
- gguf
- llama.cpp
- qwen3.5
- chat
- creative-writing
- altworld
---

# Astrea R8 Chat 9B — GGUF

Official text-only GGUF builds of
[Altworld/Astrea-R8-Chat-9B](https://huggingface.co/Altworld/Astrea-R8-Chat-9B),
a compact conversational and creative-writing model built on Qwen3.5-9B.

**[Try Astrea](https://chat.altworld.io)** ·
**[API](https://altworld.io/developer)** ·
**[Documentation](https://astrea.altworld.io/docs/)**

## Files

| Quant | Size | Use case |
|---|---:|---|
| **Q5_K_M** | 6.02 GiB | Recommended balance of quality and size |
| Q4_K_M | 5.24 GiB | Smaller and faster; passed the same chat-routing smoke tests |
| Q6_K | 6.85 GiB | Higher fidelity |
| Q8_0 | 8.87 GiB | Highest-fidelity quant in this repository |

These are single-file, text-only models. The inherited vision components are
not included because Astrea's visual behavior was not part of the release
evaluation.

## Run with llama.cpp

Use a current build of
[llama.cpp](https://github.com/ggml-org/llama.cpp); Qwen3.5 support is recent.

```bash
llama-cli \
  -hf Altworld/Astrea-R8-Chat-9B-GGUF:Q5_K_M \
  -cnv \
  --reasoning off \
  --temp 0.8 \
  --min-p 0.025 \
  --repeat-penalty 1.08
```

For an OpenAI-compatible local server:

```bash
llama-server \
  -hf Altworld/Astrea-R8-Chat-9B-GGUF:Q5_K_M \
  --reasoning off \
  --temp 0.8 \
  --min-p 0.025 \
  --repeat-penalty 1.08
```

Use `--temp 0.2` for factual chat. The model carries its official chat
template inside the GGUF and does not require a system prompt. Reasoning is
disabled in the examples because Astrea's release behavior is direct response
without a visible thinking block.

## Validation

Each file was loaded and generated from with llama.cpp. The checks covered:

- bare greeting with no system prompt;
- native Astrea/Altworld identity;
- ordinary factual chat;
- supportive conversation;
- creative-writing routing from a plain request with no scene card;
- multi-turn recall using the embedded chat template.

Q4_K_M, the most aggressive quant here, correctly answered `Good afternoon.`
as normal chat and produced a coherent story from `Write a short story about a
dragon who collects spoons.` Q5_K_M correctly recalled a name across turns.

The GGUFs declare the base architecture's 262,144-token context. Actual usable
context depends on your hardware, llama.cpp build, and runtime settings.

## Conversion details

Converted from the official merged BF16 release with llama.cpp commit
[`76f46ad`](https://github.com/ggml-org/llama.cpp/commit/76f46ad29d61fd8c1401e8221842934bf62a6064).
The source checkpoint contains no MTP tensors, so conversion used `--no-mtp`.
Quantization used llama.cpp's `Q4_K_M`, `Q5_K_M`, `Q6_K`, and `Q8_0` presets.

## About Astrea

Astrea was trained on top of Qwen3.5-9B using multi-round LoRA supervised
fine-tuning, alternating conversational and narrative rounds and checking each
round against Altworldbench. See the
[BF16 model card](https://huggingface.co/Altworld/Astrea-R8-Chat-9B) for
benchmarks, methodology, examples, limitations, and the evaluation protocol.

Astrea is English-first and can still hallucinate. Verify high-stakes claims
independently. Engine and quantization differences can affect output, so test
your own prompts and settings.

## License

Apache-2.0. See `LICENSE` and `NOTICE`.