Upload portable FP8 release

README.md

@@ -1,3 +1,111 @@
 ---
 license: apache-2.0
+base_model: Altworld/Astrea-R8-Chat-9B
+base_model_relation: quantized
+pipeline_tag: image-text-to-text
+library_name: transformers
+language:
+- en
+tags:
+- qwen3.5
+- chat
+- creative-writing
+- altworld
+- fp8
+- compressed-tensors
 ---
+
+# Altworld Astrea R8 Chat 9B — FP8
+
+This is the portable FP8 release of
+[Altworld/Astrea-R8-Chat-9B](https://huggingface.co/Altworld/Astrea-R8-Chat-9B),
+Altworld's open 9B creative-writing and chat model.
+
+**[Try Astrea](https://chat.altworld.io)** ·
+**[API](https://altworld.io/developer)** ·
+**[Documentation](https://astrea.altworld.io/docs/)**
+
+The checkpoint uses LLM Compressor's `compressed-tensors` format and can be
+served directly by compatible versions of vLLM. It contains six safetensor
+shards totalling 11.09 GiB, compared with 17.53 GiB for the BF16 weights.
+
+## Quantization
+
+The checkpoint was generated from the released merged BF16 model with:
+
+- LLM Compressor 0.12.0;
+- compressed-tensors 0.17.1;
+- `FP8_DYNAMIC`, using E4M3 weights and BF16 per-output-channel scales;
+- dynamic per-token FP8 activation quantization at inference time;
+- data-free round-to-nearest post-training quantization.
+
+All 248 two-dimensional language-model linear weights are stored in FP8. The
+following sensitive or non-linear components remain in BF16:
+
+- input token embeddings;
+- `lm_head`;
+- Qwen3.5 linear-attention convolution kernels;
+- the complete vision tower.
+
+No calibration or training dataset was used during conversion.
+
+## Running with vLLM
+
+Use a recent vLLM release with `compressed-tensors` support and an accelerator
+with supported FP8 kernels:
+
+```bash
+vllm serve Altworld/Astrea-R8-Chat-9B-FP8 \
+  --max-model-len 32768 \
+  --gpu-memory-utilization 0.90
+```
+
+The quantization configuration is embedded in `config.json`; no separate
+`--quantization` argument should be necessary.
+
+Recommended sampling for writing:
+
+```json
+{
+  "temperature": 0.8,
+  "min_p": 0.025,
+  "repetition_penalty": 1.08
+}
+```
+
+Use `temperature: 0.2` for ordinary factual chat. The bundled chat template
+works without a system prompt and defaults to hidden thinking.
+
+## Validation
+
+Before upload, this artifact passed the following local checks:
+
+- every original tensor is present with the expected shape;
+- all 248 targeted weights are E4M3 with matching BF16 channel scales;
+- all excluded components retain their original BF16 dtype;
+- the sharded safetensor index exactly matches all 1,008 stored tensors;
+- `AutoConfig`, `AutoTokenizer`, and `AutoProcessor` load successfully;
+- `Qwen3_5ForConditionalGeneration` loads the compressed checkpoint and
+  completes a text-generation smoke test.
+
+The local Transformers smoke test dequantizes weights for its CPU fallback and
+therefore does not measure FP8 serving speed. Quantization changes model
+numerics, so applications should evaluate this checkpoint on their own prompts
+before replacing the BF16 release. Quant-specific Altworldbench results are not
+claimed on this card.
+
+## Model quality and limitations
+
+See the [BF16 model card](https://huggingface.co/Altworld/Astrea-R8-Chat-9B)
+for training lineage, benchmark methodology, samples, usage guidance, and
+limitations. In particular:
+
+- Astrea is text-first; inherited vision behavior was not part of its release
+  evaluation;
+- English dominates training and evaluation;
+- outputs can be confidently wrong and must not be treated as authoritative in
+  high-stakes settings.
+
+## License
+
+Apache-2.0, matching the source model. See `LICENSE` and `NOTICE`.