README.md

---
license: apache-2.0
pipeline_tag: text-generation
---

# ReyaChat-Reasoning

<img src="https://huggingface.co/Statical-Workspace/Storage/resolve/main/DepressedGirl.png" alt="icon" height="500" width="500">

This is a low restriction, creative roleplay and conversational reasoning model based on [deepseek-ai/DeepSeek-R1-0528-Qwen3-8B](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B).

I have distilled and quantized the model through GPTQ 4-bit model (W4A16), meaning it can run on most GPUs.

This model uses ChatML chat template and can think using "\<think>" and "\</think>" tokens.

Note I made this model for personal use, I just have the repository public for everyone else to use, not expecting me answering requests.

Established by Staticaliza.

# vLLM: Use Instruction

```python
from huggingface_hub import snapshot_download
from vllm import LLM, SamplingParams

# Consider toggling "enforce_eager" to False if you want to load the model quicker, at the expense of tokens per second.
repo = snapshot_download(repo_id="Staticaliza/Reya-Reasoning", allow_patterns=["*.json", "*.bin", "*.safetensors"])
llm = LLM(model=repo, dtype="auto", tensor_parallel_size=torch.cuda.device_count(), enforce_eager=True, trust_remote_code=True)

# ChatML with think tokens is suggested
input = """<|im_start|>system
You are Reya.<|im_end|>
<|im_start|>user
Hi.<|im_end|>
<|im_start|>assistant
<think>"""

params = SamplingParams(
    max_tokens=256,
    temperature=1,
    top_p=0.95,
    top_k=50,
    min_p=0.1,
    presence_penalty=0,
    frequency_penalty=0,
    repetition_penalty=1,
    stop=["<|im_end|>"],
    seed=42,
)

result = llm.generate(input, params)[0].outputs[0].text
print(result)
```