Instructions to use Ayaka157/RetNet-0.5B-v21 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ayaka157/RetNet-0.5B-v21 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ayaka157/RetNet-0.5B-v21", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Ayaka157/RetNet-0.5B-v21", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Ayaka157/RetNet-0.5B-v21 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ayaka157/RetNet-0.5B-v21"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ayaka157/RetNet-0.5B-v21",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Ayaka157/RetNet-0.5B-v21

SGLang

How to use Ayaka157/RetNet-0.5B-v21 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ayaka157/RetNet-0.5B-v21" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ayaka157/RetNet-0.5B-v21",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ayaka157/RetNet-0.5B-v21" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ayaka157/RetNet-0.5B-v21",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Ayaka157/RetNet-0.5B-v21 with Docker Model Runner:
```
docker model run hf.co/Ayaka157/RetNet-0.5B-v21
```

RetNet-0.5B-v21 / README.md

Ayaka157

Upload folder using huggingface_hub

a9e39bc verified 22 days ago

preview code

Raw History Blame Contribute Delete

1.57 kB

---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags: [retnet, retention, custom-architecture]
---

# RetNet-0.5B-v21

RetNet 0.5B（504,179,896 参数）第 v21 代权重，从零训练。

## 这一代和上一代的区别

159,857 步 / 10.23B 训练 token / 9.81B 唯一 token（19.46 token/参数）；8 段语料自动接力（预训练→中文网页→代码→教材→代码剩余→中文编程→数学→新旧混合 SFT）；可热加载的训练计划（阶段预算 / LR / 存档节奏在线改）

## 用法

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Ayaka157/RetNet-0.5B-v21", trust_remote_code=True)
tok = AutoTokenizer.from_pretrained("Ayaka157/RetNet-0.5B-v21", trust_remote_code=True)

ids = tok("人工智能的未来是", return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=64, do_sample=True, temperature=0.8)
print(tok.decode(out[0], skip_special_tokens=True))
```

`trust_remote_code=True` 是必需的：这是自定义架构，代码随仓库发布
（`modeling_retnet.py`），不在 transformers 里。

## 训练与评测

见项目主页的 `docs/`。本代的关键数字：

| 指标 | 值 |
|---|---|
| 域内 held-out loss（5 个分片） | 3.5023（ep3 基线 3.6287） |
| 上一代（ep3）基线 | 3.6287 |
| 相对变化 | -3.5%（中途最佳 -21.5% @ step 17,500） |
| 训练 token | 9.81B 唯一 / 19.46 tok/param |
| 硬件 | 2× RTX PRO 6000（DDP） |

	---
	license: apache-2.0
	library_name: transformers
	pipeline_tag: text-generation
	tags: [retnet, retention, custom-architecture]
	---

	# RetNet-0.5B-v21

	RetNet 0.5B（504,179,896 参数）第 v21 代权重，从零训练。

	## 这一代和上一代的区别

	159,857 步 / 10.23B 训练 token / 9.81B 唯一 token（19.46 token/参数）；8 段语料自动接力（预训练→中文网页→代码→教材→代码剩余→中文编程→数学→新旧混合 SFT）；可热加载的训练计划（阶段预算 / LR / 存档节奏在线改）

	## 用法

	```python
	from transformers import AutoModelForCausalLM, AutoTokenizer

	model = AutoModelForCausalLM.from_pretrained("Ayaka157/RetNet-0.5B-v21", trust_remote_code=True)
	tok = AutoTokenizer.from_pretrained("Ayaka157/RetNet-0.5B-v21", trust_remote_code=True)

	ids = tok("人工智能的未来是", return_tensors="pt").input_ids
	out = model.generate(ids, max_new_tokens=64, do_sample=True, temperature=0.8)
	print(tok.decode(out[0], skip_special_tokens=True))
	```

	`trust_remote_code=True` 是必需的：这是自定义架构，代码随仓库发布
	（`modeling_retnet.py`），不在 transformers 里。

	## 训练与评测

	见项目主页的 `docs/`。本代的关键数字：

	\| 指标 \| 值 \|
	\|---\|---\|
	\| 域内 held-out loss（5 个分片） \| 3.5023（ep3 基线 3.6287） \|
	\| 上一代（ep3）基线 \| 3.6287 \|
	\| 相对变化 \| -3.5%（中途最佳 -21.5% @ step 17,500） \|
	\| 训练 token \| 9.81B 唯一 / 19.46 tok/param \|
	\| 硬件 \| 2× RTX PRO 6000（DDP） \|