---

title: RWKV HF Space
emoji: 🐦‍⬛
colorFrom: purple
colorTo: pink
sdk: docker
pinned: false

---

# Simple RWKV OpenAI-Compatible API

---

title: RWKV HF Space
emoji: 🐦‍⬛
colorFrom: purple
colorTo: pink
sdk: docker
pinned: false

---

# Simple RWKV OpenAI-Compatible API

## Quick Windows Setup (no Docker)

This repository was originally packaged with a Dockerfile. It now provides a `setup_windows.ps1` script that mirrors Dockerfile actions and sets up the service locally on Windows (installs Python dependencies, builds the frontend, and downloads the 0.1B model).

Prerequisites:
- Python 3.10+ installed and in PATH
- Node.js + npm (optional, required for building the frontend)
- (Optional) NVIDIA GPU and CUDA (for GPU runtime)

To setup locally on Windows (CPU-only):

```powershell
.\setup_windows.ps1 -gpu:$false -buildFrontend:$true -CONFIG_FILE config.production.yaml
```

If you have a compatible NVIDIA GPU and prefer to install GPU-enabled dependencies, run with the `-gpu` switch.

After setup, run the API:

```powershell
#$env:CONFIG_FILE='config.production.yaml'
python app.py
```

The default production config in `config.production.yaml` now contains a single model — the 0.1B model `rwkv7-g1a-0.1b-20250728-ctx4096` — set as default chat and reasoning model.

To download models defined in any config:

```powershell
python download_models.py --config config.production.yaml
```

This will store the downloaded .pth files under the `DOWNLOAD_MODEL_DIR` specified in the YAML (defaults to `./models`).

Advanced features:
 - `reasoning` is performed in-process by the same model (no external reasoning model is used). Use a request model like `rwkv-latest:thinking` or set the reasoning suffix and the requested model will run reasoning in the same model.
 - `web_search` functionality is available at the request level — set `web_search: true` and optionally `search_top_k` to inject search results from DuckDuckGo into the prompt. This is executed by the server and provided to the same model as context.
 - `tools` are executed server-side and results injected into the prompt for the same model. Supported tools: `web_search` and `calc` (calculator). Example of `tools` usage:

```json
{
	"model": "rwkv-latest",
	"prompt": "Calculate 2+3*4 and tell me the result",
	"tools": [{"name": "calc", "args": {"expression": "2+3*4"}}]
}
```

API endpoints and model listing:
- `GET /api/v1/models` — returns a JSON list of configured models, sampler defaults, and ALLOW_* flags. This lets clients build per-model UI toggles (web search, tools, reasoning) based on server-provided capabilities.

Examples:
- `curl http://127.0.0.1:7860/api/v1/models` will show configured models and their sampler defaults.
Example: POST with `web_search` and reasoning enabled

```json
{
	"model": "rwkv-latest:thinking",
	"prompt": "Who is the current president of France?",
	"max_tokens": 32,
	"web_search": true,
	"search_top_k": 3
}
```

The server will perform a web search for the prompt, aggregate the top 3 results, and inject those into the prompt, then run the model with reasoning enabled — all using the same model instead of an external reasoning or search model.

Universal tool and model-initiated tool calls:
- The `universal` tool returns a structured JSON/dict with the following fields: `action` (calc/web_search), `result` (string), and `metadata` (dict with `confidence`, query/expression, etc.).
- Example `universal` result:

```json
{
	"action": "calc",
	"result": "14",
	"metadata": {"expression": "2+3*4", "confidence": 0.98}
}
```

- The model can also request tools mid-generation by emitting a sentinel tag, e.g.:

```
<tool-call>{"name":"calc","args":{"expression":"40+2"}}</tool-call>
```

	When the model emits such a sentinel, the server will execute the requested tool, inject the results into the prompt, and continue streaming output. The server will also emit a metadata-only streaming chunk so the client is aware a tool was executed mid-stream.

Streaming behavior:
- The API streams responses token-by-token by default (`stream: true`) and persists a `state_name` for the generation if requested (or will generate one). Provide `state_name` to resume continuation from where the previous stream stopped. The server stores model state in memory under `(model, state_name)` so subsequent requests with the same `state_name` can continue generation from that exact point.