File size: 7,438 Bytes
ae99ec3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 | # Mirai Backend API
Canonical contract of the FastAPI service in `app/`. The web client never calls it directly: the
Cloudflare Worker in the `mirai-chat` repo (`src/worker.js`) sits in front, validates and clamps
requests, and adds the API key. Its browser-facing differences are listed in that repo's
`SETUP_SERVER.md`.
When you change a field, a limit or a status code here, update that file and the Worker constants
in the same change (see [Limits](#limits-and-where-they-are-enforced)).
## Authentication
| Env var | Effect |
| --- | --- |
| `SPACE_API_KEY` | When set, `/chat` and `/tts` require it as `Authorization: Bearer <key>` or `X-Api-Key: <key>`. Mismatch gives `401`. |
| `REQUIRE_API_KEY=1` | Fail closed: if `SPACE_API_KEY` is missing, every `/chat` and `/tts` request gets `503 Server is not configured`. **Set this in production.** Without it and without a key, both endpoints are open to anyone who knows the Space URL. |
`/health` never requires a key. CORS allows any origin for `GET` and `POST` (no credentials).
`/docs`, `/redoc` and `/openapi.json` are off unless `ENABLE_DOCS=1`.
## `GET /health`
```json
{ "status": "ok", "model": "mirai_model-q4_k_m.gguf" }
```
| HTTP | `status` | Meaning |
| --- | --- | --- |
| 200 | `ok` | Model loaded and ready. |
| 503 + `Retry-After: 15` | `loading` | Model still loading in the background thread. |
| 503 + `Retry-After: 15` | `error` | Model failed to load (`model` may be empty; details are in the logs). |
## `POST /chat`
Runs the LLM and returns the reply. Unknown request fields are ignored (the server never trusts
client-supplied instructions).
### Request
| Field | Type | Notes |
| --- | --- | --- |
| `message` | string, 1-2000 chars | Required. Special chat-template tokens and `User:`/`Mirai:` prefixes are stripped. |
| `history` | `{role: "user" \| "assistant", content: string (max 2000)}[]`, max 24 items | Optional. Only the last 12 are considered, and old turns are dropped to fit `PROMPT_TOKEN_BUDGET`. |
| `max_tokens` (alias `maxTokens`) | int | Optional. Clamped to **96-192**, default 128. |
| `max_words` (alias `maxWords`) | int | Optional. Clamped to **10-60**, default 40. Becomes the length hint in the persona prompt. |
| `tts` | bool, default `true` | `false` skips speech synthesis so the text returns as soon as the LLM finishes; fetch audio from `/tts`. |
### Response `200`
```json
{
"text": "Chào bạn! ...",
"reply": "Chào bạn! ...",
"emotion": "happy",
"audio": null,
"audioFormat": "mp3",
"audioError": null,
"lipSync": null,
"sources": [],
"meta": { "model": "...", "finish_reason": "stop", "truncated": false, "history_used": 4,
"prompt_tokens": 512, "completion_tokens": 40, "latency_ms": 9000, "queue_ms": 0,
"llm_ms": 8800, "first_token_ms": 4200, "tts_ms": 0 }
}
```
| Field | Notes |
| --- | --- |
| `text`, `reply` | Same string. `reply` is kept for older clients; new code should read `text`. |
| `emotion` | `happy`, `sad`, `angry`, `surprised`, `relaxed`, `neutral`, `shy`, `thinking`, or `null` when there is no clear signal (the client then leaves the face alone). A `[smile]`-style tag emitted by the model wins, otherwise keywords decide. Tags are removed from `text`. |
| `audio` | Base64 MP3 or `null` (when `tts: false` or synthesis failed). |
| `audioError` | `"tts_unavailable"` when synthesis failed, else `null`. TTS failure never fails the request. |
| `lipSync` | See [Lip-sync payload](#lip-sync-payload), or `null`. |
| `sources` | Always `[]` for now. Reserved. |
| `meta.truncated` | `true` when generation hit the token limit or deadline; the text is cut back to the last whole sentence. |
### Errors
Error bodies are `{ "detail": "<message>" }` (FastAPI). Through the Worker they become `{ "error": "<message>" }`.
| HTTP | When | Retry-After |
| --- | --- | --- |
| 400 | Message empty after trimming/sanitizing. | |
| 401 | Missing or wrong API key. | |
| 413 | Prompt does not fit the context window because the **message** is too long. | |
| 422 | Request fails schema validation (length limits, types). | |
| 500 | Model failed to load, generation failed, or the **server prompt** alone exceeds `PROMPT_TOKEN_BUDGET` (a config problem, not the user's fault). | |
| 503 | Model still loading, or queue full (one request generating plus one waiting). | 15 / 10 |
## `POST /tts`
Request `{ "text": string, 1-1000 chars }`. Response:
```json
{ "audio": "<base64 mp3>", "audioFormat": "mp3", "audioError": null, "lipSync": { } }
```
On synthesis failure the status is still `200` with `audio: null` and `audioError: "tts_unavailable"`.
`503` (+ `Retry-After: 5`) when 4 syntheses are already running. Voice is `vi-VN-HoaiMyNeural`;
markup, URLs, `[tags]` and emoji are removed from the text before synthesis.
## Lip-sync payload
```json
{
"metadata": { "duration": 3.42, "text": "...", "source": "word", "version": 2 },
"mouthCues": [
{ "start": 0.0, "end": 0.12, "value": { "A": 1.0 } },
{ "start": 0.12, "end": 0.2, "value": {} }
]
}
```
- `source` is `word`, `sentence` or `estimated`, depending on the timing marks edge-tts provided.
The client only rescales cue time for non-`word` sources.
- `value` maps vowel channels `A`, `I`, `U`, `E`, `O` to 0-1 weights (summing to at most 1). An empty object means mouth closed.
- Cues are non-overlapping and ordered by `start`; times are seconds from the start of the audio.
## Limits and where they are enforced
The same limit appears in several layers. Change all of them together.
| Limit | Worker (`mirai-chat/src/worker.js`) | Backend | Client (`chatApi.js`) |
| --- | --- | --- | --- |
| `message` length | 1000 (silently truncated) | 2000 (422) | |
| `history` items / chars each | 12 / 500 (truncated) | 24 / 2000 (422), uses last 12 | 12 (`HISTORY_MESSAGES`) |
| `max_tokens` | cap 192, default 128 | clamp 96-192, default 128 | default 128 (`VITE_MIRAI_MAX_TOKENS`) |
| `max_words` | cap 60, default 40 | clamp 10-60, default 40 | default 40 (`VITE_MIRAI_MAX_WORDS`) |
| `/tts` text | 1000 | 1000 | |
| Request body | 16 KB | | |
| Timeouts | `/chat` upstream 110 s, `/tts` 30 s, `/health` 10 s | generation deadline `GEN_DEADLINE_SECONDS` (80), TTS 15 s | `/chat` 120 s, `/tts` 30 s, `/health` 5 s |
Timeouts must stay ordered: backend < Worker < client, so the user gets a meaningful `504` instead of an abort.
## Configuration (backend env vars)
| Var | Default | Purpose |
| --- | --- | --- |
| `MODEL_REPO`, `MODEL_FILE` | set in `Dockerfile` | GGUF model to download from Hugging Face. |
| `CHAT_FORMAT` | `llama-3` | llama-cpp chat template; `auto` uses the one embedded in the GGUF. Use `chatml` for the Qwen3 model from `mirai-train`. |
| `EMOTION_TAGS` | off | `1` adds the emotion-tag rule to the system prompt. Needed for models trained by `mirai-train`; leave off for the old Llama 3 model. |
| `N_CTX` | 2048 | Context window. |
| `PROMPT_TOKEN_BUDGET` | 900 | Extra cap on prompt tokens (slow CPU prefill). |
| `N_THREADS`, `N_THREADS_BATCH`, `N_BATCH` | 2, = `N_THREADS`, 256 | CPU tuning. |
| `TEMPERATURE` | 0.5 | Sampling temperature. |
| `GEN_DEADLINE_SECONDS` | 80 | Hard stop for one generation. |
| `KNOWLEDGE_HIDE_PREFIXES` | `Tên thật` | Lines of `app/data/knowledge_data.txt` starting with these (comma separated) are kept out of the prompt. |
| `SPACE_API_KEY`, `REQUIRE_API_KEY`, `ENABLE_DOCS` | unset | See above. |
|