| # Mirai Backend API |
|
|
| Canonical contract of the FastAPI service in `app/`. The web client never calls it directly: the |
| Cloudflare Worker in the `mirai-chat` repo (`src/worker.js`) sits in front, validates and clamps |
| requests, and adds the API key. Its browser-facing differences are listed in that repo's |
| `SETUP_SERVER.md`. |
|
|
| When you change a field, a limit or a status code here, update that file and the Worker constants |
| in the same change (see [Limits](#limits-and-where-they-are-enforced)). |
|
|
| ## Authentication |
|
|
| | Env var | Effect | |
| | --- | --- | |
| | `SPACE_API_KEY` | When set, `/chat` and `/tts` require it as `Authorization: Bearer <key>` or `X-Api-Key: <key>`. Mismatch gives `401`. | |
| | `REQUIRE_API_KEY=1` | Fail closed: if `SPACE_API_KEY` is missing, every `/chat` and `/tts` request gets `503 Server is not configured`. **Set this in production.** Without it and without a key, both endpoints are open to anyone who knows the Space URL. | |
|
|
| `/health` never requires a key. CORS allows any origin for `GET` and `POST` (no credentials). |
| `/docs`, `/redoc` and `/openapi.json` are off unless `ENABLE_DOCS=1`. |
|
|
| ## `GET /health` |
|
|
| ```json |
| { "status": "ok", "model": "mirai_model-q4_k_m.gguf" } |
| ``` |
|
|
| | HTTP | `status` | Meaning | |
| | --- | --- | --- | |
| | 200 | `ok` | Model loaded and ready. | |
| | 503 + `Retry-After: 15` | `loading` | Model still loading in the background thread. | |
| | 503 + `Retry-After: 15` | `error` | Model failed to load (`model` may be empty; details are in the logs). | |
|
|
| ## `POST /chat` |
|
|
| Runs the LLM and returns the reply. Unknown request fields are ignored (the server never trusts |
| client-supplied instructions). |
|
|
| ### Request |
|
|
| | Field | Type | Notes | |
| | --- | --- | --- | |
| | `message` | string, 1-2000 chars | Required. Special chat-template tokens and `User:`/`Mirai:` prefixes are stripped. | |
| | `history` | `{role: "user" \| "assistant", content: string (max 2000)}[]`, max 24 items | Optional. Only the last 12 are considered, and old turns are dropped to fit `PROMPT_TOKEN_BUDGET`. | |
| | `max_tokens` (alias `maxTokens`) | int | Optional. Clamped to **96-192**, default 128. | |
| | `max_words` (alias `maxWords`) | int | Optional. Clamped to **10-60**, default 40. Becomes the length hint in the persona prompt. | |
| | `tts` | bool, default `true` | `false` skips speech synthesis so the text returns as soon as the LLM finishes; fetch audio from `/tts`. | |
|
|
| ### Response `200` |
|
|
| ```json |
| { |
| "text": "Chào bạn! ...", |
| "reply": "Chào bạn! ...", |
| "emotion": "happy", |
| "audio": null, |
| "audioFormat": "mp3", |
| "audioError": null, |
| "lipSync": null, |
| "sources": [], |
| "meta": { "model": "...", "finish_reason": "stop", "truncated": false, "history_used": 4, |
| "prompt_tokens": 512, "completion_tokens": 40, "latency_ms": 9000, "queue_ms": 0, |
| "llm_ms": 8800, "first_token_ms": 4200, "tts_ms": 0 } |
| } |
| ``` |
|
|
| | Field | Notes | |
| | --- | --- | |
| | `text`, `reply` | Same string. `reply` is kept for older clients; new code should read `text`. | |
| | `emotion` | `happy`, `sad`, `angry`, `surprised`, `relaxed`, `neutral`, `shy`, `thinking`, or `null` when there is no clear signal (the client then leaves the face alone). A `[smile]`-style tag emitted by the model wins, otherwise keywords decide. Tags are removed from `text`. | |
| | `audio` | Base64 MP3 or `null` (when `tts: false` or synthesis failed). | |
| | `audioError` | `"tts_unavailable"` when synthesis failed, else `null`. TTS failure never fails the request. | |
| | `lipSync` | See [Lip-sync payload](#lip-sync-payload), or `null`. | |
| | `sources` | Always `[]` for now. Reserved. | |
| | `meta.truncated` | `true` when generation hit the token limit or deadline; the text is cut back to the last whole sentence. | |
|
|
| ### Errors |
|
|
| Error bodies are `{ "detail": "<message>" }` (FastAPI). Through the Worker they become `{ "error": "<message>" }`. |
|
|
| | HTTP | When | Retry-After | |
| | --- | --- | --- | |
| | 400 | Message empty after trimming/sanitizing. | | |
| | 401 | Missing or wrong API key. | | |
| | 413 | Prompt does not fit the context window because the **message** is too long. | | |
| | 422 | Request fails schema validation (length limits, types). | | |
| | 500 | Model failed to load, generation failed, or the **server prompt** alone exceeds `PROMPT_TOKEN_BUDGET` (a config problem, not the user's fault). | | |
| | 503 | Model still loading, or queue full (one request generating plus one waiting). | 15 / 10 | |
|
|
| ## `POST /tts` |
|
|
| Request `{ "text": string, 1-1000 chars }`. Response: |
|
|
| ```json |
| { "audio": "<base64 mp3>", "audioFormat": "mp3", "audioError": null, "lipSync": { } } |
| ``` |
|
|
| On synthesis failure the status is still `200` with `audio: null` and `audioError: "tts_unavailable"`. |
| `503` (+ `Retry-After: 5`) when 4 syntheses are already running. Voice is `vi-VN-HoaiMyNeural`; |
| markup, URLs, `[tags]` and emoji are removed from the text before synthesis. |
|
|
| ## Lip-sync payload |
|
|
| ```json |
| { |
| "metadata": { "duration": 3.42, "text": "...", "source": "word", "version": 2 }, |
| "mouthCues": [ |
| { "start": 0.0, "end": 0.12, "value": { "A": 1.0 } }, |
| { "start": 0.12, "end": 0.2, "value": {} } |
| ] |
| } |
| ``` |
|
|
| - `source` is `word`, `sentence` or `estimated`, depending on the timing marks edge-tts provided. |
| The client only rescales cue time for non-`word` sources. |
| - `value` maps vowel channels `A`, `I`, `U`, `E`, `O` to 0-1 weights (summing to at most 1). An empty object means mouth closed. |
| - Cues are non-overlapping and ordered by `start`; times are seconds from the start of the audio. |
|
|
| ## Limits and where they are enforced |
|
|
| The same limit appears in several layers. Change all of them together. |
|
|
| | Limit | Worker (`mirai-chat/src/worker.js`) | Backend | Client (`chatApi.js`) | |
| | --- | --- | --- | --- | |
| | `message` length | 1000 (silently truncated) | 2000 (422) | | |
| | `history` items / chars each | 12 / 500 (truncated) | 24 / 2000 (422), uses last 12 | 12 (`HISTORY_MESSAGES`) | |
| | `max_tokens` | cap 192, default 128 | clamp 96-192, default 128 | default 128 (`VITE_MIRAI_MAX_TOKENS`) | |
| | `max_words` | cap 60, default 40 | clamp 10-60, default 40 | default 40 (`VITE_MIRAI_MAX_WORDS`) | |
| | `/tts` text | 1000 | 1000 | | |
| | Request body | 16 KB | | | |
| | Timeouts | `/chat` upstream 110 s, `/tts` 30 s, `/health` 10 s | generation deadline `GEN_DEADLINE_SECONDS` (80), TTS 15 s | `/chat` 120 s, `/tts` 30 s, `/health` 5 s | |
|
|
| Timeouts must stay ordered: backend < Worker < client, so the user gets a meaningful `504` instead of an abort. |
|
|
| ## Configuration (backend env vars) |
|
|
| | Var | Default | Purpose | |
| | --- | --- | --- | |
| | `MODEL_REPO`, `MODEL_FILE` | set in `Dockerfile` | GGUF model to download from Hugging Face. | |
| | `CHAT_FORMAT` | `llama-3` | llama-cpp chat template; `auto` uses the one embedded in the GGUF. Use `chatml` for the Qwen3 model from `mirai-train`. | |
| | `EMOTION_TAGS` | off | `1` adds the emotion-tag rule to the system prompt. Needed for models trained by `mirai-train`; leave off for the old Llama 3 model. | |
| | `N_CTX` | 2048 | Context window. | |
| | `PROMPT_TOKEN_BUDGET` | 900 | Extra cap on prompt tokens (slow CPU prefill). | |
| | `N_THREADS`, `N_THREADS_BATCH`, `N_BATCH` | 2, = `N_THREADS`, 256 | CPU tuning. | |
| | `TEMPERATURE` | 0.5 | Sampling temperature. | |
| | `GEN_DEADLINE_SECONDS` | 80 | Hard stop for one generation. | |
| | `KNOWLEDGE_HIDE_PREFIXES` | `Tên thật` | Lines of `app/data/knowledge_data.txt` starting with these (comma separated) are kept out of the prompt. | |
| | `SPACE_API_KEY`, `REQUIRE_API_KEY`, `ENABLE_DOCS` | unset | See above. | |
|
|
