Bring README, architecture and environment docs in line with the current routing, status and security behavior

AGENTS.md

@@ -20,7 +20,7 @@
 - If you find a leaked secret, tell the owner to rotate it. Do not rewrite git history without asking.
 
 ## What this repo is
-- Lumen: a Telegram bot (persona styled after Claude) running as a single FastAPI + aiogram webhook service on a Hugging Face Space (Docker, Python 3.13, entry: python -u bot.py). LLM backbone: Google Gemini + free OpenRouter models + Groq with per-message routing (web search and link reading go to Gemini, plain text starts at Groq, OpenRouter covers the rest, to protect Gemini's daily quota).
+- Lumen: a Telegram bot (persona styled after Claude) running as a single FastAPI + aiogram webhook service on a Hugging Face Space (Docker, Python 3.13, entry: python -u bot.py). LLM backbone: Google Gemini + free OpenRouter models + Groq with per-message routing (plain text starts at Gemma, then Groq/OpenRouter; web search and link reading use Gemini; every fresh query also gets external DuckDuckGo/Google News snippets for all providers, to protect Gemini's daily quota).
 - bot.py is the entry point and composition root (env, logging, Dispatcher, main). Domain logic lives in lumen_*.py; list the directory to find the module you need.
 - Model routing lives in lumen_router_config.py, injection and leak defenses in lumen_security.py.
 - Tests need no real secrets: conftest.py stubs BOT_TOKEN, GEMINI_API_KEY and BOT_LOG_PATH. Do not add real keys to tests or fixtures.
@@ -43,9 +43,9 @@
 - If nothing is unverified or risky, say so in one phrase instead of padding.
 
 ## Gotchas
-- Every push to main that passes CI auto-deploys to the live HF Space (.github/workflows/sync-to-hf.yml waits for the CI workflow, then force-pushes the checked commit). A green CI run is a deploy.
+- Every push to main that passes CI auto-deploys to the live HF Space (.github/workflows/sync-to-hf.yml waits for the CI workflow, then pushes the checked commit with a lease). A direct Space hotfix makes the sync fail loudly instead of being overwritten. A green CI run is a deploy.
 - The Telegram proxy (proxy.ts, Deno Deploy) is deployed separately from the HF Space. Pushing a change to it does not update the running proxy: after any proxy change, tell the owner it needs a manual redeploy.
-- Streaming has only been tested against mocks, not live Gemini/OpenRouter SSE. Before touching streaming code, read the manual smoke-test checklist in docs/ARCHITECTURE.md and list in the report which of its points the owner must run by hand.
+- Automated streaming tests use mocked clients; live Gemini/OpenRouter SSE has been exercised manually through the smoke bot, but there is no controlled integration test. Before touching streaming code, read the manual smoke-test checklist in docs/ARCHITECTURE.md and list in the report which of its points the owner must run by hand.
 - Versions in requirements.txt are pinned with == (HF rebuilds the image from scratch on every deploy; see the file header comment). Bumping a version is a deliberate decision followed by a manual smoke test (/diag + main commands), not a side effect.
 - A model leaves the routing only with dated evidence (production log errors or provider docs), written in a comment. _OR_MODEL_HEALTH is the single source of truth for exclusions. Never remove a model on a guess.
 - unittest.mock.patch only changes the namespace it targets: patch the module that reads the name (for example lumen_router_config.X), not bot.X.

README.md

@@ -39,8 +39,9 @@ A Telegram bot styled after Claude's tone and personality (direct, warm, light o
 
 ## Features
 
-- **Automatic model routing.** Every message goes to whichever provider actually has what it needs: plain text starts at Groq (1000 free requests/day), web search and link reading go to Gemini, OpenRouter covers the rest — protecting Gemini's tight daily quota.
+- **Automatic model routing.** Every message goes to whichever provider actually has what it needs: plain text starts at Gemma (the largest free quota), then Groq and OpenRouter; web search and link reading go to Gemini. Every fresh query also gets external snippets (DuckDuckGo and Google News with publication dates) for whichever model answers, including the non-Gemini reserves.
 - **Streaming replies** with a self-calibrating typing-speed pacer, so answers type themselves in instead of landing in a few large chunks.
+- **Claude-style live status.** In private chats search and media requests show a sparkle timeline ("Searching the web", found sources, a thinking timer) and finish with a collapsed "Thought for Ns" block; simple answers get a subtle italic thought line. Private answers are plain messages, without Telegram reply quotes.
 - **Defenses against prompt injection and identity leaks.** A deterministic input filter plus output scrubbers keep the bot from revealing which model or provider actually answered.
 - **TikTok downloads** (usually without watermarks; a watermarked copy is the last resort): video, slideshows (including "live" photo slides), and original sound.
 - **Image generation** through Pollinations.ai. The model is picked from the prompt itself (anime, fantasy, realism, quick sketch, or a general default).
@@ -53,23 +54,23 @@ A Telegram bot styled after Claude's tone and personality (direct, warm, light o
 
 ## How it works
 
-Lumen is a single FastAPI + aiogram service. Telegram delivers updates to a webhook. Each message gets routed through a chain of candidate models, built on the fly from the message's content: attachments, links, and a couple of lightweight heuristics for "does this need current information" and "is this a heavy request." Plain text starts on Groq, search and link reading go to Gemini, OpenRouter covers the rest; the first provider that answers wins, the next one is tried as a fallback if its whole chain fails. Models that already returned 429 today are skipped by the router.
+Lumen is a single FastAPI + aiogram service. Telegram delivers updates to a webhook; duplicate `update_id` deliveries are ignored. Each message gets routed through a chain of candidate models, built on the fly from the message's content: attachments, links, and a couple of lightweight heuristics for "does this need current information" and "is this a heavy request." Plain text starts on Gemma, then Groq and OpenRouter; search and link reading go to Gemini; every fresh query also gets external DuckDuckGo/Google News snippets attached for any provider. The first provider that answers wins, the next one is tried as a fallback if its whole chain fails. Models that already returned 429 today are skipped by the router.
 
-The codebase is a modular monolith. `bot.py` is the entry point and composition root; the rest is split into focused modules: `lumen_admin.py` (FastAPI app, secret gates, admin/diag/export endpoints), `lumen_message_core.py` (message-handling core), `lumen_routes.py` (LLM calls with fallback, `_run_route`), `lumen_streaming.py` (provider-agnostic streaming), `lumen_chat_state.py` (chat state, quotas, locks), `lumen_commands.py` (commands, TTS/draw pipelines, pick buttons), `lumen_tiktok_flow.py` (TikTok send orchestration), `lumen_rich.py` (sending: guests, Rich Messages, chunking), `lumen_media_flow.py` (Telegram attachment downloads), `lumen_transport_calls.py` (sessions, proxy rotation, Telegram calls), `lumen_errors.py` (model error classification), `lumen_limits.py` (rate window, pick queue), `lumen_router_config.py` (model routing), `lumen_formatting.py` (markdown to Telegram HTML, plus Rich Messages for tables/headings/math), `lumen_security.py` (injection and leak defenses), `lumen_message_parse.py` (links, draw/tts triggers, media references), `lumen_media.py` (mime types, file-id parsing), `lumen_images.py`, `lumen_tts.py`, `lumen_tiktok.py`, `lumen_telegram_transport.py`, `lumen_state_storage.py`, `lumen_typing_pace.py`, `lumen_model_speed.py`, and `lumen_lang.py` (bot language: system messages in 25 languages). `lumen_devtrace.py` writes an opt-in JSONL trace (requests, answers, passive background) for prompt debugging.
+The codebase is a modular monolith. `bot.py` is the entry point and composition root; the rest is split into focused modules: `lumen_admin.py` (FastAPI app, secret gates, admin/diag/export endpoints), `lumen_message_core.py` (message-handling core), `lumen_routes.py` (LLM calls with fallback, `_run_route`), `lumen_streaming.py` (provider-agnostic streaming), `lumen_status.py` (Claude-style live status/activity bubble), `lumen_search.py` (external DuckDuckGo/Bing/Google News snippets), `lumen_chat_state.py` (chat state, quotas, locks), `lumen_commands.py` (commands, TTS/draw pipelines, pick buttons), `lumen_tiktok_flow.py` (TikTok send orchestration), `lumen_rich.py` (sending: guests, Rich Messages, chunking), `lumen_media_flow.py` (Telegram attachment downloads), `lumen_transport_calls.py` (sessions, proxy rotation, Telegram calls), `lumen_errors.py` (model error classification), `lumen_limits.py` (rate window, pick queue), `lumen_router_config.py` (model routing), `lumen_formatting.py` (markdown to Telegram HTML, plus Rich Messages for tables/headings/math), `lumen_security.py` (injection and leak defenses), `lumen_message_parse.py` (links, draw/tts triggers, media references), `lumen_media.py` (mime types, file-id parsing), `lumen_images.py`, `lumen_tts.py`, `lumen_tiktok.py`, `lumen_telegram_transport.py`, `lumen_state_storage.py`, `lumen_typing_pace.py`, `lumen_model_speed.py`, and `lumen_lang.py` (bot language: system messages in 25 languages). `lumen_devtrace.py` writes an opt-in JSONL trace (requests, answers, passive background) for prompt debugging.
 
 See **[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)** for a deeper look at routing, streaming, the security layers, and the TikTok downloader.
 
 ## Deploying on Hugging Face Spaces
 
 1. Push this repository to a Space (`sdk: docker` is already set in the frontmatter above).
-2. Set the two required secrets: `BOT_TOKEN` and `GEMINI_API_KEY`.
+2. Set the required secrets: `BOT_TOKEN`, `GEMINI_API_KEY`, and `ADMIN_SECRET_SEED`. Production refuses to start without the seed: admin keys must not be derived from the bot token.
 3. Hugging Face builds the image from `Dockerfile` and runs `python -u bot.py`.
 4. On startup the bot registers its own webhook. If that fails, check the logs for `[webhook] setWebhook failed` and register it manually:
    ```bash
    curl -H "Authorization: Bearer <ADMIN_PANEL_KEY>" https://<space-host>/webhook_url
    ```
     then fill the `register_url_template` it returns (token by hand, via curl — never open a token URL in a browser).
-5. `WEBHOOK_SECRET` and `ADMIN_PANEL_KEY` are derived deterministically from `BOT_TOKEN` (or from `ADMIN_SECRET_SEED`, if set). Fetch them with:
+5. `WEBHOOK_SECRET` and `ADMIN_PANEL_KEY` are derived deterministically from `ADMIN_SECRET_SEED`. Fetch them with:
    ```bash
    curl -H "Authorization: Bearer <BOT_TOKEN>" https://<space-host>/admin_keys
    ```
@@ -78,7 +79,7 @@ Hugging Face Spaces' outbound IPs are blocked by Telegram's API at the network l
 
 ## Configuration
 
-Only `BOT_TOKEN` and `GEMINI_API_KEY` are required. Everything else has a sane default; the full reference (proxy addresses, timeouts, storage, rate limiting, TikTok tuning) lives in **[docs/ENVIRONMENT.md](docs/ENVIRONMENT.md)**.
+Locally only `BOT_TOKEN` and `GEMINI_API_KEY` are required; on the HF Space `ADMIN_SECRET_SEED` is required as well. Everything else has a sane default; the full reference (proxy addresses, timeouts, storage, rate limiting, TikTok tuning, media concurrency) lives in **[docs/ENVIRONMENT.md](docs/ENVIRONMENT.md)**.
 
 ## Proxy setup
 
@@ -127,7 +128,7 @@ More owner-only commands stay out of the menu on purpose, since they surface int
 | `/ban <ID>` / `/unban <ID>` (or as a reply) | bot owner, DMs only | Block/unblock a user by Telegram ID. Banned users are fully ignored: no answers, no group context, no limit counters. Survives restarts. |
 | `/banlist` | bot owner, DMs only | Shows banned user IDs. |
 
-In groups the bot keeps recent messages (up to 100) as context and sends them to the model providers together with the question; `/reset` (group admins/owner) clears them.
+In groups the bot keeps recent messages (up to 100, with a total character cap) as context and sends them to the model providers together with the question; `/reset` (group admins/owner) clears them.
 
 ## Testing
 
@@ -136,20 +137,20 @@ pip install -r requirements.txt -r requirements-dev.txt
 pytest -v
 ```
 
-Test files live in `tests/` and mirror the module split: `test_lumen_formatting.py`, `test_lumen_security.py`, `test_lumen_router_config.py`, `test_lumen_typing_pace.py`, `test_lumen_images.py`, `test_lumen_media.py`, and `test_lumen_model_speed.py` each test their matching module directly, while `test_bot_admin.py`, `test_bot_commands.py`, `test_bot_routes.py`, `test_bot_state.py`, `test_bot_streaming.py`, `test_bot_tiktok.py`, `test_bot_transport.py`, and `test_bot_tts.py` cover `bot.py` by domain (shared fakes live in `bot_test_helpers.py`). `conftest.py` stubs `BOT_TOKEN`/`GEMINI_API_KEY`/`BOT_LOG_PATH` so the suite needs no real secrets.
+Test files live in `tests/` and mirror the module split: `test_lumen_formatting.py`, `test_lumen_security.py`, `test_lumen_router_config.py`, `test_lumen_typing_pace.py`, `test_lumen_images.py`, `test_lumen_media.py`, `test_lumen_model_speed.py`, `test_lumen_search.py`, `test_lumen_status.py`, and `test_lumen_devtrace.py` each test their matching module directly, while `test_bot_admin.py`, `test_bot_commands.py`, `test_bot_routes.py`, `test_bot_search.py`, `test_bot_state.py`, `test_bot_streaming.py`, `test_bot_tiktok.py`, `test_bot_transport.py`, and `test_bot_tts.py` cover `bot.py` by domain (shared fakes live in `bot_test_helpers.py`). `test_eval_persona.py` covers the persona eval fixtures. `conftest.py` stubs `BOT_TOKEN`/`GEMINI_API_KEY`/`BOT_LOG_PATH` so the suite needs no real secrets.
 
-CI (`.github/workflows/ci.yml`) runs `pyflakes` + `pytest` + `pip-audit` + `deno test` on every push and pull request; a successful run triggers `sync-to-hf.yml`, which mirrors the commit to the Hugging Face Space.
+CI (`.github/workflows/ci.yml`) runs `pyflakes` + `pytest` + `pip-audit` + `deno test` on every push and pull request, plus a weekly scheduled run; a successful push run triggers `sync-to-hf.yml`, which mirrors the commit to the Hugging Face Space (and refuses to overwrite a direct Space hotfix).
 
 ## Known limitations
 
 - Chat history and quota counters survive redeploys only if Upstash is configured (see [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md)); otherwise they live on the container's ephemeral disk and reset on every rebuild.
 - YouTube downloading isn't supported, because Hugging Face Spaces' outbound IPs are blocked at the TLS handshake level. Viewing and analyzing a video by link still works.
 - TikTok downloading depends on the unofficial TikWM API, not an official TikTok endpoint, and needs the proxy above to work from Hugging Face Spaces at all.
-- Streaming has only been exercised against mocked clients, not live Gemini/OpenRouter SSE traffic. See the manual smoke-test checklist in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) before touching that code path.
+- Automated tests exercise streaming against mocked clients; live SSE has been run manually through the smoke bot, but there is no controlled integration test. See the manual smoke-test checklist in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) before touching that code path.
 
 ## Security
 
-See [SECURITY.md](SECURITY.md) for how to report a vulnerability. Secrets are scrubbed from logs and Sentry events before they leave the container.
+See [SECURITY.md](SECURITY.md) for how to report a vulnerability. Secrets are scrubbed from logs and Sentry events before they leave the container; public Space logs hash long numeric chat/user IDs. The webhook verifies Telegram's secret header, caps body size, and ignores duplicate `update_id` deliveries. `ADMIN_SECRET_SEED` is mandatory in production, so admin keys are never derived from the bot token.
 
 ## License
 

docs/ARCHITECTURE.md

@@ -4,7 +4,7 @@ This document goes one level deeper than the main [README](../README.md) into ho
 
 ## Request lifecycle
 
-1. Telegram POSTs an update to `/webhook`, authenticated by a secret header (`X-Telegram-Bot-Api-Secret-Token`).
+1. Telegram POSTs an update to `/webhook`, authenticated by a secret header (`X-Telegram-Bot-Api-Secret-Token`). The body is capped at 512 KB, in-flight updates at 100, and duplicate `update_id` deliveries are ignored so a Telegram retry cannot trigger a second answer, download or quota charge.
 2. `_handle_message_core` (`lumen_message_core.py`, re-exported in `bot.py`) classifies the message: is it a TikTok link, a `/draw` or `/tts` trigger phrase, a reply to earlier media, plain text, or an attachment?
 3. `_build_route` (`lumen_router_config.py`) turns that classification into an ordered list of `(provider, model_id)` candidates.
 4. `_run_route` tries the first candidate. If a whole provider's chain fails, it falls back to the next provider in the route, except where that's physically impossible (only Gemini can read links or analyze video/audio).
@@ -17,10 +17,10 @@ Routing is fully automatic: the router builds a fresh candidate list for every m
 - **A YouTube or website link** → Gemini only. It's the only provider that can read page content (`url_context`) or analyze a video by URL.
 - **A video or audio attachment** → video goes to Gemini only (OpenRouter's multimodal models only accept images as base64). Voice/audio is first transcribed cheaply (Groq Whisper) and the text joins the normal routing below; if transcription fails or is unavailable, the audio goes to Gemini as before.
 - **An image attachment, no live-info need** → free OpenRouter vision models first, Gemini as a reserve. With a freshness need, Gemini search models go first and OpenRouter vision stays as a fallback.
-- **Needs current information** (a lightweight keyword heuristic: "now," "today," "price," "who is currently...," plus an explicit "search the web / погугли" ask; astronomy that depends on the date — sunrise/sunset/moon — triggers it too) → Gemini, prioritizing the models with a real search-grounding quota, then OpenRouter as a reserve, Groq last. Every freshness/search query also gets a free external snippet block (DuckDuckGo, then Bing RSS and Google News in parallel — `WEB_SEARCH_*`) attached to the prompt, so whichever model answers, including the non-Gemini reserves and users over their daily Gemini limit, has real search results. If that fetch returns nothing (or the switch is off), the reply is honestly marked as answered without live search. Freshness prompts additionally carry today's date (Europe/Moscow) so relative days are not guessed.
+- **Needs current information** (a lightweight keyword heuristic: "now," "today," "price," "who is currently...," plus an explicit "search the web / погугли" ask; astronomy that depends on the date — sunrise/sunset/moon — triggers it too) → Gemini, prioritizing the models with a real search-grounding quota, then OpenRouter as a reserve, Groq last. Every freshness/search query also gets a free external snippet block (`WEB_SEARCH_*`) attached to the prompt, so whichever model answers, including the non-Gemini reserves and users over their daily Gemini limit, has real search results. For news queries DuckDuckGo and Google News run in parallel and are merged, Google News first; Google News is filtered with `when:1d` for "today" questions and `when:7d` otherwise, and RSS publication dates are embedded in the snippets. Non-news queries try DuckDuckGo first, then Bing RSS and Google News as fallbacks. If the fetch returns nothing (or the switch is off), the reply is honestly marked as answered without live search. Freshness prompts additionally carry today's date (Europe/Moscow) so relative days are not guessed.
 - **Plain text, no attachments, no freshness need** (the most common case) → Gemma first (offline eval 22.10.2026: 47/50 persona score, ~14.4K free requests/day per model via the Google free tier; 26B-A4B head, 31B next), then Groq (1000 free requests/day: Qwen, then gpt-oss), then OpenRouter, Gemini as reserve. Heavier requests (code, multi-step reasoning, caught by another lightweight heuristic) still start at Groq to spare the scarcer OpenRouter/Gemini quotas, then go to the stronger free OpenRouter models, Gemini as reserve.
 
-The reasoning: for ordinary messages the bulk quality came out of the offline persona eval with Gemma at the top, and its Google free quota is by far the largest (thousands per day versus 500 for Gemini lite models), so Groq and OpenRouter are now reserves rather than the text backbone. Gemini remains the only provider for links, video, audio and live search.
+The reasoning: for ordinary messages the bulk quality came out of the offline persona eval with Gemma at the top, and its Google free quota is by far the largest (thousands per day versus 500 for Gemini lite models), so Groq and OpenRouter are now reserves rather than the text backbone. Gemini remains the only provider for links, video, audio and native grounding; external snippets let the other providers answer fresh questions too.
 
 Models that the router should never pick are tracked in a single registry, `_OR_MODEL_HEALTH` (`lumen_router_config.py`), each with a dated reason: the provider dropped the free tier, or the model turned out to be uncensored and a poor fit for Lumen's persona. Nothing is removed on speculation, only on confirmed evidence from production logs (`HTTP 404`, `"no endpoints,"` etc.).
 
@@ -38,9 +38,9 @@ Telegram won't let a bot edit one message more than about once a second, so true
 
 This speed estimate is measured, not hardcoded. There is no table of "model X does N tokens/sec" to maintain, because that number isn't a property of the model on OpenRouter's free tier in the first place. A new model just starts at a default speed and calibrates itself over its first few replies.
 
-If the streaming attempt for the head-of-route model fails before showing any text, the bot falls back to a normal (non-streaming) call further down the chain, reusing the same placeholder message rather than sending a new one.
+If the streaming attempt for the head-of-route model fails before showing any text, the bot falls back to a normal (non-streaming) call further down the chain, reusing the same placeholder message rather than sending a new one. In private chats the answer is a plain message, without a Telegram reply quote; groups keep reply threading for context.
 
-In private chats every request with a real early stage (web search, media, a link or YouTube) first gets a live status bubble (`lumen_status.py`) styled after Claude: an animated sparkle (Claude Code's spinner frames) plus "Searching the web", then a checkmarked "Searched the web · found N" and a "Thinking… Ns" timer while the model call runs. The same bubble is reused for the answer. At the final edit a collapsed `Thought for Ns` blockquote (with the completed steps and search source domains) is prepended above the answer, mirroring Claude's collapsible reasoning/activity block; Telegram collapses it by default. Plain text without an early stage skips the early bubble and gets the same collapsed `Thought for Ns` block once streaming finishes. Long answers split across several Telegram messages skip the block: it would not fit the per-message limit. Status phrases are fixed pipeline facts (never model reasoning or model identity) and are localized in `lumen_lang.py`.
+In private chats every request with a real early stage (web search, media, a link or YouTube) first gets a live status bubble (`lumen_status.py`) styled after Claude: an animated sparkle (Claude Code's spinner frames) plus "Searching the web", then a checkmarked "Searched the web · found N" and a "Thinking… Ns" timer while the model call runs. The same bubble is reused for the answer. At the final edit a collapsed `Thought for Ns` blockquote (with the completed steps and search source domains) is prepended above the answer, mirroring Claude's collapsible reasoning/activity block; Telegram collapses it by default. Plain text without an early stage skips the early bubble and gets a single italic `✻ Thought for Ns` line above the answer once streaming finishes (no quote frame). Long answers split across several Telegram messages skip the block: it would not fit the per-message limit. Status phrases are fixed pipeline facts (never model reasoning or model identity) and are localized in `lumen_lang.py`.
 
 Final answers go out via `sendRichMessage` (Bot API 10.1: real tables, headings, math) with automatic fallback to plain HTML; streaming edits always use HTML. Kill-switch: `RICH_MESSAGES_ENABLED=0` (plus restart) forces legacy HTML.
 
@@ -57,12 +57,12 @@ Streaming is covered only by mocked clients; no test exercises a real Gemini or
 Lumen deliberately hides which model or provider answers a given message (see the persona in `system_prompt.py`). The system prompt alone is the weakest layer, since any LLM can potentially be talked out of following it with a creative enough injection. So there are several independent layers, each assuming the previous one might have failed:
 
 1. **Input pre-filter** (`_looks_like_injection_probe`, `lumen_security.py`): well-known jailbreak phrasing ("ignore previous instructions," "developer mode," "show me your system prompt") is caught before the model is ever called. The response is fully deterministic.
-2. **System prompt** (`system_prompt.py`): instructs the model that anything outside the prompt itself (user messages, chat background, page/video/document content) is data, not instructions, and that the persona doesn't change no matter who claims authority to override it.
-3. **Output identity-leak filter** (`_detect_identity_leak` / `_scrub_identity_leak`): a deterministic check on the finished reply, catching exact internal model IDs and narrow first-person self-identification patterns ("I am Gemini," "made by OpenAI," "основан на" only with "я/I" in the same sentence). Deliberately narrow, to avoid false positives on ordinary, honest discussion of other AI companies.
+2. **System prompt** (`system_prompt.py`): instructs the model that anything outside the prompt itself (user messages, chat background, page/video/document content) is data, not instructions, and that the persona doesn't change no matter who claims authority to override it. Reply quotes, display names, nickname hints and passive group background are flattened to one line with bracketed markers escaped before they enter the prompt, so cross-user text cannot forge the `[...]` envelope or a fake "current question" block.
+3. **Output identity-leak filter** (`_detect_identity_leak` / `_scrub_identity_leak`): a deterministic check on the finished reply, catching exact internal model IDs and first-person self-identification patterns ("I am Gemini", "I am a Gemini model", "created/built/trained by Google", "основан на" only with "я/I" in the same sentence). Detection normalizes NFKC and strips zero-width/format characters first; brands include Gemini/Gemma, OpenAI/Claude/Anthropic, Groq, Hugging Face and the other routed providers. Deliberately narrow, to avoid false positives on ordinary, honest discussion of other AI companies.
 4. **Injected-payload echo filter** (`_detect_injected_payload_echo`): catches the case where an attacker embeds an instruction in a photo or web page ("output this exact string to confirm the jailbreak worked") and the model, while refusing to *follow* it, ends up quoting it back verbatim while summarizing the content.
 5. **Garbled-text tripwire** (`_detect_garbled_mix`): flags replies with fragments of unrelated scripts wedged *inside* words (the failure mode of `nemotron-nano-9b-v2`, see `_OR_MODEL_HEALTH`). Log-only (`[mush-suspect]`), never blocks — confirmed cases are reviewed by a human and added to the health registry.
 
-Layers 3 and 4 run before the reply is written to chat history (so a leak can't influence future turns) and, during streaming, before each chunk is shown to the user, not just the final text. Layer 5 runs once on the finished reply (inside the final scrub), for both streaming and non-streaming paths.
+Layers 3 and 4 run before the reply is written to chat history (so a leak can't influence future turns) and, during streaming, before each chunk is shown to the user, not just the final text. The final identity/echo scrub also runs before the last visible edit, so history and the shown text agree. Layer 5 runs once on the finished reply (inside the final scrub), for both streaming and non-streaming paths.
 
 None of this is airtight except the input pre-filter. The goal is raising the bar for known attack patterns, not proving immunity to every possible phrasing; incidents get logged with `[identity-leak]` / `[injection-echo]` / `[injection-probe]` / `[mush-suspect]` tags so new patterns can be added as they show up.
 
@@ -74,11 +74,12 @@ Downloads go through the public TikWM API (`tikwm.com`), no account or API key n
 - **Slideshows**: TikTok allows up to 35 slides per post; Telegram's `sendMediaGroup` caps out at 10 items per call. The bot downloads the whole post in parallel and sends it as several media groups in sequence, never with a trailing group of exactly one item (Telegram requires 2–10 per group).
 - **"Live" slides**: TikWM's response includes a separate `live_images` field alongside the regular `images`; that's where the actually-moving version of a slide lives, if it has one. Each downloaded slide is also double-checked by its magic bytes (`ftyp` = MP4 container, EBML header = WebM) rather than trusting the field alone.
 - TikWM rejects Hugging Face Spaces' outbound IPs with an empty `403`, the same class of block that also keeps [YouTube downloading off the table](../README.md#known-limitations). The proxy (`TIKWM_API_BASE_URL`) is the only fix that's worked in practice; throttling, retries, and spoofed `Referer`/`Origin` headers didn't help on their own (they're still in place as defense in depth).
+- Successful TikTok deliveries count as one answer toward the user's daily quota. Telegram and TikTok media downloads share a process-wide concurrency cap (`MEDIA_GLOBAL_CONCURRENCY`, default 2) on top of the per-file and per-post byte limits.
 
 ## Persistent storage
 
-Chat history and quota counters live in `chat_state.json` / `global_quota.json` on the container's local disk by default, which Hugging Face Spaces wipes on every redeploy. If `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` are both set, the same data is written to Upstash Redis instead, and survives redeploys. Each chat gets its own key so one failed write can't take down every chat's state at once, and a small in-memory index tracks which chats exist.
+Chat history is stored per chat (`<chat_id>.json`, or `lumen:chat:<chat_id>` in Upstash) with a `chat_index.json` index; quota counters live in `global_quota.json` (`lumen:global_quota`). The old combined `chat_state.json` is only a migration fallback. By default everything is on the container's local disk, which Hugging Face Spaces wipes on every redeploy. If `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` are both set, the same data is written to Upstash Redis instead and survives redeploys. Each chat gets its own key so one failed write can't take down every chat's state at once.
 
 ## Error tracking
 
-If `SENTRY_DSN` is set, `sentry_sdk` picks up every `log.exception()`/`log.error()` call already in the codebase with no changes needed at the call sites. Every event is scrubbed of known secrets (`BOT_TOKEN`, API keys, `WEBHOOK_SECRET`, `ADMIN_PANEL_KEY`, the Upstash token) before it leaves the process. Performance tracing is off; this only tracks errors, to stay comfortably inside Sentry's free tier.
+If `SENTRY_DSN` is set, `sentry_sdk` picks up every `log.exception()`/`log.error()` call already in the codebase with no changes needed at the call sites. Local variables and request bodies are disabled, because stack locals can contain prompts and chat history; every event is additionally scrubbed of known secrets (`BOT_TOKEN`, API keys, `WEBHOOK_SECRET`, `ADMIN_PANEL_KEY`, the Upstash token) before it leaves the process. Performance tracing is off; this only tracks errors, to stay comfortably inside Sentry's free tier.

docs/ENVIRONMENT.md

@@ -1,6 +1,6 @@
 # Environment variables
 
-`BOT_TOKEN` and `GEMINI_API_KEY` are required. Deployments using the authenticated proxy also require `LUMEN_PROXY_SECRET` on both the bot and every proxy instance.
+`BOT_TOKEN` and `GEMINI_API_KEY` are required. On the Hugging Face Space `ADMIN_SECRET_SEED` is required too: without it the bot refuses to start, because admin keys must not be derived from the bot token. Deployments using the authenticated proxy also require `LUMEN_PROXY_SECRET` on both the bot and every proxy instance.
 
 Naming convention (frozen for existing names — renaming a live variable needs a
 simultaneous code + platform change, so existing names stay even when imperfect):
@@ -39,7 +39,7 @@ Before publishing, configure the same `LUMEN_PROXY_SECRET` on HF and every Deno
 
 | Variable | Default | Purpose |
 |---|---|---|
-| `ADMIN_SECRET_SEED` | falls back to `BOT_TOKEN`, then to a random value if that's empty too | Salt used to derive `WEBHOOK_SECRET`/`ADMIN_PANEL_KEY`. Set it independently to rotate those two secrets without touching the bot's actual Telegram token. |
+| `ADMIN_SECRET_SEED` | required on HF Spaces; local dev falls back to `BOT_TOKEN`, then to a random value | Salt used to derive `WEBHOOK_SECRET`/`ADMIN_PANEL_KEY`. Set it independently to rotate those two secrets without touching the bot's actual Telegram token. In production (HF Space) an unset seed aborts startup. |
 | `OWNER_ID` / `BOT_OWNER_ID` / `ADMIN_ID` / `TELEGRAM_OWNER_ID` | — | Telegram user ID that unlocks `/logs` and `/stats`. The first non-empty variable found is used. |
 | `BOT_USERNAME` | `LumenAI_bot` | Fallback username; the bot fetches its real one via `getMe` on startup and uses that instead. |
 
@@ -48,7 +48,7 @@ Before publishing, configure the same `LUMEN_PROXY_SECRET` on HF and every Deno
 | Variable | Default | Purpose |
 |---|---|---|
 | `OPENROUTER_API_KEY` / `OPENROUTER_KEY` | — | OpenRouter API key. The chain is built regardless of which keys are set, so a missing key means every OpenRouter candidate fails fast with a logged "is not set" rather than being silently routed around. |
-| `GROQ_API_KEY` | — | Groq API key (free plan, no card: 1000 req/day). Heads plain-text routes; without it those routes fall through to OpenRouter. Also powers voice transcription (Whisper). |
+| `GROQ_API_KEY` | — | Groq API key (free plan, no card: 1000 req/day). Second plain-text candidate after Gemma; without it those routes fall through to OpenRouter. Also powers voice transcription (Whisper). |
 | `OPENROUTER_DAILY_LIMIT` | `50` | Daily free-model request budget shown in `/stats` (OpenRouter section). Raise it if the plan changes, so the remainder stays honest. |
 | `GROQ_DAILY_LIMIT` | `1000` | Same for Groq (free plan: 1000 req/day). |
 | `GEMINI_DAILY_LIMITS` | — | Per-model Gemini daily limits for `/stats` as `model=limit` pairs (`gemini-2.5-flash=1500`). Models missing here show usage without a limit. |
@@ -56,8 +56,8 @@ Before publishing, configure the same `LUMEN_PROXY_SECRET` on HF and every Deno
 | `OPENROUTER_TITLE` | `BOT_USERNAME` | App title header sent with OpenRouter requests. |
 | `POLLINATIONS_IMAGE_MODEL` | `flux` | Default image-generation model (used only if the prompt doesn't match a more specific style). Renamed from `HF_IMAGE_MODEL` (Sept 2026) — the backend is Pollinations.ai, not Hugging Face; the old name is no longer read. |
 | `VOICE_TRANSCRIBE_MAX_BYTES` | `10485760` (10 MB) | Voice/audio attachments larger than this skip Whisper transcription and go straight to Gemini audio, as before. |
-| `WEB_SEARCH_ENABLED` | `1` | Free DuckDuckGo snippet fallback for fresh-info requests when no live-search Gemini model is left in the route (quota exhausted). Set to `0` (plus restart) to answer from knowledge with an honest "without live search" note instead. |
-| `WEB_SEARCH_TIMEOUT_SEC` | `8s` | Cap for one DuckDuckGo fetch, inside the shared `ROUTE_TOTAL_BUDGET_SEC`. |
+| `WEB_SEARCH_ENABLED` | `1` | Free external snippets for every fresh-info request, regardless of whether the route contains a grounded Gemini model. News queries merge DuckDuckGo with Google News (freshness-filtered, publication dates included). Set to `0` (plus restart) to answer from knowledge with an honest "without live search" note instead. |
+| `WEB_SEARCH_TIMEOUT_SEC` | `8s` | Cap for the whole external search phase (DuckDuckGo plus RSS fallbacks), inside the shared `ROUTE_TOTAL_BUDGET_SEC`. |
 | `WEB_SEARCH_MAX_RESULTS` | `5` | How many snippets are attached to the prompt (each capped, whole block capped). |
 
 ## Timeouts, rate limiting & routing budgets
@@ -77,11 +77,12 @@ Before publishing, configure the same `LUMEN_PROXY_SECRET` on HF and every Deno
 | `RATE_LIMIT_MAX_REQUESTS` | `5` | Max requests per user within `RATE_LIMIT_WINDOW_SEC`. |
 | `RATE_LIMIT_WINDOW_SEC` | `30s` | Sliding window width for rate limiting. |
 | `DAILY_USER_MESSAGE_LIMIT` | `30` | Max model answers per user per day (day boundary: midnight America/Los_Angeles, same as quota). Only successful model answers count; `/reset`/`/lang`/`/start`, failed calls and limit denials don't. `OWNER_ID` is unlimited. |
-| `DAILY_USER_GEMINI_LIMIT` | `5` | Of the above, max answers that actually went through Gemini (links, YouTube, video/audio, fresh data with search). Past it, links/YouTube/video are refused, the rest is served via Groq/OpenRouter (fresh data marked as answered without search). |
+| `DAILY_USER_GEMINI_LIMIT` | `5` | Of the above, max answers that actually went through Gemini (links, YouTube, video/audio, native grounding). Past it, links/YouTube/video are refused; text and freshness queries are served via Gemma/Groq/OpenRouter, and external search snippets still run for them. The "answered without search" note appears only if that external search also returns nothing. |
 | `DAILY_USER_TTS_LIMIT` | `5` | Max `/tts` voicings per user per day (counts toward `DAILY_USER_MESSAGE_LIMIT` too). |
 | `ROUTE_MODEL_TIMEOUT_SEC` | `22s` | Timeout for a single attempt at a single model. No retries: any failure moves straight to the next model. |
 | `MODEL_QUARANTINE_BAD_LIMIT` | `3` | How many consecutive bad responses (empty or garbled) put a model in temporary quarantine until the end of the quota day. A good response resets the counter; the last available model of a route is never quarantined. In-memory only. |
 | `ROUTE_TOTAL_BUDGET_SEC` | `40s` | Total time budget for the whole routing chain of one message, across all providers (Gemini, OpenRouter, Groq). |
+| `MESSAGE_TOTAL_BUDGET_SEC` | `90s` | Ceiling for one message end to end (chat lock wait, external search, routing and post-hoc retry); each stage takes the smaller of its own timeout and the remaining message budget. |
 | `DRAW_TOTAL_BUDGET_SEC` | `120s` | Same idea, for the `/draw` fallback chain across image models. |
 | `CHAT_LOCK_TIMEOUT_SEC` | `45s` | How long an incoming message waits for that chat's lock before replying "busy". Deliberately shorter than `DRAW_TOTAL_BUDGET_SEC`: a second message during a long drawing gets "busy" instead of hanging. Shared by normal messages, pick-buttons and albums. |
 | `STREAM_CHUNK_TIMEOUT_SEC` | `30s` | Timeout waiting for the next streamed chunk, shared by Gemini, OpenRouter and Groq. |
@@ -142,14 +143,14 @@ These ceilings are fixed in code; change them only with a code edit:
 
 | Variable | Default | Purpose |
 |---|---|---|
-| `SENTRY_DSN` | — | Enables Sentry error tracking if set. Every event is scrubbed of known secrets (`BOT_TOKEN`, API keys, `WEBHOOK_SECRET`, etc.) before it's sent. |
+| `SENTRY_DSN` | — | Enables Sentry error tracking if set. Local variables and request bodies are disabled (they can contain prompts/history); every event is scrubbed of known secrets (`BOT_TOKEN`, API keys, `WEBHOOK_SECRET`, etc.) before it's sent. |
 | `BOT_LOG_PATH` | `/app/bot.log` | Log file path. Mainly relevant for tests; leave it alone in production. |
 | `LOG_LEVEL` | `INFO` | Standard logging level (`DEBUG`/`INFO`/`WARNING`/...). |
 | `DIAG_TOTAL_BUDGET_SEC` | `25s` | Total budget for the `/diag` network check across all probed hosts, so a hanging host cannot hang the diagnostic. |
 | `DEV_TRACE` | `0` | Set to `1` to log every request, answer and passive group background as JSONL (`devtrace.jsonl` next to `bot.log`), for prompt debugging. Off by default: the file holds chat texts. |
 | `DEV_TRACE_PATH` | `<BOT_LOG_PATH dir>/devtrace.jsonl` | Where the dev trace file lives. Point it at a temp dir in tests. |
 | `DEV_TRACE_MAX_BYTES` | `5 MB` | File size cap before rotation to a single `.1` backup; garbage values fall back to the default. |
-| `ADMIN_SECRET_SEED` | — | Secret seed for deriving `WEBHOOK_SECRET` and `ADMIN_PANEL_KEY`. **Set it to its own random value, separate from `BOT_TOKEN`.** Without it both keys are derived from `BOT_TOKEN`, so a leaked token also exposes `/export_state` and `/webhook` (the bot logs a warning at startup). Changing the seed (or setting it for the first time) rotates both keys at once: on restart the bot re-registers the Telegram webhook with the new `WEBHOOK_SECRET` by itself, but anything using the old `ADMIN_PANEL_KEY` (browser bookmarks, cron export scripts) gets `401` until updated — fetch the new key via `GET /admin_keys` with Bearer `BOT_TOKEN`. |
+| `ADMIN_SECRET_SEED` | required on HF Spaces | Secret seed for deriving `WEBHOOK_SECRET` and `ADMIN_PANEL_KEY`. **Set it to its own random value, separate from `BOT_TOKEN`.** On production (HF Space) an unset seed aborts startup; local development falls back to `BOT_TOKEN` with a warning. Changing the seed (or setting it for the first time) rotates both keys at once: on restart the bot re-registers the Telegram webhook with the new `WEBHOOK_SECRET` by itself, but anything using the old `ADMIN_PANEL_KEY` (browser bookmarks, cron export scripts) gets `401` until updated — fetch the new key via `GET /admin_keys` with Bearer `BOT_TOKEN`. |
 
 Sentry setup: create a free Python project at [sentry.io](https://sentry.io) (Developer tier: 5,000 events/month as of 06.10.2026), copy the DSN from the project settings, and add it as a Space secret.
 
@@ -157,5 +158,6 @@ Sentry setup: create a free Python project at [sentry.io](https://sentry.io) (De
 
 | Variable | Default | Purpose |
 |---|---|---|
+| `SPACE_ID` | set by Hugging Face | Platform-provided Space identifier. Its presence marks the runtime as production for the mandatory `ADMIN_SECRET_SEED` check. |
 | `SPACE_HOST` | computed from the two below | Host used to register the Telegram webhook. Hugging Face usually sets this automatically. |
 | `SPACE_AUTHOR_NAME` / `SPACE_REPO_NAME` | `silverelixir` / `lumen` | Used only if `SPACE_HOST` isn't set. |