Spaces:
Sleeping
Sleeping
| title: Sidechat | |
| emoji: 💬 | |
| colorFrom: yellow | |
| colorTo: purple | |
| sdk: gradio | |
| sdk_version: 6.18.0 | |
| app_file: app.py | |
| pinned: false | |
| license: apache-2.0 | |
| short_description: Completely normal text assistant, with talking on the side | |
| tags: | |
| - track:wood | |
| - sponsor:openbmb | |
| - achievement:offgrid | |
| - achievement:sharing | |
| - achievement:fieldnotes | |
| # Side chat | |
| A Gradio port of the browser steganacrostics app. A completely normal text | |
| assistant — except every line of the answer secretly starts with the next | |
| letter of a hidden **secret** word (an acrostic). It does this with | |
| grammar-constrained decoding over a small local model (`openbmb/MiniCPM5-1B` by | |
| default; set `SIDECHAT_MODEL=LiquidAI/LFM2.5-350M` for the smaller, faster | |
| original), running on **CPU** via PyTorch `transformers`. | |
| What's ported from the JavaScript original (`../../src/`): | |
| - **Grammar engine** (`grammar.py`) — a tiny NFA that pins each line to its | |
| forced first letter, with optional ` * ` bullets and a max line length. | |
| - **Constrained generation** (`logits.py` + `masking.py`) — a `LogitsProcessor` | |
| that masks every token that would break the acrostic; EOS only at an accept | |
| state. A state-keyed cache makes the per-step vocab scan cheap. | |
| - **List-vs-prose classifier** (`classifier.py`) — an optimized prompt, | |
| grammar-constrained to `list.` / `story.`, that auto-picks the render mode. | |
| The prompt is tuned per model: failure modes are model-specific, so | |
| `eval_classifier.py` (50 list + 50 prose prompts) and `sweep_minicpm.py` | |
| re-optimize it for whatever model is in use. | |
| - **Local-crossing search** (`crossing_search.py`) — the "extra attention at | |
| the constraint": generate each prose line greedily, then choose where to | |
| break it so a short window straddling the crossing (last *k* tokens + forced | |
| letter + next *j* tokens) reads best. Plus stealth lowercase casing and a | |
| minimum line length. | |
| Run locally: | |
| ``` | |
| pip install -r requirements.txt | |
| python app.py | |
| ``` | |
| Then open the printed URL, type a prompt, set a secret in ⚙️ Settings, and click | |
| Generate. The list-vs-prose classifier runs automatically on each Generate (turn | |
| it off in ⚙️ Settings to set the render mode by hand, or use 🔎 Detect to preview | |
| it). Because everything runs on CPU, generation takes seconds (more for the | |
| larger model); the crossing search trades extra time for smoother prose. | |
| The model is downloaded from the Hugging Face Hub on first run. Custom logits | |
| processing requires the model to run in-process, so this app does not use the | |
| remote Inference API. | |