README.md

metadata

title: Sidechat
emoji: 💬
colorFrom: yellow
colorTo: purple
sdk: gradio
sdk_version: 6.18.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: Completely normal text assistant, with talking on the side
tags:
  - track:wood
  - sponsor:openbmb
  - achievement:offgrid
  - achievement:sharing
  - achievement:fieldnotes

Side chat

A Gradio port of the browser steganacrostics app. A completely normal text assistant — except every line of the answer secretly starts with the next letter of a hidden secret word (an acrostic). It does this with grammar-constrained decoding over a small local model (openbmb/MiniCPM5-1B by default; set SIDECHAT_MODEL=LiquidAI/LFM2.5-350M for the smaller, faster original), running on CPU via PyTorch transformers.

What's ported from the JavaScript original (../../src/):

• Grammar engine (grammar.py) — a tiny NFA that pins each line to its forced first letter, with optional * bullets and a max line length.
• Constrained generation (logits.py + masking.py) — a LogitsProcessor that masks every token that would break the acrostic; EOS only at an accept state. A state-keyed cache makes the per-step vocab scan cheap.
• List-vs-prose classifier (classifier.py) — an optimized prompt, grammar-constrained to list. / story., that auto-picks the render mode. The prompt is tuned per model: failure modes are model-specific, so eval_classifier.py (50 list + 50 prose prompts) and sweep_minicpm.py re-optimize it for whatever model is in use.
• Local-crossing search (crossing_search.py) — the "extra attention at the constraint": generate each prose line greedily, then choose where to break it so a short window straddling the crossing (last k tokens + forced letter + next j tokens) reads best. Plus stealth lowercase casing and a minimum line length.

Run locally:

pip install -r requirements.txt
python app.py

Then open the printed URL, type a prompt, set a secret in ⚙️ Settings, and click Generate. The list-vs-prose classifier runs automatically on each Generate (turn it off in ⚙️ Settings to set the render mode by hand, or use 🔎 Detect to preview it). Because everything runs on CPU, generation takes seconds (more for the larger model); the crossing search trades extra time for smoother prose.

The model is downloaded from the Hugging Face Hub on first run. Custom logits processing requires the model to run in-process, so this app does not use the remote Inference API.