---
title: Sidechat
emoji: 💬
colorFrom: yellow
colorTo: purple
sdk: gradio
sdk_version: 6.18.0
app_file: app.py
pinned: false
license: apache-2.0
short_description: Completely normal text assistant, with talking on the side
tags:
  - track:wood
  - sponsor:openbmb
  - achievement:offgrid
  - achievement:sharing
  - achievement:fieldnotes
---

# Side chat

A Gradio port of the browser steganacrostics app. A completely normal text
assistant — except every line of the answer secretly starts with the next
letter of a hidden **secret** word (an acrostic). It does this with
grammar-constrained decoding over a small local model (`openbmb/MiniCPM5-1B` by
default; set `SIDECHAT_MODEL=LiquidAI/LFM2.5-350M` for the smaller, faster
original), running on **CPU** via PyTorch `transformers`.

What's ported from the JavaScript original (`../../src/`):

- **Grammar engine** (`grammar.py`) — a tiny NFA that pins each line to its
  forced first letter, with optional ` * ` bullets and a max line length.
- **Constrained generation** (`logits.py` + `masking.py`) — a `LogitsProcessor`
  that masks every token that would break the acrostic; EOS only at an accept
  state. A state-keyed cache makes the per-step vocab scan cheap.
- **List-vs-prose classifier** (`classifier.py`) — an optimized prompt,
  grammar-constrained to `list.` / `story.`, that auto-picks the render mode.
  The prompt is tuned per model: failure modes are model-specific, so
  `eval_classifier.py` (50 list + 50 prose prompts) and `sweep_minicpm.py`
  re-optimize it for whatever model is in use.
- **Local-crossing search** (`crossing_search.py`) — the "extra attention at
  the constraint": generate each prose line greedily, then choose where to
  break it so a short window straddling the crossing (last *k* tokens + forced
  letter + next *j* tokens) reads best. Plus stealth lowercase casing and a
  minimum line length.

Run locally:

```
pip install -r requirements.txt
python app.py
```

Then open the printed URL, type a prompt, set a secret in ⚙️ Settings, and click
Generate. The list-vs-prose classifier runs automatically on each Generate (turn
it off in ⚙️ Settings to set the render mode by hand, or use 🔎 Detect to preview
it). Because everything runs on CPU, generation takes seconds (more for the
larger model); the crossing search trades extra time for smoother prose.

The model is downloaded from the Hugging Face Hub on first run. Custom logits
processing requires the model to run in-process, so this app does not use the
remote Inference API.