---
title: The Shape Of Words
emoji: 🔥
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.18.0
python_version: '3.12'
app_file: app.py
pinned: false
short_description: Tell a story. Watch its feelings take shape.
license: mit
tags:
  - track:wood
  - sponsor:openbmb
  - sponsor:modal
  - achievement:offbrand
  - achievement:llama
  - achievement:fieldnotes
---

# The Shape of Words

A collaborative storytelling toy: tell a story one beat at a time, and the model
turns the *feeling* of each beat into an abstract shape. At the end, hidden
colors bloom through the shapes one by one, the model adds its own companion
shapes, and everything composes into a single arrangement you can rearrange,
sonify, save, or turn into a painting.

- [`Live Space`](https://huggingface.co/spaces/build-small-hackathon/the_shape_of_words)
- [`Demo Video`](https://youtu.be/I0XPUqYRvYE)
- [`Social Media Post`](https://www.linkedin.com/posts/resakemal_the-shape-of-words-field-notes-activity-7472374372591153152-Kipg)
- **Field notes (deep dive):** [`Hugging Face Article`](https://huggingface.co/blog/resakemal/the-shape-of-words) or [`field_notes.md`](field_notes.md)

Built on **gradio.Server** (FastAPI + Gradio's API engine, gradio 6.x) so the
custom HTML/JS frontend gets queuing, streaming, and Hugging Face Spaces hosting.

---

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│ Browser — static/index.html (custom HTML/JS, no build step) │
│  • renderer · mappings · layout · sound · share card · UI   │
│  • all rendering is client-side (JS port of engine/)        │
│  • calls the backend for MODEL JUDGMENT only                │
└───────────────┬─────────────────────────────────────────────┘
                │  /gradio_api/call/<name>   (queued, SSE)
┌───────────────▼─────────────────────────────────────────────┐
│ app.py — gradio.Server                                      │
│  @app.api  judge_beat · judge_beat_segmented ·              │
│            continue_story · reveal · title_story · paint    │
│  @app.get "/"  serves the frontend                          │
└───────┬──────────────────────────────────────┬──────────────┘
        │ model/backend.py                     │ model/painter.py
        │  STORY_SHAPES_BACKEND                │  STORY_SHAPES_PAINT_BACKEND
        ▼                                      ▼
  llamacpp  GGUF in-process (llama.cpp)   modal       Modal endpoint (FLUX.2 Klein)
  modal_llm Modal endpoint (transformers) flux_local  in-process diffusers
```

The **model only judges affect** (valence / arousal / dominance + a few flags +
a voice line). Everything visual — geometry, color (OKLCH), layout, scoring — is
deterministic code (in `engine/`, mirrored client-side for rendering), grounded
in perception research (see the spec's references).

---

## Run locally (llama.cpp + MiniCPM4.1-8B)

The default backend runs the LLM in-process via `llama-cpp-python` — no daemon,
no separate model server.

```bash
pip install -r requirements.txt
python app.py            # serves http://localhost:7860
```

On first run it pulls a quantized GGUF (default `openbmb/MiniCPM4.1-8B-GGUF`,
Q4_K_M, ~5 GB). With a GPU it offloads all layers automatically; **CPU-only?**
set `STORY_SHAPES_LLAMACPP_GPU_LAYERS=0`. Then open http://localhost:7860.

Config via env vars (all optional):
| var | default | meaning |
|-----|---------|---------|
| `STORY_SHAPES_BACKEND` | `llamacpp` | `llamacpp` (in-process) or `modal_llm` (Modal endpoint) |
| `STORY_SHAPES_LLAMACPP_REPO` | `openbmb/MiniCPM4.1-8B-GGUF` | GGUF Hub repo |
| `STORY_SHAPES_LLAMACPP_FILE` | `*Q4_K_M.gguf` | GGUF filename glob |
| `STORY_SHAPES_LLAMACPP_GPU_LAYERS` | `-1` | layers on GPU (`-1` all, `0` CPU-only) |
| `PORT` | `7860` | server port |

> The frontend falls back to a built-in keyword **stub** if the backend is
> unreachable, so `static/index.html` also opens standalone for quick UI testing.

---

## Deploy to a Hugging Face Space

The README YAML frontmatter above is the HF Space config (`sdk: gradio`,
`sdk_version: 6.18.0`, `app_file: app.py`). The gradio version is pinned there
because `gr.Server` requires gradio 6.x. Push this repo to a Space and set the
env vars below.

**Recommended setup — both models on one 24 GB GPU (e.g. a dedicated L4):**

| var | value | notes |
|-----|-------|-------|
| `STORY_SHAPES_BACKEND` | `llamacpp` | MiniCPM4.1-8B as a GGUF via llama.cpp (~5 GB at Q4_K_M) |
| `STORY_SHAPES_PAINT_BACKEND` | `flux_local` | FLUX.2 Klein 4B in-process with CPU offload |

A quantized LLM (~5 GB) + cpu-offloaded FLUX (~8–12 GB peak) fit comfortably
together on 24 GB. A **dedicated** GPU Space is also not ZeroGPU, which avoids
ZeroGPU's per-request quota attribution.

**Alternative — offload the heavy work to Modal** (Space then needs no GPU):

| var | value | notes |
|-----|-------|-------|
| `STORY_SHAPES_BACKEND` | `modal_llm` | LLM on a Modal GPU (`modal deploy modal_llm.py`) |
| `STORY_SHAPES_LLM_MODAL_URL` | *(from `modal deploy`)* | the printed `modal_llm` URL |
| `STORY_SHAPES_PAINT_BACKEND` | `modal` | painter on a Modal GPU (`modal deploy modal_painter.py`) |
| `STORY_SHAPES_PAINT_MODAL_URL` | *(from `modal deploy`)* | the printed `modal_painter` URL |

**DISCLAIMER: I was attempting the Off the Grid challenge (local models only) but I could not test the model deployment on ZeroGPU due to consistently encountering 429 Too Many Requests error (more details in the "deployment story" in the Architecture section in [`field_notes.md`](field_notes.md)). For stable/expected results on the Space I'm routing both models to Modal (shape judgment & painter; this will increase initial inference time due to cold start); I apologize for the inconvenience. I have attached a local_run_logs.txt for proof that the shape judgment model runs locally.**

> **Why not run MiniCPM4.1 in-process under plain `transformers`?** gradio 6
> forces `transformers ≥ 5.0`, and MiniCPM4.1's `trust_remote_code` modeling
> code is incompatible with transformers 5.x (it loads with a shim but crashes
> during generation). That's exactly why the LLM runs via **llama.cpp** (no
> transformers at all) or on **Modal** with transformers pinned to 4.x.

See `requirements.txt` for the llama.cpp install (a prebuilt CUDA wheel, with a
one-comment CPU-fallback toggle) and the diffusers-from-source line FLUX needs.

---

## Layout

```
app.py                 gradio.Server: API endpoints + serves frontend
requirements.txt
engine/                deterministic core (also mirrored in the frontend JS)
  mappings.py          affect -> geometry, affect -> color (OKLCH, research-grounded)
  renderer.py          geometry -> shape points
  scorer.py            zone-based puzzle scoring (+ band + hint)
model/
  backend.py           LLM abstraction: llamacpp / modal_llm
  painter.py           painting abstraction: modal / flux_local
modal_llm.py           Modal GPU endpoint for the LLM (deploy once)
modal_painter.py       Modal GPU endpoint for FLUX.2 Klein (deploy once)
prompts/               system prompts + JSON schemas / GBNF grammars per call
static/
  index.html           the entire frontend (custom HTML/JS)
docs/
  make_spec_figures.py renders the spec/field-notes illustrations from the engine
```

---

## Features

- ✅ Exploration mode: full loop, hidden-color reveal, AI companion shapes,
  narrative composition, drag / rotate / layers / hide / save.
- ✅ **Per-segment essence**: shape morphs through each phrase's felt character.
- ✅ **OKLCH color**: perceptually-uniform lightness (Python + JS parity).
- ✅ **Reveal sound** (Web Audio): pentatonic chimes, shape-born tick, atmospheric
  pad, replay — togglable in Settings.
- ✅ **Share card** (canvas + painting): model-named title, popup, PNG export.
- ✅ **Origin tooltip**: hover/tap a shape or layer row to see its source beat.
- ✅ **Painting step (FLUX.2 Klein img2img)**: "Paint this"; free style field +
  preset styles + 🎲 Surprise me; randomized seed; `modal` or `flux_local`.
- ✅ **LLM backends**: `llamacpp` (GGUF, Llama Champion) and `modal_llm`. Both
  produce schema-constrained JSON.
- ✅ **Felt-quality eval** (`eval_felt_quality.py`): A/B models on a fixed set.
- ✅ **HF Space deployment** (gradio 6.18, `gr.Server`).