---
title: InkWitch
emoji: 🪄
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.16.0
python_version: '3.12'
app_file: app.py
pinned: false
tags:
  - track:wood
  - sponsor:openbmb
  - achievement:offgrid
  - achievement:offbrand
---

# 🪄 InkWitch

> **Build Small Hackathon submission**. *An Adventure in Thousand Token Wood* Track

Your research grant is running dry, and the loan shark has sent you an ultimatum for the debt. 
He repossessed your lab and turned it into a witchcraft store. 

To keep your roof and clear the debt, you serve the customers who wander in — each wants an object with a few specific
qualities, *a key with wings*, *a mug with a cat face*, etc.

Sketch glyphs and transmute them to fulfill the customers' orders, so you can receive payment and build your reputation. But watch out, every stroke drains your mana. Reinvest wisely, sketch sparingly, and DON'T GO BANKRUPT!

## Model Pipeline
1. **Glyph appraisal** — `MiniCPM-V-4.6` looks at your raw glyph and scores it
   for *recognition* (is it the right object at all?) plus a confidence on each
   requested *quality*.
2. **Transmutation** — if the object is recognized, your glyph is fed as a
   scribble control image into an `SDXL-Flash + Scribble-ControlNet` pipeline,
   which renders a finished illustration that follows your lines.
3. **Re-check** — the VLM then re-judges the *transmuted image*, but only on the
   qualities your glyph **failed**.
   Your gold and reputation reflect this final artwork — so a rough glyph that
   the transmutation fleshes out faithfully can win back reputation it missed.

In other words, you're judged not just on what you drew, but on what your glyph
*becomes*. The AI is load-bearing twice over: once as the judge, once as the artist.

## Models (all running on the Space — Off the Grid)

| Role | Model | Params |
|---|---|---|
| Vision judge | [`openbmb/MiniCPM-V-4.6`](https://huggingface.co/openbmb/MiniCPM-V-4.6) | ~1.3B |
| Image generator | [`sd-community/sdxl-flash`](https://huggingface.co/sd-community/sdxl-flash) | ~2.6B |
| Scribble control | [`xinsir/controlnet-scribble-sdxl-1.0`](https://huggingface.co/xinsir/controlnet-scribble-sdxl-1.0) | ~1.25B |
| VAE | [`madebyollin/sdxl-vae-fp16-fix`](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix) | ~84M |

The image pipeline is adapted from
[`linoyts/scribble-sdxl-flash`](https://huggingface.co/spaces/linoyts/scribble-sdxl-flash).
No external inference APIs are called — both heavy models are eagerly loaded at
Space startup and every appraisal + transmutation happens inside one `@spaces.GPU`
acquisition.

## Tech Notes

- **Custom frontend on `gr.Server`** — `/` serves a hand-written `index.html` +
  `app.js` + `style.css` (the drawing canvas, shop, and night phase), driven
  through the official `@gradio/client` so ZeroGPU per-user quota is attributed
  correctly. `app.launch(ssr_mode=False)` keeps Gradio's SSR from shadowing it.
- **Per-user state** is keyed on `gr.Request.session_hash`, so simultaneous
  players get independent games.
- **One GPU call per Transmute** — glyph appraisal, transmutation, and re-check
  are fused into a single `@spaces.GPU` function.
- **Hidden test mode** — click **Reset** 5 times in a row to flip on a test session: 
  unlimited gold/mana, the full request pool, and the image-gen tuning panel. 
  Refresh the page to return to a normal run.

## Team Member
- [gizzleon](https://huggingface.co/gizzleon)

## 🎥 Demo video

https://youtu.be/3R139SneUzA?si=zGK5RLvHhFSDOs44

## 📣 Social post

https://www.reddit.com/r/aigamedev/comments/1u55h2u/webbased_ai_doodle_game/

## Run it locally

Only the **image generation** stage (SDXL-Flash + ControlNet) requires a CUDA
GPU — it does not run on CPU/MPS. The MiniCPM judge runs fine without one, so on
a machine with no CUDA GPU, disable image-gen with `WITCH_WOOD_GEN_ENABLED=0`.

```bash
# Set up the project with uv (https://github.com/astral-sh/uv)
uv sync

# Full pipeline — real MiniCPM judge + image generation (image-gen needs a CUDA GPU):
python app.py

# No CUDA GPU? Disable image-gen only — the real MiniCPM judge still runs
# (on CPU/MPS) and the SDXL pipe is never loaded:
WITCH_WOOD_GEN_ENABLED=0 python app.py

# Fully model-free — skips ALL model calls (mock judge, no image generation):
WITCH_WOOD_MOCK=1 python app.py
```

Runs on Hugging Face Spaces **ZeroGPU** in production.