README.md

metadata

title: InkWitch
emoji: 🪄
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.16.0
python_version: '3.12'
app_file: app.py
pinned: false
tags:
  - track:wood
  - sponsor:openbmb
  - achievement:offgrid
  - achievement:offbrand

🪄 InkWitch

Build Small Hackathon submission. An Adventure in Thousand Token Wood Track

Your research grant is running dry, and the loan shark has sent you an ultimatum for the debt. He repossessed your lab and turned it into a witchcraft store.

To keep your roof and clear the debt, you serve the customers who wander in — each wants an object with a few specific qualities, a key with wings, a mug with a cat face, etc.

Sketch glyphs and transmute them to fulfill the customers' orders, so you can receive payment and build your reputation. But watch out, every stroke drains your mana. Reinvest wisely, sketch sparingly, and DON'T GO BANKRUPT!

Model Pipeline

1. Glyph appraisal — MiniCPM-V-4.6 looks at your raw glyph and scores it for recognition (is it the right object at all?) plus a confidence on each requested quality.
2. Transmutation — if the object is recognized, your glyph is fed as a scribble control image into an SDXL-Flash + Scribble-ControlNet pipeline, which renders a finished illustration that follows your lines.
3. Re-check — the VLM then re-judges the transmuted image, but only on the qualities your glyph failed. Your gold and reputation reflect this final artwork — so a rough glyph that the transmutation fleshes out faithfully can win back reputation it missed.

In other words, you're judged not just on what you drew, but on what your glyph becomes. The AI is load-bearing twice over: once as the judge, once as the artist.

Models (all running on the Space — Off the Grid)

Role	Model	Params
Vision judge	openbmb/MiniCPM-V-4.6	~1.3B
Image generator	sd-community/sdxl-flash	~2.6B
Scribble control	xinsir/controlnet-scribble-sdxl-1.0	~1.25B
VAE	madebyollin/sdxl-vae-fp16-fix	~84M

The image pipeline is adapted from linoyts/scribble-sdxl-flash. No external inference APIs are called — both heavy models are eagerly loaded at Space startup and every appraisal + transmutation happens inside one @spaces.GPU acquisition.

Tech Notes

• Custom frontend on gr.Server — / serves a hand-written index.html + app.js + style.css (the drawing canvas, shop, and night phase), driven through the official @gradio/client so ZeroGPU per-user quota is attributed correctly. app.launch(ssr_mode=False) keeps Gradio's SSR from shadowing it.
• Per-user state is keyed on gr.Request.session_hash, so simultaneous players get independent games.
• One GPU call per Transmute — glyph appraisal, transmutation, and re-check are fused into a single @spaces.GPU function.
• Hidden test mode — click Reset 5 times in a row to flip on a test session: unlimited gold/mana, the full request pool, and the image-gen tuning panel. Refresh the page to return to a normal run.

Team Member

• gizzleon

🎥 Demo video

https://youtu.be/3R139SneUzA?si=zGK5RLvHhFSDOs44

📣 Social post

https://www.reddit.com/r/aigamedev/comments/1u55h2u/webbased_ai_doodle_game/

Run it locally

Only the image generation stage (SDXL-Flash + ControlNet) requires a CUDA GPU — it does not run on CPU/MPS. The MiniCPM judge runs fine without one, so on a machine with no CUDA GPU, disable image-gen with WITCH_WOOD_GEN_ENABLED=0.

# Set up the project with uv (https://github.com/astral-sh/uv)
uv sync

# Full pipeline — real MiniCPM judge + image generation (image-gen needs a CUDA GPU):
python app.py

# No CUDA GPU? Disable image-gen only — the real MiniCPM judge still runs
# (on CPU/MPS) and the SDXL pipe is never loaded:
WITCH_WOOD_GEN_ENABLED=0 python app.py

# Fully model-free — skips ALL model calls (mock judge, no image generation):
WITCH_WOOD_MOCK=1 python app.py

Runs on Hugging Face Spaces ZeroGPU in production.