---
title: SnapQuest
emoji: โ๏ธ
colorFrom: yellow
colorTo: gray
sdk: gradio
sdk_version: 5.9.1
app_file: app.py
pinned: true
tags:
- thousand-token-wood
- off-brand
- tiny-titan
- best-demo
- best-agent
- openbmb
- minicpm
- game
- rpg
- voice
- photo-to-rpg
- modal
license: mit
---
---
## What SnapQuest Does
Upload 1โ3 photos of any real space. Each photo becomes one room of a dungeon. The last room always spawns a **boss** โ an entity formed from the most prominent object the vision model detects.
```
๐ธ Photo 1 โ Room 1: Entry Hall
๐ธ Photo 2 โ Room 2: Inner Chamber
๐ธ Photo 3 โ Room 3: BOSS LAIR โ
```
MiniCPM-V 4.6 (1.3B parameters) reads each image directly. It sees your specific objects. It builds a scene grounded in what it actually observes โ not a template.
| Real Object | Becomes |
|---|---|
| Red chair | Throne of the Forgotten Scholar |
| Black backpack | Wanderer's Cursed Pack |
| Stuffed bear | **โ The Bear Sentinel โ BOSS** |
| Desk lamp | Flickering Oracle |
| Curtain | Veil of Shadow |
---
## Five Classes, Five Worldviews
| Class | How They See Your Room |
|---|---|
| โ๏ธ Swordsman | Cover, threats, chokepoints |
| ๐น Archer | Vantage points, sightlines, escape routes |
| ๐ Healer | Vulnerability, what needs protecting |
| ๐ก๏ธ Rogue | Shadows, hiding spots, things worth stealing |
| ๐ฎ Mage | Arcane energy, omens, symbolic meaning |
---
## Game Loop
1. Upload 1โ3 photos โ pick class โ **Begin Quest**
2. MiniCPM-V reads each photo, builds rooms
3. Click choices or type custom actions (try "attack the chair")
4. Combat: attack enemies to clear rooms, take damage in return
5. Type **"Go deeper"** to advance to the next room
6. Final room: **BOSS FIGHT** โ the boss is built from your last photo's main object
7. Defeat the boss โ dungeon cleared, loot collected
---
## Architecture
```
Photos (1โ3)
โ
โผ
vision.py โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ Modal A10G GPU
Base64 encode + POST to Modal endpoint MiniCPM-V 4.6 (1.3B)
3x retry with backoff Returns: scene JSON per photo
โ
โผ
dungeon.py
build_rooms() โ 1โ3 rooms with difficulty scaling
generate_boss() โ boss from photo's main object
apply_combat() โ lightweight combat resolution
minimap_html() โ pixel minimap display
โ
โผ
engine_photo.py
HF Inference API โ Qwen2.5-3B-Instruct (free, serverless)
Rule-based fallback (always works if HF is slow)
Multi-room state management, history window
โ
โผ
ui_photo.py + voice.py
Gradio 5.x CRT interface Whisper STT + edge-tts TTS
Press Start 2P pixel font en-GB-RyanNeural voice
Boss banner + minimap + XP bar
```
---
## Tech Stack
| Layer | Technology | Notes |
|---|---|---|
| Vision | MiniCPM-V 4.6 (OpenBMB) | 1.3B params, reads real photos |
| GPU | Modal A10G | On-demand, scales to zero |
| DM Narration | Qwen2.5-3B-Instruct via HF Inference | Free serverless, no key needed |
| DM Fallback | Rule-based generator | Always playable even if API is slow |
| Voice In | OpenAI Whisper (local) | Speech โ text |
| Voice Out | edge-tts RyanNeural | Text โ speech |
| UI | Gradio 5.x + custom CSS | Press Start 2P font, CRT scanlines |
| Deployment | HuggingFace Spaces | Free tier |
---
## Run Locally
```bash
git clone https://github.com/ashish-doing/snapquest
cd snapquest
pip install -r requirements-local.txt
# Pull DM model locally (optional โ HF Inference is used on Space)
ollama pull qwen2.5vl:3b
# Set Modal endpoint (get from: modal deploy modal_app.py)
export SNAPQUEST_MODAL_ENDPOINT=https://ashish-kumar-doing--snapquest-minicpm-v-46-minicpmvservi-12daf4.modal.run
python app.py
```
### HF Space Secrets Required
```
SNAPQUEST_MODAL_ENDPOINT โ Modal GPU endpoint URL
```
---
## Prize Tracks Targeted
| Prize | Qualification |
|---|---|
| ๐ **Thousand Token Wood** ($4k) | Creative whimsical AI game โ photo โ dungeon |
| ๐ฌ **OpenBMB Best MiniCPM Build** ($2.5k) | MiniCPM-V 4.6 doing real multimodal vision |
| ๐จ **Off Brand Badge** ($1.5k) | Custom pixel UI, Press Start 2P font, boss banner, minimap |
| ๐ **Tiny Titan Badge** ($1.5k) | Both models โค4B: MiniCPM-V 1.3B + Qwen2.5 3B |
| ๐ฌ **Best Demo Badge** ($1k) | App + video + social post |
| ๐ค **Best Agent Badge** ($1k) | Multi-step: vision โ room build โ DM narration โ combat |
---
## Author
**Ashish Kumar** โ B.Tech ECE, IIIT Guwahati (Batch 2024)
[](https://github.com/ashish-doing)
[](https://huggingface.co/ashish-doing)
---
Built for **HuggingFace Build Small Hackathon 2026**
*MiniCPM-V 4.6 ยท Modal ยท Qwen2.5-3B ยท Whisper ยท edge-tts ยท Gradio*
**Small model. Real vision. Your room becomes the dungeon.**
