README.md

metadata

title: SnapQuest
emoji: ⚔️
colorFrom: yellow
colorTo: gray
sdk: gradio
sdk_version: 5.9.1
app_file: app.py
pinned: true
tags:
  - thousand-token-wood
  - off-brand
  - tiny-titan
  - best-demo
  - best-agent
  - openbmb
  - minicpm
  - game
  - rpg
  - voice
  - photo-to-rpg
  - modal
license: mit

Build Small Hackathon 2026 — Track: Thousand Token Wood 🍄 Upload any real photo. MiniCPM-V 4.6 reads it. Your room becomes a dungeon you can actually play.

Demo Video: YouTube → (link added after recording) Social Post: LinkedIn → (link added after posting)

---

What SnapQuest Does

Upload 1–3 photos of any real space. Each photo becomes one room of a dungeon. The last room always spawns a boss — an entity formed from the most prominent object the vision model detects.

📸 Photo 1  →  Room 1: Entry Hall
📸 Photo 2  →  Room 2: Inner Chamber  
📸 Photo 3  →  Room 3: BOSS LAIR ☠

MiniCPM-V 4.6 (1.3B parameters) reads each image directly. It sees your specific objects. It builds a scene grounded in what it actually observes — not a template.

Real Object	Becomes
Red chair	Throne of the Forgotten Scholar
Black backpack	Wanderer's Cursed Pack
Stuffed bear	☠ The Bear Sentinel — BOSS
Desk lamp	Flickering Oracle
Curtain	Veil of Shadow

---

Five Classes, Five Worldviews

Class	How They See Your Room
⚔️ Swordsman	Cover, threats, chokepoints
🏹 Archer	Vantage points, sightlines, escape routes
💚 Healer	Vulnerability, what needs protecting
🗡️ Rogue	Shadows, hiding spots, things worth stealing
🔮 Mage	Arcane energy, omens, symbolic meaning

---

Game Loop

1. Upload 1–3 photos → pick class → Begin Quest
2. MiniCPM-V reads each photo, builds rooms
3. Click choices or type custom actions (try "attack the chair")
4. Combat: attack enemies to clear rooms, take damage in return
5. Type "Go deeper" to advance to the next room
6. Final room: BOSS FIGHT — the boss is built from your last photo's main object
7. Defeat the boss → dungeon cleared, loot collected

---

Architecture

Photos (1–3)
    │
    ▼
vision.py  ─────────────────────────────────→  Modal A10G GPU
  Base64 encode + POST to Modal endpoint          MiniCPM-V 4.6 (1.3B)
  3x retry with backoff                          Returns: scene JSON per photo
    │
    ▼
dungeon.py
  build_rooms()  → 1–3 rooms with difficulty scaling
  generate_boss() → boss from photo's main object
  apply_combat()  → lightweight combat resolution
  minimap_html()  → pixel minimap display
    │
    ▼
engine_photo.py
  HF Inference API → Qwen2.5-3B-Instruct (free, serverless)
  Rule-based fallback (always works if HF is slow)
  Multi-room state management, history window
    │
    ▼
ui_photo.py  +  voice.py
  Gradio 5.x CRT interface        Whisper STT + edge-tts TTS
  Press Start 2P pixel font       en-GB-RyanNeural voice
  Boss banner + minimap + XP bar

---

Tech Stack

Layer	Technology	Notes
Vision	MiniCPM-V 4.6 (OpenBMB)	1.3B params, reads real photos
GPU	Modal A10G	On-demand, scales to zero
DM Narration	Qwen2.5-3B-Instruct via HF Inference	Free serverless, no key needed
DM Fallback	Rule-based generator	Always playable even if API is slow
Voice In	OpenAI Whisper (local)	Speech → text
Voice Out	edge-tts RyanNeural	Text → speech
UI	Gradio 5.x + custom CSS	Press Start 2P font, CRT scanlines
Deployment	HuggingFace Spaces	Free tier

---

Run Locally

git clone https://github.com/ashish-doing/snapquest
cd snapquest
pip install -r requirements-local.txt

# Pull DM model locally (optional — HF Inference is used on Space)
ollama pull qwen2.5vl:3b

# Set Modal endpoint (get from: modal deploy modal_app.py)
export SNAPQUEST_MODAL_ENDPOINT=https://ashish-kumar-doing--snapquest-minicpm-v-46-minicpmvservi-12daf4.modal.run

python app.py

HF Space Secrets Required

SNAPQUEST_MODAL_ENDPOINT  — Modal GPU endpoint URL

---

Prize Tracks Targeted

Prize	Qualification
🍄 Thousand Token Wood ($4k)	Creative whimsical AI game — photo → dungeon
🔬 OpenBMB Best MiniCPM Build ($2.5k)	MiniCPM-V 4.6 doing real multimodal vision
🎨 Off Brand Badge ($1.5k)	Custom pixel UI, Press Start 2P font, boss banner, minimap
🐜 Tiny Titan Badge ($1.5k)	Both models ≤4B: MiniCPM-V 1.3B + Qwen2.5 3B
🎬 Best Demo Badge ($1k)	App + video + social post
🤖 Best Agent Badge ($1k)	Multi-step: vision → room build → DM narration → combat

---

Author

Ashish Kumar — B.Tech ECE, IIIT Guwahati (Batch 2024)

---

Built for HuggingFace Build Small Hackathon 2026

MiniCPM-V 4.6 · Modal · Qwen2.5-3B · Whisper · edge-tts · Gradio

Small model. Real vision. Your room becomes the dungeon.