Spaces:
Sleeping
Sleeping
| title: ChatCraft | |
| emoji: 🎮 | |
| colorFrom: red | |
| colorTo: yellow | |
| sdk: docker | |
| app_port: 7860 | |
| pinned: false | |
| license: mit | |
| storage: small | |
| short_description: 1v1 RTS game controlled by voice via Mistral | |
| tags: | |
| - game | |
| - voice | |
| - mistral | |
| - rts | |
| - hackathon | |
| - svelte | |
| - fastapi | |
| ChatCraft is a 1v1 real-time strategy game (inspired by StarCraft's Terran race) entirely controlled by voice. Speak your commands — build barracks, train marines, attack the enemy — and Mistral AI interprets them into live game actions. No mouse clicks to play, just your microphone. | |
| <div align="center"> | |
| # 🎮 ChatCraft | |
| ### Voice-Controlled Real-Time Strategy Game | |
| [](https://mistral.ai) | |
| [](https://elevenlabs.io) | |
| [](https://huggingface.co/spaces) | |
| [](https://kit.svelte.dev) | |
| [](https://fastapi.tiangolo.com) | |
| > **Mistral × ElevenLabs Hackathon** — Command your army. With your voice. | |
| *A 1v1 real-time strategy game entirely controlled by voice. Inspired by StarCraft's Terran race — playable on mobile with just a microphone.* | |
| </div> | |
| --- | |
| ## 🎯 Concept | |
| > *"Build barracks, train 4 marines, attack the enemy base"* | |
| Speak. Command. Conquer. | |
| The AI (**Mistral**) interprets your natural language orders and translates them into real-time game actions. Voice feedback from the game (*"Construction complete", "Under attack!"*) is delivered instantly via the **Web Speech API**. | |
| Touch / mouse is only used to navigate the map, inspect units or techs — **never to play**. | |
| --- | |
| ## ⚡ Tech Stack | |
| | Layer | Technology | | |
| |:------|:-----------| | |
| | 🖥️ Frontend | SvelteKit + socket.io-client | | |
| | ⚙️ Backend | FastAPI + python-socketio (asyncio) | | |
| | 🧠 Command AI | Mistral API (`mistral-large-latest`) | | |
| | 🎙️ Voice → Text | Voxtral (`voxtral-mini-latest`) | | |
| | 🔊 Text → Voice | Web Speech API (browser, instant, free) | | |
| | ☁️ Hosting | Hugging Face Spaces (Docker) | | |
| --- | |
| ## 🗣️ Voice Command Flow | |
| ``` | |
| 🎤 [Player holds PTT] | |
| │ | |
| ▼ | |
| MediaRecorder (browser) | |
| │ audio blob | |
| ▼ | |
| Socket.IO ──voice_input──► backend | |
| │ | |
| ▼ | |
| 🤖 Voxtral STT | |
| "train 3 marines" | |
| │ | |
| ▼ | |
| ✨ Mistral API (system prompt = game state) | |
| → { action: "train", unit: "marine", count: 3 } | |
| │ | |
| ▼ | |
| engine.apply_command() | |
| │ | |
| ┌───────┴───────┐ | |
| ▼ ▼ | |
| game_state feedback_text | |
| broadcast "Training 3 marines started" | |
| │ │ | |
| ▼ ▼ | |
| ◄──game_update── 🔊 Web Speech API (browser) | |
| ``` | |
| --- | |
| ## 🗺️ Game — Simplified Terran Race | |
| ### 💎 Resources | |
| | Resource | Source | Usage | | |
| |:---------|:-------|:------| | |
| | 🔵 Minerals | Patches collected by SCV | Everything | | |
| | 🟢 Vespene Gas | Geysers collected by SCV | Advanced units, upgrades | | |
| | 👥 Supply | Supply Depots + Command Center | Population cap | | |
| ### 🏗️ Buildings | |
| | Building | Prerequisite | Role | | |
| |:---------|:-------------|:-----| | |
| | 🏛️ Command Center | — | Starting building, produces SCV | | |
| | 📦 Supply Depot | — | +8 supply | | |
| | 🪖 Barracks | — | Marine, Medic | | |
| | 🔧 Engineering Bay | Barracks | Infantry upgrades | | |
| | 🏭 Factory | Barracks | Goliath, Tank | | |
| | ⚔️ Armory | Factory | Vehicle upgrades | | |
| | 🚀 Starport | Factory | Wraith | | |
| ### ⚔️ Units | |
| | Unit | Cost | Prerequisite | Role | | |
| |:-----|:-----|:-------------|:-----| | |
| | 🔨 SCV | 50m | CC | Gathering, construction | | |
| | 🪖 Marine | 50m | Barracks | Basic infantry, anti-air | | |
| | 💊 Medic | 50m / 25g | Barracks + Academy* | Heals infantry | | |
| | 🤖 Goliath | 100m / 50g | Factory | Anti-air + ground | | |
| | 💣 Siege Tank | 150m / 100g | Factory + Machine Shop* | Ground DPS, siege mode | | |
| | ✈️ Wraith | 150m / 100g | Starport | Air, cloaking | | |
| *\*simplified: add-ons replaced by simple building prerequisites* | |
| ### 🎤 Voice Command Examples | |
| ``` | |
| "Build a supply depot near the command center" | |
| "Train four marines" | |
| "Send all marines to attack top-left" | |
| "Put the tank in siege mode" | |
| "Patrol the wraiths around my base" | |
| "How much gas do I have?" | |
| "Scout with an SCV toward the center" | |
| ``` | |
| --- | |
| ## 🎮 Core Game Rules | |
| - **🖥️ HUD layout**: resources top-left, production top-right, timer center; minimap bottom-left; control groups bottom-center and right; voice/text commands on the right. | |
| - **🌫️ Fog of war**: visibility based on the player's units and buildings (vision radii per type); explored but not visible areas semi-transparent, never-seen areas in black. | |
| - **🗺️ Minimap**: 40×40 map representation with fog, main map viewport, click to recenter. | |
| - **🎯 Control groups**: 3 groups; add/remove units or buildings, or create empty; unified representation by icons (buildings, technologies, units). | |
| --- | |
| ## 🚀 Local Setup | |
| ```bash | |
| # 1. Environment variables | |
| cp .env.example .env | |
| # Fill in MISTRAL_API_KEY | |
| # 2. Backend | |
| cd backend | |
| pip install -r requirements.txt | |
| uvicorn main:app --reload --port 8000 | |
| # 3. Frontend | |
| cd frontend | |
| npm install | |
| npm run dev | |
| ``` | |
| ### 🔑 Environment Variables | |
| | Variable | Description | | |
| |:---------|:------------| | |
| | `MISTRAL_API_KEY` | Mistral API key (Voxtral STT + mistral-large chat) | | |
| | `SECRET_KEY` | Key to sign lobby sessions | | |
| --- | |
| ## 🏗️ Project Architecture | |
| ``` | |
| hackaton/ | |
| ├── Dockerfile # HF Spaces — exposes port 7860 | |
| ├── docker-compose.yml # Local dev | |
| │ | |
| ├── backend/ | |
| │ ├── main.py # FastAPI + Socket.IO entry point | |
| │ ├── lobby/manager.py # Room creation / joining, matchmaking | |
| │ ├── game/ | |
| │ │ ├── engine.py # Game loop (tick ~250ms) | |
| │ │ ├── state.py # GameState: resources, units, buildings | |
| │ │ ├── units.py # Marine / Medic / Goliath / Tank / Wraith / SCV | |
| │ │ ├── buildings.py # Command Center / Barracks / Factory… | |
| │ │ ├── tech_tree.py # Build prerequisites | |
| │ │ └── map.py # Grid + A* pathfinding | |
| │ └── voice/ | |
| │ ├── stt.py # Voxtral STT: audio → text | |
| │ └── command_parser.py # Mistral: text + game state → actions | |
| │ | |
| └── frontend/ | |
| └── src/ | |
| ├── routes/ | |
| │ ├── +page.svelte # Lobby | |
| │ └── game/+page.svelte # Main game view | |
| └── lib/ | |
| ├── socket.ts # socket.io-client singleton | |
| ├── voice.ts # PTT mic capture → backend | |
| └── components/ | |
| ├── Map.svelte # Scrollable/zoomable SVG map | |
| ├── ResourceBar.svelte # Minerals / Gas / Supply | |
| └── VoiceButton.svelte # Push-to-talk button | |
| ``` | |
| --- | |
| ## 🧠 Design Notes | |
| - **python-socketio** with asyncio is chosen for room management — it natively handles rooms, namespaces and reconnection out of the box. | |
| - The game loop runs **server-side** (tick every 250ms). Clients receive a full state snapshot each tick — no client-side reconciliation. | |
| - Mistral receives a **summary of the player's current state** (resources, active buildings, supply) as system context for consistent command interpretation. | |
| - **PTT (push-to-talk)** avoids false positives on mobile and reduces latency by limiting audio chunk size. | |