README.md

metadata

title: ChatCraft
emoji: 🎮
colorFrom: red
colorTo: yellow
sdk: docker
app_port: 7860
pinned: false
license: mit
storage: small
short_description: 1v1 RTS game controlled by voice via Mistral
tags:
  - game
  - voice
  - mistral
  - rts
  - hackathon
  - svelte
  - fastapi

ChatCraft is a 1v1 real-time strategy game (inspired by StarCraft's Terran race) entirely controlled by voice. Speak your commands — build barracks, train marines, attack the enemy — and Mistral AI interprets them into live game actions. No mouse clicks to play, just your microphone.

🎮 ChatCraft

Voice-Controlled Real-Time Strategy Game

Mistral × ElevenLabs Hackathon — Command your army. With your voice.

A 1v1 real-time strategy game entirely controlled by voice. Inspired by StarCraft's Terran race — playable on mobile with just a microphone.

---

🎯 Concept

"Build barracks, train 4 marines, attack the enemy base"

Speak. Command. Conquer.

The AI (Mistral) interprets your natural language orders and translates them into real-time game actions. Voice feedback from the game ("Construction complete", "Under attack!") is delivered instantly via the Web Speech API.

Touch / mouse is only used to navigate the map, inspect units or techs — never to play.

---

⚡ Tech Stack

Layer	Technology
🖥️ Frontend	SvelteKit + socket.io-client
⚙️ Backend	FastAPI + python-socketio (asyncio)
🧠 Command AI	Mistral API (mistral-large-latest)
🎙️ Voice → Text	Voxtral (voxtral-mini-latest)
🔊 Text → Voice	Web Speech API (browser, instant, free)
☁️ Hosting	Hugging Face Spaces (Docker)

---

🗣️ Voice Command Flow

🎤 [Player holds PTT]
        │
        ▼
  MediaRecorder (browser)
        │  audio blob
        ▼
  Socket.IO  ──voice_input──►  backend
                                  │
                                  ▼
                          🤖 Voxtral STT
                          "train 3 marines"
                                  │
                                  ▼
                      ✨ Mistral API  (system prompt = game state)
                      → { action: "train", unit: "marine", count: 3 }
                                  │
                                  ▼
                          engine.apply_command()
                                  │
                          ┌───────┴───────┐
                          ▼               ▼
                    game_state       feedback_text
                    broadcast        "Training 3 marines started"
                          │               │
                          ▼               ▼
                  ◄──game_update──  🔊 Web Speech API (browser)

---

🗺️ Game — Simplified Terran Race

💎 Resources

Resource	Source	Usage
🔵 Minerals	Patches collected by SCV	Everything
🟢 Vespene Gas	Geysers collected by SCV	Advanced units, upgrades
👥 Supply	Supply Depots + Command Center	Population cap

🏗️ Buildings

Building	Prerequisite	Role
🏛️ Command Center	—	Starting building, produces SCV
📦 Supply Depot	—	+8 supply
🪖 Barracks	—	Marine, Medic
🔧 Engineering Bay	Barracks	Infantry upgrades
🏭 Factory	Barracks	Goliath, Tank
⚔️ Armory	Factory	Vehicle upgrades
🚀 Starport	Factory	Wraith

⚔️ Units

Unit	Cost	Prerequisite	Role
🔨 SCV	50m	CC	Gathering, construction
🪖 Marine	50m	Barracks	Basic infantry, anti-air
💊 Medic	50m / 25g	Barracks + Academy*	Heals infantry
🤖 Goliath	100m / 50g	Factory	Anti-air + ground
💣 Siege Tank	150m / 100g	Factory + Machine Shop*	Ground DPS, siege mode
✈️ Wraith	150m / 100g	Starport	Air, cloaking

*simplified: add-ons replaced by simple building prerequisites

🎤 Voice Command Examples

"Build a supply depot near the command center"
"Train four marines"
"Send all marines to attack top-left"
"Put the tank in siege mode"
"Patrol the wraiths around my base"
"How much gas do I have?"
"Scout with an SCV toward the center"

---

🎮 Core Game Rules

• 🖥️ HUD layout: resources top-left, production top-right, timer center; minimap bottom-left; control groups bottom-center and right; voice/text commands on the right.
• 🌫️ Fog of war: visibility based on the player's units and buildings (vision radii per type); explored but not visible areas semi-transparent, never-seen areas in black.
• 🗺️ Minimap: 40×40 map representation with fog, main map viewport, click to recenter.
• 🎯 Control groups: 3 groups; add/remove units or buildings, or create empty; unified representation by icons (buildings, technologies, units).

---

🚀 Local Setup

# 1. Environment variables
cp .env.example .env
# Fill in MISTRAL_API_KEY

# 2. Backend
cd backend
pip install -r requirements.txt
uvicorn main:app --reload --port 8000

# 3. Frontend
cd frontend
npm install
npm run dev

🔑 Environment Variables

Variable	Description
MISTRAL_API_KEY	Mistral API key (Voxtral STT + mistral-large chat)
SECRET_KEY	Key to sign lobby sessions

---

🏗️ Project Architecture

hackaton/
├── Dockerfile                  # HF Spaces — exposes port 7860
├── docker-compose.yml          # Local dev
│
├── backend/
│   ├── main.py                 # FastAPI + Socket.IO entry point
│   ├── lobby/manager.py        # Room creation / joining, matchmaking
│   ├── game/
│   │   ├── engine.py           # Game loop (tick ~250ms)
│   │   ├── state.py            # GameState: resources, units, buildings
│   │   ├── units.py            # Marine / Medic / Goliath / Tank / Wraith / SCV
│   │   ├── buildings.py        # Command Center / Barracks / Factory…
│   │   ├── tech_tree.py        # Build prerequisites
│   │   └── map.py              # Grid + A* pathfinding
│   └── voice/
│       ├── stt.py              # Voxtral STT: audio → text
│       └── command_parser.py   # Mistral: text + game state → actions
│
└── frontend/
    └── src/
        ├── routes/
        │   ├── +page.svelte        # Lobby
        │   └── game/+page.svelte   # Main game view
        └── lib/
            ├── socket.ts           # socket.io-client singleton
            ├── voice.ts            # PTT mic capture → backend
            └── components/
                ├── Map.svelte          # Scrollable/zoomable SVG map
                ├── ResourceBar.svelte  # Minerals / Gas / Supply
                └── VoiceButton.svelte  # Push-to-talk button

---

🧠 Design Notes

• python-socketio with asyncio is chosen for room management — it natively handles rooms, namespaces and reconnection out of the box.
• The game loop runs server-side (tick every 250ms). Clients receive a full state snapshot each tick — no client-side reconciliation.
• Mistral receives a summary of the player's current state (resources, active buildings, supply) as system context for consistent command interpretation.
• PTT (push-to-talk) avoids false positives on mobile and reduces latency by limiting audio chunk size.