# BUILD PROMPT — paste everything below this line into Codex / OpenCode
═══════════════════════════════════════════════════════════════════════════════

You are a senior Python + Gradio engineer. Build a Hugging Face Space called **"Third Eye"**: a
fully voice-driven accessibility app for blind / low-vision users. The user points a webcam at
something (menu, medicine label, sign, scene), speaks a question, and hears the answer back in
their language. **Zero typing required** on the happy path.

Work in the current project folder. Build **incrementally** and **verify at every checkpoint**.
Do not write the whole app at once. Do not invent APIs. Follow these rules exactly.

───────────────────────────────────────────────────────────────────────────────
## ABSOLUTE RULES (do not break these)
1. **Verify before coding inference.** Model load/call APIs differ by version. For every model,
   open its Hugging Face model card and confirm the exact model ID, load call, and inference call
   BEFORE writing that stage. If a tool to fetch docs is available (e.g. context7), use it. If a
   model ID does not resolve, **STOP and report it** — never silently substitute another model.
2. **Sponsor models ONLY** (table below). No OpenAI/Whisper/Google/etc. anywhere.
3. **Build in phases. Stop at each CHECKPOINT and confirm it passes before continuing.**
4. **Never show a raw traceback to the user.** Every stage is wrapped; failures become
   `gr.Warning("friendly message")` and a graceful fallback.
5. **MUST-HAVE before SHOULD-HAVE before NICE-TO-HAVE.** A working minimal app beats a broken
   complete one. If you run low on time, ship the vertical slice.
6. When unsure about an API after reading the card, write the smallest possible test script and run
   it before wiring that stage into the UI.

───────────────────────────────────────────────────────────────────────────────
## MODELS (only these — verify each card before use)
| Role | Model ID | Params | Sponsor |
|---|---|---|---|
| Vision + OCR (PRIMARY) | `openbmb/MiniCPM-V-2_0` | 2.8B | OpenBMB |
| Vision + OCR (FALLBACK only if quality unacceptable) | `openbmb/MiniCPM-V-4_5` | 8B | OpenBMB |
| Speech-to-text | `CohereLabs/cohere-transcribe-03-2026` | 2B | Cohere |
| Text-to-speech | `openbmb/VoxCPM2` | 2B | OpenBMB |

Primary param budget = 2.8B (≤ 4B → qualifies for Tiny Titan). If you must use the 8B fallback,
write in the README: "fallback used, Tiny Titan badge forfeited." Never swap silently.

───────────────────────────────────────────────────────────────────────────────
## TECH STACK
- Gradio 5.x (`gr.Blocks`) on a Hugging Face Space (`sdk: gradio`).
- Modal serverless GPU (A10G) runs vision + TTS + STT.
- Python 3.11. No other cloud APIs.

───────────────────────────────────────────────────────────────────────────────
## TARGET FILE TREE
```
app.py              # Gradio UI + orchestration
modal_backend.py    # Modal app: describe_scene(), speak(), transcribe_audio()
cohere_stt.py       # Cohere Transcribe wrapper (imported by modal_backend)
utils.py            # image<->bytes, bytes<->wav, safe_call wrapper
requirements.txt
.env.example        # MODAL_TOKEN_ID, MODAL_TOKEN_SECRET, HF_TOKEN
README.md           # HF frontmatter + story + edge section
BLOG.md             # Field Notes draft
DEMO_SCRIPT.md      # 45s shot list
assets/
  custom.css        # "Iris" design system
  sample_menu.jpg   sample_label.jpg   sample_sign.jpg   (use 3 royalty-free / your own photos)
```

═══════════════════════════════════════════════════════════════════════════════
# PHASE 0 — VERIFY REALITY (do this first, write NO inference code yet)
For each model (`MiniCPM-V-2_0`, `VoxCPM2`, `cohere-transcribe-03-2026`):
- Confirm the model ID resolves on Hugging Face.
- Read the card's usage example. Record the EXACT: import, `from_pretrained` args
  (`trust_remote_code`, dtype, etc.), and the inference call signature.
- Note especially: MiniCPM-V's `model.chat(...)` signature varies — some versions take
  `image=<PIL>, msgs=[{"role":"user","content":<str>}]`; others take
  `image=None, msgs=[{"role":"user","content":[<PIL>, <str>]}]`. Use whatever THIS card shows.
- For VoxCPM2: find the real synthesis call (it may need a reference voice / a `generate` method,
  not `model.synthesize`). For Cohere Transcribe: confirm whether it loads via `transformers`
  `pipeline("automatic-speech-recognition", ...)` or needs a custom call.

**CHECKPOINT 0:** Output a short table of the verified API for each model (load call + infer call).
If anything can't be verified, list it explicitly and propose the smallest fix. Then continue.

═══════════════════════════════════════════════════════════════════════════════
# PHASE 1 — SCAFFOLD (UI renders with fake data)
Create all files. Use STUB functions that return canned data so the UI runs with no GPU.

`requirements.txt`:
```
gradio>=5.0
modal
pillow
soundfile
numpy
```
(Add `transformers`, `torch`, `accelerate`, `sentencepiece`, `timm` to the **Modal image**, not the
Space requirements — the Space does not run the models locally.)

`utils.py` — implement:
```python
import io, base64, tempfile, numpy as np
from PIL import Image

def image_to_bytes(image) -> bytes:
    if isinstance(image, np.ndarray):
        image = Image.fromarray(image)
    buf = io.BytesIO(); image.convert("RGB").save(buf, format="JPEG"); return buf.getvalue()

def bytes_to_wav(audio_bytes: bytes) -> str:
    f = tempfile.NamedTemporaryFile(delete=False, suffix=".wav")
    f.write(audio_bytes); f.close(); return f.name

def safe_call(fn, *args, fallback=None, warn="Something went wrong.", **kwargs):
    import gradio as gr
    try:
        return fn(*args, **kwargs)
    except Exception as e:
        gr.Warning(f"{warn} ({type(e).__name__})")
        return fallback
```

`app.py` — build `gr.Blocks(css=open("assets/custom.css").read())` with:
- A header with the **Iris orb** (a `gr.HTML` div, class `iris idle`) + an ARIA live status line.
- A language `gr.Dropdown` (English, Hindi, German, Tamil, Telugu, Kannada), default English.
- Three `gr.Tab`s: **Describe**, **Ask**, **Read Text**. Each has a `gr.Image(sources=["webcam","upload"])`,
  Ask also has `gr.Audio(sources=["microphone"])`, plus a large primary button, a `gr.Audio` output
  (set `autoplay=True`), and a large-font `gr.Textbox` output for the transcript.
- The 3 sample images wired as `gr.Examples` so judges can test with no webcam.
- For now, button click calls a STUB `run_pipeline(...)` that returns a placeholder wav path + text.

**CHECKPOINT 1:** `python app.py` launches locally; UI loads; clicking a button shows placeholder
text and the Iris orb is visible. No GPU involved yet.

═══════════════════════════════════════════════════════════════════════════════
# PHASE 2 — MODAL VISION (real description)
`modal_backend.py`: create the Modal app + GPU image, implement `describe_scene` using the
**verified** MiniCPM-V API from Phase 0.
```python
import modal
app = modal.App("third-eye-backend")
vision_image = modal.Image.debian_slim().pip_install(
    "transformers>=4.40","torch","pillow","accelerate","sentencepiece","timm","soundfile")

@app.function(gpu="A10G", image=vision_image, timeout=180)
def describe_scene(image_bytes: bytes, question: str, lang: str = "en") -> str:
    import io, torch
    from PIL import Image
    from transformers import AutoModel, AutoTokenizer
    model_id = "openbmb/MiniCPM-V-2_0"
    model = AutoModel.from_pretrained(model_id, trust_remote_code=True,
              torch_dtype=torch.float16).cuda().eval()
    tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
    image = Image.open(io.BytesIO(image_bytes)).convert("RGB")
    prompt = question.strip() or "Describe everything you see in detail."
    # >>> USE THE EXACT .chat() SIGNATURE YOU VERIFIED IN PHASE 0 <<<
    return model.chat(image=image, msgs=[{"role":"user","content":prompt}], tokenizer=tok)
```
Deploy: `modal deploy modal_backend.py`. Then write `test_vision.py` that reads `sample_menu.jpg`,
calls `describe_scene.remote(...)`, prints the answer.

**CHECKPOINT 2:** Real, sensible text comes back for the menu image. If quality is unusable, switch
to `MiniCPM-V-4_5` and note the forfeit in README (do not do this lightly).

═══════════════════════════════════════════════════════════════════════════════
# PHASE 3 — MODAL TTS (real speech)
Add `speak(text, lang)` to `modal_backend.py` using the **verified** VoxCPM2 API. Return WAV bytes.
Write `test_tts.py` that synthesizes "Hello, this is Third Eye." and saves `out.wav`.

**CHECKPOINT 3:** `out.wav` plays intelligible speech. (If multilingual needs a lang/voice arg, wire
`lang` through now.)

═══════════════════════════════════════════════════════════════════════════════
# PHASE 4 — WIRE "DESCRIBE" END-TO-END  ← MINIMUM VALID SUBMISSION
Replace the stub `run_pipeline` in `app.py`:
```python
def run_pipeline(image, audio_path, mode, lang):
    if image is None:
        gr.Warning("No image captured. Point the camera and try again.")
        return None, "No image captured.", ""
    img_bytes = image_to_bytes(image)
    if mode == "Ask" and audio_path:
        question = safe_call(transcribe_audio.remote, audio_path,
                             warn="Couldn't hear you — type your question instead.", fallback="")
    elif mode == "Read Text":
        question = "Read all text visible in this image, word by word, exactly as written."
    else:
        question = "Describe everything in this image in detail for a blind user."
    answer = safe_call(describe_scene.remote, img_bytes, question, lang,
                       warn="Vision model is waking up — try once more.", fallback="")
    if not answer:
        return None, "Could not analyze the image.", question
    audio_bytes = safe_call(speak.remote, answer, lang, warn="Voice unavailable — showing text.",
                            fallback=None)
    audio_out = bytes_to_wav(audio_bytes) if audio_bytes else None
    return audio_out, answer, question
```
Show `gr.Progress` with "Loading AI models (first run: ~30s)…" around the first heavy call.

**CHECKPOINT 4:** In the running Space/app, pick `sample_menu.jpg` in the Describe tab → audio
auto-plays a description + the transcript shows. THIS IS THE MINIMUM VALID SUBMISSION. Commit here.

═══════════════════════════════════════════════════════════════════════════════
# PHASE 5 — STT + "ASK" (zero-typing loop)
Implement `transcribe_audio(audio_path)` in `modal_backend.py` (delegates to `cohere_stt.py`) using
the verified Cohere Transcribe API. Wire the Ask tab: mic → transcribe → describe → speak.

**CHECKPOINT 5:** Record a spoken question about the sample image → hear a spoken answer. No typing.

═══════════════════════════════════════════════════════════════════════════════
# PHASE 6 — "READ TEXT" + LANGUAGE + CSS POLISH
- Read Text tab uses the fixed OCR prompt (already in run_pipeline).
- Confirm the language dropdown changes TTS output language (test English + Hindi minimum).
- Build the real **Iris** `assets/custom.css` (see DESIGN SPEC below). Drive orb state from `app.py`
  by updating the orb HTML's class (idle / listening / seeing / thinking / speaking) at each stage.

**CHECKPOINT 6:** All three tabs work; Hindi TTS works; UI matches the Iris spec.

═══════════════════════════════════════════════════════════════════════════════
# PHASE 7 — HARDENING
- Cold-start progress shown on first call.
- Mic failure → reveal a `gr.Textbox` typed-question fallback (never block).
- TTS failure → large-font text output only, with a `gr.Warning`.
- Every stage wrapped in `safe_call`; no traceback ever reaches the user.
- Confirm 3 examples load and run with no webcam.

**CHECKPOINT 7:** Manually break the mic and break TTS — the app degrades gracefully, never crashes.

═══════════════════════════════════════════════════════════════════════════════
# PHASE 8 — SUBMISSION ASSETS
`README.md` — start with this frontmatter VERBATIM:
```yaml
---
title: Third Eye
emoji: 👁️
colorFrom: indigo
colorTo: blue
sdk: gradio
sdk_version: "5.0"
app_file: app.py
pinned: false
tags:
  - hackathon
  - build-small
  - backyard-ai
  - accessibility
  - blind
  - openbmb/MiniCPM-V-2_0
  - openbmb/VoxCPM2
  - cohere/cohere-transcribe-03-2026
  - tiny-titan
  - off-brand
---
```
Then write: what it is, who it's for, how to use, models+sizes table (call out 2.8B Tiny Titan),
architecture paragraph, **on-device/edge section** (honest claim + roadmap: these models quantize to
int4 GGUF and can run offline on a phone via llama.cpp — framed as roadmap, not a shipped phone
build), accessibility & Iris design, run-it-yourself (env vars + `modal deploy` + Space secrets:
MODAL_TOKEN_ID / MODAL_TOKEN_SECRET / HF_TOKEN), credits (OpenBMB, Cohere, Modal, HF).

`BLOG.md` — a Field Notes draft: "What VLM quality really feels like at 2.8B" (what worked, where
MiniCPM-V-2 struggled vs the 8B fallback, OCR accuracy notes). `DEMO_SCRIPT.md` — the 45–60s shot
list (eye opens → blindfolded menu read aloud in Hindi → label/sign cuts → tagline).

**CHECKPOINT 8:** Space builds clean, loads on a cold visit, all assets present.

═══════════════════════════════════════════════════════════════════════════════
# PHASE 9 — NICE-TO-HAVE (only if time remains, in this order)
1. Bounding-box "Zoom & Read" tab via `gr.ImageEditor`: user draws a rectangle, the crop is sent to
   MiniCPM-V with "Read the text in this image exactly as written."
2. Cache model weights on a `modal.Volume` to cut cold-start time.
3. A small GGUF int4 on-device proof + a benchmark table in README (params, int4 size, target device).

═══════════════════════════════════════════════════════════════════════════════
# DESIGN SPEC — "Iris" custom CSS (Off-Brand track)
Accessibility constraints rendered as a futuristic aesthetic. Pure CSS so it's reliable to build.

- **Background** `#06070A` + faint radial vignette; optional subtle grain.
- **Accent gradient** `#5B7CFA → #3DE0FF`; glows via layered `box-shadow`.
- **Text** `#F5F7FA`; base font 20px, output text 24px+, line-height 1.7; contrast ≥ WCAG AA.
- **Iris orb**: a centered circular `div` (~140px) with the accent radial gradient and a soft outer
  glow. Define keyframe animations per state class:
  - `.iris.idle` slow breathing scale 1.0↔1.04 (~4s).
  - `.iris.listening` pulsing ring.
  - `.iris.seeing` a scan-line sweep.
  - `.iris.thinking` faster, tighter pulse.
  - `.iris.speaking` waveform-like glow pulse synced loosely to playback.
- **Primary button**: large pill / circle, min 96px hit target, accent gradient, thick cyan focus ring.
- **Surfaces**: glass panels — `backdrop-filter: blur(12px)`, 1px hairline border, subtle inner glow.
- **Motion**: wrap ALL animations in `@media (prefers-reduced-motion: reduce)` to disable them.
- **Focus**: visible thick cyan focus ring on every interactive element (serves keyboard + the look).
- The ARIA `live=polite` status line mirrors the orb state in words for screen-reader users.

═══════════════════════════════════════════════════════════════════════════════
# DONE = ALL OF THIS TRUE
- [ ] Describe / Ask / Read Text all work end-to-end, zero typing on the happy path.
- [ ] Audio answers auto-play; transcript shown large.
- [ ] Language dropdown drives multilingual TTS (English + Hindi verified).
- [ ] 3 bundled examples run with no webcam.
- [ ] Cold-start progress + mic/TTS fallbacks + `gr.Warning` everywhere; no raw tracebacks.
- [ ] Iris custom CSS live (Off-Brand); WCAG-AA contrast.
- [ ] README frontmatter verbatim; edge/on-device section honest; BLOG.md + DEMO_SCRIPT.md present.
- [ ] Primary model 2.8B ≤ 4B (Tiny Titan) — stated in README.
- [ ] Space builds clean and loads cold.

Report which checkpoints passed and paste the final file tree when done.
═══════════════════════════════════════════════════════════════════════════════
```