---
title: Live Code Camera Engine
emoji: 🎥
colorFrom: blue
colorTo: purple
sdk: docker
pinned: false
---

# Live Code Camera Engine

A live **camera + microphone → evidence compression → multimodal LLM → code stream** engine.

This is the upgraded version of the click-to-capture prototype. It continuously samples the camera, tracks evidence density, collects speech transcript text, and periodically asks a multimodal LLM to generate or revise code based on what the camera and audio are exposing.

## Core primitive

```text
iPhone / laptop camera
→ live frame stream
→ evidence compressor
→ rolling observation buffer
→ LLM code synthesizer
→ live code updates
→ receipt chain
```

## What "continuity" means here

The practical version is:

```text
iPhone Safari / laptop browser
→ HTTPS Hugging Face Space
→ FastAPI WebSocket endpoint
→ multimodal LLM backend
```

If your iPhone and laptop are near each other, the shared continuity is not magic hardware continuity; it is a shared live endpoint. You open the same Space URL from either device, grant camera/mic permissions, and the stream becomes an evidence source.

## Run locally

```bash
cd live-code-camera-engine
python3 -m venv .venv
source .venv/bin/activate
pip install -r backend/requirements.txt

export PROVIDER=ollama
export OLLAMA_MODEL=llava
uvicorn backend.main:app --host 0.0.0.0 --port 8000
```

Open:

```text
http://localhost:8000
```

For iPhone on same Wi-Fi, use your laptop LAN IP:

```text
http://YOUR_LAPTOP_IP:8000
```

Camera/mic access on mobile is most reliable over HTTPS, except localhost. For remote iPhone use, deploy to Hugging Face Spaces or use an HTTPS tunnel.

## Deploy to Hugging Face Spaces

Create a Docker Space and upload this folder. Required files are already included:

```text
Dockerfile
requirements.txt
backend/main.py
frontend/index.html
```

Set secrets/environment variables in the Space:

```text
PROVIDER=ollama or openai
OPENAI_API_KEY=...
OPENAI_MODEL=<current multimodal model>
```

For local Ollama you normally run the backend locally, not inside Hugging Face, unless you provide a reachable Ollama endpoint.

## Modes

- `continuous_code`: generate code as the camera moves.
- `debug_visible_error`: point camera at terminal/browser error and generate a fix.
- `derive_pipeline`: infer a pipeline from scene + speech.
- `soft_qr_xlc`: decode/encode QR-like exoatomic carriers.
- `research_prism`: convert video/audio observations into code + research hypotheses.

## Receipts

Every synthesis has:

```text
session_id
frame hashes
transcript hash
observation entropy
frame-delta score
prompt hash
provider
timestamp
```

This makes the stream auditable without storing raw frames by default.

## Safety boundaries

- The app does not execute generated code automatically.
- It does not secretly record; the browser asks for permissions.
- It does not store raw camera frames unless you add persistence.
- It should not be used for covert surveillance, credential theft, malware, or unauthorized automation.
- Multi-speaker separation is represented as an extension point. True speaker diarization needs an ASR/diarization model such as Whisper-family ASR plus a diarization stack.