title: Live Code Camera Engine
emoji: 🎥
colorFrom: blue
colorTo: purple
sdk: docker
pinned: false
Live Code Camera Engine
A live camera + microphone → evidence compression → multimodal LLM → code stream engine.
This is the upgraded version of the click-to-capture prototype. It continuously samples the camera, tracks evidence density, collects speech transcript text, and periodically asks a multimodal LLM to generate or revise code based on what the camera and audio are exposing.
Core primitive
iPhone / laptop camera
→ live frame stream
→ evidence compressor
→ rolling observation buffer
→ LLM code synthesizer
→ live code updates
→ receipt chain
What "continuity" means here
The practical version is:
iPhone Safari / laptop browser
→ HTTPS Hugging Face Space
→ FastAPI WebSocket endpoint
→ multimodal LLM backend
If your iPhone and laptop are near each other, the shared continuity is not magic hardware continuity; it is a shared live endpoint. You open the same Space URL from either device, grant camera/mic permissions, and the stream becomes an evidence source.
Run locally
cd live-code-camera-engine
python3 -m venv .venv
source .venv/bin/activate
pip install -r backend/requirements.txt
export PROVIDER=ollama
export OLLAMA_MODEL=llava
uvicorn backend.main:app --host 0.0.0.0 --port 8000
Open:
http://localhost:8000
For iPhone on same Wi-Fi, use your laptop LAN IP:
http://YOUR_LAPTOP_IP:8000
Camera/mic access on mobile is most reliable over HTTPS, except localhost. For remote iPhone use, deploy to Hugging Face Spaces or use an HTTPS tunnel.
Deploy to Hugging Face Spaces
Create a Docker Space and upload this folder. Required files are already included:
Dockerfile
requirements.txt
backend/main.py
frontend/index.html
Set secrets/environment variables in the Space:
PROVIDER=ollama or openai
OPENAI_API_KEY=...
OPENAI_MODEL=<current multimodal model>
For local Ollama you normally run the backend locally, not inside Hugging Face, unless you provide a reachable Ollama endpoint.
Modes
continuous_code: generate code as the camera moves.debug_visible_error: point camera at terminal/browser error and generate a fix.derive_pipeline: infer a pipeline from scene + speech.soft_qr_xlc: decode/encode QR-like exoatomic carriers.research_prism: convert video/audio observations into code + research hypotheses.
Receipts
Every synthesis has:
session_id
frame hashes
transcript hash
observation entropy
frame-delta score
prompt hash
provider
timestamp
This makes the stream auditable without storing raw frames by default.
Safety boundaries
- The app does not execute generated code automatically.
- It does not secretly record; the browser asks for permissions.
- It does not store raw camera frames unless you add persistence.
- It should not be used for covert surveillance, credential theft, malware, or unauthorized automation.
- Multi-speaker separation is represented as an extension point. True speaker diarization needs an ASR/diarization model such as Whisper-family ASR plus a diarization stack.