---
title: SceneSolver
emoji: 🕵️
colorFrom: yellow
colorTo: gray
sdk: gradio
sdk_version: 5.49.1
app_file: app.py
python_version: "3.12"
license: mit
short_description: Multimodal surveillance analysis and evidence synthesis
---

# SceneSolver — CrimeSceneSolver (Scene Solver v3)

SceneSolver is a multimodal surveillance-analysis system. This public Space runs the
real nine-stage pipeline from the original notebook, together with the Sherl0ck
agentic escalation layer (Cohere reasoning + LFM2.5-Audio voice) and the full
black/white/orange four-tab frontend (Home · Analyse · Sherl0ck Agent · Contact).

## Pipeline (nine stages, all real models)

0. **AutoEncoder (AE)** — primary anomaly signal (batched reconstruction error)
1. **TimeSformer Binary** — semantic gate (Normal / Suspicious)
2. **TimeSformer Multi-class** — fine-grained anomaly classification (7 classes)
3. **YOLO Weapon Detection** — hard safety trigger (melee / firearms)
4. **Audio Feature Extraction** — ZCR / MFCC variance / spectral flux / RMS / centroid
5. **Tracking & Behavior** — BYTETrack or optical-flow motion analysis
6. **Fusion Layer** — rule-based reasoning core with a full reasoning trace
7. **RL3033** — PPO temporal-reasoning agent (final risk)
8. **LLaVA** — optional 4-frame contextual narrative (checkbox on the Analyse tab)

Outputs: interactive risk report, peak anomaly frame, 11-page forensic PDF dossier,
and structured JSON.

## Sherl0ck Agent

Text chat and hands-free voice call (browser VAD → LFM2.5-Audio ASR → Cohere →
LFM2.5-Audio TTS), incoming-call overlay for HIGH/CRITICAL results, and evidence
dispatch to Telegram / Discord / email when those secrets are configured.

## Model sources (public)

- [`samxkevin/scenesolver-assets`](https://huggingface.co/samxkevin/scenesolver-assets) — AE, TimeSformer, RL3033, YOLO checkpoints + agent audio
- [`samxkevin/SceneDescriptionModel`](https://huggingface.co/samxkevin/SceneDescriptionModel) — LLaVA VLM (~14 GB, downloaded on first use of Stage 8)
- `facebook/timesformer-base-finetuned-k400` — TimeSformer base weights
- `LiquidAI/LFM2.5-Audio-1.5B` — Sherl0ck voice (ASR/TTS)

## Hardware — ZeroGPU

This Space is designed for **ZeroGPU** (select *ZeroGPU* under Settings → Space
hardware; ZeroGPU is not expressible in this README). All GPU-heavy inference runs
inside `@spaces.GPU` regions (full pipeline analysis with dynamic duration, and the
Sherl0ck voice ASR/TTS turns). Python is pinned to 3.12 (a ZeroGPU-supported
runtime).

## Secrets (Settings → Variables and secrets)

All integrations are optional and read from the environment — no credentials are
stored in this repository:

| Secret | Purpose |
| --- | --- |
| `COHERE_API_KEY` | Sherl0ck text/voice reasoning (offline fallback replies when absent) |
| `TELEGRAM_BOT_TOKEN`, `TELEGRAM_CHAT_ID` | Telegram evidence dispatch |
| `DISCORD_WEBHOOK_URL` | Discord evidence dispatch |
| `ALERT_EMAIL_FROM`, `ALERT_EMAIL_PASS`, `ALERT_EMAIL_TO` | Email alerts |
| `SENDER_EMAIL`, `SENDER_PASS`, `ADMIN_EMAIL` | Contact form delivery |

## Disclaimer

Experimental research project. ML outputs are investigative aids, not definitive
conclusions, and require human expert review before any operational action.