README.md

metadata

title: SceneSolver
emoji: 🕵️
colorFrom: yellow
colorTo: gray
sdk: gradio
sdk_version: 5.49.1
app_file: app.py
python_version: '3.12'
license: mit
short_description: Multimodal surveillance analysis and evidence synthesis

SceneSolver — CrimeSceneSolver (Scene Solver v3)

SceneSolver is a multimodal surveillance-analysis system. This public Space runs the real nine-stage pipeline from the original notebook, together with the Sherl0ck agentic escalation layer (Cohere reasoning + LFM2.5-Audio voice) and the full black/white/orange four-tab frontend (Home · Analyse · Sherl0ck Agent · Contact).

Pipeline (nine stages, all real models)

0. AutoEncoder (AE) — primary anomaly signal (batched reconstruction error)
1. TimeSformer Binary — semantic gate (Normal / Suspicious)
2. TimeSformer Multi-class — fine-grained anomaly classification (7 classes)
3. YOLO Weapon Detection — hard safety trigger (melee / firearms)
4. Audio Feature Extraction — ZCR / MFCC variance / spectral flux / RMS / centroid
5. Tracking & Behavior — BYTETrack or optical-flow motion analysis
6. Fusion Layer — rule-based reasoning core with a full reasoning trace
7. RL3033 — PPO temporal-reasoning agent (final risk)
8. LLaVA — optional 4-frame contextual narrative (checkbox on the Analyse tab)

Outputs: interactive risk report, peak anomaly frame, 11-page forensic PDF dossier, and structured JSON.

Sherl0ck Agent

Text chat and hands-free voice call (browser VAD → LFM2.5-Audio ASR → Cohere → LFM2.5-Audio TTS), incoming-call overlay for HIGH/CRITICAL results, and evidence dispatch to Telegram / Discord / email when those secrets are configured.

Model sources (public)

• samxkevin/scenesolver-assets — AE, TimeSformer, RL3033, YOLO checkpoints + agent audio
• samxkevin/SceneDescriptionModel — LLaVA VLM (~14 GB, downloaded on first use of Stage 8)
• facebook/timesformer-base-finetuned-k400 — TimeSformer base weights
• LiquidAI/LFM2.5-Audio-1.5B — Sherl0ck voice (ASR/TTS)

Hardware — ZeroGPU

This Space is designed for ZeroGPU (select ZeroGPU under Settings → Space hardware; ZeroGPU is not expressible in this README). All GPU-heavy inference runs inside @spaces.GPU regions (full pipeline analysis with dynamic duration, and the Sherl0ck voice ASR/TTS turns). Python is pinned to 3.12 (a ZeroGPU-supported runtime).

Secrets (Settings → Variables and secrets)

All integrations are optional and read from the environment — no credentials are stored in this repository:

Secret	Purpose
COHERE_API_KEY	Sherl0ck text/voice reasoning (offline fallback replies when absent)
TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID	Telegram evidence dispatch
DISCORD_WEBHOOK_URL	Discord evidence dispatch
ALERT_EMAIL_FROM, ALERT_EMAIL_PASS, ALERT_EMAIL_TO	Email alerts
SENDER_EMAIL, SENDER_PASS, ADMIN_EMAIL	Contact form delivery

Disclaimer

Experimental research project. ML outputs are investigative aids, not definitive conclusions, and require human expert review before any operational action.