Spaces:
Sleeping
Sleeping
| title: SceneSolver | |
| emoji: 🕵️ | |
| colorFrom: yellow | |
| colorTo: gray | |
| sdk: gradio | |
| sdk_version: 5.49.1 | |
| app_file: app.py | |
| python_version: "3.12" | |
| license: mit | |
| short_description: Multimodal surveillance analysis and evidence synthesis | |
| # SceneSolver — CrimeSceneSolver (Scene Solver v3) | |
| SceneSolver is a multimodal surveillance-analysis system. This public Space runs the | |
| real nine-stage pipeline from the original notebook, together with the Sherl0ck | |
| agentic escalation layer (Cohere reasoning + LFM2.5-Audio voice) and the full | |
| black/white/orange four-tab frontend (Home · Analyse · Sherl0ck Agent · Contact). | |
| ## Pipeline (nine stages, all real models) | |
| 0. **AutoEncoder (AE)** — primary anomaly signal (batched reconstruction error) | |
| 1. **TimeSformer Binary** — semantic gate (Normal / Suspicious) | |
| 2. **TimeSformer Multi-class** — fine-grained anomaly classification (7 classes) | |
| 3. **YOLO Weapon Detection** — hard safety trigger (melee / firearms) | |
| 4. **Audio Feature Extraction** — ZCR / MFCC variance / spectral flux / RMS / centroid | |
| 5. **Tracking & Behavior** — BYTETrack or optical-flow motion analysis | |
| 6. **Fusion Layer** — rule-based reasoning core with a full reasoning trace | |
| 7. **RL3033** — PPO temporal-reasoning agent (final risk) | |
| 8. **LLaVA** — optional 4-frame contextual narrative (checkbox on the Analyse tab) | |
| Outputs: interactive risk report, peak anomaly frame, 11-page forensic PDF dossier, | |
| and structured JSON. | |
| ## Sherl0ck Agent | |
| Text chat and hands-free voice call (browser VAD → LFM2.5-Audio ASR → Cohere → | |
| LFM2.5-Audio TTS), incoming-call overlay for HIGH/CRITICAL results, and evidence | |
| dispatch to Telegram / Discord / email when those secrets are configured. | |
| ## Model sources (public) | |
| - [`samxkevin/scenesolver-assets`](https://huggingface.co/samxkevin/scenesolver-assets) — AE, TimeSformer, RL3033, YOLO checkpoints + agent audio | |
| - [`samxkevin/SceneDescriptionModel`](https://huggingface.co/samxkevin/SceneDescriptionModel) — LLaVA VLM (~14 GB, downloaded on first use of Stage 8) | |
| - `facebook/timesformer-base-finetuned-k400` — TimeSformer base weights | |
| - `LiquidAI/LFM2.5-Audio-1.5B` — Sherl0ck voice (ASR/TTS) | |
| ## Hardware — ZeroGPU | |
| This Space is designed for **ZeroGPU** (select *ZeroGPU* under Settings → Space | |
| hardware; ZeroGPU is not expressible in this README). All GPU-heavy inference runs | |
| inside `@spaces.GPU` regions (full pipeline analysis with dynamic duration, and the | |
| Sherl0ck voice ASR/TTS turns). Python is pinned to 3.12 (a ZeroGPU-supported | |
| runtime). | |
| ## Secrets (Settings → Variables and secrets) | |
| All integrations are optional and read from the environment — no credentials are | |
| stored in this repository: | |
| | Secret | Purpose | | |
| | --- | --- | | |
| | `COHERE_API_KEY` | Sherl0ck text/voice reasoning (offline fallback replies when absent) | | |
| | `TELEGRAM_BOT_TOKEN`, `TELEGRAM_CHAT_ID` | Telegram evidence dispatch | | |
| | `DISCORD_WEBHOOK_URL` | Discord evidence dispatch | | |
| | `ALERT_EMAIL_FROM`, `ALERT_EMAIL_PASS`, `ALERT_EMAIL_TO` | Email alerts | | |
| | `SENDER_EMAIL`, `SENDER_PASS`, `ADMIN_EMAIL` | Contact form delivery | | |
| ## Disclaimer | |
| Experimental research project. ML outputs are investigative aids, not definitive | |
| conclusions, and require human expert review before any operational action. | |