README.md

metadata

title: SAM-MM Multimodal Demo
emoji: 🧠
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.18.0
app_file: app.py
pinned: false
license: other
short_description: 58M offline multimodal model for vision and physics

SAM-MM — multimodal reasoning demo

Interactive demo of SAM-MM (AMFORGE), a ~58M-parameter offline multimodal model that perceives synthetic frames (and a log-mel spectrogram for audio scenes) and answers with either a [CHAT] reasoning trace or a [ACTION] JSON record.

Pick a scene; each one is freshly generated and the model decodes token-by-token — nothing is hard-coded. Physics and motion are the model's strength (its latent world-model carries real dynamics); OCR generalizes to unseen numbers; the cross-modal action family is weaker; audio is the weak modality (trained on synthetic pseudo-mel).

Setup

The model weights live in private HuggingFace repos. Add a Space secret named HF_TOKEN (a token with read access to the AMFORGE/sam-mm-* repos). The app pulls the checkpoint and tokenizer from the Hub on first run.

Architecture internals, tokenizer construction, and data generators are proprietary and not exposed by this demo.