--- license: apache-2.0 datasets: - violetcliff/SmartHome-Bench language: - en base_model: - Qwen/Qwen2.5-3B-Instruct --- # Model Card for Memories-S0 **Memories-S0** is a highly efficient, 3-billion-parameter video understanding model designed specifically for the security and surveillance domain. It leverages synthetic data generation (via Veo 3) and extreme optimization strategies to achieve state-of-the-art performance on edge devices. ## Model Details * **Model Name:** Memories-S0 * **Organization:** Memories.ai Research * **Model Architecture:** 3B Parameter VideoLLM * **Release Date:** Jan 2026 * **License:** Apache 2.0 * **Paper:** [Memories-SO: An Efficient and Accurate Framework for Security Video Understanding](https://memories.ai/research/Camera) * **Code Repository:** [https://github.com/Memories-ai-labs/memories-s0](https://github.com/Memories-ai-labs/memories-s0) ### Model Description Memories-S0 is designed to address two key challenges in security video understanding: data scarcity and deployment efficiency on resource-constrained devices. * **Data Innovation:** The model is pre-trained on a massive, diverse set of synthetic surveillance videos generated by advanced video generation models (like Veo 3). This allows for pixel-perfect annotations and covers diverse scenarios (e.g., dimly lit hallways, unattended packages). * **Extreme Efficiency:** It utilizes an innovative input token compression algorithm that dynamically prunes redundant background tokens, focusing computation on foreground objects and motion. This allows the 3B model to run efficiently on mobile/edge hardware. * **Post-Training:** The model employs a unique post-training strategy using Reinforcement Learning (RL) and event-based temporal shuffling to enhance sequential understanding without expensive full fine-tuning. ## Installation ```bash conda create -n memories-s0 python=3.10 -y conda activate memories-s0 # Install PyTorch with CUDA support pip install torch torchvision torchaudio --index-url # Install dependencies for Qwen2.5-VL architecture and Flash Attention pip install transformers>=4.37.0 accelerate qwen_vl_utils pip install flash-attn --no-build-isolation ``` ## Inference The following script demonstrates how to run the **Memories-S0** model. It automatically handles the loading of weights from the official Hugging Face repository. ```python import torch import argparse from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor from qwen_vl_utils import process_vision_info # Official Model Repository MODEL_ID = "Memories-ai/security_model" def run_inference(video_path, model_id=MODEL_ID): # Load Model with Flash Attention 2 for efficiency model = Qwen2_5_VLForConditionalGeneration.from_pretrained( model_id, torch_dtype=torch.bfloat16, attn_implementation="flash_attention_2", device_map="auto", ) processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True) # Define Security Analysis Prompt prompt_text = """YOUR_PROMPT""" messages = [ { "role": "user", "content": [ {"type": "video", "video": video_path}, {"type": "text", "text": prompt_text}, ], } ] # Preprocessing text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) image_inputs, video_inputs, video_kwargs = process_vision_info(messages, return_video_kwargs=True) inputs = processor( text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt", **video_kwargs, ) inputs = inputs.to("cuda") # Generate generated_ids = model.generate(**inputs, max_new_tokens=768) generated_ids_trimmed = [ out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids) ] output_text = processor.batch_decode( generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False ) print(output_text[0]) if __name__ == "__main__": parser = argparse.ArgumentParser() parser.add_argument("--video_path", type=str, required=True, help="Path to input video") args = parser.parse_args() run_inference(args.video_path) ``` ## Intended Use ### Primary Use Cases * **Security & Surveillance:** Detecting anomalies, tracking suspicious activities, and monitoring public safety. * **Smart Home Monitoring:** Analyzing video feeds for unusual events (e.g., falls, intruders) as benchmarked on SmartHomeBench. * **Edge Computing:** Deploying high-performance video analysis directly on cameras or local gateways with limited memory and compute power. ### Out-of-Scope Use Cases * General open-domain video understanding (e.g., movie classification) may not be optimal as the model is specialized for surveillance angles and events. * Biometric identification (Face Recognition) is not the primary design goal; the focus is on action and event understanding. ## Performance (SmartHomeBench) We evaluated Memories-S0(3B) on the **SmartHomeBench** dataset, a recognized benchmark for smart home video anomaly detection. Despite having only **3B parameters**, our model achieves an **F1-score of 79.21** using a simple **Zero-shot** prompt, surpassing larger models like VILA-13b and performing competitively against GPT-4o and Claude-3.5-Sonnet (which require complex Chain-of-Thought prompting). | Model | Params | Prompting Method | Accuracy | Precision | Recall | **F1-score** | | --- | --- | --- | --- | --- | --- | --- | | **Memories-S0 (Ours)** | **3B** | **Zero-shot** | **71.33** | **73.04** | **86.51** | **79.21** | | VILA-13b | 13B | Few-shot CoT | 67.17 | 69.18 | 70.57 | 69.87 | | GPT-4o | Closed | Zero-shot | 68.41 | 80.09 | 55.16 | 65.33 | | Gemini-1.5-Pro | Closed | Zero-shot | 57.36 | 84.34 | 25.73 | 39.43 | ## Citation If you use this model or framework in your research, please cite our technical report: ```bibtex @techreport{memories_s0_2025, title = {{Memories-S0}: An Efficient and Accurate Framework for Security Video Understanding}, author = {{Memories.ai Research}}, institution = {Memories.ai}, year = {2025}, month = oct, url = {https://huggingface.co/Memories-ai/security_model}, note = {Accessed: 2025-11-20} } ```