Improve model card: Add metadata, links, usage, and visuals

README.md

@@ -1,4 +1,86 @@
 ---
 license: apache-2.0
+pipeline_tag: video-text-to-text
+library_name: transformers
 ---
-paper link: https://arxiv.org/abs/2508.09736
+
+# Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory
+
+This repository contains a model checkpoint from the M3-Agent framework, which was introduced in the paper [Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory](https://arxiv.org/abs/2508.09736).
+
+-   🌐 [**Project Page**](https://m3-agent.github.io)
+-   💻 [**GitHub Repository**](https://github.com/hyc2026/M3-Agent-Training)
+
+<div align=left>
+    <img src="https://github.com/user-attachments/assets/c42e675e-497c-4508-8bb9-093ad4d1f216" width=40%>
+</div>
+
+## Abstract
+We introduce M3-Agent, a novel multimodal agent framework equipped with long-term memory. Like humans, M3-Agent can process real-time visual and auditory inputs to build and update its long-term memory. Beyond episodic memory, it also develops semantic memory, enabling it to accumulate world knowledge over time. Its memory is organized in an entity-centric, multimodal format, allowing deeper and more consistent understanding of the environment. Given an instruction, M3-Agent autonomously performs multi-turn, iterative reasoning and retrieves relevant information from memory to accomplish the task. To evaluate memory effectiveness and memory-based reasoning in multimodal agents, we develop M3-Bench, a new long-video question answering benchmark. M3-Bench comprises 100 newly recorded real-world videos captured from a robot's perspective (M3-Bench-robot) and 929 web-sourced videos across diverse scenarios (M3-Bench-web). We annotate question-answer pairs designed to test key capabilities essential for agent applications, such as human understanding, general knowledge extraction, and cross-modal reasoning. Experimental results show that M3-Agent, trained via reinforcement learning, outperforms the strongest baseline, a prompting agent using Gemini-1.5-pro and GPT-4o, achieving 6.7%, 7.7%, and 5.3% higher accuracy on M3-Bench-robot, M3-Bench-web and VideoMME-long, respectively. Our work advances the multimodal agents toward more human-like long-term memory and provides insights into their practical design. Model, code and data are available at this https URL
+
+## M3-Agent Demo Video
+Explore M3-Agent's capabilities as a personal assistant in this demo video:
+
+[![Watch the video](https://github.com/ByteDance-Seed/M3-Agent/raw/main/figs/demo.png)](https://www.youtube.com/watch?v=XUx31cBanfo)
+
+## Usage
+
+This model is designed to be a component of the larger M3-Agent framework, typically used for tasks related to memory generation or control. It can be loaded using the Hugging Face `transformers` library.
+
+For detailed usage within the full M3-Agent pipeline, including processing video and audio inputs to generate memory graphs and performing reasoning tasks, please refer to the [official GitHub repository](https://github.com/hyc2026/M3-Agent-Training).
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Assuming this repository contains the 'M3-Agent-Memorization' or a similar Qwen-based checkpoint
+# If your repository has a different ID, replace it below.
+model_id = "ByteDance-Seed/M3-Agent-Memorization" # This might need to be adjusted to the actual repo ID
+
+# Load tokenizer and model
+# trust_remote_code=True is often required for custom architectures like Qwen3
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16, # Use torch.float16 if bfloat16 is not supported by your hardware
+    device_map="auto",
+    trust_remote_code=True
+)
+
+# Set model to evaluation mode
+model.eval()
+
+print(f"Model {model_id} loaded successfully.")
+print("This model is a component of the M3-Agent framework. For full agent pipeline usage,")
+print("including video processing for memory generation and control tasks,")
+print("please refer to the official GitHub repository: https://github.com/hyc2026/M3-Agent-Training")
+
+# Example for basic text generation (demonstrates model loading)
+# The actual use case for this model is within the M3-Agent pipeline,
+# involving multimodal inputs and structured outputs.
+# messages = [
+#     {"role": "user", "content": "Hello! How are you today?"},
+# ]
+# text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+# input_ids = tokenizer(text, return_tensors="pt").to(model.device)
+#
+# with torch.inference_mode():
+#     outputs = model.generate(input_ids, max_new_tokens=50, do_sample=True, temperature=0.7)
+# generated_text = tokenizer.decode(outputs[0][len(input_ids[0]):], skip_special_tokens=True)
+# print(f"
+Generated response (general text generation): {generated_text}")
+```
+
+## Citation
+If you find this model or the M3-Agent project helpful, please cite the following paper:
+
+```bibtex
+@misc{long2025seeing,
+      title={Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory}, 
+      author={Lin Long, Yichen He, Wentao Ye, Yiyuan Pan, Yuan Lin, Hang Li, Junbo Zhao, Wei Li},
+      year={2025},
+      eprint={2508.09736},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV}
+}
+```