Update README.md

README.md

@@ -14,71 +14,16 @@ pipeline_tag: video-text-to-text
 # ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding
 
 <div align="center">
-  <img src="[INSERT LINK TO FIGURE 2 (ARCHITECTURE) HERE]" width="800"/>
+  <img src="architecture.png" width="800"/>
   <p>Figure: ROMA processes streaming inputs as aligned multimodal units, using a 'Speak Head' to decide when to respond.</p>
 </div>
 
 ## Model Summary
 
-[cite_start]**ROMA** is a Real-time Omni-Multimodal Assistant designed for unified streaming audio-video understanding[cite: 9, 46]. [cite_start]Unlike traditional video LLMs that only answer after a query, ROMA integrates both **Reactive** (Question Answering) and **Proactive** (Event-Driven Alert, Real-Time Narration) capabilities within a single framework[cite: 58].
+**ROMA** is a Real-time Omni-Multimodal Assistant designed for unified streaming audio-video understanding. Unlike traditional videoLLMs that only answer after a query, ROMA integrates both **Reactive** (Question Answering) and **Proactive** (Event-Driven Alert, Real-Time Narration) capabilities within a single framework.
 
-[cite_start]ROMA introduces a "Speak Head" mechanism to decouple response timing from content generation, allowing it to autonomously decide *when* to speak based on the continuous audio-visual stream[cite: 11, 49].
+ROMA introduces a "Speak Head" mechanism to decouple response timing from content generation, allowing it to autonomously decide *when* to speak based on the continuous audio-visual stream.
 
 - **Paper:** [ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding](https://arxiv.org/abs/250x.xxxxx)
-- [cite_start]**Project Page:** [Link](https://eureka-maggie.github.io/ROMA_show/) [cite: 20]
-- **Repository:** [INSERT GITHUB LINK]
-- **Developed by:** Institute of Computing Technology, CAS; UCAS; [cite_start]Tsinghua University[cite: 2, 3, 4].
-
-## Key Capabilities
-
-[cite_start]ROMA excels in three main interaction modes[cite: 53]:
-
-1.  [cite_start]**Event-Driven Alert (Proactive):** Monitors the stream and notifies the user immediately when a specific event occurs (e.g., "Notify me when a bird pops out")[cite: 23, 210].
-2.  [cite_start]**Real-Time Narration (Proactive):** Continuously describes the evolving video and audio context without needing user prompts[cite: 25, 223].
-3.  [cite_start]**Reactive QA:** Answers questions based on the past context, handling synchronized audio and video inputs[cite: 27, 227].
-
-## Model Architecture
-
-[cite_start]ROMA processes continuous inputs as synchronized **Multimodal Units** (1-second intervals aligning dense audio with video frames)[cite: 10, 128].
-
-Key architectural innovations include:
-- [cite_start]**Chunked TMROPE:** Ensures proper temporal position encoding across streaming chunks[cite: 48, 134].
-- [cite_start]**Speak Head:** A lightweight module parallel to the LM head that predicts a binary probability to trigger a response, solving the issue of task conflict between listening and speaking[cite: 144, 145].
-
-## Performance
-
-[cite_start]ROMA achieves state-of-the-art performance on proactive benchmarks while remaining competitive on reactive settings[cite: 14].
-
-| Benchmark Type | Task | ROMA Performance | State-of-the-Art? |
-| :--- | :--- | :--- | :--- |
-| **Proactive** | Event-Driven Alert (QVHighlights) | [cite_start]**53.7 mAP** [cite: 241] | ✅ Yes |
-| **Proactive** | Real-Time Narration (YouCook2) | [cite_start]**35.21 F1** [cite: 251] | ✅ Yes |
-| **Reactive** | Omni-Source Understanding (StreamingBench) | [cite_start]**Top Rank** [cite: 255] | ✅ Yes |
-| **Reactive** | Full-Modality QA (Video-MME w/ Audio) | [cite_start]**33.30 Accuracy** [cite: 260] | ✅ Yes |
-
-## Quick Start
-
-```python
-# Note: This is a pseudo-code example. Please refer to the official GitHub repo for the exact inference loop.
-
-from transformers import AutoModel, AutoTokenizer
-
-model = AutoModel.from_pretrained("Your-HF-Username/ROMA")
-tokenizer = AutoTokenizer.from_pretrained("Your-HF-Username/ROMA")
-
-# Input: Streaming chunks of Video and Audio (1-second units)
-# The model uses a "Speak Head" to decide when to output text.
-
-stream = load_video_audio_stream("example_video.mp4")
-
-history_cache = None
-
-for multimodal_unit in stream:
-    # Process 1-second unit
-    response, history_cache = model.streaming_inference(
-        multimodal_unit, 
-        past_key_values=history_cache
-    )
-    
-    if response:
-        print(f"ROMA says: {response}")
+- **Project Page:** [Link](https://eureka-maggie.github.io/ROMA_show/)
+- **Repository:** [[Github (Coming Soon)](https://github.com/Eureka-Maggie/GLOW)]