update

README.md

@@ -1,3 +1,34 @@
 ---
 license: gpl-3.0
 ---
+# Stream-Omni: Simultaneous Multimodal Interactions with Large Language-Vision-Speech Model
+
+[![arXiv](https://img.shields.io/badge/arXiv-XXXX.XXXX-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/XXXX.XXXXX)
+[![model](https://img.shields.io/badge/%F0%9F%A4%97%20Huggingface%20-stream--omni--8b-orange.svg)](https://huggingface.co/ICTNLP/stream-omni-8b)
+[![data](https://img.shields.io/badge/%F0%9F%93%91%20Datasets%20-InstructOmni-green.svg)](https://huggingface.co/datasets/ICTNLP/InstructOmni)
+[![Badge](https://hitscounter.dev/api/hit?url=https%3A%2F%2Fgithub.com%2Fictnlp%2FStream-Omni&label=Visitors&icon=graph-up&color=%23dc3545)](https://github.com/ictnlp/Stream-Omni)
+
+> [**Shaolei Zhang**](https://zhangshaolei1998.github.io/), [**Shoutao Guo**](https://scholar.google.com.hk/citations?user=XwHtPyAAAAAJ), [**Qingkai Fang**](https://fangqingkai.github.io/), [**Yan Zhou**](https://zhouyan19.github.io/zhouyan/), [**Yang Feng**](https://people.ucas.edu.cn/~yangfeng?language=en)\*
+
+
+Stream-Omni is an end-to-end language-vision-speech chatbot that simultaneously supports interaction across various modality combinations, with the following features💡:
+- **Omni Interaction**: Support any multimodal inputs including text, vision, and speech, and generate both text and speech responses.
+- **Seamless "see-while-hear" Experience**: Simultaneously output *intermediate textual results* (e.g., ASR transcriptions and model responses) during speech interactions, like the advanced voice service of GPT-4o.
+- **Efficient Training**: Require only a small amount of omni-modal data for training.
+
+<p align="center" width="100%">
+<img src="https://github.com/ictnlp/Stream-Omni/blob/main/assets/stream-omni.png" alt="stream-omni" style="width: 90%; min-width: 300px; display: block; margin: auto;">
+</p>
+
+## 🖥 Demo
+| Microphone Input                                                | File Input                                                      |
+| ------------------------------------------------------------ | ------------------------------------------------------------ |
+| <video src='https://github.com/user-attachments/assets/99325a91-b81c-4fd1-a68e-c0628d5784fe' width="100%"/> | <video src='https://github.com/user-attachments/assets/bcb37b15-6c8c-4eae-845e-5da10aaac11e' width="100%"/> |
+
+
+> [!NOTE]
+>
+> **Stream-Omni can produce intermediate textual results (ASR transcription and text response) during speech interaction, offering users a seamless "see-while-hear" experience.**
+
+
+The introduction and usage of Stream-Omni refer to [https://github.com/ictnlp/Stream-Omni](https://github.com/ictnlp/Stream-Omni).