File size: 1,215 Bytes
b779915 4f40398 b779915 19de515 b779915 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | ---
library_name: transformers
license: apache-2.0
---
[Model checkpoints will be released soon.]
### Model Details
We introduce a new streaming paradigm that enables large language models to achieve strong performance and generalization in streaming settings, without requiring any architectural modifications.
* Streaming-processing: The LLMs process the input as it arrives, incrementally and in real time.
<img src="streaming.gif" alt="Streaming-processing" width="900"/>
* Batch-processing: The LLMs process inputs all at once after receiving the full sequence.
<img src="batch.gif" alt="Batch-processing" width="900"/>
### Model Sources
- **Paper:** https://arxiv.org/abs/2505.16983
- **Repository:** https://github.com/EIT-NLP/StreamingLLM
### Citation
```tex
@misc{tong2025llmeffectivestreamingprocessor,
title={LLM as Effective Streaming Processor: Bridging Streaming-Batch Mismatches with Group Position Encoding},
author={Junlong Tong and Jinlan Fu and Zixuan Lin and Yingqi Fan and Anhao Zhao and Hui Su and Xiaoyu Shen},
year={2025},
eprint={2505.16983},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.16983},
}
``` |