MaineCoon: Pursuing A Real-Time Audio-Visual Social World Model

Catnip AI Team

🌐 Project	https://mainecoon.tech/
🕹️ Experience	https://mainecoon.tech/experience-platform
📄 Paper (arXiv)	https://arxiv.org/abs/2606.17800
📝 Blog	https://mainecoon.tech/blogs
💻 GitHub	https://github.com/catnip-ai-tech/MaineCoon

Abstract

As an increasing majority of global video content is consumed on social platforms for interactive social purposes, video generation models built for social worlds are important but largely overlooked by previous studies. In this work, we define the position of social world models and build a prototype model as the first step towards this goal. While previous world models successfully simulate physical environments or gaming world exploration, they remain fundamentally detached from human-centric social dynamics. They typically omit critical auditory information or fail to capture the high-engagement pacing, emotional resonance, and rapid conversational flow that define viral social media. To bridge this gap as the first step to social world models, we present MaineCoon, the first real-time audio-visual autoregressive model that has 22B parameters and is capable of real-time streaming generation and sub-second interaction, with a record-breaking frame rate of up to 47.5 FPS, on a single GPU. To the best of our knowledge, MaineCoon is also the first real-time audio-visual generation model specifically optimized for social-interactive applications. To enable efficient and stable training, we introduce several novel techniques into MaineCoon, including self-resampling, cross-modal representation alignment, domain-aware preference optimization, and reinforced online-policy distillation (ROPD). We also design the first agentic streaming inference framework that supports thousand-second-scale or even longer generation while mitigating drift with agentic cache management and prompt planning. These innovations significantly accelerate training while optimizing real-time inference performance. We believe this work not only sets a new state-of-the-art (SOTA) performance benchmark for high-quality, low-latency, and long-horizon audio-visual autoregressive models, but also points out the paradigm shift desired for next-generation AI-native social platforms.

Highlights

• ⚡ Real-time on a single GPU. A 22B interactive audio-visual autoregressive model capable of streaming generation and sub-second interaction, with a record-breaking frame rate of up to 47.5 FPS on a single H100. Generation cost drops well below $0.001 per second — and keeps falling.
• 🌍 A new paradigm: social world models. MaineCoon positions and serves as the first generative core for social world models, a technical foundation for next-generation AI-native social platforms.
• 🎓 Forcing-free streaming training. A multi-stage training paradigm — self-resampling, cross-modal representation alignment, domain-aware preference optimization, and reinforced online-policy distillation (ROPD) — that enables native, efficient streaming audio-visual training at 22B scale.
• 🧠 Agentic streaming inference. An agentic inference framework that supports thousand-second-scale generation while mitigating drift through agentic cache management, chunk commitment, long-context rollout, and prompt planning.
• 📊 SocialVideo-Bench. A new benchmark focused on audio-visual social-video generation, with 9 representative metrics covering visual quality, motion, audio quality, audio-visual alignment, and social-video harmony. MaineCoon outperforms 7 representative open audio-visual models while achieving the fastest generation speed — a new state of the art for real-time social video generation.

Showcase

Hand-picked MaineCoon generations (audio-visual, with sound) play directly in the GitHub repository.

🎬 Minute-scale, long-form demos are best viewed on our blog.   🕹️ Try MaineCoon live at the experience platform.

Benchmark — SocialVideo-Bench

Table 2. Main quantitative results on SocialVideo-Bench. 🐱 MaineCoon (Ours) achieves the best average score and wins most metrics, including the two most comprehensive ones — Audio-Visual Harmony (AVH) and Joint Audio-Visual Integrated Score (JAVIS) — over both streaming and bidirectional baselines.

Type	Model	Vis↑	Mot↑	Aud↑	IB-TV↑	IB-TA↑	IB-AV↑	AV-Al↑	AVH↑	JAVIS↑	Average↑
Bidirectional T2AV	JavisDiT++	4.39	2.22	4.06	0.134	0.070	0.151	0.312	0.136	0.112	0.711
	Ovi	4.44	1.89	3.76	0.138	0.079	0.191	0.412	0.188	0.162	0.779
	JoyAI-Echo	4.61	1.17	3.47	0.147	0.088	0.226	0.319	0.196	0.173	0.749
	MoVA	4.66	1.68	3.69	0.133	0.105	0.258	0.359	0.245	0.216	0.842
	LTX-2.3	4.10	0.99	4.06	0.132	0.111	0.311	0.334	0.287	0.247	0.848
Streaming TA2V	LiveAvatar	4.60	1.46	4.13	0.131	0.120	0.316	0.326	0.291	0.246	0.892
	SoulX-FlashTalk	4.65	1.99	4.07	0.128	0.120	0.307	0.279	0.283	0.238	0.895
Streaming T2AV	🐱 MaineCoon (Ours)	4.71	1.62	4.35	0.127	0.130	0.318	0.334	0.308	0.272	0.934 🥇

_{🐱 = our method  ·  bold = best, italic = second best.   Metrics — Vis: visual quality · Mot: motion · Aud: audio quality · IB-TV / IB-TA / IB-AV: ImageBind Text–Video / Text–Audio / Audio–Video alignment · AV-Al: audio–visual alignment · AVH: Audio-Visual Harmony · JAVIS: Joint Audio-Visual Integrated Score. See the technical report for the full benchmark and metric definitions.}

Table 3. Latency and model size comparison. Sampling throughput (FPS) is measured for 480P 20-second generation on a single H100 GPU. 🐱 MaineCoon (Ours) has the largest model yet by far the fastest speed — up to 7× faster than other streaming audio-visual generators, and faster even than a 1.3B streaming video model.

Type	Model	Params	FPS↑
Bidirectional T2AV	JavisDiT++	1.8B	0.87
	Ovi	11B	0.58
	JoyAI-Echo	23B	18.0
	MoVA	32B	0.26
	LTX-2.3	22B	1.40
	LTX-2.3-Distilled	22B	20.7
Streaming T2V	Causal-Forcing	1.3B	19.1
	Helios-Distilled	14B	18.2
	Krea	14B	6.1
Streaming TA2V	LiveAvatar	14B	6.7
	SoulX-FlashTalk	14B	6.6
Streaming T2AV	🐱 MaineCoon (Ours)	22B	47.5 🥇

_{🐱 = our method  ·  bold = best, italic = second best. FPS for 480P-20s on a single H100.}

Paper

The full paper is available on arXiv:2606.17800. A PDF copy is also included in this repository: MaineCoon_Technical_Report.pdf. It covers the social-video data infrastructure, the native streaming autoregressive training recipe, the agentic streaming inference framework, SocialVideo-Bench, and a position/outlook on social world models.

Acknowledgements

MaineCoon stands on the shoulders of the open-source community. We are especially grateful to:

• 🎬 LTX-2.3 & the LTX series — Lightricks. MaineCoon's audio-visual backbone builds on the excellent open LTX-2.3 model. Huge credit to the LTX team and the broader LTX-Video series.
  • LTX-2 (incl. LTX-2.3): https://github.com/Lightricks/LTX-2
  • LTX-Video: https://github.com/Lightricks/LTX-Video
• ⚡ DMD series & the distribution-matching distillation community. Our reinforced online-policy distillation (ROPD) builds on the Distribution Matching Distillation (DMD / DMD2) line of work and the wider few-step / real-time distillation community.
  • DMD2: https://github.com/tianweiy/DMD2
  • DMD (project page): https://tianweiy.github.io/dmd/

We thank these projects and their communities for advancing real-time, few-step, and streaming video generation.

Citation

@article{catnip2026mainecoon,
  title        = {MaineCoon: Pursuing A Real-Time Audio-Visual Social World Model},
  author       = {Catnip AI Team},
  year         = {2026},
  journal      = {arXiv preprint arXiv:2606.17800},
  url          = {https://arxiv.org/abs/2606.17800}
}