update readme & fix inference error

README.md

@@ -1,3 +1,35 @@
 ---
 license: apache-2.0
 ---
+# MOSS-TTS Family
+
+## Overview
+MOSS‑TTS Family is an open‑source **speech and sound generation model family** from [MOSI.AI](https://mosi.cn/#hero) and the [OpenMOSS team](https://www.open-moss.com/). It is designed for **high‑fidelity**, **high‑expressiveness**, and **complex real‑world scenarios**, covering stable long‑form speech, multi‑speaker dialogue, voice/character design, environmental sound effects, and real‑time streaming TTS.
+
+
+## Introduction
+
+<p align="center">
+  <img src="./assets/moss_tts_family.jpeg" width="85%" />
+</p>
+
+When a single piece of audio needs to **sound like a real person**, **pronounce every word accurately**, **switch speaking styles across content**, **remain stable over tens of minutes**, and **support dialogue, role‑play, and real‑time interaction**, a single TTS model is often not enough. The **MOSS‑TTS Family** breaks the workflow into five production‑ready models that can be used independently or composed into a complete pipeline.
+
+- **MOSS‑TTS**: MOSS-TTS is the flagship, production-ready Text-to-Speech foundation model in the MOSS-TTS Family, built to ship, scale, and deliver real-world voice applications beyond demos. It provides high-fidelity zero-shot voice cloning as the core capability, along with ultra-long speech generation, token-level duration control, multilingual and code-switched synthesis, and fine-grained Pinyin/phoneme pronunciation control. Together, these features make it a robust base model for scalable narration, dubbing, and voice-driven products.
+- **MOSS‑TTSD**: MOSS-TTSD is a production-oriented long-form spoken dialogue generation model for creating highly expressive, multi-party conversational audio at scale. It supports continuous long-duration generation, flexible multi-speaker turn-taking control, and zero-shot voice cloning from short reference audio, enabling natural conversations with rich interaction dynamics. It is designed for real-world long-form content such as podcasts, audiobooks, commentary, dubbing, and entertainment dialogue.
+- **MOSS‑VoiceGenerator**: MOSS-VoiceGenerator is an open-source voice design system that generates speaker timbres directly from free-form text descriptions, enabling fast creation of voices for characters, personalities, and emotions—without requiring reference audio. It unifies timbre design, style control, and content synthesis in a single instruction-driven model, producing high-fidelity, emotionally expressive speech that feels naturally human. It can be used standalone for creative production, or as a voice design layer that improves integration and usability for downstream TTS systems.
+- **MOSS‑SoundEffect**: MOSS-SoundEffect is a high-fidelity sound effect generation model built for real-world content creation, offering strong environmental richness, broad category coverage, and reliable duration controllability. Trained on large-scale, high-quality data, it generates consistent audio from text prompts across natural ambience, urban scenes, creatures, human actions, and music-like clips. It is well suited for film and game production, interactive experiences, and data synthesis pipelines.
+- **MOSS‑TTS‑Realtime**: MOSS-TTS-Realtime is a context-aware, multi-turn streaming TTS foundation model designed for real-time voice agents. Unlike conventional TTS that synthesizes replies in isolation, it conditions generation on multi-turn dialogue history—including both textual and acoustic signals from prior user speech—so responses stay coherent, consistent, and natural across turns. With low-latency incremental synthesis and strong voice stability, it enables truly conversational, human-like real-time speech experiences.
+
+
+## Released Models
+
+| Model | Architecture | Size | Model Card | Hugging Face |
+|---|---|---:|---|---|
+| **MOSS-TTS** | MossTTSDelay | 8B | [moss_tts_model_card.md](https://github.com/OpenMOSS/MOSS-TTS/blob/main/moss_tts_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-TTS) |
+|  | MossTTSLocal | 1.7B | [moss_tts_model_card.md](moss_tts_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Local-Transformer) |
+| **MOSS‑TTSD‑V1.0** | MossTTSDelay | 8B | [moss_ttsd_model_card.md](https://github.com/OpenMOSS/MOSS-TTS/blob/main/moss_ttsd_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-TTSD-v1.0) |
+| **MOSS‑VoiceGenerator** | MossTTSDelay | 1.7B | [moss_voice_generator_model_card.md](https://github.com/OpenMOSS/MOSS-TTS/blob/main/moss_voice_generator_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-Voice-Generator) |
+| **MOSS‑SoundEffect** | MossTTSDelay | 8B | [moss_sound_effect_model_card.md](https://github.com/OpenMOSS/MOSS-TTS/blob/main/moss_sound_effect_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-SoundEffect) |
+| **MOSS‑TTS‑Realtime** | MossTTSRealtime | 1.7B | [moss_tts_realtime_model_card.md](https://github.com/OpenMOSS/MOSS-TTS/blob/main/moss_tts_realtime_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Realtime) |
+

modeling_moss_tts.py

@@ -398,9 +398,9 @@ class MossTTSDelayModel(MossTTSDelayPreTrainedModel):
         text_temperature: float = 1.5,
         text_top_p: float = 1.0,
         text_top_k: int = 50,
-        audio_temperature: float = 1.5,
+        audio_temperature: float = 1.7,
         audio_top_p: float = 0.8,
-        audio_top_k: int = 50,
+        audio_top_k: int = 25,
         audio_repetition_penalty: float = 1.0,
     ):
         if text_temperature > 0:
@@ -495,11 +495,21 @@ class MossTTSDelayModel(MossTTSDelayPreTrainedModel):
             next_audio_tokens[~sampling_audio_mask] = self.config.audio_pad_code
             
             if sampling_audio_mask.sum() > 0:
-                audio_logits = torch.stack(next_token_logits[1:], dim=1)[sampling_audio_mask] # torch.stack -> [batch_size, n_vq - 1, vocab_size]
+                audio_ch0_logits = next_token_logits[1][sampling_audio_mask[:, 0]]
+                audio_logits = torch.stack(next_token_logits[2:], dim=1)[sampling_audio_mask[:, 1:]]
+                audio_ch0_logits[..., self.config.audio_pad_code] = float('-inf')
                 audio_logits[..., self.config.audio_pad_code] = float('-inf')
-                next_audio_tokens[sampling_audio_mask] = sample_token(
+                next_audio_tokens[:, 0][sampling_audio_mask[:, 0]] = sample_token(
+                    logits=audio_ch0_logits,
+                    prev_tokens=generation_ids[:, :, 1],
+                    repetition_penalty=audio_repetition_penalty,
+                    top_p=audio_top_p, 
+                    top_k=audio_top_k, 
+                    do_sample=audio_do_sample
+                )
+                next_audio_tokens[:, 1:][sampling_audio_mask[:, 1:]] = sample_token(
                     logits=audio_logits,
-                    prev_tokens=generation_ids[:, :, 1:],
+                    prev_tokens=generation_ids[:, :, 2:],
                     repetition_penalty=audio_repetition_penalty,
                     top_p=audio_top_p, 
                     top_k=audio_top_k, 

processing_moss_tts.py

@@ -88,7 +88,7 @@ class UserMessage(Message):
             reference = []
             for speaker_idx, speaker_reference in enumerate(self.reference):
                 if speaker_reference is not None:
-                    reference.append(f"[S{speaker_idx}]:\n{AUDIO_PLACEHOLDER}")
+                    reference.append(f"[S{speaker_idx+1}]:\n{AUDIO_PLACEHOLDER}")
             reference = "\n".join(reference)
             audio_codes_list = [
                 speaker_reference
@@ -548,6 +548,7 @@ class MossTTSDelayProcessor(ProcessorMixin):
         """
         if role == "user":
             audio_gen_slot_token = audio_delay_slot_token = self.audio_user_slot_token
+            truncation = False
         else:
             audio_gen_slot_token = self.audio_assistant_gen_slot_token
             audio_delay_slot_token = self.audio_assistant_delay_slot_token
@@ -895,30 +896,25 @@ class MossTTSDelayProcessor(ProcessorMixin):
             codes.transpose(0, 1).contiguous().to(device=device, dtype=torch.long)
             for codes in audio_tokens_list
         ]
-
-        if hasattr(audio_tokenizer, "batch_decode"):
-            dec = audio_tokenizer.batch_decode(codes_list)
-            audio = dec.audio  # (B, C, T)
-            audio_lengths = dec.audio_lengths  # (B,)
-        else:
-            # Fallback: pad to (NQ, B, T) + mask, then decode.
-            nq = int(codes_list[0].shape[0])
-            max_t = max(int(c.shape[1]) for c in codes_list)
-            audio_codes = torch.zeros(
-                nq, len(codes_list), max_t, device=device, dtype=torch.long
-            )
-            padding_mask = torch.zeros(
-                len(codes_list), max_t, device=device, dtype=torch.bool
-            )
-            for i, c in enumerate(codes_list):
-                t = int(c.shape[1])
-                audio_codes[:, i, :t] = c
-                padding_mask[i, :t] = True
-            dec = audio_tokenizer.decode(
-                audio_codes, padding_mask=padding_mask, return_dict=True
-            )
-            audio = dec.audio
-            audio_lengths = dec.audio_lengths
+    
+        # Fallback: pad to (NQ, B, T) + mask, then decode.
+        nq = int(codes_list[0].shape[0])
+        max_t = max(int(c.shape[1]) for c in codes_list)
+        audio_codes = torch.zeros(
+            nq, len(codes_list), max_t, device=device, dtype=torch.long
+        )
+        padding_mask = torch.zeros(
+            len(codes_list), max_t, device=device, dtype=torch.bool
+        )
+        for i, c in enumerate(codes_list):
+            t = int(c.shape[1])
+            audio_codes[:, i, :t] = c
+            padding_mask[i, :t] = True
+        dec = audio_tokenizer.decode(
+            audio_codes, padding_mask=padding_mask, return_dict=True, chunk_duration=8
+        )
+        audio = dec.audio
+        audio_lengths = dec.audio_lengths
 
         if audio is None or audio_lengths is None:
             raise RuntimeError(