Replace DJ intro TTS with MMS models

README.md

@@ -11,7 +11,10 @@ pinned: false
 license: mit
 models:
   - ACE-Step/Ace-Step1.5
-  - hexgrad/Kokoro-82M
+  - facebook/mms-tts-eng
+  - facebook/mms-tts-deu
+  - facebook/mms-tts-fra
+  - facebook/mms-tts-spa
   - CohereLabs/tiny-aya-global
 tags:
   - build-small
@@ -37,7 +40,7 @@ Turntable Time Machine is a whimsical Gradio app built for the Hugging Face x Gr
 
 *The twist:* this is not a famous-song remix machine 🚫🎤. The app uses broad musical-era vocabularies and safety constraints to make new little audio artifacts: 1960s soul through a 1990s house lens 🪩, 1980s synth-pop polished into 2000s filtered disco house 💿, or a 2020s lo-fi signal drifting backward through old radio static 📡.
 
-This submission targets the **Thousand Token Wood** track 🪵: playful, compact, AI-native, and built around small/open model targets. ACE-Step, Kokoro, and Tiny Aya are used as optional model hooks, while deterministic templates and fallback audio keep the app demonstrable when model access or GPU runtime is limited.
+This submission targets the **Thousand Token Wood** track 🪵: playful, compact, AI-native, and built around small/open model targets. ACE-Step, MMS TTS, and Tiny Aya are used as optional model hooks, while deterministic templates and fallback audio keep the app demonstrable when model access or GPU runtime is limited.
 
 ## 🔗 Submission Links
 
@@ -60,7 +63,7 @@ A concise demo flow:
 8. 🌈 Keep **nostalgic and warm**, choose **Clean digital master**, and use the **10 second** duration.
 9. ⚡ Click **Bend the Timeline**.
 10. ▶️ Play the **Final broadcast**, then compare it with **Music only**.
-11. 🎙️ Show the **Fictional DJ intro** text card. If Kokoro TTS is unavailable, note that the app shows the intro as text instead of pretending a tone is spoken word.
+11. 🎙️ Show the **Fictional DJ intro** text card and the **Spoken DJ intro** audio when MMS TTS is available.
 12. ✍️ Show the **Generated micro-lyrics** card and the **Mixtape Card** with the route, model status, BPM, seed, texture, and safety note.
 
 🎬 Recommended one-video case: **1960s Folk pop -> 2000s indie rock** with **The Warehouse Announcer**, **French**, **Original micro-lyrics**, and **English + broadcast language**. It shows the app's full loop: era routing, fictional host text, multilingual prompt routing, generated micro-lyrics, safety-aware prompt, texture processing, and final audio.
@@ -93,7 +96,7 @@ User selections
 → BPM selection from genre ranges
 → optional Tiny Aya micro-lyrics
 → safety-aware music prompt
-→ optional Kokoro DJ narration
+→ optional MMS TTS DJ narration
 → ACE-Step music hook or fallback synth audio
 → nostalgia texture processing
 → final broadcast and mixtape card
@@ -106,11 +109,11 @@ User selections
 | Component | Model | Parameters | Used for |
 | --- | --- | ---: | --- |
 | Music generation target | [ACE-Step/Ace-Step1.5](https://huggingface.co/ACE-Step/Ace-Step1.5) | under 32B target stack | Original short music clips |
-| Spoken DJ intro target | [hexgrad/Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) | 82M | Optional fictional DJ narration |
+| Spoken DJ intro target | [facebook/mms-tts-eng](https://huggingface.co/facebook/mms-tts-eng), `deu`, `fra`, `spa` variants | 36.3M each for English model card | Optional fictional DJ narration in the selected broadcast language |
 | Text / micro-lyrics target | [CohereLabs/tiny-aya-global](https://huggingface.co/CohereLabs/tiny-aya-global) | small multilingual model | Optional two-line original lyrics |
 | Template and synth fallback | none | 0 | Keeps the app usable in restricted runtimes |
 
-ACE-Step and Kokoro are lazy optional hooks. If model packages, weights, or hardware are unavailable, the app falls back to playable synthetic demo audio and text-only DJ intros. Tiny Aya falls back to deterministic micro-lyrics when access is restricted.
+ACE-Step and MMS TTS are lazy optional hooks. If model packages, weights, or hardware are unavailable, the app falls back to playable synthetic demo audio and text-only DJ intros. Tiny Aya falls back to deterministic micro-lyrics when access is restricted.
 
 ## 🗂️ Data
 

app.py

@@ -383,7 +383,7 @@ with gr.Blocks(title="Turntable Time Machine", css=CSS) as demo:
                         final_audio = gr.Audio(label="Final broadcast", elem_classes=["audio-card"])
                         music_audio = gr.Audio(label="Music only", elem_classes=["audio-card"])
                         intro_audio = gr.Audio(label="Spoken DJ intro", elem_classes=["audio-card"])
-                        intro_audio_status = gr.Markdown("Spoken DJ intro appears here when Kokoro TTS is available.")
+                        intro_audio_status = gr.Markdown("Spoken DJ intro appears here when MMS TTS is available.")
                         mixtape_card = gr.HTML("<div class='mixtape-card'><div class='mixtape-title'>Mixtape Card</div><h3>Waiting for a timeline bend.</h3></div>")
                 with gr.Accordion("Generated text and status", open=False):
                     dj_intro_text = gr.Textbox(label="Fictional DJ intro text", lines=3)

src/prompt_builder.py

@@ -143,7 +143,7 @@ def build_mixtape_card(
         ("Texture", texture.get("label")),
         ("Mood", mood),
         ("BPM / Seed", f"{bpm} / {seed}"),
-        ("Model stack", "ACE-Step 1.5 music, Kokoro-82M spoken DJ, Tiny Aya text; graceful local fallbacks"),
+        ("Model stack", "ACE-Step 1.5 music, MMS TTS spoken DJ, Tiny Aya text; graceful local fallbacks"),
         ("Status", " | ".join(statuses)),
         ("Safety", copyright_safety_note()),
     ]

src/tts.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+from functools import lru_cache
 from pathlib import Path
 import os
 
@@ -8,6 +9,23 @@ import numpy as np
 from .audio_utils import save_audio
 
 
+MMS_TTS_MODEL_IDS = {
+    "en": "facebook/mms-tts-eng",
+    "de": "facebook/mms-tts-deu",
+    "fr": "facebook/mms-tts-fra",
+    "es": "facebook/mms-tts-spa",
+}
+DEFAULT_TTS_MODEL_ID = MMS_TTS_MODEL_IDS["en"]
+
+
+@lru_cache(maxsize=4)
+def load_tts_pipeline(model_id: str):
+    from transformers import pipeline
+
+    token = os.getenv("HF_TOKEN") or os.getenv("HUGGING_FACE_HUB_TOKEN")
+    return pipeline("text-to-speech", model=model_id, token=token)
+
+
 def generate_dj_voice(
     text: str,
     language_id: str,
@@ -15,7 +33,11 @@ def generate_dj_voice(
     output_path: str = "outputs/dj_intro.wav",
 ) -> dict:
     Path(output_path).parent.mkdir(parents=True, exist_ok=True)
+    model_id = os.getenv("TTS_MODEL_ID") or MMS_TTS_MODEL_IDS.get(language_id, DEFAULT_TTS_MODEL_ID)
     try:
+        if os.getenv("TTM_DISABLE_TTS_MODEL") == "1":
+            raise RuntimeError("TTS model disabled by TTM_DISABLE_TTS_MODEL.")
+
         if os.getenv("TTM_TEST_MODEL_CALLS") == "1":
             sr = 24000
             duration = min(2.0, max(0.4, len(text.split()) * 0.08))
@@ -25,23 +47,31 @@ def generate_dj_voice(
             return {
                 "path": output_path,
                 "sample_rate": sr,
-                "status": "Kokoro spoken-DJ model call simulated for tests.",
+                "status": f"MMS TTS spoken-DJ model call simulated for tests with {model_id}.",
                 "tts_succeeded": True,
                 "model_attempted": True,
-                "model_id": "hexgrad/Kokoro-82M",
+                "model_id": model_id,
             }
 
-        # Kokoro is optional and lazily imported. This app never clones or imitates
-        # voices; the text is only fictional DJ narration.
-        import kokoro  # type: ignore  # noqa: F401
-
-        raise RuntimeError("Kokoro wrapper hook is present, but local TTS inference is not configured.")
+        generator = load_tts_pipeline(model_id)
+        speech = generator(text[:420])
+        audio = np.asarray(speech["audio"], dtype=np.float32).squeeze()
+        sample_rate = int(speech["sampling_rate"])
+        save_audio(output_path, audio, sample_rate)
+        return {
+            "path": output_path,
+            "sample_rate": sample_rate,
+            "status": f"Spoken DJ intro generated with {model_id}.",
+            "tts_succeeded": True,
+            "model_attempted": True,
+            "model_id": model_id,
+        }
     except Exception as exc:
         return {
             "path": None,
             "sample_rate": None,
-            "status": f"Spoken DJ intro unavailable; showing text intro only. Kokoro TTS unavailable or unsupported here: {exc}",
+            "status": f"Spoken DJ intro unavailable; showing text intro only. MMS TTS unavailable or unsupported here: {exc}",
             "tts_succeeded": False,
             "model_attempted": True,
-            "model_id": "hexgrad/Kokoro-82M",
+            "model_id": model_id,
         }

tests/test_app_logic.py

@@ -17,6 +17,7 @@ from src.tts import generate_dj_voice
 class TurntableTimeMachineLogicTests(unittest.TestCase):
     def tearDown(self):
         os.environ.pop("TTM_TEST_MODEL_CALLS", None)
+        os.environ.pop("TTM_DISABLE_TTS_MODEL", None)
         patterns = [
             "outputs/test_*.wav",
             "outputs/final_mix_*.wav",
@@ -148,9 +149,22 @@ class TurntableTimeMachineLogicTests(unittest.TestCase):
         self.assertFalse(lyrics["fallback_used"])
         self.assertIn("simulated for tests", music["status"])
         self.assertIn("simulated for tests", tts["status"])
+        self.assertEqual(tts["model_id"], "facebook/mms-tts-eng")
         self.assertIn("simulated for tests", lyrics["status"])
 
+    def test_tts_model_routes_by_language(self):
+        os.environ["TTM_TEST_MODEL_CALLS"] = "1"
+        tts = generate_dj_voice(
+            "Bonsoir, voyageurs du temps.",
+            "fr",
+            "warehouse_announcer",
+            "outputs/test_model_tts_fr.wav",
+        )
+        self.assertTrue(tts["tts_succeeded"])
+        self.assertEqual(tts["model_id"], "facebook/mms-tts-fra")
+
     def test_generate_pipeline_all_vocal_modes_create_final_audio(self):
+        os.environ["TTM_DISABLE_TTS_MODEL"] = "1"
         for index, vocal_mode in enumerate(app.VOCAL_MODES):
             with self.subTest(vocal_mode=vocal_mode):
                 result = app.generate_time_machine_mix(
@@ -231,7 +245,7 @@ class TurntableTimeMachineLogicTests(unittest.TestCase):
         self.assertTrue(Path(intro_path).exists())
         self.assertEqual(intro_status, "Spoken DJ intro generated.")
         self.assertIn("ACE-Step model call simulated", status)
-        self.assertIn("Kokoro spoken-DJ model call simulated", status)
+        self.assertIn("MMS TTS spoken-DJ model call simulated", status)
         self.assertIn("tiny-aya-global text model call simulated", status)
         self.assertIn("Lyrics:", prompt)
         self.assertGreaterEqual(len(lyrics_text.splitlines()), 2)