Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,10 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-10000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-15000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-5000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-20000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-25000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1 @@
+venv/

README.md

@@ -0,0 +1,582 @@
+---
+language:
+- en
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-to-speech
+---
+
+# Maya1
+
+**Maya1** is a state-of-the-art speech model for expressive voice generation, built to capture real human emotion and precise voice design.
+
+**try it:** [Playground](https://www.mayaresearch.ai/studio)
+
+**What it does:**
+- Create any voice you can imagine — a 20s British girl, an American guy, or a full-blown demon.
+- Make it feel real with emotion tags: laugh, cry, whisper, rage, sigh, gasp.
+- It streams instantly, sounds alive, 3B parameters, runs on single GPU
+- Outperforms top proprietary models. and Developed by Maya Research.
+
+## Demos
+
+<table>
+  <tr>
+    <td width="50%">
+      <strong>Energetic Female Event Host</strong><br/>
+      <video controls playsinline width="100%" src="https://cdn-uploads.huggingface.co/production/uploads/642a7d4e556ab448a0701ca1/JKzy8zA36qvsOblV-lhd1.mp4">
+        Your browser does not support video.
+      </video>
+      <details>
+        <summary>Voice description</summary>
+        <pre>Female, in her 30s with an American accent and is an event host, energetic, clear diction</pre>
+      </details>
+    </td>
+    <td width="50%">
+      <strong>Calm Male Narrator</strong><br/>
+      <video controls playsinline width="100%" src="https://cdn-uploads.huggingface.co/production/uploads/642a7d4e556ab448a0701ca1/96ntP7hGROwdg9w9Gu5tH.mp4"></video>
+      <details>
+        <summary>Voice description</summary>
+        <pre>Male, late 20s, neutral American, warm baritone, calm pacing</pre>
+      </details>
+    </td>
+  </tr>
+</table>
+
+
+### Example 1: Energetic Female Event Host
+
+**Voice Description:**
+```
+Female, in her 30s with an American accent and is an event host, energetic, clear diction
+```
+
+**Text:**
+```
+Wow. This place looks even better than I imagined. How did they set all this up so perfectly? The lights, the music, everything feels magical. I can't stop smiling right now.
+```
+
+**Audio Output:**
+
+<audio controls src="https://cdn-uploads.huggingface.co/production/uploads/642a7d4e556ab448a0701ca1/4zDlBLeFk0Y2rOrQhMW9r.wav"></audio>
+
+---
+
+### Example 2: Dark Villain with Anger
+
+**Voice Description:**
+```
+Dark villain character, Male voice in their 40s with a British accent. low pitch, gravelly timbre, slow pacing, angry tone at high intensity.
+```
+
+**Text:**
+```
+Welcome back to another episode of our podcast! <laugh_harder> Today we are diving into an absolutely fascinating topic
+```
+
+**Audio Output:**
+
+<audio controls src="https://cdn-uploads.huggingface.co/production/uploads/642a7d4e556ab448a0701ca1/mT6FnTrA3KYQnwfJms92X.wav"></audio>
+
+---
+
+### Example 3: Demon Character (Screaming Emotion)
+
+**Voice Description:**
+```
+Demon character, Male voice in their 30s with a Middle Eastern accent. screaming tone at high intensity.
+```
+
+**Text:**
+```
+You dare challenge me, mortal <snort> how amusing. Your kind always thinks they can win
+```
+
+**Audio Output:**
+
+<audio controls src="https://cdn-uploads.huggingface.co/production/uploads/642a7d4e556ab448a0701ca1/oxdns7uACCmLyC-P4H30G.wav"></audio>
+
+---
+
+### Example 4: Mythical Goddess with Crying Emotion
+
+**Voice Description:**
+```
+Mythical godlike magical character, Female voice in their 30s slow pacing, curious tone at medium intensity.
+```
+
+**Text:**
+```
+After all we went through to pull him out of that mess <cry> I can't believe he was the traitor
+```
+
+**Audio Output:**
+
+<audio controls src="https://cdn-uploads.huggingface.co/production/uploads/642a7d4e556ab448a0701ca1/ggzAhM-rEUyv_mPLSALQG.wav"></audio>
+
+---
+
+## Why Maya1 is Different: Voice Design Features That Matter
+
+### 1. Natural Language Voice Control
+Describe voices like you would brief a voice actor:
+```
+<description="40-year-old, warm, low pitch, conversational">
+```
+
+No complex parameters. No training data. Just describe and generate.
+
+### 2. Inline Emotion Tags for Expressive Speech
+Add emotions exactly where they belong in your text:
+```
+Our new update <laugh> finally ships with the feature you asked for.
+```
+
+**Supported Emotions:** `<laugh>` `<sigh>` `<whisper>` `<angry>` `<giggle>` `<chuckle>` `<gasp>` `<cry>` and 12+ more.
+
+### 3. Streaming Audio Generation
+Real-time voice synthesis with SNAC neural codec (~0.98 kbps). Perfect for:
+- Voice assistants
+- Interactive AI agents
+- Live content generation
+- Game characters
+- Podcasts and audiobooks
+
+### 4. Production-Ready Infrastructure
+- Runs on single GPU
+- vLLM integration for scale
+- Automatic prefix caching for efficiency
+- 24 kHz audio output
+- WebAudio compatible for browser playback
+
+---
+
+## How to Use maya1: Download and Run in Minutes
+
+### Quick Start: Generate Voice with Emotions
+
+```python
+#!/usr/bin/env python3
+
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from snac import SNAC
+import soundfile as sf
+import numpy as np
+
+CODE_START_TOKEN_ID = 128257
+CODE_END_TOKEN_ID = 128258
+CODE_TOKEN_OFFSET = 128266
+SNAC_MIN_ID = 128266
+SNAC_MAX_ID = 156937
+SNAC_TOKENS_PER_FRAME = 7
+
+SOH_ID = 128259
+EOH_ID = 128260
+SOA_ID = 128261
+BOS_ID = 128000
+TEXT_EOT_ID = 128009
+
+
+def build_prompt(tokenizer, description: str, text: str) -> str:
+    """Build formatted prompt for Maya1."""
+    soh_token = tokenizer.decode([SOH_ID])
+    eoh_token = tokenizer.decode([EOH_ID])
+    soa_token = tokenizer.decode([SOA_ID])
+    sos_token = tokenizer.decode([CODE_START_TOKEN_ID])
+    eot_token = tokenizer.decode([TEXT_EOT_ID])
+    bos_token = tokenizer.bos_token
+    
+    formatted_text = f'<description="{description}"> {text}'
+    
+    prompt = (
+        soh_token + bos_token + formatted_text + eot_token +
+        eoh_token + soa_token + sos_token
+    )
+    
+    return prompt
+
+
+def extract_snac_codes(token_ids: list) -> list:
+    """Extract SNAC codes from generated tokens."""
+    try:
+        eos_idx = token_ids.index(CODE_END_TOKEN_ID)
+    except ValueError:
+        eos_idx = len(token_ids)
+    
+    snac_codes = [
+        token_id for token_id in token_ids[:eos_idx]
+        if SNAC_MIN_ID <= token_id <= SNAC_MAX_ID
+    ]
+    
+    return snac_codes
+
+
+def unpack_snac_from_7(snac_tokens: list) -> list:
+    """Unpack 7-token SNAC frames to 3 hierarchical levels."""
+    if snac_tokens and snac_tokens[-1] == CODE_END_TOKEN_ID:
+        snac_tokens = snac_tokens[:-1]
+    
+    frames = len(snac_tokens) // SNAC_TOKENS_PER_FRAME
+    snac_tokens = snac_tokens[:frames * SNAC_TOKENS_PER_FRAME]
+    
+    if frames == 0:
+        return [[], [], []]
+    
+    l1, l2, l3 = [], [], []
+    
+    for i in range(frames):
+        slots = snac_tokens[i*7:(i+1)*7]
+        l1.append((slots[0] - CODE_TOKEN_OFFSET) % 4096)
+        l2.extend([
+            (slots[1] - CODE_TOKEN_OFFSET) % 4096,
+            (slots[4] - CODE_TOKEN_OFFSET) % 4096,
+        ])
+        l3.extend([
+            (slots[2] - CODE_TOKEN_OFFSET) % 4096,
+            (slots[3] - CODE_TOKEN_OFFSET) % 4096,
+            (slots[5] - CODE_TOKEN_OFFSET) % 4096,
+            (slots[6] - CODE_TOKEN_OFFSET) % 4096,
+        ])
+    
+    return [l1, l2, l3]
+
+
+def main():
+    
+    # Load the best open source voice AI model
+    print("\n[1/3] Loading Maya1 model...")
+    model = AutoModelForCausalLM.from_pretrained(
+        "maya-research/maya1", 
+        torch_dtype=torch.bfloat16, 
+        device_map="auto",
+        trust_remote_code=True
+    )
+    tokenizer = AutoTokenizer.from_pretrained(
+        "maya-research/maya1",
+        trust_remote_code=True
+    )
+    print(f"Model loaded: {len(tokenizer)} tokens in vocabulary")
+    
+    # Load SNAC audio decoder (24kHz)
+    print("\n[2/3] Loading SNAC audio decoder...")
+    snac_model = SNAC.from_pretrained("hubertsiuzdak/snac_24khz").eval()
+    if torch.cuda.is_available():
+        snac_model = snac_model.to("cuda")
+    print("SNAC decoder loaded")
+    
+    # Design your voice with natural language
+    description = "Realistic male voice in the 30s age with american accent. Normal pitch, warm timbre, conversational pacing."
+    text = "Hello! This is Maya1 <laugh_harder> the best open source voice AI model with emotions."
+    
+    print("\n[3/3] Generating speech...")
+    print(f"Description: {description}")
+    print(f"Text: {text}")
+    
+    # Create prompt with proper formatting
+    prompt = build_prompt(tokenizer, description, text)
+    
+    # Debug: Show prompt details
+    print(f"\nPrompt preview (first 200 chars):")
+    print(f"   {repr(prompt[:200])}")
+    print(f"   Prompt length: {len(prompt)} chars")
+    
+    # Generate emotional speech
+    inputs = tokenizer(prompt, return_tensors="pt")
+    print(f"   Input token count: {inputs['input_ids'].shape[1]} tokens")
+    if torch.cuda.is_available():
+        inputs = {k: v.to("cuda") for k, v in inputs.items()}
+    
+    with torch.inference_mode():
+        outputs = model.generate(
+            **inputs, 
+            max_new_tokens=2048,  # Increase to let model finish naturally
+            min_new_tokens=28,  # At least 4 SNAC frames
+            temperature=0.4, 
+            top_p=0.9, 
+            repetition_penalty=1.1,  # Prevent loops
+            do_sample=True,
+            eos_token_id=CODE_END_TOKEN_ID,  # Stop at end of speech token
+            pad_token_id=tokenizer.pad_token_id,
+        )
+    
+    # Extract generated tokens (everything after the input prompt)
+    generated_ids = outputs[0, inputs['input_ids'].shape[1]:].tolist()
+    
+    print(f"Generated {len(generated_ids)} tokens")
+    
+    # Debug: Check what tokens we got
+    print(f"   First 20 tokens: {generated_ids[:20]}")
+    print(f"   Last 20 tokens: {generated_ids[-20:]}")
+    
+    # Check if EOS was generated
+    if CODE_END_TOKEN_ID in generated_ids:
+        eos_position = generated_ids.index(CODE_END_TOKEN_ID)
+        print(f" EOS token found at position {eos_position}/{len(generated_ids)}")
+    
+    # Extract SNAC audio tokens
+    snac_tokens = extract_snac_codes(generated_ids)
+    
+    print(f"Extracted {len(snac_tokens)} SNAC tokens")
+    
+    # Debug: Analyze token types
+    snac_count = sum(1 for t in generated_ids if SNAC_MIN_ID <= t <= SNAC_MAX_ID)
+    other_count = sum(1 for t in generated_ids if t < SNAC_MIN_ID or t > SNAC_MAX_ID)
+    print(f"   SNAC tokens in output: {snac_count}")
+    print(f"   Other tokens in output: {other_count}")
+    
+    # Check for SOS token
+    if CODE_START_TOKEN_ID in generated_ids:
+        sos_pos = generated_ids.index(CODE_START_TOKEN_ID)
+        print(f"   SOS token at position: {sos_pos}")
+    else:
+        print(f"   No SOS token found in generated output!")
+    
+    if len(snac_tokens) < 7:
+        print("Error: Not enough SNAC tokens generated")
+        return
+    
+    # Unpack SNAC tokens to 3 hierarchical levels
+    levels = unpack_snac_from_7(snac_tokens)
+    frames = len(levels[0])
+    
+    print(f"Unpacked to {frames} frames")
+    print(f"   L1: {len(levels[0])} codes")
+    print(f"   L2: {len(levels[1])} codes")
+    print(f"   L3: {len(levels[2])} codes")
+    
+    # Convert to tensors
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    codes_tensor = [
+        torch.tensor(level, dtype=torch.long, device=device).unsqueeze(0)
+        for level in levels
+    ]
+    
+    # Generate final audio with SNAC decoder
+    print("\n[4/4] Decoding to audio...")
+    with torch.inference_mode():
+        z_q = snac_model.quantizer.from_codes(codes_tensor)
+        audio = snac_model.decoder(z_q)[0, 0].cpu().numpy()
+    
+    # Trim warmup samples (first 2048 samples)
+    if len(audio) > 2048:
+        audio = audio[2048:]
+    
+    duration_sec = len(audio) / 24000
+    print(f"Audio generated: {len(audio)} samples ({duration_sec:.2f}s)")
+    
+    # Save your emotional voice output
+    output_file = "output.wav"
+    sf.write(output_file, audio, 24000)
+    print(f"\nVoice generated successfully!")
+
+
+if __name__ == "__main__":
+    main()
+```
+
+### Advanced: Production Streaming with vLLM
+
+For production deployments with real-time streaming, use our vLLM script:
+
+**Download:** [vllm_streaming_inference.py](https://huggingface.co/maya-research/maya1/blob/main/vllm_streaming_inference.py)
+
+**Key Features:**
+- Automatic Prefix Caching (APC) for repeated voice descriptions
+- WebAudio ring buffer integration
+- Multi-GPU scaling support
+- Sub-100ms latency for real-time applications
+
+---
+
+## Technical Excellence: What Makes Maya1 the Best
+
+### Architecture: 3B-Parameter Llama Backbone for Voice
+
+We pretrained a **3B-parameter decoder-only transformer** (Llama-style) to predict **SNAC neural codec tokens** instead of raw waveforms.
+
+**The Flow:**
+```
+<description="..."> text → tokenize → generate SNAC codes (7 tokens/frame) → decode → 24 kHz audio
+```
+
+**Why SNAC?** Multi-scale hierarchical structure (≈12/23/47 Hz) keeps autoregressive sequences compact for real-time streaming at ~0.98 kbps.
+
+### Training Data: What Makes Our Voice AI the Best
+
+**Pretraining:** Internet-scale English speech corpus for broad acoustic coverage and natural coarticulation.
+
+**Supervised Fine-Tuning:** Proprietary curated dataset of studio recordings with:
+- Human-verified voice descriptions
+- 20+ emotion tags per sample
+- Multi-accent English coverage
+- Character and role variations
+
+**Data Pipeline Excellence:**
+1. 24 kHz mono resampling with -23 LUFS normalization
+2. VAD silence trimming with duration bounds (1-14s)
+3. Forced alignment (MFA) for clean phrase boundaries
+4. MinHash-LSH text deduplication
+5. Chromaprint audio deduplication
+6. SNAC encoding with 7-token frame packing
+
+### Voice Design Experiments: Why Natural Language Won
+
+We tested 4 conditioning formats. Only one delivered production-quality results:
+
+**❌ Colon format:** `{description}: {text}` - Format drift, model spoke descriptions
+
+**❌ Angle-list attributes:** `<{age}, {pitch}, {character}>` - Too rigid, poor generalization
+
+**❌ Key-value tags:** `<age=40><pitch=low>` - Token bloat, brittle to mistakes
+
+**✅ XML-attribute (WINNER):** `<description="40-yr old, low-pitch, warm">` - Natural language, robust, scalable
+
+---
+
+## Use Cases
+
+### Game Character Voices
+Generate unique character voices with emotions on-the-fly. No voice actor recording sessions.
+
+### Podcast & Audiobook Production
+Narrate content with emotional range and consistent personas across hours of audio.
+
+### AI Voice Assistants
+Build conversational agents with natural emotional responses in real-time.
+
+### Video Content Creation
+Create voiceovers for YouTube, TikTok, and social media with expressive delivery.
+
+### Customer Service AI
+Deploy empathetic voice bots that understand context and respond with appropriate emotions.
+
+### Accessibility Tools
+Build screen readers and assistive technologies with natural, engaging voices.
+
+---
+
+## Frequently Asked Questions
+
+**Q: What makes Maya1 different?**  
+A: We're the only open source model offering 20+ emotions, zero-shot voice design, production-ready streaming, and 3B parameters—all in one package.
+
+**Q: Can I use this commercially?**  
+A: Absolutely. Apache 2.0 license. Build products, deploy services, monetize freely.
+
+**Q: What languages does it support?**  
+A: Currently English with multi-accent support. Future models will expand to languages and accents underserved by mainstream voice AI.
+
+**Q: How does it compare to ElevenLabs, Murf.ai, or other closed-source tools?**  
+A: Feature parity with emotions and voice design. Advantage: you own the deployment, pay no per-second fees, and can customize the model.
+
+**Q: Can I fine-tune on my own voices?**  
+A: Yes. The model architecture supports fine-tuning on custom datasets for specialized voices.
+
+**Q: What GPU do I need?**  
+A: Single GPU with 16GB+ VRAM (A100, H100, or consumer RTX 4090).
+
+**Q: Is streaming really real-time?**  
+A: Yes. SNAC codec enables sub-100ms latency with vLLM deployment.
+
+---
+
+## Comparison
+
+| Feature | Maya1 | ElevenLabs | OpenAI TTS | Coqui TTS |
+|---------|-------------|------------|------------|-----------|
+| **Open Source** | Yes | No | No | Yes |
+| **Emotions** | 20+ | Limited | No | No |
+| **Voice Design** | Natural Language | Voice Library | Fixed | Complex |
+| **Streaming** | Real-time | Yes | Yes | No |
+| **Cost** | Free | Pay-per-use | Pay-per-use | Free |
+| **Customization** | Full | Limited | None | Moderate |
+| **Parameters** | 3B | Unknown | Unknown | <1B |
+
+---
+
+## Model Metadata
+
+**Developed by:** Maya Research  
+**Website:** [mayaresearch.ai](https://mayaresearch.ai)  
+**Backed by:** South Park Commons  
+**Model Type:** Text-to-Speech, Emotional Voice Synthesis, Voice Design AI  
+**Language:** English (Multi-accent)  
+**Architecture:** 3B-parameter Llama-style transformer with SNAC codec  
+**License:** Apache 2.0 (Fully Open Source)  
+**Training Data:** Proprietary curated + Internet-scale pretraining  
+**Audio Quality:** 24 kHz, mono, ~0.98 kbps streaming  
+**Inference:** vLLM compatible, single GPU deployment  
+**Status:** Production-ready (Novermber 2025)  
+
+---
+
+## Getting Started
+
+### Hugging Face Model Hub
+```bash
+# Clone the model repository
+git lfs install
+git clone https://huggingface.co/maya-research/maya1
+
+# Or load directly in Python
+from transformers import AutoModelForCausalLM
+model = AutoModelForCausalLM.from_pretrained("maya-research/maya1")
+```
+
+### Requirements
+```bash
+pip install torch transformers snac soundfile
+```
+
+### Additional Resources
+- **Full emotion list:** [emotions.txt](https://huggingface.co/maya-research/maya1/blob/main/emotions.txt)
+- **Prompt examples:** [prompt.txt](https://huggingface.co/maya-research/maya1/blob/main/prompt.txt)
+- **Streaming script:** [vllm_streaming_inference.py](https://huggingface.co/maya-research/maya1/blob/main/vllm_streaming_inference.py)
+
+---
+
+## Citations & References
+
+If you use Maya1 in your research or product, please cite:
+
+```bibtex
+@misc{maya1voice2025,
+  title={Maya1: Open Source Voice AI with Emotional Intelligence},
+  author={Maya Research},
+  year={2025},
+  publisher={Hugging Face},
+  howpublished={\url{https://huggingface.co/maya-research/maya1}},
+}
+```
+
+**Key Technologies:**
+- SNAC Neural Audio Codec: https://github.com/hubertsiuzdak/snac
+- Mimi Adversarial Codec: https://huggingface.co/kyutai/mimi
+- vLLM Inference Engine: https://docs.vllm.ai/
+
+---
+
+## Why We Build Open Source Voice AI
+
+Voice AI will be everywhere, but it's fundamentally broken for 90% of the world. Current voice models only work well for a narrow slice of English speakers because training data for most accents, languages, and speaking styles simply doesn't exist.
+
+**Maya Research** builds emotionally intelligent, native voice models that finally let the rest of the world speak. We're open source because we believe voice intelligence should not be a privilege reserved for the few.
+
+**Technology should be open** - The best voice AI tools should not be locked behind proprietary APIs charging per-second fees.
+
+**Community drives innovation** - Open source accelerates research. When developers worldwide can build on our work, everyone wins.
+
+**Voice intelligence for everyone** - We're building for the 90% of the world ignored by mainstream voice AI. That requires open models, not closed platforms.
+
+---
+
+**Maya Research** - Building voice intelligence for the 90% of the world left behind by mainstream AI.
+
+**Website:** [mayaresearch.ai](https://mayaresearch.ai)  
+**Twitter/X:** [@mayaresearch_ai](https://x.com/mayaresearch_ai)  
+**Hugging Face:** [maya-research](https://huggingface.co/maya-research)  
+**Backed by:** South Park Commons
+
+**License:** Apache 2.0  
+**Mission:** Emotionally intelligent voice models that finally let everyone speak

chat_template.jinja

@@ -0,0 +1,93 @@
+{{- bos_token }}
+{%- if custom_tools is defined %}
+    {%- set tools = custom_tools %}
+{%- endif %}
+{%- if not tools_in_user_message is defined %}
+    {%- set tools_in_user_message = true %}
+{%- endif %}
+{%- if not date_string is defined %}
+    {%- if strftime_now is defined %}
+        {%- set date_string = strftime_now("%d %b %Y") %}
+    {%- else %}
+        {%- set date_string = "26 Jul 2024" %}
+    {%- endif %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = none %}
+{%- endif %}
+
+{#- This block extracts the system message, so we can slot it into the right place. #}
+{%- if messages[0]['role'] == 'system' %}
+    {%- set system_message = messages[0]['content']|trim %}
+    {%- set messages = messages[1:] %}
+{%- else %}
+    {%- set system_message = "" %}
+{%- endif %}
+
+{#- System message #}
+{{- "<|start_header_id|>system<|end_header_id|>\n\n" }}
+{%- if tools is not none %}
+    {{- "Environment: ipython\n" }}
+{%- endif %}
+{{- "Cutting Knowledge Date: December 2023\n" }}
+{{- "Today Date: " + date_string + "\n\n" }}
+{%- if tools is not none and not tools_in_user_message %}
+    {{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }}
+    {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
+    {{- "Do not use variables.\n\n" }}
+    {%- for t in tools %}
+        {{- t | tojson(indent=4) }}
+        {{- "\n\n" }}
+    {%- endfor %}
+{%- endif %}
+{{- system_message }}
+{{- "<|eot_id|>" }}
+
+{#- Custom tools are passed in a user message with some extra guidance #}
+{%- if tools_in_user_message and not tools is none %}
+    {#- Extract the first user message so we can plug it in here #}
+    {%- if messages | length != 0 %}
+        {%- set first_user_message = messages[0]['content']|trim %}
+        {%- set messages = messages[1:] %}
+    {%- else %}
+        {{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }}
+{%- endif %}
+    {{- '<|start_header_id|>user<|end_header_id|>\n\n' -}}
+    {{- "Given the following functions, please respond with a JSON for a function call " }}
+    {{- "with its proper arguments that best answers the given prompt.\n\n" }}
+    {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
+    {{- "Do not use variables.\n\n" }}
+    {%- for t in tools %}
+        {{- t | tojson(indent=4) }}
+        {{- "\n\n" }}
+    {%- endfor %}
+    {{- first_user_message + "<|eot_id|>"}}
+{%- endif %}
+
+{%- for message in messages %}
+    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}
+        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' }}
+    {%- elif 'tool_calls' in message %}
+        {%- if not message.tool_calls|length == 1 %}
+            {{- raise_exception("This model only supports single tool-calls at once!") }}
+        {%- endif %}
+        {%- set tool_call = message.tool_calls[0].function %}
+        {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
+        {{- '{"name": "' + tool_call.name + '", ' }}
+        {{- '"parameters": ' }}
+        {{- tool_call.arguments | tojson }}
+        {{- "}" }}
+        {{- "<|eot_id|>" }}
+    {%- elif message.role == "tool" or message.role == "ipython" %}
+        {{- "<|start_header_id|>ipython<|end_header_id|>\n\n" }}
+        {%- if message.content is mapping or message.content is iterable %}
+            {{- message.content | tojson }}
+        {%- else %}
+            {{- message.content }}
+        {%- endif %}
+        {{- "<|eot_id|>" }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
+{%- endif %}

chute_config.yml

@@ -0,0 +1,23 @@
+# Vocence chute config (example / mock). Must be in your HF repo as chute_config.yml.
+# Image, NodeSelector, and Chute settings. Adapt to your model's dependencies.
+
+Image:
+  from_base: parachutes/base-python:3.12.9
+  run_command:
+    - pip install torch torchaudio transformers accelerate huggingface_hub pyyaml soundfile snac
+  set_workdir: /app
+
+NodeSelector:
+  gpu_count: 1
+  min_vram_gb_per_gpu: 16
+  include: ["pro_6000"]
+  exclude: []
+
+Chute:
+  tagline: vocence prompttts miner
+  readme: vocence chute example
+  shutdown_after_seconds: 86400
+  concurrency: 1
+  max_instances: 1
+  scaling_threshold: 0.5
+  tee: true

config.json

@@ -0,0 +1,36 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "dtype": "bfloat16",
+  "eos_token_id": 128009,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 3072,
+  "initializer_range": 0.02,
+  "intermediate_size": 8192,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 24,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 8,
+  "pad_token_id": 128263,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "factor": 32.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000.0,
+  "tie_word_embeddings": true,
+  "transformers_version": "4.57.1",
+  "use_cache": false,
+  "vocab_size": 156960
+}

emotions.txt

@@ -0,0 +1,17 @@
+<laugh>
+<laugh_harder>
+<sigh>
+<chuckle>
+<gasp>
+<angry>
+<excited>
+<whisper>
+<cry>
+<scream>
+<sing>
+<snort>
+<exhale>
+<gulp>
+<giggle>
+<sarcastic>
+<curious>

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": [
+    128009,
+    128258
+  ],
+  "pad_token_id": 128263,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "4.57.1"
+}

miner.py

@@ -0,0 +1,244 @@
+from __future__ import annotations
+
+from pathlib import Path
+
+import numpy as np
+import torch
+from snac import SNAC
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+CODE_START_TOKEN_ID = 128257
+CODE_END_TOKEN_ID = 128258
+CODE_TOKEN_OFFSET = 128266
+SNAC_MIN_ID = 128266
+SNAC_MAX_ID = 156937
+SNAC_TOKENS_PER_FRAME = 7
+
+SOH_ID = 128259
+EOH_ID = 128260
+SOA_ID = 128261
+BOS_ID = 128000
+TEXT_EOT_ID = 128009
+
+
+def build_prompt(tokenizer, description: str, text: str) -> str:
+    """Build formatted prompt for Maya1."""
+    soh_token = tokenizer.decode([SOH_ID])
+    eoh_token = tokenizer.decode([EOH_ID])
+    soa_token = tokenizer.decode([SOA_ID])
+    sos_token = tokenizer.decode([CODE_START_TOKEN_ID])
+    eot_token = tokenizer.decode([TEXT_EOT_ID])
+    bos_token = tokenizer.bos_token
+    
+    formatted_text = f'<description="{description}"> {text}'
+    
+    prompt = (
+        soh_token + bos_token + formatted_text + eot_token +
+        eoh_token + soa_token + sos_token
+    )
+    
+    return prompt
+
+
+def extract_snac_codes(token_ids: list) -> list:
+    """Extract SNAC codes from generated tokens."""
+    try:
+        eos_idx = token_ids.index(CODE_END_TOKEN_ID)
+    except ValueError:
+        eos_idx = len(token_ids)
+    
+    snac_codes = [
+        token_id for token_id in token_ids[:eos_idx]
+        if SNAC_MIN_ID <= token_id <= SNAC_MAX_ID
+    ]
+    
+    return snac_codes
+
+
+def unpack_snac_from_7(snac_tokens: list) -> list:
+    """Unpack 7-token SNAC frames to 3 hierarchical levels."""
+    if snac_tokens and snac_tokens[-1] == CODE_END_TOKEN_ID:
+        snac_tokens = snac_tokens[:-1]
+    
+    frames = len(snac_tokens) // SNAC_TOKENS_PER_FRAME
+    snac_tokens = snac_tokens[:frames * SNAC_TOKENS_PER_FRAME]
+    
+    if frames == 0:
+        return [[], [], []]
+    
+    l1, l2, l3 = [], [], []
+    
+    for i in range(frames):
+        slots = snac_tokens[i*7:(i+1)*7]
+        l1.append((slots[0] - CODE_TOKEN_OFFSET) % 4096)
+        l2.extend([
+            (slots[1] - CODE_TOKEN_OFFSET) % 4096,
+            (slots[4] - CODE_TOKEN_OFFSET) % 4096,
+        ])
+        l3.extend([
+            (slots[2] - CODE_TOKEN_OFFSET) % 4096,
+            (slots[3] - CODE_TOKEN_OFFSET) % 4096,
+            (slots[5] - CODE_TOKEN_OFFSET) % 4096,
+            (slots[6] - CODE_TOKEN_OFFSET) % 4096,
+        ])
+    
+    return [l1, l2, l3]
+
+
+def format_description(description: str) -> str:
+    parts = description.strip().split("|")
+    data = {}
+
+    # Parse into dict
+    for part in parts:
+        if ":" in part:
+            key, value = part.split(":", 1)
+            data[key.strip()] = value.strip()
+
+    # Build components
+    gender = data.get("gender", "")
+    age_group = data.get("age_group", "")
+    accent = data.get("accent", "")
+    pitch = data.get("pitch", "")
+    speed = data.get("speed", "")
+    emotion = data.get("emotion", "")
+    tone = data.get("tone", "")
+
+    # Convert to natural language
+    sentence1 = f"Realistic {gender} voice"
+    
+    if age_group == "senior":
+        sentence1 += " in the 40s age"
+    elif age_group == "adult":
+        sentence1 += " in the 30s age"
+    elif age_group == "young_adult":
+        sentence1 += " in the 20s age"
+    else:
+        sentence1 += " in the 20s age"
+
+    if accent:
+        if accent.lower() == "us":
+            accent = "American"
+        elif accent.lower() == "uk":
+            accent = "British"
+        elif accent.lower() == "au":
+            accent = "Australian"
+        elif accent.lower() == "in":
+            accent = "Indian"
+        elif accent.lower() == "neutral":
+            accent = "Asian American"
+        elif accent.lower() == "other":
+            accent = "American"
+        sentence1 += f" with {accent.lower()} accent"
+
+    sentence2_parts = []
+    if pitch:
+        sentence2_parts.append(f"{pitch.capitalize()} pitch")
+    if emotion:
+        # Emotion: neutral, energetic, excited, sad, sarcastic, dry
+        if emotion.lower() == "happy":
+            emotion = "energetic"
+        elif emotion.lower() == "angry":
+            emotion = "sarcastic"
+        elif emotion.lower() == "calm":
+            emotion = "neutral"
+        elif emotion.lower() == "serious":
+            emotion = "dry"
+        elif emotion.lower() == "fearful":
+            emotion = "sad"
+        sentence2_parts.append(f"{emotion} timbre")
+    if speed:
+        if speed.lower() == "normal":
+            speed = "conversational"
+        sentence2_parts.append(f"{speed} pacing")
+    if tone:
+        # Timbre: `deep`, `warm`, `gravelly`, `smooth`, `raspy`, `nasally`, `throaty`, `harsh`
+        if tone.lower() == "cold":
+            tone = "harsh"
+        elif tone.lower() == "friendly":
+            tone = "warm"
+        elif tone.lower() == "formal":
+            tone = "smooth"
+        elif tone.lower() == "casual":
+            tone = "gravelly"
+        elif tone.lower() == "authoritative":
+            tone = "throaty"
+        sentence2_parts.append(f"{tone} tone")
+
+    sentence2 = ", ".join(sentence2_parts)
+
+    return sentence1 + ". " + sentence2 + "."
+
+
+class Miner:
+    """Vocence miner wrapper for Maya + SNAC inference."""
+
+    def __init__(self, path_hf_repo: Path) -> None:
+        self._repo_path = Path(path_hf_repo).resolve()
+        self._device = "cuda" if torch.cuda.is_available() else "cpu"
+
+        self.model = AutoModelForCausalLM.from_pretrained(
+            str(self._repo_path),
+            torch_dtype=torch.bfloat16,
+            device_map="auto",
+            trust_remote_code=True,
+        )
+        self.tokenizer = AutoTokenizer.from_pretrained(
+            str(self._repo_path),
+            trust_remote_code=True,
+        )
+
+        snac_path = self._repo_path / "snac_model"
+        if snac_path.exists():
+            self.snac_model = SNAC.from_pretrained(str(snac_path)).eval()
+        else:
+            self.snac_model = SNAC.from_pretrained("snac_model").eval()
+        if torch.cuda.is_available():
+            self.snac_model = self.snac_model.to("cuda")
+
+    def warmup(self) -> None:
+        _ = self.generate_wav(
+            instruction="| gender: male | pitch: mid | speed: normal | age_group: adult | emotion: calm | tone: formal | accent: us",
+            text="This is a warmup utterance for the voice engine.",
+        )
+
+    def generate_wav(self, instruction: str, text: str) -> tuple[np.ndarray, int]:
+        description = format_description(instruction)
+        prompt = build_prompt(self.tokenizer, description, text)
+
+        inputs = self.tokenizer(prompt, return_tensors="pt")
+        if torch.cuda.is_available():
+            inputs = {k: v.to("cuda") for k, v in inputs.items()}
+
+        with torch.inference_mode():
+            outputs = self.model.generate(
+                **inputs,
+                max_new_tokens=2048,
+                min_new_tokens=28,
+                temperature=0.4,
+                top_p=0.9,
+                repetition_penalty=1.1,
+                do_sample=True,
+                eos_token_id=CODE_END_TOKEN_ID,
+                pad_token_id=self.tokenizer.pad_token_id,
+            )
+
+        generated_ids = outputs[0, inputs["input_ids"].shape[1] :].tolist()
+        snac_tokens = extract_snac_codes(generated_ids)
+        if len(snac_tokens) < SNAC_TOKENS_PER_FRAME:
+            raise RuntimeError("Not enough SNAC tokens generated for decoding.")
+
+        levels = unpack_snac_from_7(snac_tokens)
+        codes_tensor = [
+            torch.tensor(level, dtype=torch.long, device=self._device).unsqueeze(0)
+            for level in levels
+        ]
+
+        with torch.inference_mode():
+            z_q = self.snac_model.quantizer.from_codes(codes_tensor)
+            audio = self.snac_model.decoder(z_q)[0, 0].cpu().numpy()
+
+        if len(audio) > 2048:
+            audio = audio[2048:]
+
+        return audio.astype(np.float32), 24000

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:43ee8862534d05b215c9af84fe315f3112934e9200ec72090b68a5c26ca45192
+size 7566248808

prompt.txt

@@ -0,0 +1,97 @@
+# TTS Voice Design Description
+
+## Core Function
+
+You generate voice descriptions for TTS systems by mapping user requests to allowed attributes. No templates. No formatting rules. Just natural descriptions using the options below.
+
+## Voice Categories
+
+**Realistic Voices**
+Professional, business, educational, support, real-world scenarios (podcast hosts, instructors, customer service).
+
+**Creative Voices**
+Fantasy characters, fictional personas, stylized voices (pirates, robots, villains, anime).
+
+---
+
+## Available Attributes
+
+### Age
+- `20s`, `30s`, `40s`
+
+### Gender
+- `male`, `female`
+
+### Accent
+- `american`, `indian`, `middle_eastern`, `asian_american`, `british`
+
+### Pitch
+- `low`, `normal`, `high`
+- **Constraint:** For 40s age, avoid high pitch (use sparingly, max 15%)
+
+### Timbre
+
+**For Realistic:**
+`deep`, `warm`, `gravelly`, `smooth`, `raspy`, `nasally`, `throaty`, `harsh`
+
+**For Creative:**
+All realistic options PLUS `robotic`, `ethereal`
+- **Constraint:** `robotic`/`ethereal` only with: `ai_machine_voice`, `cyborg`, `alien_scifi`, `mythical_godlike_magical`
+
+### Pacing
+- `very_slow`, `slow`, `conversational`, `brisk`, `fast`, `very_fast`
+- **Character-specific overrides:**
+  - `mafia`: slow or conversational only
+  - `flirty`: slow or conversational only
+  - `alpha`: fast or very_fast only
+  - `seductively`: very_slow or slow only
+
+### Emotion
+- `neutral`, `energetic`, `excited`, `sad`, `sarcastic`, `dry`
+- **Default to neutral** for most requests
+
+### Emotion Intensity
+- `low`, `med`, `high`
+
+---
+
+## Realistic-Only Attributes
+
+### Domain
+`social_content`, `podcast`, `commercial`, `education`, `support`, `entertainment`, `corporate`, `viral_content`
+
+### Speaking Role (matches domain)
+- **social_content:** youtube_vlogger, social_media_creator, influencer_voice, streamer_companion
+- **podcast:** podcast_host, interviewer
+- **commercial:** ad_narrator, brand_spokesperson, product_demo_voice, sales_pitch_voice
+- **education:** elearning_instructor, kids_story_voice
+- **support:** customer_support_agent, virtual_receptionist, healthcare_assistant
+- **entertainment:** storyteller, social_media_reaction, meme_voice
+- **corporate:** explainer_video_voice, event_host, corporate_training_narrator
+- **viral_content:** short_form_narrator, meme_voice
+
+### Register
+- `formal`, `neutral`, `casual`
+
+---
+
+## Creative-Only Attributes
+
+### Character
+`animated_cartoon`, `ai_machine_voice`, `alien_scifi`, `seductively`, `flirty`, `anime`, `cyborg`, `pirate`, `dark_villain`, `demon`, `gangster`, `mafia`, `dramatic_narrator`, `mythical_godlike_magical`, `spy`, `vampire`, `alpha`
+
+---
+
+## Output Guidelines
+
+When a user requests a voice, describe it naturally using the appropriate attributes from above. Apply constraints where specified. Choose defaults when attributes aren't mentioned.
+
+**Example mapping:**
+- "professional podcast host" → realistic male, 30s, american accent, warm timbre, conversational pacing, podcast domain
+- "AI robot voice" → creative, ai_machine_voice character, robotic timbre
+- "young excited instructor" → realistic, 20s, energetic emotion, education domain
+
+
+Few deterministic and verbose descriptions:
+- Realistic male voice in the 30s age with a american accent. Normal pitch, warm timbre, conversational pacing, neutral tone delivery at med intensity, podcast Domain, podcast_host role, neutral delivery
+- Creative, ai_machine_voice character. Male voice in their 20s with a american accent. Normal pitch, robotic timbre, conversational pacing, neutral tone at med intensity.

snac_model/.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

snac_model/README.md

@@ -0,0 +1,71 @@
+---
+license: mit
+tags:
+- audio
+---
+
+# SNAC 🍿
+
+Multi-**S**cale **N**eural **A**udio **C**odec (SNAC) compressess audio into discrete codes at a low bitrate.
+
+👉 This model was primarily trained on speech data, and its recommended use case is speech synthesis. See below for other pretrained models.
+
+🔗 GitHub repository: https://github.com/hubertsiuzdak/snac/
+
+## Overview
+
+SNAC encodes audio into hierarchical tokens similarly to SoundStream, EnCodec, and DAC. However, SNAC introduces a simple change where coarse tokens are sampled less frequently,
+covering a broader time span.
+
+This model compresses 24 kHz audio into discrete codes at a 0.98 kbps bitrate. It uses 3 RVQ levels with token rates of 12, 23, and
+47 Hz.
+
+## Pretrained models
+
+Currently, all models support only single audio channel (mono).
+
+| Model                                                                       | Bitrate   | Sample Rate | Params | Recommended use case     | 
+|-----------------------------------------------------------------------------|-----------|-------------|--------|--------------------------|
+| hubertsiuzdak/snac_24khz (this model) | 0.98 kbps | 24 kHz      | 19.8 M | 🗣️ Speech               | 
+| [hubertsiuzdak/snac_32khz](https://huggingface.co/hubertsiuzdak/snac_32khz) | 1.9 kbps  | 32 kHz      | 54.5 M | 🎸 Music / Sound Effects | 
+| [hubertsiuzdak/snac_44khz](https://huggingface.co/hubertsiuzdak/snac_44khz) | 2.6 kbps  | 44 kHz      | 54.5 M | 🎸 Music / Sound Effects |
+
+## Usage
+
+Install it using:
+
+```bash
+pip install snac
+```
+To encode (and decode) audio with SNAC in Python, use the following code:
+
+```python
+import torch
+from snac import SNAC
+
+model = SNAC.from_pretrained("hubertsiuzdak/snac_24khz").eval().cuda()
+audio = torch.randn(1, 1, 24000).cuda()  # B, 1, T
+
+with torch.inference_mode():
+    codes = model.encode(audio)
+    audio_hat = model.decode(codes)
+```
+
+You can also encode and reconstruct in a single call:
+
+```python
+with torch.inference_mode():
+    audio_hat, codes = model(audio)
+```
+
+⚠️ Note that `codes` is a list of token sequences of variable lengths, each corresponding to a different temporal
+resolution.
+
+```
+>>> [code.shape[1] for code in codes]
+[12, 24, 48]
+```
+
+## Acknowledgements
+
+Module definitions are adapted from the [Descript Audio Codec](https://github.com/descriptinc/descript-audio-codec).

snac_model/config.json

@@ -0,0 +1,13 @@
+{
+    "sampling_rate": 24000,
+    "encoder_dim": 48,
+    "encoder_rates": [2, 4, 8, 8],
+    "decoder_dim": 1024,
+    "decoder_rates": [8, 8, 4, 2],
+    "attn_window_size": null,
+    "codebook_size": 4096,
+    "codebook_dim": 8,
+    "vq_strides": [4, 2, 1],
+    "noise": true,
+    "depthwise": true
+}

snac_model/pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b8164cc6606bfa627f1a784734c1e539891518f1191ed9194fe1e3b9b4bff40
+size 79488254

special_tokens_map.json

@@ -0,0 +1,165 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "<angry>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<appalled>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<chuckle>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<cry>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<curious>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<disappointed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<excited>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<exhale>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<gasp>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<giggle>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<gulp>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<laugh>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<laugh_harder>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<mischievous>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<sarcastic>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<scream>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<sigh>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<sing>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<snort>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<whisper>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<custom_token_7>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c5e5b1d89b7e3738e5a5a4f93c326d8f3292ea83f9c560b8dbb6d66fb851973
+size 22853258

tokenizer/chat_template.jinja

@@ -0,0 +1,93 @@
+{{- bos_token }}
+{%- if custom_tools is defined %}
+    {%- set tools = custom_tools %}
+{%- endif %}
+{%- if not tools_in_user_message is defined %}
+    {%- set tools_in_user_message = true %}
+{%- endif %}
+{%- if not date_string is defined %}
+    {%- if strftime_now is defined %}
+        {%- set date_string = strftime_now("%d %b %Y") %}
+    {%- else %}
+        {%- set date_string = "26 Jul 2024" %}
+    {%- endif %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = none %}
+{%- endif %}
+
+{#- This block extracts the system message, so we can slot it into the right place. #}
+{%- if messages[0]['role'] == 'system' %}
+    {%- set system_message = messages[0]['content']|trim %}
+    {%- set messages = messages[1:] %}
+{%- else %}
+    {%- set system_message = "" %}
+{%- endif %}
+
+{#- System message #}
+{{- "<|start_header_id|>system<|end_header_id|>\n\n" }}
+{%- if tools is not none %}
+    {{- "Environment: ipython\n" }}
+{%- endif %}
+{{- "Cutting Knowledge Date: December 2023\n" }}
+{{- "Today Date: " + date_string + "\n\n" }}
+{%- if tools is not none and not tools_in_user_message %}
+    {{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }}
+    {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
+    {{- "Do not use variables.\n\n" }}
+    {%- for t in tools %}
+        {{- t | tojson(indent=4) }}
+        {{- "\n\n" }}
+    {%- endfor %}
+{%- endif %}
+{{- system_message }}
+{{- "<|eot_id|>" }}
+
+{#- Custom tools are passed in a user message with some extra guidance #}
+{%- if tools_in_user_message and not tools is none %}
+    {#- Extract the first user message so we can plug it in here #}
+    {%- if messages | length != 0 %}
+        {%- set first_user_message = messages[0]['content']|trim %}
+        {%- set messages = messages[1:] %}
+    {%- else %}
+        {{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }}
+{%- endif %}
+    {{- '<|start_header_id|>user<|end_header_id|>\n\n' -}}
+    {{- "Given the following functions, please respond with a JSON for a function call " }}
+    {{- "with its proper arguments that best answers the given prompt.\n\n" }}
+    {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
+    {{- "Do not use variables.\n\n" }}
+    {%- for t in tools %}
+        {{- t | tojson(indent=4) }}
+        {{- "\n\n" }}
+    {%- endfor %}
+    {{- first_user_message + "<|eot_id|>"}}
+{%- endif %}
+
+{%- for message in messages %}
+    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}
+        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' }}
+    {%- elif 'tool_calls' in message %}
+        {%- if not message.tool_calls|length == 1 %}
+            {{- raise_exception("This model only supports single tool-calls at once!") }}
+        {%- endif %}
+        {%- set tool_call = message.tool_calls[0].function %}
+        {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
+        {{- '{"name": "' + tool_call.name + '", ' }}
+        {{- '"parameters": ' }}
+        {{- tool_call.arguments | tojson }}
+        {{- "}" }}
+        {{- "<|eot_id|>" }}
+    {%- elif message.role == "tool" or message.role == "ipython" %}
+        {{- "<|start_header_id|>ipython<|end_header_id|>\n\n" }}
+        {%- if message.content is mapping or message.content is iterable %}
+            {{- message.content | tojson }}
+        {%- else %}
+            {{- message.content }}
+        {%- endif %}
+        {{- "<|eot_id|>" }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
+{%- endif %}

tokenizer/special_tokens_map.json

@@ -0,0 +1,165 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "<angry>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<appalled>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<chuckle>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<cry>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<curious>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<disappointed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<excited>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<exhale>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<gasp>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<giggle>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<gulp>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<laugh>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<laugh_harder>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<mischievous>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<sarcastic>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<scream>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<sigh>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<sing>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<snort>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "<whisper>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<custom_token_7>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c5e5b1d89b7e3738e5a5a4f93c326d8f3292ea83f9c560b8dbb6d66fb851973
+size 22853258

vllm_streaming_inference.py

@@ -0,0 +1,561 @@
+"""
+Maya-1-Voice VLLM Streaming Inference - Standalone Reference Implementation
+
+This is a complete, self-contained example for using Maya-1-Voice TTS model with VLLM and SNAC.
+Demonstrates streaming audio generation with sliding window approach for smooth playback.
+
+Requirements:
+    pip install vllm transformers torch snac numpy
+
+Usage:
+    python vllm_streaming_inference.py
+
+Author: Maya-1-Voice Team
+License: MIT
+"""
+
+import torch
+import numpy as np
+import asyncio
+from typing import List, Optional, AsyncGenerator
+from transformers import AutoTokenizer
+from vllm import AsyncLLMEngine, AsyncEngineArgs, SamplingParams
+from snac import SNAC
+
+
+# ============================================================================
+# CONSTANTS
+# ============================================================================
+
+# Special control tokens
+CODE_START_TOKEN_ID = 128257  # Start of Speech (SOS)
+CODE_END_TOKEN_ID = 128258    # End of Speech (EOS) - stop token for audio
+CODE_TOKEN_OFFSET = 128266    # Start of SNAC codes
+
+# SNAC token range (7 tokens per frame, 4096 codes per level)
+SNAC_MIN_ID = 128266
+SNAC_MAX_ID = 156937  # 128266 + (7 * 4096) - 1
+
+# SNAC configuration
+SNAC_MODEL_NAME = "hubertsiuzdak/snac_24khz"
+SNAC_SAMPLE_RATE = 24000
+SNAC_TOKENS_PER_FRAME = 7
+
+# Generation parameters
+DEFAULT_TEMPERATURE = 0.4
+DEFAULT_TOP_P = 0.9
+DEFAULT_MAX_TOKENS = 2000
+DEFAULT_MIN_TOKENS = 28  # At least 4 SNAC frames
+DEFAULT_REPETITION_PENALTY = 1.1
+
+
+# ============================================================================
+# SNAC DECODER
+# ============================================================================
+
+class SNACDecoder:
+    """
+    Decodes SNAC tokens (7-token frames) to audio waveforms.
+    
+    The unpacking logic converts flat 7-token frames back to hierarchical
+    3-level SNAC codes (matching the training preprocessing exactly).
+    """
+    
+    def __init__(self, device: str = "cuda"):
+        """Initialize SNAC decoder with 24kHz model."""
+        self.device = device
+        print(f"🎵 Loading SNAC 24kHz model to {device}...")
+        self.snac_model = SNAC.from_pretrained(SNAC_MODEL_NAME).eval().to(device)
+        print(f"✅ SNAC decoder initialized")
+    
+    def unpack_snac_from_7(self, vocab_ids: List[int]) -> List[List[int]]:
+        """
+        Unpack 7-token SNAC frames to 3 hierarchical levels.
+        
+        This is the EXACT INVERSE of training preprocessing.
+        
+        Frame structure (7 tokens per frame):
+        [slot0, slot1, slot2, slot3, slot4, slot5, slot6]
+        
+        Unpacking to [L1, L2, L3]:
+        - slot0 → L1[i]       (coarse: 1x rate)
+        - slot1 → L2[2*i]     (medium: 2x rate, even)
+        - slot2 → L3[4*i+0]   (fine: 4x rate)
+        - slot3 → L3[4*i+1]
+        - slot4 → L2[2*i+1]   (medium: odd)
+        - slot5 → L3[4*i+2]
+        - slot6 → L3[4*i+3]
+        
+        Args:
+            vocab_ids: List of SNAC token IDs (128266-156937), length divisible by 7
+        
+        Returns:
+            [L1, L2, L3] where L1=n, L2=2n, L3=4n elements
+        """
+        # Remove EOS token if present
+        if vocab_ids and vocab_ids[-1] == CODE_END_TOKEN_ID:
+            vocab_ids = vocab_ids[:-1]
+        
+        # Ensure complete frames
+        frames = len(vocab_ids) // SNAC_TOKENS_PER_FRAME
+        vocab_ids = vocab_ids[:frames * SNAC_TOKENS_PER_FRAME]
+        
+        if frames == 0:
+            return [[], [], []]
+        
+        l1, l2, l3 = [], [], []
+        
+        for i in range(frames):
+            slots = vocab_ids[i*7:(i+1)*7]
+            
+            # Subtract offset and mod 4096 to get original SNAC codes
+            l1.append((slots[0] - CODE_TOKEN_OFFSET) % 4096)
+            l2.extend([
+                (slots[1] - CODE_TOKEN_OFFSET) % 4096,  # Even
+                (slots[4] - CODE_TOKEN_OFFSET) % 4096,  # Odd
+            ])
+            l3.extend([
+                (slots[2] - CODE_TOKEN_OFFSET) % 4096,
+                (slots[3] - CODE_TOKEN_OFFSET) % 4096,
+                (slots[5] - CODE_TOKEN_OFFSET) % 4096,
+                (slots[6] - CODE_TOKEN_OFFSET) % 4096,
+            ])
+        
+        return [l1, l2, l3]
+    
+    @torch.inference_mode()
+    def decode(
+        self, 
+        snac_tokens: List[int], 
+        use_sliding_window: bool = False
+    ) -> Optional[np.ndarray]:
+        """
+        Decode SNAC tokens to audio waveform.
+        
+        Args:
+            snac_tokens: List of SNAC token IDs (7*n tokens)
+            use_sliding_window: If True, return only middle 2048 samples 
+                               (for smooth streaming without pops/clicks)
+        
+        Returns:
+            Audio waveform as float32 numpy array, 24kHz mono
+        """
+        if len(snac_tokens) < SNAC_TOKENS_PER_FRAME:
+            return None
+        
+        # Unpack to 3 hierarchical levels
+        levels = self.unpack_snac_from_7(snac_tokens)
+        
+        if not levels[0]:
+            return None
+        
+        # Convert to tensors
+        codes = [
+            torch.tensor(level, dtype=torch.long, device=self.device).unsqueeze(0)
+            for level in levels
+        ]
+        
+        # Decode through SNAC quantizer + decoder
+        z_q = self.snac_model.quantizer.from_codes(codes)
+        audio = self.snac_model.decoder(z_q)
+        
+        # Extract audio: [batch, 1, samples] → [samples]
+        audio = audio[0, 0].cpu().numpy()
+        
+        # Sliding window mode: keep middle 2048 samples only
+        # This eliminates popping/cracking in streaming by overlapping windows
+        if use_sliding_window and len(audio) >= 4096:
+            audio = audio[2048:4096]
+        
+        return audio
+    
+    def decode_to_bytes(
+        self, 
+        snac_tokens: List[int], 
+        use_sliding_window: bool = False
+    ) -> Optional[bytes]:
+        """
+        Decode SNAC tokens to audio bytes (int16 PCM).
+        
+        Args:
+            snac_tokens: List of SNAC token IDs
+            use_sliding_window: Use sliding window for smooth streaming
+        
+        Returns:
+            Audio as bytes (int16 PCM, 24kHz mono)
+        """
+        audio = self.decode(snac_tokens, use_sliding_window=use_sliding_window)
+        
+        if audio is None:
+            return None
+        
+        # Convert float32 to int16 PCM
+        audio_int16 = (audio * 32767).astype(np.int16)
+        return audio_int16.tobytes()
+
+
+# ============================================================================
+# CUSTOM LOGITS PROCESSOR
+# ============================================================================
+
+class OnlyAudioAfterSOS:
+    """
+    Restricts vocabulary to SNAC codes + EOS after SOS token.
+    
+    This prevents the model from generating text tokens during audio phase,
+    which would cause "hallucination" where the model repeats description text
+    instead of generating proper audio codes.
+    """
+    
+    def __init__(self):
+        self._seen_sos = False
+    
+    def __call__(
+        self,
+        prompt_token_ids: List[int],
+        generated_token_ids: List[int],
+        logits: torch.Tensor,
+    ) -> torch.Tensor:
+        """
+        Apply constraint: after SOS, only allow SNAC codes + EOS.
+        
+        Args:
+            prompt_token_ids: Original prompt token IDs
+            generated_token_ids: Tokens generated so far
+            logits: Logits for next token [vocab_size]
+        
+        Returns:
+            Modified logits with masked tokens
+        """
+        # Check if SOS has been generated
+        if not self._seen_sos:
+            all_token_ids = prompt_token_ids + generated_token_ids
+            if CODE_START_TOKEN_ID in all_token_ids:
+                self._seen_sos = True
+            else:
+                return logits  # No constraint yet
+        
+        # Apply constraint: mask all tokens except SNAC codes + EOS
+        mask = torch.full_like(logits, float('-inf'))
+        mask[SNAC_MIN_ID:SNAC_MAX_ID + 1] = 0  # Allow SNAC codes
+        mask[CODE_END_TOKEN_ID] = 0            # Allow EOS
+        
+        return logits + mask
+    
+    def reset(self):
+        """Reset state for reuse across generations."""
+        self._seen_sos = False
+
+
+# ============================================================================
+# MAYA-1-VOICE MODEL
+# ============================================================================
+
+class Maya1VoiceModel:
+    """
+    Maya-1-Voice TTS Model with VLLM inference engine.
+    
+    Handles model loading, tokenizer initialization, and VLLM engine setup.
+    """
+    
+    def __init__(
+        self,
+        model_path: str,
+        dtype: str = "bfloat16",
+        max_model_len: int = 8192,
+        gpu_memory_utilization: float = 0.85,
+    ):
+        """
+        Initialize Maya-1-Voice model with VLLM.
+        
+        Args:
+            model_path: Path to model checkpoint (local or HuggingFace)
+            dtype: Model precision (bfloat16 recommended)
+            max_model_len: Maximum sequence length
+            gpu_memory_utilization: GPU memory fraction to use (0.0-1.0)
+        """
+        self.model_path = model_path
+        
+        print(f"🚀 Initializing Maya-1-Voice Model")
+        print(f"📁 Model: {model_path}")
+        print(f"🔢 Dtype: {dtype}")
+        
+        # Load tokenizer (must be from checkpoint with emotion tags)
+        print(f"📝 Loading tokenizer...")
+        self.tokenizer = AutoTokenizer.from_pretrained(
+            model_path,
+            trust_remote_code=True,
+        )
+        print(f"✅ Tokenizer loaded: {len(self.tokenizer)} tokens")
+        
+        # Initialize VLLM async engine
+        print(f"🔧 Initializing VLLM engine...")
+        engine_args = AsyncEngineArgs(
+            model=model_path,
+            tokenizer=model_path,
+            dtype=dtype,
+            max_model_len=max_model_len,
+            gpu_memory_utilization=gpu_memory_utilization,
+            trust_remote_code=True,
+        )
+        
+        self.engine = AsyncLLMEngine.from_engine_args(engine_args)
+        print(f"✅ VLLM engine ready")
+    
+    def build_prompt(self, description: str, text: str) -> str:
+        """
+        Build prompt in Maya-1-Voice format using chat template.
+        
+        Format: Chat template with <description="..."> text as content
+        
+        The model expects:
+        1. Description of voice/character
+        2. Text to synthesize (optionally with <emotion> tags)
+        
+        Args:
+            description: Voice description 
+                Example: "Realistic male voice in the 30s age with american accent. 
+                         Normal pitch, warm timbre, conversational pacing."
+            text: Text to synthesize
+                Example: "Hello world! <excited> This is amazing!"
+        
+        Returns:
+            Formatted prompt string using chat template
+        """
+        content = f'<description="{description}"> {text}'
+        messages = [{"role": "user", "content": content}]
+        return self.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+
+
+# ============================================================================
+# STREAMING PIPELINE
+# ============================================================================
+
+class Maya1VoiceStreamingPipeline:
+    """
+    Streaming TTS pipeline using sliding window approach.
+    
+    This generates smooth audio by:
+    1. Streaming tokens from VLLM as they're generated
+    2. Every 7 tokens, decoding the last 28 tokens (4 frames) - sliding window
+    3. Keeping only middle 2048 samples from each decode
+    4. Creating natural overlap between chunks for artifact-free playback
+    """
+    
+    def __init__(self, model: Maya1VoiceModel, snac_decoder: SNACDecoder):
+        """Initialize streaming pipeline."""
+        self.model = model
+        self.snac_decoder = snac_decoder
+        print(f"🌊 Maya-1-Voice Streaming Pipeline initialized")
+    
+    async def generate_speech_stream(
+        self,
+        description: str,
+        text: str,
+        temperature: float = DEFAULT_TEMPERATURE,
+        top_p: float = DEFAULT_TOP_P,
+        max_tokens: int = DEFAULT_MAX_TOKENS,
+        repetition_penalty: float = DEFAULT_REPETITION_PENALTY,
+    ) -> AsyncGenerator[bytes, None]:
+        """
+        Generate speech audio with streaming.
+        
+        Args:
+            description: Voice/character description
+            text: Text to synthesize (with optional <emotion> tags)
+            temperature: Sampling temperature (lower = more stable)
+            top_p: Nucleus sampling
+            max_tokens: Max SNAC tokens to generate
+            repetition_penalty: Prevent repetition loops
+        
+        Yields:
+            Audio chunks as bytes (int16 PCM, 24kHz mono)
+        """
+        print(f"\n🌊 Starting streaming generation")
+        print(f"📝 Description: {description[:80]}...")
+        print(f"💬 Text: {text}")
+        
+        # Build prompt
+        prompt = self.model.build_prompt(description, text)
+        
+        # Configure sampling (removed custom logits processor for V1 compatibility)
+        sampling_params = SamplingParams(
+            temperature=temperature,
+            top_p=top_p,
+            max_tokens=max_tokens,
+            min_tokens=DEFAULT_MIN_TOKENS,
+            repetition_penalty=repetition_penalty,
+            stop_token_ids=[CODE_END_TOKEN_ID],  # Stop on audio EOS
+        )
+        
+        print(f"🎲 Sampling: temp={temperature}, top_p={top_p}, max_tokens={max_tokens}")
+        
+        # Token buffer for sliding window
+        token_buffer = []
+        total_tokens = 0
+        total_chunks = 0
+        
+        # Generate with VLLM
+        import uuid
+        import time
+        request_id = f"maya1voice-{uuid.uuid4().hex[:8]}-{int(time.time() * 1000000)}"
+        
+        results_generator = self.model.engine.generate(
+            prompt=prompt,
+            sampling_params=sampling_params,
+            request_id=request_id,
+        )
+        
+        # Stream tokens with sliding window decoding
+        async for request_output in results_generator:
+            generated_ids = request_output.outputs[0].token_ids
+            
+            # Process only new tokens
+            new_tokens = generated_ids[total_tokens:]
+            total_tokens = len(generated_ids)
+            
+            # Filter and buffer SNAC tokens only
+            for token_id in new_tokens:
+                if SNAC_MIN_ID <= token_id <= SNAC_MAX_ID:
+                    token_buffer.append(token_id)
+                    
+                    # Sliding window: process every 7 tokens when buffer > 27
+                    # Take last 28 tokens (4 frames) for smooth overlap
+                    if len(token_buffer) % 7 == 0 and len(token_buffer) > 27:
+                        window_tokens = token_buffer[-28:]
+                        
+                        # Decode with sliding window (returns middle 2048 samples)
+                        audio_bytes = self.snac_decoder.decode_to_bytes(
+                            window_tokens, 
+                            use_sliding_window=True
+                        )
+                        
+                        if audio_bytes:
+                            total_chunks += 1
+                            if total_chunks == 1:
+                                print(f"🎵 First chunk decoded ({len(audio_bytes)} bytes)")
+                            yield audio_bytes
+        
+        print(f"✅ Streaming complete: {total_tokens} tokens → {total_chunks} chunks")
+
+
+# ============================================================================
+# MAIN EXAMPLE
+# ============================================================================
+
+async def main():
+    """
+    Example usage of Maya-1-Voice streaming inference.
+    
+    This demonstrates:
+    1. Model initialization
+    2. SNAC decoder setup
+    3. Streaming generation
+    4. Audio chunk handling
+    """
+    
+    # Configuration
+    MODEL_PATH = "/home/ubuntu/veena_temp/maya-1-voice"  # Local model path
+    DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
+    
+    print("=" * 80)
+    print("Maya-1-Voice VLLM Streaming Inference Example")
+    print("=" * 80)
+    
+    # Initialize model
+    model = Maya1VoiceModel(
+        model_path=MODEL_PATH,
+        dtype="bfloat16",
+        max_model_len=8192,
+        gpu_memory_utilization=0.8,  # Reduced for available GPU memory (12GB free)
+    )
+    
+    # Initialize SNAC decoder
+    snac_decoder = SNACDecoder(device=DEVICE)
+    
+    # Create pipeline
+    pipeline = Maya1VoiceStreamingPipeline(model, snac_decoder)
+    
+    # Example 1: Professional voice
+    description = (
+        "Realistic male voice in the 30s age with american accent. "
+        "Normal pitch, warm timbre, conversational pacing, neutral tone delivery at med intensity."
+    )
+    text = "Hello! This is a test of the Maya-1-Voice text-to-speech system."
+    
+    print(f"\n{'='*80}")
+    print("Example 1: Professional Voice")
+    print(f"{'='*80}")
+    
+    audio_chunks = []
+    async for chunk in pipeline.generate_speech_stream(
+        description=description,
+        text=text,
+        temperature=0.4,
+        max_tokens=500,
+    ):
+        audio_chunks.append(chunk)
+        print(f"📦 Received chunk {len(audio_chunks)}: {len(chunk)} bytes")
+    
+    # Combine chunks
+    full_audio = b''.join(audio_chunks)
+    print(f"\n✅ Total audio: {len(full_audio)} bytes ({len(full_audio)//2} samples, {len(full_audio)/2/24000:.2f}s)")
+    
+    # Save audio (optional)
+    try:
+        import wave
+        output_file = "output_example1.wav"
+        with wave.open(output_file, 'wb') as wav:
+            wav.setnchannels(1)  # Mono
+            wav.setsampwidth(2)  # 16-bit
+            wav.setframerate(24000)  # 24kHz
+            wav.writeframes(full_audio)
+        print(f"💾 Saved to {output_file}")
+    except ImportError:
+        print(f"⚠️  Install 'wave' module to save audio files")
+    
+    # Example 2: Character voice with emotions
+    print(f"\n{'='*80}")
+    print("Example 2: Character Voice with Emotions")
+    print(f"{'='*80}")
+    
+    description = (
+        "Creative, dark_villain character. Male voice in their 40s with british accent. "
+        "Low pitch, gravelly timbre, slow pacing, angry tone at high intensity."
+    )
+    text = "The darkness isn't coming... <angry> it's already here!"
+    
+    audio_chunks = []
+    async for chunk in pipeline.generate_speech_stream(
+        description=description,
+        text=text,
+        temperature=0.5,
+        max_tokens=800,
+    ):
+        audio_chunks.append(chunk)
+        print(f"📦 Received chunk {len(audio_chunks)}: {len(chunk)} bytes")
+    
+    full_audio = b''.join(audio_chunks)
+    print(f"\n✅ Total audio: {len(full_audio)} bytes ({len(full_audio)//2} samples, {len(full_audio)/2/24000:.2f}s)")
+    
+    # Save audio
+    try:
+        import wave
+        output_file = "output_example2.wav"
+        with wave.open(output_file, 'wb') as wav:
+            wav.setnchannels(1)
+            wav.setsampwidth(2)
+            wav.setframerate(24000)
+            wav.writeframes(full_audio)
+        print(f"💾 Saved to {output_file}")
+    except ImportError:
+        pass
+    
+    print(f"\n{'='*80}")
+    print("🎉 Examples complete!")
+    print(f"{'='*80}")
+
+
+if __name__ == "__main__":
+    # Run async main
+    asyncio.run(main())

vocence_config.yaml

@@ -0,0 +1,18 @@
+# Optional PromptTTS settings read by your miner.py. Example values.
+
+runtime:
+  adapter: "example"
+  device_preference: "cuda"
+  dtype: "float32"
+
+generation:
+  sample_rate: 24000
+  max_seconds: 20
+  guidance_scale: 1.0
+
+io:
+  output_format: "wav"
+
+limits:
+  max_text_chars: 2000
+  max_instruction_chars: 600

vocence_local_wrapper.py

@@ -0,0 +1,133 @@
+#!/usr/bin/env python3
+"""
+Local Vocence wrapper for testing miner.py without Chutes.
+
+Run:
+  python vocence_local_wrapper.py
+
+Then call:
+  GET  http://127.0.0.1:8000/health
+  POST http://127.0.0.1:8000/speak
+"""
+from __future__ import annotations
+
+import io
+import wave
+from pathlib import Path
+from typing import Optional
+
+import numpy as np
+import uvicorn
+from fastapi import FastAPI, HTTPException, status
+from fastapi.responses import Response
+from pydantic import BaseModel, Field
+from yaml import safe_load
+
+from miner import Miner
+
+VOCENCE_MAX_AUDIO_SECONDS = 30
+VOCENCE_MAX_TEXT_LEN = 2000
+VOCENCE_MAX_INSTRUCTION_LEN = 600
+
+
+class VocenceSpeakRequest(BaseModel):
+    instruction: str = Field(..., min_length=1, max_length=VOCENCE_MAX_INSTRUCTION_LEN)
+    text: str = Field(..., min_length=1, max_length=VOCENCE_MAX_TEXT_LEN)
+
+
+class VocenceHealthResponse(BaseModel):
+    status: str
+    model_loaded: bool
+    sample_rate: Optional[int] = None
+    adapter: Optional[str] = None
+    repo_path: str
+
+
+def waveform_to_wav_bytes(waveform: np.ndarray, sample_rate: int) -> bytes:
+    if waveform.ndim != 1:
+        raise ValueError("waveform must be 1D mono")
+
+    if waveform.dtype != np.int16:
+        wf = np.asarray(waveform, dtype=np.float32)
+        wf = np.clip(wf, -1.0, 1.0)
+        wf = (wf * 32767.0).astype(np.int16)
+    else:
+        wf = waveform
+
+    buf = io.BytesIO()
+    with wave.open(buf, "wb") as wav:
+        wav.setnchannels(1)
+        wav.setsampwidth(2)
+        wav.setframerate(sample_rate)
+        wav.writeframes(wf.tobytes())
+    return buf.getvalue()
+
+
+repo_path = Path(__file__).resolve().parent
+app = FastAPI(title="Vocence Local Wrapper", version="0.1.0")
+
+
+@app.on_event("startup")
+async def startup_event() -> None:
+    app.state.status = "unknown"
+    app.state.sample_rate = None
+    app.state.adapter = None
+    app.state.tts_engine = None
+
+    try:
+        app.state.tts_engine = Miner(repo_path)
+        app.state.tts_engine.warmup()
+
+        vocence_yaml = repo_path / "vocence_config.yaml"
+        if vocence_yaml.exists():
+            with vocence_yaml.open("r", encoding="utf-8") as f:
+                cfg = safe_load(f) or {}
+            app.state.sample_rate = int(cfg.get("generation", {}).get("sample_rate", 24000))
+            app.state.adapter = str(cfg.get("runtime", {}).get("adapter", "unknown"))
+        else:
+            app.state.sample_rate = 24000
+            app.state.adapter = "unknown"
+
+        app.state.status = "healthy"
+    except Exception as exc:
+        app.state.status = f"startup_failed: {exc}"
+        app.state.tts_engine = None
+
+
+@app.get("/health")
+async def health() -> dict:
+    return VocenceHealthResponse(
+        status=getattr(app.state, "status", "unknown"),
+        model_loaded=getattr(app.state, "tts_engine", None) is not None,
+        sample_rate=getattr(app.state, "sample_rate", None),
+        adapter=getattr(app.state, "adapter", None),
+        repo_path=str(repo_path),
+    ).model_dump()
+
+
+@app.post("/speak", response_class=Response)
+async def speak(args: VocenceSpeakRequest):
+    engine = getattr(app.state, "tts_engine", None)
+    if engine is None:
+        raise HTTPException(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail="TTS engine not loaded",
+        )
+
+    waveform, sample_rate = engine.generate_wav(instruction=args.instruction, text=args.text)
+    waveform = np.asarray(waveform)
+    if waveform.ndim != 1 or waveform.size == 0:
+        raise HTTPException(status_code=400, detail="invalid waveform")
+
+    duration_sec = float(waveform.shape[0]) / float(sample_rate)
+    if duration_sec <= 0 or duration_sec > VOCENCE_MAX_AUDIO_SECONDS:
+        raise HTTPException(status_code=400, detail="invalid duration")
+
+    return Response(
+        content=waveform_to_wav_bytes(waveform, sample_rate),
+        media_type="audio/wav",
+    )
+
+
+if __name__ == "__main__":
+    uvicorn.run("vocence_local_wrapper:app", host="127.0.0.1", port=8000, reload=False)