Initial commit of readme/Model Card

README.md

@@ -1,5 +1,251 @@
 ---
 license: other
-license_name: modified-mit
-license_link: LICENSE
+library_name: pytorch
+pipeline_tag: text-to-speech
+tags:
+  - text-to-speech
+  - speech-synthesis
+  - voice
+  - audio
+  - sesame
+  - mimi
+  - llama
 ---
+
+<div align="center">
+
+<img src="images/repo_banner.png" alt="Miso TTS 8B" width="100%">
+
+# Miso TTS 8B
+
+### An 8B-parameter Sesame-style text-to-speech model from Miso Labs
+
+<p>
+  <a href="https://misolabs.ai"><img alt="Website" src="https://img.shields.io/badge/Website-misolabs.ai-black?style=for-the-badge"></a>
+  <a href="https://huggingface.co/MisoLabs/MisoTTS"><img alt="Hugging Face" src="https://img.shields.io/badge/Hugging%20Face-MisoTTS-yellow?style=for-the-badge"></a>
+  <a href="https://github.com/MisoLabsAI"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-MisoLabsAI-181717?style=for-the-badge&logo=github&labelColor=555555"></a>
+  <a href="https://x.com/MisoLabsAI"><img alt="X" src="https://img.shields.io/badge/-MisoLabsAI-181717?style=for-the-badge&logo=x&labelColor=555555"></a>
+</p>
+
+<p>
+  <a href="#quickstart">Quickstart</a> |
+  <a href="#model-introduction">Model Introduction</a> |
+  <a href="#model-summary">Model Summary</a> |
+  <a href="#usage">Usage</a> |
+  <a href="#safety">Safety</a>
+</p>
+
+</div>
+
+---
+
+## Model Introduction
+
+Miso TTS 8B is a text-to-speech model based on the Sesame CSM architecture. It
+generates Mimi audio codes from text and optional audio context, using a large
+Llama 3.2-style backbone and a smaller autoregressive audio decoder.
+
+The model is designed for high-quality conversational speech generation and
+voice continuation from prompt audio. This repository contains the inference
+code, model definition, and setup instructions for running Miso TTS locally.
+
+---
+
+## Quickstart
+
+If you do not have `uv` installed yet:
+
+```bash
+curl -LsSf https://astral.sh/uv/install.sh | sh
+```
+
+Then clone the repository and create the environment:
+
+```bash
+git clone https://github.com/MisoLabsAI/OpenMiso.git
+cd OpenMiso
+uv sync --python 3.10
+```
+
+Point the loader at the public model weights:
+
+```bash
+export MISO_TTS_8B_MODEL=MisoLabs/MisoTTS
+```
+
+Then run the example conversation:
+
+```bash
+uv run python run_csm.py
+```
+
+The script writes `full_conversation.wav` in the repository root.
+
+For local checkpoints, set `MISO_TTS_8B_MODEL` to the checkpoint path instead:
+
+```bash
+export MISO_TTS_8B_MODEL=/path/to/miso-tts-8b.safetensors
+uv run python run_csm.py
+```
+
+With `pip` instead of `uv`:
+
+```bash
+python3.10 -m venv .venv
+source .venv/bin/activate
+pip install -e .
+export MISO_TTS_8B_MODEL=MisoLabs/MisoTTS
+python run_csm.py
+```
+
+---
+
+## Model Summary
+
+| Item                | Value            |
+| ------------------- | ---------------- |
+| Model               | Miso TTS 8B      |
+| Organization        | Miso Labs        |
+| Task                | Text-to-speech   |
+| Architecture        | Sesame-style CSM |
+| Backbone            | `llama-8B`       |
+| Audio decoder       | `llama-300M`     |
+| Text vocabulary     | `128,256`        |
+| Audio vocabulary    | `2,051`          |
+| Audio codebooks     | `32`             |
+| Audio tokenizer     | Mimi             |
+| Max sequence length | `2,048`          |
+
+### Architecture
+
+Miso TTS 8B uses two transformer components:
+
+- A large backbone transformer that consumes text/audio-frame embeddings.
+- A smaller decoder transformer that autoregressively predicts higher-order
+  audio codebooks within each frame.
+
+The model follows Sesame's compute-amortized decoder design: codebook 0 is
+predicted from the backbone hidden state, while codebooks 1 through 31 are
+predicted by the audio decoder.
+
+---
+
+## Usage
+
+### Python
+
+```python
+import torch
+import torchaudio
+
+from generator import load_miso_8b
+
+device = "cuda" if torch.cuda.is_available() else "cpu"
+
+generator = load_miso_8b(
+    device=device,
+    model_path_or_repo_id="MisoLabs/MisoTTS",
+)
+
+audio = generator.generate(
+    text="Hello from Miso.",
+    speaker=0,
+    context=[],
+    max_audio_length_ms=10_000,
+)
+
+torchaudio.save("miso.wav", audio.unsqueeze(0).cpu(), generator.sample_rate)
+```
+
+### Prompted generation
+
+Miso TTS performs best when provided with prompt audio and matching transcript
+context:
+
+```python
+import torchaudio
+
+from generator import Segment, load_miso_8b
+
+generator = load_miso_8b(device="cuda")
+
+prompt_audio, sample_rate = torchaudio.load("prompt.wav")
+prompt_audio = torchaudio.functional.resample(
+    prompt_audio.squeeze(0),
+    orig_freq=sample_rate,
+    new_freq=generator.sample_rate,
+)
+
+context = [
+    Segment(
+        speaker=0,
+        text="This is the transcript for the prompt audio.",
+        audio=prompt_audio,
+    )
+]
+
+audio = generator.generate(
+    text="This is the next sentence to synthesize.",
+    speaker=0,
+    context=context,
+    max_audio_length_ms=10_000,
+)
+```
+
+---
+
+## Weights
+
+The model weights are hosted publicly on Hugging Face:
+
+```bash
+export MISO_TTS_8B_MODEL=MisoLabs/MisoTTS
+```
+
+For development, the loader also supports local checkpoint files:
+
+```bash
+export MISO_TTS_8B_MODEL=/path/to/model.pt
+export MISO_TTS_8B_MODEL=/path/to/model.safetensors
+```
+
+If you are working from an internal S3 checkpoint, download it first:
+
+```bash
+aws s3 cp s3://your-bucket/path/to/miso-tts-8b.safetensors ./miso-tts-8b.safetensors
+export MISO_TTS_8B_MODEL=./miso-tts-8b.safetensors
+```
+
+---
+
+## Deployment Notes
+
+Miso TTS 8B is a large model. For best results, use a CUDA GPU with sufficient
+VRAM for the checkpoint precision you are loading. The default inference path
+uses `torch.bfloat16`.
+
+The example script uses public Sesame prompt assets by default. After publishing
+Miso-specific prompt assets, set:
+
+```bash
+export MISO_TTS_PROMPT_REPO=MisoLabs/MisoTTS
+```
+
+---
+
+## Safety
+
+Miso TTS is a speech generation model. Do not use it to impersonate people,
+create deceptive audio, commit fraud, or generate harmful content.
+
+Generated audio is watermarked by default. If you deploy this model in another
+application, use your own private watermark key and keep it secret.
+
+---
+
+## Links
+
+- Website: [misolabs.ai](https://misolabs.ai)
+- Hugging Face: [MisoLabs/MisoTTS](https://huggingface.co/MisoLabs/MisoTTS)
+- GitHub: [MisoLabsAI](https://github.com/MisoLabsAI)
+- X: [@MisoLabsAI](https://x.com/MisoLabsAI)