Upload Breeze TTS 2 model and model card

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,427 @@
+BREEZEBLUE RESEARCH AND NON-COMMERCIAL LICENSE AGREEMENT
+
+Version 1.0
+Last Updated: August 25, 2026
+
+This BreezeBlue Research and Non-Commercial License Agreement (the
+"Agreement") is entered into between RESONIA, INC., doing business as
+BreezeBlue ("BreezeBlue," "Licensor," "we," or "us"), and the individual or
+entity accessing, using, modifying, or distributing any Model Materials
+("You," "Your," or "Licensee").
+
+By accessing, downloading, using, modifying, or distributing any portion of
+the Model Materials or a Derivative Model, You acknowledge that You have read,
+understood, and agreed to this Agreement. If You act on behalf of an entity,
+You represent that You have authority to bind that entity, and "You" includes
+both You and that entity.
+
+This Agreement permits research and non-commercial use of the Model Materials
+free of charge. It does not grant any commercial rights and is not an open
+source license. Commercial use requires a separate written license from
+BreezeBlue.
+
+
+1. DEFINITIONS
+
+1.1 "Model" means the Breeze TTS 2 model.
+
+1.2 "Model Materials" means the Model weights, checkpoints, parameters,
+adapters, model-specific tokenizer or codec weights, embeddings, and other
+machine-learning artifacts that BreezeBlue expressly makes available under
+this Agreement, together with any accompanying documentation expressly
+designated as part of the Model Materials.
+
+Model Materials do not include source code expressly licensed under the
+Apache License, Version 2.0, or any third-party component made available under
+its own license.
+
+1.3 "Derivative Model" means any modification, adaptation, fine-tune, LoRA,
+merge, quantization, distillation, or other model that is based on or derived
+from the Model Materials, including a model created using the weights,
+parameters, activations, embeddings, or Outputs of the Model Materials to
+replicate or materially transfer their speech or audio generation
+capabilities. Outputs themselves are not Derivative Models.
+
+1.4 "Output" means audio, speech, text, metadata, or other content generated
+by the Model Materials or a Derivative Model in response to an input.
+
+1.5 "Research Purpose" means academic or scientific research that is not
+primarily intended for commercial advantage or monetary compensation,
+including peer-reviewed research, reproducibility studies, benchmarking,
+evaluation, security testing, and red-teaming.
+
+1.6 "Non-Commercial Purpose" means personal, educational, hobbyist, or other
+use that is not primarily intended for commercial advantage or monetary
+compensation, including limited evaluation and testing.
+
+1.7 "Commercial Purpose" means any use primarily intended for or directed
+toward commercial advantage, business benefit, or monetary compensation,
+whether direct or indirect. Commercial Purpose includes, without limitation:
+
+(a) production use or use in a product, service, or business operation;
+
+(b) hosting or making the Model Materials, a Derivative Model, or their
+functionality available through an API, SaaS offering, application, plug-in,
+website, or other service;
+
+(c) creating or providing content, products, or services for a fee, for a
+client, in connection with advertising, sponsorship, subscription revenue,
+or any other monetized activity;
+
+(d) using the Model Materials, a Derivative Model, or Outputs for an
+organization's internal operations beyond limited evaluation; or
+
+(e) distributing, licensing, selling, or otherwise commercializing the Model
+Materials or a Derivative Model.
+
+The status of a Licensee as a for-profit entity does not, by itself, make
+limited internal research, benchmarking, evaluation, security testing, or
+red-teaming a Commercial Purpose, provided that the use is not deployed in
+production, made available to customers or other external end users, used to
+generate revenue, or used to train or improve a non-BreezeBlue generative
+model.
+
+1.8 "Affiliate" means an entity that directly or indirectly controls, is
+controlled by, or is under common control with a party, where "control" means
+ownership or control of more than fifty percent (50%) of the voting interests
+of that entity.
+
+
+2. RESEARCH AND NON-COMMERCIAL LICENSE
+
+Subject to Your continued compliance with this Agreement, BreezeBlue grants
+You a limited, non-exclusive, worldwide, non-transferable, non-sublicensable,
+royalty-free license under BreezeBlue's intellectual property rights in the
+Model Materials to:
+
+(a) access, use, reproduce, and modify the Model Materials;
+
+(b) create Derivative Models; and
+
+(c) distribute the Model Materials and Derivative Models as expressly
+permitted by Section 4,
+
+in each case solely for a Research Purpose or Non-Commercial Purpose.
+
+No rights are granted except as expressly stated in this Agreement.
+
+
+3. COMMERCIAL USE
+
+No Commercial Purpose is permitted under this Agreement. Any Commercial
+Purpose involving the Model Materials, a Derivative Model, or an Output
+requires a separate written commercial license from BreezeBlue.
+
+For clarity, this Agreement does not include a creator, content monetization,
+small-business, revenue-threshold, or other implied commercial-use exception.
+
+This Agreement governs only the open-weight Model Materials and their
+self-hosted use. Outputs generated through BreezeBlue's hosted products or
+APIs are governed by the applicable Terms of Service, plan terms, and any
+separate written agreement, rather than this Agreement.
+
+For commercial licensing inquiries, contact:
+
+Website: https://breezeblue.ai
+Email: contact@breeze.blue
+
+
+4. DISTRIBUTION AND ATTRIBUTION
+
+You may distribute the Model Materials or a Derivative Model solely for a
+Research Purpose or Non-Commercial Purpose and only if You:
+
+(a) provide every recipient with a complete copy of this Agreement and any
+NOTICE file accompanying the Model Materials;
+
+(b) retain all copyright, attribution, provenance, and proprietary notices;
+
+(c) include the following notice in a NOTICE file distributed with the Model
+Materials or Derivative Model:
+
+"Breeze TTS 2 is licensed under the BreezeBlue Research and Non-Commercial
+License Agreement. Copyright (c) 2026 RESONIA, INC. All Rights Reserved.";
+
+(d) clearly identify any modifications You made and, for a Derivative Model,
+state prominently in its model card, repository, and documentation:
+
+"Derived from Breeze TTS 2 by BreezeBlue and licensed for research and
+non-commercial use only.";
+
+(e) license the distributed Model Materials or Derivative Model under this
+Agreement without adding terms that conflict with or reduce the protections
+of this Agreement; and
+
+(f) ensure that each recipient independently accepts this Agreement before
+accessing or using the Model Materials or Derivative Model.
+
+You may not use "BreezeBlue," "Breeze TTS 2," or any confusingly similar name
+as the primary name of a Derivative Model, product, or service, or in any way
+that suggests endorsement, affiliation, official status, or successor status.
+Reasonable descriptive attribution required by this Section is permitted.
+
+Publication of non-commercial research results, benchmark results, and
+illustrative samples in papers, reports, conference presentations, model
+cards, repositories, or non-monetized demonstrations is permitted, provided
+that You reasonably acknowledge Breeze TTS 2 and comply with this Agreement.
+
+
+5. USE RESTRICTIONS
+
+You will not, and will not permit any third party to:
+
+(a) use the Model Materials, a Derivative Model, or any Output for a
+Commercial Purpose without a separate written commercial license;
+
+(b) use the Model Materials, a Derivative Model, or any Output to create,
+train, fine-tune, distill, evaluate for training selection, or otherwise
+improve any non-BreezeBlue foundational or generative speech, audio,
+language, or multimodal model, except for creating a Derivative Model solely
+for a Research Purpose or Non-Commercial Purpose under this Agreement;
+
+(c) use the Model Materials, a Derivative Model, or any Output to clone,
+simulate, imitate, or impersonate a real person's voice, identity, likeness,
+or persona without that person's explicit and legally sufficient consent;
+
+(d) submit or use reference audio, speaker embeddings, voice prompts,
+recordings, performances, or other materials unless You have all rights,
+licenses, permissions, and consents necessary for that use;
+
+(e) use the Model Materials, a Derivative Model, or any Output to defraud,
+deceive, harass, stalk, threaten, defame, exploit, or cause harm to any person;
+
+(f) generate or distribute sexual, exploitative, or otherwise harmful content
+involving a minor;
+
+(g) engage in deceptive political persuasion, election interference, voter
+suppression, synthetic robocalls, or false impersonation of a candidate,
+public official, or other person in a political context;
+
+(h) identify, authenticate, track, surveil, or profile a person using voice or
+other biometric characteristics without a lawful basis and all consent
+required by applicable law;
+
+(i) distribute synthetic audio in circumstances where a reasonable person
+would be materially misled about its origin or authenticity, or fail to make
+any disclosure required by applicable law;
+
+(j) remove, obscure, alter, or circumvent any copyright notice, attribution,
+watermark, provenance signal, safety mechanism, or other proprietary notice
+included in the Model Materials or Outputs;
+
+(k) attempt to extract or reconstruct non-public training data, source data,
+trade secrets, or proprietary components from the Model Materials, except to
+the extent such a restriction is prohibited by applicable law; or
+
+(l) use, export, re-export, transfer, or make available the Model Materials or
+a Derivative Model in violation of any applicable law, regulation, export
+control, or trade sanction.
+
+Nothing in Section 5(k) prohibits ordinary inspection, loading, conversion,
+quantization, modification, or fine-tuning of the released Model Materials as
+expressly permitted by this Agreement.
+
+
+6. LEGAL COMPLIANCE AND THIRD-PARTY RIGHTS
+
+You are solely responsible for ensuring that Your use of the Model Materials,
+Derivative Models, inputs, and Outputs complies with all applicable laws and
+regulations and does not infringe or violate any third-party right, including
+intellectual property, privacy, publicity, biometric privacy,
+recording-consent, consumer-protection, election, and other applicable rights.
+
+This Agreement grants no right to use, reproduce, clone, simulate, imitate,
+or otherwise exploit any third-party voice, identity, likeness, persona,
+performance, recording, copyrighted work, trademark, or other protected
+material. You are solely responsible for obtaining all necessary rights,
+licenses, permissions, notices, and consents.
+
+BreezeBlue does not represent or warrant that any Model Material or Output is
+cleared for Your intended use, unique, accurate, non-infringing, or suitable
+for any particular purpose.
+
+
+7. INTELLECTUAL PROPERTY
+
+7.1 Model Materials. BreezeBlue and its licensors retain all right, title, and
+interest in and to the Model Materials. Except for the limited license granted
+in Section 2, no right or interest in the Model Materials is transferred to
+You.
+
+7.2 Derivative Models. As between You and BreezeBlue, You own the original
+modifications You contribute to a Derivative Model, to the extent permitted by
+applicable law. That ownership is subject to BreezeBlue's ownership of the
+underlying Model Materials and to every restriction in this Agreement. A
+Derivative Model remains governed by this Agreement.
+
+7.3 Outputs. As between You and BreezeBlue, and to the extent permitted by
+applicable law, You own Outputs You lawfully generate. Output ownership does
+not remove or limit the non-commercial restrictions in this Agreement, grant
+rights in third-party material, or authorize any use prohibited by this
+Agreement.
+
+7.4 Trademarks. No trademark license is granted except the limited right to
+make accurate descriptive references required for attribution under Section
+4. All goodwill arising from permitted use of BreezeBlue's marks belongs to
+BreezeBlue.
+
+7.5 Feedback. If You voluntarily provide suggestions, comments, or other
+feedback about the Model Materials, You grant BreezeBlue a perpetual,
+irrevocable, worldwide, royalty-free, fully paid, transferable, sublicensable,
+non-exclusive license to use and exploit that feedback without restriction or
+compensation.
+
+7.6 Intellectual Property Claims. If You or Your Affiliate brings a claim or
+legal proceeding alleging that the Model Materials, a Derivative Model, or an
+Output infringes intellectual property rights owned or licensable by You or
+Your Affiliate, the licenses granted to You under this Agreement terminate
+automatically when the claim or proceeding is filed.
+
+
+8. REPRESENTATIONS
+
+You represent and warrant that:
+
+(a) You have legal capacity and authority to enter into this Agreement;
+
+(b) You will use and distribute the Model Materials, Derivative Models, and
+Outputs only as permitted by this Agreement;
+
+(c) You have all necessary rights and consents for Your inputs and intended
+uses; and
+
+(d) You are not prohibited from receiving or using the Model Materials under
+applicable export-control, sanctions, or other laws.
+
+
+9. DISCLAIMER OF WARRANTIES
+
+TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, THE MODEL MATERIALS,
+DERIVATIVE MODELS, OUTPUTS, AND ALL RELATED MATERIALS ARE PROVIDED "AS IS" AND
+"AS AVAILABLE," WITH ALL FAULTS AND WITHOUT WARRANTIES OF ANY KIND, WHETHER
+EXPRESS, IMPLIED, STATUTORY, OR OTHERWISE.
+
+BREEZEBLUE DISCLAIMS ALL WARRANTIES, INCLUDING WARRANTIES OF TITLE,
+NON-INFRINGEMENT, MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE,
+ACCURACY, RELIABILITY, SECURITY, AND ANY WARRANTY THAT THE MODEL MATERIALS OR
+OUTPUTS WILL BE ERROR-FREE, UNINTERRUPTED, UNIQUE, OR SUITABLE FOR YOUR USE.
+
+YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS AND LAWFULNESS
+OF YOUR USE AND ASSUME ALL RISKS ARISING FROM THAT USE.
+
+
+10. LIMITATION OF LIABILITY
+
+TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, BREEZEBLUE AND ITS
+AFFILIATES, LICENSORS, OFFICERS, DIRECTORS, EMPLOYEES, CONTRACTORS, AND AGENTS
+WILL NOT BE LIABLE UNDER ANY THEORY OF LIABILITY FOR ANY INDIRECT, INCIDENTAL,
+SPECIAL, CONSEQUENTIAL, EXEMPLARY, OR PUNITIVE DAMAGES, OR FOR ANY LOSS OF
+PROFITS, REVENUE, BUSINESS, GOODWILL, DATA, OR USE, ARISING OUT OF OR RELATING
+TO THIS AGREEMENT, THE MODEL MATERIALS, ANY DERIVATIVE MODEL, OR ANY OUTPUT,
+EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
+
+TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, BREEZEBLUE'S TOTAL
+AGGREGATE LIABILITY ARISING OUT OF OR RELATING TO THIS AGREEMENT WILL NOT
+EXCEED ONE HUNDRED U.S. DOLLARS (US$100).
+
+
+11. INDEMNIFICATION
+
+You agree to defend, indemnify, and hold harmless BreezeBlue and its
+Affiliates, licensors, officers, directors, employees, contractors, and agents
+from and against any third-party claim, demand, action, proceeding, damage,
+loss, liability, judgment, settlement, cost, or expense, including reasonable
+attorneys' fees, arising out of or relating to:
+
+(a) Your access to, use, modification, or distribution of the Model Materials
+or a Derivative Model;
+
+(b) Your inputs or Outputs;
+
+(c) Your violation of this Agreement or applicable law; or
+
+(d) Your infringement, misappropriation, or violation of any third-party
+right.
+
+
+12. TERM AND TERMINATION
+
+This Agreement begins when You first access or use the Model Materials and
+continues until terminated.
+
+BreezeBlue may terminate this Agreement if You materially breach it. For a
+material breach that is reasonably capable of cure, BreezeBlue will provide
+written notice and at least thirty (30) days to cure before termination.
+BreezeBlue may terminate immediately for a breach of Section 3, Section 5, or
+applicable law, or where delay would reasonably create legal, security, fraud,
+or safety risk.
+
+Upon termination, You must immediately cease using and distributing the Model
+Materials and Derivative Models, delete all copies under Your control, and,
+upon reasonable request, confirm deletion in writing. Termination does not
+require deletion of an Output lawfully created before termination, but no
+Output may be used in violation of this Agreement or applicable law.
+
+BreezeBlue may stop distributing the Model Materials prospectively. Such
+discontinuation alone does not terminate rights already granted to a Licensee
+that remains in compliance with this Agreement.
+
+Sections 3, 5, 6, 7, 9, 10, 11, 12, and 13 survive termination to the extent
+necessary to give them effect.
+
+
+13. GOVERNING LAW AND JURISDICTION
+
+This Agreement is governed by and construed in accordance with the laws of the
+State of Delaware, without regard to its conflict-of-laws principles. The
+United Nations Convention on Contracts for the International Sale of Goods
+does not apply.
+
+To the extent permitted by applicable law, any dispute arising out of or
+relating to this Agreement shall be brought exclusively in the state or
+federal courts located in the State of Delaware, and each party consents to
+the personal jurisdiction and venue of those courts.
+
+
+14. MISCELLANEOUS
+
+14.1 Entire Agreement. This Agreement and any NOTICE file accompanying the
+Model Materials constitute the entire agreement concerning the Model
+Materials and supersede prior or contemporaneous understandings concerning
+the same subject matter.
+
+14.2 Separate Software Licenses. Source code and third-party components may be
+made available under separate licenses. Those separate licenses govern only
+the materials expressly identified as subject to them and do not grant any
+right to use the Model Materials.
+
+14.3 Assignment. You may not assign or transfer this Agreement or any right
+under it without BreezeBlue's prior written consent. BreezeBlue may assign
+this Agreement in connection with a merger, acquisition, reorganization, sale
+of assets, or operation of law.
+
+14.4 Severability. If any provision is held invalid or unenforceable, it will
+be enforced to the maximum extent permitted, and the remaining provisions
+will remain in effect.
+
+14.5 No Waiver. Failure to enforce a provision is not a waiver of the right to
+enforce it later.
+
+14.6 No Endorsement. Nothing in this Agreement creates an agency,
+partnership, joint venture, employment, fiduciary, sponsorship, or endorsement
+relationship between the parties.
+
+14.7 Updates. BreezeBlue may publish new versions of this Agreement. A new
+version applies only if You expressly accept it or access Model Materials
+released under that new version. Your continued use of a previously obtained
+version of the Model Materials remains governed by the version You accepted,
+unless applicable law requires otherwise.
+
+
+15. CONTACT
+
+Questions and commercial licensing requests may be sent to:
+
+RESONIA, INC., doing business as BreezeBlue
+Website: https://breezeblue.ai
+Email: contact@breeze.blue

README.md

@@ -1,5 +1,206 @@
 ---
+language:
+- en
+- zh
+library_name: transformers
+pipeline_tag: text-to-speech
 license: other
 license_name: breezeblue-research-and-non-commercial-license
 license_link: LICENSE
+tags:
+- text-to-speech
+- speech-generation
+- voice-clone
+- voice-design
+- voice-direction
+- pytorch
+- cuda
 ---
+<div align="center">
+  <a href="https://breezeblue.ai/"><img src="assets/breezeblue-logo.png" alt="BreezeBlue" width="35%"></a>
+  <br><br>
+  <a href="https://github.com/breezeblue-ai/breeze-tts"><img src="https://img.shields.io/badge/GitHub-breeze--tts-181717?logo=github&logoColor=white" alt="GitHub"></a>
+  <a href="https://breezeblue.ai/breeze-tts-2"><img src="https://img.shields.io/badge/Blog-Breeze%20TTS%202-2563EB" alt="Blog"></a>
+  <a href="https://breezeblue.ai/"><img src="https://img.shields.io/badge/Website-BreezeBlue-0EA5E9" alt="Website"></a>
+  <a href="https://discord.com/invite/6H7AgPe9pA"><img src="https://img.shields.io/badge/Discord-Join%20us-5865F2?logo=discord&logoColor=white" alt="Discord"></a>
+  <a href="https://x.com/BreezeBlueX"><img src="https://img.shields.io/badge/X-Follow%20BreezeBlue-000000?logo=x&logoColor=white" alt="X"></a>
+</div>
+
+> [!IMPORTANT]
+> Source code is licensed under Apache 2.0. Breeze TTS 2 model weights, derivative models, and self-hosted outputs are for research and non-commercial use only. See [License](#license-and-responsible-use).
+
+## 📰 News
+
+- **[2026.08.25]** 🎉 We open-source [Breeze TTS 2](https://huggingface.co/BreezeBlue/breeze-tts-2) model weights and the [PyTorch inference code](https://github.com/breezeblue-ai/breeze-tts).
+- **[2026.08.07]** 🔥 We release the TTS benchmark suite for [voice design](https://github.com/breezeblue-ai/tts-voice-design-benchmark), [voice direction](https://github.com/breezeblue-ai/TTS-Voice-Direction-Benchmark), and [latency evaluation](https://github.com/breezeblue-ai/TTS-Latency-Benchmark).
+
+## 📖 Introduction
+
+Breeze TTS 2 is an open-weight text-to-speech model built for real-time interaction. It ranks #1 among open-weight models on the Artificial Analysis TTS leaderboard, while outperforming frontier proprietary systems. Its open-ended natural-language instruction-following capability supports reference-free voice design and reference-guided voice direction, while ultra-low-latency streaming enables responsive, expressive interaction.
+
+<div align="center">
+  <a href="https://artificialanalysis.ai/text-to-speech/leaderboard/provider-voice">
+    <img src="assets/tts-elo-leaderboard.svg" alt="Text-to-speech models ranked by Artificial Analysis Elo score" width="90%">
+  </a>
+</div>
+
+## ✨ Highlights
+
+- 🎙️ **Voice Clone** — Uses reference audio with its exact transcript to preserve timbre, rhythm, emotion, and style.
+- 🎨 **Voice Design** — Creates a distinctive voice from a natural-language description, without reference audio.
+- 🎛️ **Voice Direction** — Clones a voice from reference audio while steering tone, emotion, pace, and delivery.
+- 🎭 **Vocal Events** — Adds expressive inline events directly in the text: use parentheses in English, such as `(laugh)`, `(cough)`, `(clears throat)`, and `(sigh)`; use square brackets in Chinese, such as `[笑]`, `[咳嗽]`, `[清嗓子]`, and `[叹气]`.
+- ⚡ **Ultra-Low Latency** — Achieves under 40 ms time to first audio (TTFA) with the warmed-up fast path on an NVIDIA H100.
+- 🌊 **Real-Time Streaming** — Reaches a 0.32 real-time factor (RTF), generating audio at approximately 3.1× real time with the warmed-up fast path on an NVIDIA H100.
+- 💾 **GPU-Efficient** — Eager inference uses approximately 7.7 GiB of GPU memory; a 12 GB GPU is the minimum recommended configuration.
+- 🌏 **Bilingual Support** — Generates natural English and Chinese speech with a single model.
+
+## 🚀 Quick Start
+
+### Requirements
+
+- Linux and Python 3.10 or newer
+- A CUDA-capable NVIDIA GPU
+- GPU memory: approximately 7.7 GiB for eager inference or 14.4 GiB with `--fast-all`; use a 12 GB GPU for eager or a 24 GB GPU for the fast path
+- The Breeze TTS 2 checkpoint
+
+### Installation
+
+Download the inference code:
+
+```bash
+git clone https://github.com/breezeblue-ai/breeze-tts.git
+cd breeze-tts
+```
+
+Install the dependencies:
+
+```bash
+python -m pip install -r requirements.txt
+```
+
+All required model components are included in the Breeze TTS 2 checkpoint.
+
+For the tested CUDA environment, build the included Docker image:
+
+```bash
+bash docker/build.sh
+```
+
+The default image targets H100/Hopper (sm90). For A100:
+
+```bash
+FLASH_ATTN_CUDA_ARCHS=80 bash docker/build.sh
+```
+
+### 🎙️ Voice Clone
+
+Clone a speaker from clean reference audio and its exact transcript.
+
+#### English
+
+```bash
+python infer.py ../breeze-tts-2 \
+  --ref-audio reference_en.wav \
+  --ref-text "This is the exact transcript of the English reference audio." \
+  --text "(sigh) It is good to hear your voice again after all this time." \
+  --output outputs/voice_clone_en.wav
+```
+
+#### Chinese
+
+```bash
+python infer.py ../breeze-tts-2 \
+  --ref-audio reference_zh.wav \
+  --ref-text "这是中文参考音频的准确文字稿。" \
+  --text "[叹气] 没想到过了这么久，你还记得我的声音。" \
+  --output outputs/voice_clone_zh.wav
+```
+
+Reference audio should contain clean speech with minimal background noise.
+
+### 🎨 Voice Design
+
+Create a voice from a natural-language description without reference audio. Match the instruction language to the target text. Use `--cfg-scale 4` to strengthen instruction-following.
+
+#### English
+
+```bash
+python infer.py ../breeze-tts-2 \
+  --text "(sigh) Welcome aboard. Your journey begins now." \
+  --instruction "A warm, thoughtful young woman with a clear voice and a calm, reflective delivery." \
+  --cfg-scale 4 \
+  --output outputs/voice_design_en.wav
+```
+
+#### Chinese
+
+```bash
+python infer.py ../breeze-tts-2 \
+  --text "[笑] 欢迎来到今晚的故事时间，让我们一起开始吧。" \
+  --instruction "一位温柔自信的年轻女性，声音清晰，语气亲切，表达轻快而富有感染力。" \
+  --cfg-scale 4 \
+  --output outputs/voice_design_zh.wav
+```
+
+### 🎛️ Voice Direction
+
+Keep the identity of a reference speaker while directing tone, emotion, pace, and delivery. Use `--cfg-scale 4` to strengthen instruction-following.
+
+```bash
+python infer.py ../breeze-tts-2 \
+  --ref-audio reference.wav \
+  --ref-text "This is the exact transcript of the reference audio." \
+  --text "(clears throat) We need to discuss what happened last night." \
+  --instruction "Preserve the speaker identity, but speak slowly with a restrained, serious tone." \
+  --cfg-scale 4 \
+  --output outputs/voice_direction.wav
+```
+
+### 🌐 Streaming API
+
+Start the single-concurrency streaming API. It uses the same PyTorch runtime and eager execution by default:
+
+```bash
+python -m breeze_infer.api ../breeze-tts-2 --host 0.0.0.0 --port 7860
+```
+
+Send a Voice Direction request with reference audio and CFG 4:
+
+```bash
+curl -X POST http://127.0.0.1:7860/v1/audio/speech \
+  -F "cfg_scale=4" \
+  -F "ref_audio=@reference.wav" \
+  -F "ref_text=This is the exact transcript of the reference audio." \
+  -F "text=(clears throat) We need to discuss what happened last night." \
+  -F "instruction=Preserve the speaker identity, but speak slowly with a restrained, serious tone." \
+  -F "seed=42" \
+  --output voice_direction.pcm
+```
+
+The response is streaming mono 24 kHz signed 16-bit little-endian PCM. Start the API with `--fast-all` to enable the fast path.
+
+### ⚡ Fast Inference Options
+
+Both the CLI and API use eager streaming by default and skip graph warmup. Pass `--fast-all` to enable the best configuration for every inference stage when the additional cold-start time is acceptable. Each stage can also be controlled independently:
+
+| Stage | Fast parameter | Disabled | Enabled |
+| --- | --- | --- | --- |
+| Text encoder | `--[no-]fast-text-encoder` | Native eager forward | Static CUDA Graph selected by CFG shape and text-length bucket |
+| Backbone prefill | `--[no-]fast-backbone-prefill` | Native eager prefill | CUDA Graph selected by CFG shape and prompt-length bucket |
+| Backbone decode | `--[no-]fast-backbone-decode` | Native eager token step | StaticCache-backed graph selected by CFG shape |
+| Depth decoder | `--[no-]fast-depth-decoder` | Native eager depth loop | Full-graph compilation with CFG-shape CUDA Graphs |
+| Codec | `--[no-]fast-codec` | Eager streaming decode | Single-request streaming CUDA Graph with one-frame chunks |
+
+Individual stage flags are intended for profiling and debugging.
+
+
+## License and Responsible Use
+
+The source code is licensed under the [Apache License, Version 2.0](https://github.com/breezeblue-ai/breeze-tts/blob/main/LICENSE). Model weights, checkpoints, adapters, derivative models, and self-hosted outputs are governed separately by the [BreezeBlue Research and Non-Commercial License](./LICENSE). The Apache License does not grant rights to use the model commercially.
+
+Commercial use requires written authorization from RESONIA, INC. Hosted BreezeBlue services are governed by their applicable service terms. For commercial licensing, contact [contact@breeze.blue](mailto:contact@breeze.blue).
+
+You are responsible for complying with applicable laws and obtaining all necessary rights and consents for inputs, reference audio, voices, and outputs. Unauthorized voice cloning, impersonation, fraud, and other unlawful or harmful uses are prohibited.
+
+The code and Model Materials are provided "AS IS," without warranties or liability to the maximum extent permitted by law. Third-party components remain subject to their respective licenses.

audio_tokenizer/config.json

@@ -0,0 +1,94 @@
+{
+  "architectures": [
+    "Qwen3TTSTokenizerV2Model"
+  ],
+  "model_type": "qwen3_tts_tokenizer_12hz",
+  "encoder_valid_num_quantizers": 16,
+  "input_sample_rate": 24000,
+  "output_sample_rate": 24000,
+  "decode_upsample_rate": 1920,
+  "encode_downsample_rate": 1920,
+  "decoder_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "latent_dim": 1024,
+    "codebook_dim": 512,
+    "codebook_size": 2048,
+    "decoder_dim": 1536,
+    "hidden_act": "silu",
+    "hidden_size": 512,
+    "intermediate_size": 1024,
+    "layer_scale_initial_scale": 0.01,
+    "max_position_embeddings": 8000,
+    "head_dim": 64,
+    "num_attention_heads": 16,
+    "num_hidden_layers": 8,
+    "num_key_value_heads": 16,
+    "num_quantizers": 16,
+    "num_semantic_quantizers": 1,
+    "rms_norm_eps": 1e-05,
+    "rope_theta": 10000,
+    "semantic_codebook_size": 4096,
+    "sliding_window": 72,
+    "upsample_rates": [
+      8,
+      5,
+      4,
+      3
+    ],
+    "upsampling_ratios": [
+      2,
+      2
+    ],
+    "vector_quantization_hidden_dimension": 512
+  },
+  "encoder_config": {
+    "_frame_rate": 12.5,
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "audio_channels": 1,
+    "codebook_dim": 256,
+    "codebook_size": 2048,
+    "compress": 2,
+    "dilation_growth_rate": 2,
+    "dtype": "float32",
+    "head_dim": 64,
+    "hidden_act": "gelu",
+    "hidden_size": 512,
+    "initializer_range": 0.02,
+    "intermediate_size": 2048,
+    "kernel_size": 7,
+    "last_kernel_size": 3,
+    "layer_scale_initial_scale": 0.01,
+    "max_position_embeddings": 8000,
+    "norm_eps": 1e-05,
+    "normalize": false,
+    "num_attention_heads": 8,
+    "num_filters": 64,
+    "num_hidden_layers": 8,
+    "num_key_value_heads": 8,
+    "num_quantizers": 32,
+    "num_residual_layers": 1,
+    "num_semantic_quantizers": 1,
+    "pad_mode": "constant",
+    "residual_kernel_size": 3,
+    "rope_theta": 10000.0,
+    "sampling_rate": 24000,
+    "sliding_window": 250,
+    "transformers_version": "4.57.0.dev0",
+    "trim_right_ratio": 1.0,
+    "upsample_groups": 512,
+    "upsampling_ratios": [
+      8,
+      6,
+      5,
+      4
+    ],
+    "use_cache": false,
+    "use_causal_conv": true,
+    "use_conv_shortcut": false,
+    "use_streaming": false,
+    "vector_quantization_hidden_dimension": 256
+  },
+  "transformers_version": "4.57.3"
+}

audio_tokenizer/configuration.json

@@ -0,0 +1 @@
+{"framework": "pytorch", "task": "feature-extraction", "allow_remote": true}

audio_tokenizer/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:836b7b357f5ea43e889936a3709af68dfe3751881acefe4ecf0dbd30ba571258
+size 682293092

audio_tokenizer/preprocessor_config.json

@@ -0,0 +1,10 @@
+{
+  "chunk_length_s": null,
+  "feature_extractor_type": "EncodecFeatureExtractor",
+  "feature_size": 1,
+  "overlap": null,
+  "padding_side": "right",
+  "padding_value": 0.0,
+  "return_attention_mask": true,
+  "sampling_rate": 24000
+}

config.json

@@ -0,0 +1,441 @@
+{
+  "architectures": [
+    "BreezeForConditionalGeneration"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "audio_embed_size": 2048,
+  "audio_eos_token_id": 262145,
+  "audio_num_codebooks": 16,
+  "audio_token_id": 262144,
+  "audio_vocab_size": 2051,
+  "backbone_flavor": "llama-1B",
+  "backbone_model_name_or_path": null,
+  "backbone_model_type": "qwen3",
+  "bos_token_id": 2,
+  "codebook_eos_token_id": 0,
+  "codebook_pad_token_id": 2050,
+  "codec_config": {
+    "_frame_rate": 12.5,
+    "_name_or_path": "kyutai/mimi",
+    "architectures": [
+      "MimiModel"
+    ],
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "audio_channels": 1,
+    "codebook_dim": 256,
+    "codebook_size": 2048,
+    "compress": 2,
+    "dilation_growth_rate": 2,
+    "dtype": "bfloat16",
+    "head_dim": 64,
+    "hidden_act": "gelu",
+    "hidden_size": 512,
+    "initializer_range": 0.02,
+    "intermediate_size": 2048,
+    "kernel_size": 7,
+    "last_kernel_size": 3,
+    "layer_scale_initial_scale": 0.01,
+    "max_position_embeddings": 8000,
+    "model_type": "mimi",
+    "norm_eps": 0.00001,
+    "normalize": false,
+    "num_attention_heads": 8,
+    "num_filters": 64,
+    "num_hidden_layers": 8,
+    "num_key_value_heads": 8,
+    "num_quantizers": 32,
+    "num_residual_layers": 1,
+    "num_semantic_quantizers": 1,
+    "pad_mode": "constant",
+    "residual_kernel_size": 3,
+    "rope_theta": 10000.0,
+    "sampling_rate": 24000,
+    "sliding_window": 250,
+    "trim_right_ratio": 1.0,
+    "upsample_groups": 512,
+    "upsampling_ratios": [
+      8,
+      6,
+      5,
+      4
+    ],
+    "use_cache": false,
+    "use_causal_conv": true,
+    "use_conv_shortcut": false,
+    "use_streaming": false,
+    "vector_quantization_hidden_dimension": 256
+  },
+  "decoder_flavor": "llama-100M",
+  "depth_decoder_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "audio_embed_size": 2048,
+    "backbone_hidden_size": 2048,
+    "codebook_loss_weights": [
+      3
+    ],
+    "dtype": "bfloat16",
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 1024,
+    "initializer_range": 0.02,
+    "intermediate_size": 8192,
+    "max_position_embeddings": 33,
+    "mlp_bias": false,
+    "model_type": "breeze_depth_decoder_model",
+    "num_attention_heads": 8,
+    "num_codebooks": 16,
+    "num_hidden_layers": 12,
+    "num_key_value_heads": 2,
+    "rms_norm_eps": 0.00001,
+    "rope_scaling": {
+      "factor": 32.0,
+      "high_freq_factor": 0.0078125,
+      "low_freq_factor": 0.001953125,
+      "original_max_position_embeddings": 16,
+      "rope_type": "llama3"
+    },
+    "rope_theta": 500000,
+    "use_cache": true,
+    "vocab_size": 2051
+  },
+  "depth_header_loss_weight": 1.0,
+  "dtype": "bfloat16",
+  "eos_token_id": 1,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 6144,
+  "max_position_embeddings": 2048,
+  "mlp_bias": false,
+  "model_type": "breeze",
+  "num_attention_heads": 16,
+  "num_codebooks": 16,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 8,
+  "pad_token_id": 0,
+  "rms_norm_eps": 0.00001,
+  "rope_scaling": {
+    "factor": 32.0,
+    "high_freq_factor": 0.5,
+    "low_freq_factor": 0.125,
+    "original_max_position_embeddings": 1024,
+    "rope_type": "llama3"
+  },
+  "rope_theta": 500000,
+  "text_encoder_bucket_max_length_ratio": 4.0,
+  "text_encoder_config": {
+    "add_cross_attention": false,
+    "architectures": [
+      "T5Gemma2TextEncoder"
+    ],
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_logit_softcapping": null,
+    "bad_words_ids": null,
+    "begin_suppress_tokens": null,
+    "bos_token_id": 2,
+    "chunk_size_feed_forward": 0,
+    "cross_attention_hidden_size": null,
+    "decoder_start_token_id": null,
+    "diversity_penalty": 0.0,
+    "do_sample": false,
+    "dropout_rate": 0.0,
+    "dtype": "bfloat16",
+    "early_stopping": false,
+    "encoder_no_repeat_ngram_size": 0,
+    "eoi_token_index": 256000,
+    "eos_token_id": 1,
+    "exponential_decay_length_penalty": null,
+    "final_logit_softcapping": null,
+    "finetuning_task": null,
+    "forced_bos_token_id": null,
+    "forced_eos_token_id": null,
+    "head_dim": 256,
+    "hidden_activation": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_range": 0.02,
+    "intermediate_size": 6912,
+    "is_decoder": false,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention"
+    ],
+    "length_penalty": 1.0,
+    "max_length": 20,
+    "max_position_embeddings": 32768,
+    "min_length": 0,
+    "model_type": "t5gemma2_text",
+    "no_repeat_ngram_size": 0,
+    "num_attention_heads": 4,
+    "num_beam_groups": 1,
+    "num_beams": 1,
+    "num_hidden_layers": 26,
+    "num_key_value_heads": 1,
+    "num_return_sequences": 1,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "output_scores": false,
+    "pad_token_id": 0,
+    "preferred_attn_implementation": "flash_attention_2",
+    "prefix": null,
+    "problem_type": null,
+    "pruned_heads": {},
+    "query_pre_attn_scalar": 256,
+    "remove_invalid_values": false,
+    "repetition_penalty": 1.0,
+    "requires_grad": false,
+    "return_dict": true,
+    "return_dict_in_generate": false,
+    "rms_norm_eps": 0.000001,
+    "rope_parameters": {
+      "full_attention": {
+        "factor": 8.0,
+        "rope_theta": 1000000,
+        "rope_type": "linear"
+      },
+      "sliding_attention": {
+        "rope_theta": 10000,
+        "rope_type": "default"
+      }
+    },
+    "sep_token_id": null,
+    "sliding_window": 512,
+    "suppress_tokens": null,
+    "task_specific_params": null,
+    "temperature": 1.0,
+    "tf_legacy_loss": false,
+    "tie_encoder_decoder": false,
+    "tie_word_embeddings": true,
+    "tokenizer_class": null,
+    "top_k": 50,
+    "top_p": 1.0,
+    "torchscript": false,
+    "typical_p": 1.0,
+    "use_bfloat16": false,
+    "use_bidirectional_attention": false,
+    "use_cache": true,
+    "vocab_size": 262158
+  },
+  "text_encoder_lora_config": {
+    "alpha": 16.0,
+    "dropout": 0.05,
+    "enabled": false,
+    "freeze_base_model": true,
+    "layers": [
+      0,
+      1,
+      2,
+      3,
+      4,
+      5,
+      6,
+      7,
+      8,
+      9,
+      10,
+      11,
+      12,
+      13,
+      14,
+      15,
+      16,
+      17,
+      18,
+      19,
+      20,
+      21,
+      22,
+      23,
+      24,
+      25
+    ],
+    "merged_into_base": true,
+    "rank": 8,
+    "target_modules": [
+      "q_proj",
+      "v_proj",
+      "o_proj"
+    ]
+  },
+  "text_encoder_proj_type": "linear",
+  "text_encoder_special_tokens_config": {
+    "enabled": false,
+    "freeze_base_embeddings": true,
+    "initializer_token_ids": {
+      "<ins_bos>": [
+        236820,
+        22768,
+        236813
+      ],
+      "<ins_eos>": [
+        954,
+        22768,
+        236813
+      ],
+      "[S0]": [
+        262146
+      ],
+      "[S1]": [
+        262147
+      ],
+      "[S2]": [
+        262148
+      ],
+      "[S3]": [
+        262149
+      ],
+      "[S4]": [
+        262150
+      ],
+      "[S5]": [
+        262151
+      ],
+      "[S6]": [
+        262152
+      ],
+      "[S7]": [
+        262153
+      ],
+      "[S8]": [
+        262154
+      ],
+      "[S9]": [
+        262155
+      ]
+    },
+    "initializers": {
+      "<ins_bos>": "<instruction>",
+      "<ins_eos>": "</instruction>",
+      "[S0]": "[S0]",
+      "[S1]": "[S1]",
+      "[S2]": "[S2]",
+      "[S3]": "[S3]",
+      "[S4]": "[S4]",
+      "[S5]": "[S5]",
+      "[S6]": "[S6]",
+      "[S7]": "[S7]",
+      "[S8]": "[S8]",
+      "[S9]": "[S9]"
+    },
+    "instruction_tokens": {
+      "bos": "<ins_bos>",
+      "eos": "<ins_eos>"
+    },
+    "merged_into_base": true,
+    "num_added_tokens": 12,
+    "speaker_tokens": [
+      "[S0]",
+      "[S1]",
+      "[S2]",
+      "[S3]",
+      "[S4]",
+      "[S5]",
+      "[S6]",
+      "[S7]",
+      "[S8]",
+      "[S9]"
+    ],
+    "token_ids": {
+      "<ins_bos>": 262156,
+      "<ins_eos>": 262157,
+      "[S0]": 262146,
+      "[S1]": 262147,
+      "[S2]": 262148,
+      "[S3]": 262149,
+      "[S4]": 262150,
+      "[S5]": 262151,
+      "[S6]": 262152,
+      "[S7]": 262153,
+      "[S8]": 262154,
+      "[S9]": 262155
+    },
+    "tokens": [
+      "[S0]",
+      "[S1]",
+      "[S2]",
+      "[S3]",
+      "[S4]",
+      "[S5]",
+      "[S6]",
+      "[S7]",
+      "[S8]",
+      "[S9]",
+      "<ins_bos>",
+      "<ins_eos>"
+    ],
+    "trainable": true
+  },
+  "text_vocab_size": 262158,
+  "tie_codebooks_embeddings": true,
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.3",
+  "use_cache": true,
+  "vocab_size": 2051,
+  "backbone_config": {
+    "architectures": [
+      "Qwen3ForCausalLM"
+    ],
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 151643,
+    "eos_token_id": 151645,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 2048,
+    "initializer_range": 0.02,
+    "intermediate_size": 6144,
+    "max_position_embeddings": 40960,
+    "max_window_layers": 28,
+    "model_type": "qwen3",
+    "num_attention_heads": 16,
+    "num_hidden_layers": 28,
+    "num_key_value_heads": 8,
+    "rms_norm_eps": 0.000001,
+    "rope_scaling": null,
+    "rope_theta": 1000000,
+    "sliding_window": null,
+    "tie_word_embeddings": true,
+    "torch_dtype": "bfloat16",
+    "transformers_version": "4.51.0",
+    "use_cache": true,
+    "use_sliding_window": false,
+    "vocab_size": 151936
+  }
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 2,
+  "depth_decoder_do_sample": true,
+  "depth_decoder_temperature": 0.9,
+  "do_sample": true,
+  "eos_token_id": [
+    1
+  ],
+  "max_new_tokens": 750,
+  "pad_token_id": 0,
+  "temperature": 0.9,
+  "transformers_version": "4.57.3"
+}

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:abf813781256e10cbe81f2dbb415f897556225d4dfa0282d67aa8ea164e114a9
+size 4961989890

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36aa73b1a11361e1774db90d9c63c63303b294c022de112aa51904d940edcef1
+size 2004567152

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "additional_special_tokens": [
+    "[S0]",
+    "[S1]",
+    "[S2]",
+    "[S3]",
+    "[S4]",
+    "[S5]",
+    "[S6]",
+    "[S7]",
+    "[S8]",
+    "[S9]",
+    "<ins_bos>",
+    "<ins_eos>"
+  ],
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3ec9ac3eb2392389b9f5112e85d8b43316494addb587ba7b7a9d61eac23af96
+size 33386945