Modularcomputing/dynamic-accent-pro-v1.1

Dynamic Accent Pro v1.1

Experimental prompt-controlled Nigerian TTS and code-switching technology by Modular. English, Nigerian Pidgin, Igbo, Yoruba and Hausa, mixed inside one sentence and spoken by one consistent voice, with a plain-English style instruction (--instruct) that steers pace and pausing.

Pro v1.1 is a research release, not a production system. It is the Flash recipe (CosyVoice3 LM fine-tuned on licensed Nigerian-language speech, then DPO) with a new instruction layer on top. Read What works and what does not before using it.

What works and what does not

control	status	how
Pace ("speak slowly", "speak fast")	works	read by the LM from the instruction, sharpened by classifier-free guidance (--guidance 2, the default). Slow vs fast is ordered correctly on 18/18 held-out items; slow ≈ 0.89× and fast ≈ 1.15× the default articulation rate
Pauses ("plenty of pauses", "hardly any pauses")	works through a text step	the instruction also edits the text before synthesis (pause_text.py: inserts the model's learned [pause] tag for "many", strips commas and pause tags for "few"). Many ≈ 1.28× default pause rate; few ≈ 0.91×. The LM alone barely moves pauses, so --no_text_pauses mostly turns this off
Calm / lively tone	not advertised	trained, but indistinguishable from chance in our measurements. The words are accepted and ignored in effect
Emotion (happy, sad, angry, …)	no	not trained, no data
Code-switching from tags (<ibo>…</ibo> <eng>…</eng>)	works (the core Dynamic Accent capability)	language embeddings + a speaker token keep one voice across switches
Code-switching from a request (--codeswitch_prompt "mostly English, Igbo greetings")	experimental, separate step	a text LLM (Qwen/Qwen3-32B, Apache-2.0, not bundled) rewrites plain text into tagged code-switched text before synthesis. It is not a TTS capability. Its Pidgin and Hausa are usable; its Igbo and Yoruba beyond simple greetings are weak and often wrong, so always read the printed tagged text and fix it

Without --instruct, infer.py uses Flash-DPO (the default version). Passing --instruct switches to Pro automatically unless you set --version; --version pro without an instruction uses Pro's default prompt, trained on native-like clips.

Versions and voices

version	LM	notes
pro (used automatically with --instruct)	checkpoints/pro/flash_instruct	Pro v1.1: Flash-DPO → instruction SFT → DPO polish. Instruction control, CFG 2.0 on the instruction, text pause step
flash (default)	checkpoints/flash/flash_dpo	Flash-DPO (no instruction control), the default when no --instruct is given: it sounds more natural on plain code-switched text. A/B: --compare renders the same text with both

Both use the stock Fun-CosyVoice3-0.5B-2512 flow and vocoder, zero-shot from the voice clip, and the same text front-end (numbers verbalised; missing Yoruba/Igbo diacritics restored by checkpoints/flash/diacritics_byt5).

voice (--voice)	who
anchor_male (default)	WAXAL Media Trust Yoruba speaker 4 (male), the voice both models were trained around
waxal_ibo_5_female	WAXAL Igbo speaker 5 (female), zero-shot from a 6.6 s clip
waxal_yor_5_female	WAXAL Yoruba speaker 5 (female), zero-shot; short prompt, so short pauses and more clipped endings

Setup (single CUDA GPU)

git clone --recursive https://github.com/FunAudioLLM/CosyVoice.git
cd CosyVoice && git checkout 074ca6dc9e80a2f424f1f74b48bdd7d3fea531cc && git submodule update --init --recursive && cd ..
huggingface-cli download Modularcomputing/dynamic-accent-pro-v1.1 --local-dir dynamic-accent-pro
pip install -r dynamic-accent-pro/requirements.txt

Fun-CosyVoice3-0.5B-2512 downloads on first run (--cv3_dir / CV3_DIR to reuse a copy). Tested on an H100 in bf16; a 16 GB GPU is enough for TTS. The optional code-switch step needs a GPU that holds Qwen3-32B in bf16 (~65 GB); --cs_llm Qwen/Qwen3-14B is lighter and worse.

Usage

cd dynamic-accent-pro
# default prompt (native-like pace and pauses)
python infer.py --cosyvoice_dir ../CosyVoice --text "<ibo>Ndewo nụ, ụmụnna m.</ibo> <eng>Tonight we will talk about the new market road.</eng>" --out out.wav

# pace
python infer.py --cosyvoice_dir ../CosyVoice --instruct "Please speak slowly and take your time." --text "..." --out slow.wav
python infer.py --cosyvoice_dir ../CosyVoice --instruct "Speak fast, like a quick everyday conversation." --text "..." --out fast.wav

# pauses (text step + instruction)
python infer.py --cosyvoice_dir ../CosyVoice --instruct "Speak with plenty of pauses between phrases." --text "..." --out many.wav
python infer.py --cosyvoice_dir ../CosyVoice --instruct "Say it in one smooth flow with hardly any pauses." --text "..." --out few.wav

# female Igbo voice
python infer.py --cosyvoice_dir ../CosyVoice --voice waxal_ibo_5_female --text "<ibo>Kedu ka ị mere?</ibo> <pcm>Abeg no vex, I go call you for evening.</pcm>" --out ibo_f.wav

# experimental: plain text + a code-switch request -> Qwen3-32B writes the tagged text (check it!) -> TTS
python infer.py --cosyvoice_dir ../CosyVoice --codeswitch_prompt "Nigerian Pidgin with an Igbo greeting" \
  --text "My brother, the work is too much today, but we will finish it." --out cs.wav

# A/B the same text, seed and voice with Flash and Pro -> cmp_flash.wav, cmp_pro.wav
python infer.py --cosyvoice_dir ../CosyVoice --compare --compare_versions flash,pro --text "..." --out cmp.wav

flag	what it does
--instruct "..."	pro: natural-language style instruction. Reliable: slow / fast; plenty of / hardly any pauses. Phrasings close to the examples above work best
--guidance G	pro: classifier-free guidance strength on the instruction (default 2.0; 0 = off, weaker pace control)
--no_text_pauses	pro: do not edit the text for pause instructions
--codeswitch_prompt "..."	optional text-LLM step before synthesis (see above); output is validated (tags, length, loops, back-translation meaning check) and falls back to the input text
--cs_llm	the LLM for that step (default Qwen/Qwen3-32B)
--version pro|flash, --compare, --compare_versions	model version / A/B rendering
--voice, --list_voices	voice selection
--auto_tag, --tag_only, --force_lang, --lang_map	language tagging of untagged text (see tagger.py)
--frontend codeswitch|native|accent, --frontend_lang, --no_diacritics	text front-end
--seed N	sampling seed (default 0). Try 2–3 seeds and keep the best take
--tail_pad N	append N silence speech tokens before the vocoder so the last syllable is not clipped (opt-in; 6 tested)

Text: wrap each language span in <eng>, <pcm>, <yor>, <ibo> or <hau>; untagged text is read as English unless --auto_tag / --force_lang is given. Write Yoruba and Igbo with diacritics where you can. GPU generation is not bit-reproducible (bf16 near-ties).

Samples

samples/ holds MP3 renders for all five languages and a code-switched sentence, both main voices, with the default prompt and the slow / fast / few-pauses / many-pauses instructions, plus a Flash-DPO render of the same text and seed for A/B (samples/meta.json lists text, voice and instruction per file). samples/codeswitch_frontend/ shows the experimental Qwen3-32B step: input text, request, the tagged text it produced, and the audio. The Igbo and Yoruba demo sentences have not yet been checked by a native speaker.

Results

Automatic regression set: 197 held-out sentences per voice (30 each of Igbo, Yoruba, Hausa, Pidgin, English; 24 code-switched; 23 paragraphs), default prompt, seed 0. CER from automatic ASR judges (per language, same judges for every system), pace and pause rate relative to native recordings of the same languages, clipped ending = less than 0.1 s of trailing silence, SIM = speaker similarity to the prompt clip, UTMOSv2 = predicted naturalness.

voice	group	system	CER	CER no-tone	pace × native	pauses × native	clipped endings	SIM	UTMOSv2
anchor_male	all	Flash-DPO	0.162	0.135	0.99	0.75	7%	0.784	3.17
anchor_male	all	Pro v1.1	0.167	0.141	0.99	1.02	8%	0.789	3.16
anchor_male	ibo	Flash-DPO	0.189	0.158	1.06	0.40	7%	0.762	3.06
anchor_male	ibo	Pro v1.1	0.216	0.181	1.02	0.62	3%	0.779	3.01
anchor_male	yor	Flash-DPO	0.230	0.144	1.08	0.30	3%	0.762	2.99
anchor_male	yor	Pro v1.1	0.218	0.141	1.06	0.68	3%	0.770	3.02
anchor_male	hau	Flash-DPO	0.123	0.123	0.83	0.77	0%	0.788	3.26
anchor_male	hau	Pro v1.1	0.147	0.147	0.81	1.19	3%	0.796	3.19
anchor_male	pcm	Flash-DPO	0.227	0.227	0.97	0.72	10%	0.782	3.26
anchor_male	pcm	Pro v1.1	0.199	0.199	1.01	0.97	3%	0.779	3.25
anchor_male	eng	Flash-DPO	0.026	0.026	0.94	1.85	17%	0.790	3.26
anchor_male	eng	Pro v1.1	0.031	0.031	0.94	1.89	7%	0.796	3.26
anchor_male	cs	Flash-DPO	0.208	0.159	1.02	1.00	12%	0.773	3.09
anchor_male	cs	Pro v1.1	0.213	0.162	1.05	1.27	25%	0.766	3.22
anchor_male	para	Flash-DPO	0.131	0.106	1.06	1.08	0%	0.843	3.28
anchor_male	para	Pro v1.1	0.149	0.126	1.05	1.29	13%	0.843	3.16
waxal_ibo_5_female	all	Flash-DPO	0.165	0.134	1.03	0.95	11%	0.700	3.51
waxal_ibo_5_female	all	Pro v1.1	0.152	0.123	1.01	0.84	12%	0.698	3.45
waxal_ibo_5_female	ibo	Flash-DPO	0.215	0.171	1.09	0.52	7%	0.697	3.41
waxal_ibo_5_female	ibo	Pro v1.1	0.178	0.145	1.07	0.58	0%	0.682	3.40
waxal_ibo_5_female	yor	Flash-DPO	0.251	0.150	1.18	0.86	7%	0.661	3.45
waxal_ibo_5_female	yor	Pro v1.1	0.224	0.118	1.11	0.38	10%	0.671	3.36
waxal_ibo_5_female	hau	Flash-DPO	0.117	0.117	0.86	1.20	7%	0.700	3.51
waxal_ibo_5_female	hau	Pro v1.1	0.132	0.132	0.85	0.74	13%	0.676	3.47
waxal_ibo_5_female	pcm	Flash-DPO	0.184	0.183	1.02	0.92	13%	0.674	3.63
waxal_ibo_5_female	pcm	Pro v1.1	0.152	0.151	1.02	0.83	20%	0.690	3.48
waxal_ibo_5_female	eng	Flash-DPO	0.031	0.031	0.99	1.34	10%	0.726	3.60
waxal_ibo_5_female	eng	Pro v1.1	0.031	0.031	0.93	1.62	13%	0.745	3.52
waxal_ibo_5_female	cs	Flash-DPO	0.230	0.179	1.01	1.32	29%	0.682	3.50
waxal_ibo_5_female	cs	Pro v1.1	0.220	0.174	1.07	1.22	17%	0.660	3.40
waxal_ibo_5_female	para	Flash-DPO	0.133	0.106	1.07	0.99	4%	0.773	3.44
waxal_ibo_5_female	para	Pro v1.1	0.141	0.114	1.05	1.20	9%	0.772	3.50

Native-speaker reference: pace 1.00 / pauses 1.00 by definition. Summary: Pro v1.1 matches Flash-DPO on overall CER for the anchor voice (0.167 vs 0.162) and beats it for the female Igbo voice (0.152 vs 0.165); it pauses much closer to native in Igbo and Yoruba on the anchor voice (0.62× / 0.68× vs 0.40× / 0.30×); it is worse than Flash-DPO on anchor Igbo CER (0.216 vs 0.189) and on Hausa for both voices.

Instruction adherence (anchor voice, 18 held-out Igbo/Yoruba/Hausa/Pidgin/English/code-switch items, CFG 2, text pause step on; "direction" = share of items moving the requested way vs the default prompt):

instruction	articulation rate × default	pause rate × default	direction correct
Please speak slowly and take your time.	0.89	1.16	89%
Speak fast, like a quick everyday conversation.	1.15	0.87	94%
Say it in one smooth flow with hardly any pauses.	0.92	0.91	56%
Speak with plenty of pauses between phrases.	0.90	1.28	78%
Use a calm, gentle, even tone. (not advertised)	0.98	1.02	50%
Sound lively and expressive. (not advertised)	0.96	0.94	44%

Slow vs fast on the same item: correct order on 100% of items; many vs few pauses: 67%; lively vs calm: 50% (chance). Note that "slow" also adds pauses (1.16× pause rate). 18 items is a small set; treat these as indicative.

Limitations

• Experimental. Pace control is real; pause control depends on the text step; calm/lively and emotion are not available.
• Anchor Igbo CER is still higher than Flash-DPO (0.216 vs 0.189); Hausa CER is worse than Flash-DPO for both voices. Flash-DPO remains available as --version flash.
• English stays over-paused relative to native English (about 1.9× native pause rate on the anchor voice, the same as Flash-DPO).
• Clipped endings remain on code-switched sentences and paragraphs (13–25% on the anchor voice). --tail_pad 6 (six silence tokens appended before the vocoder, inference-only) removed every measured clipped ending in testing with no consistent CER change (+0.005 overall, within seed noise); it is opt-in until it has been checked by ear.
• The code-switch request step (Qwen3-32B) is a separate text model: weak Igbo/Yoruba beyond greetings, sometimes refuses to switch and falls back to the input text.
• Pidgin has little training data and no reliable ASR judge; Yoruba tone quality needs native listening.
• Female voices are zero-shot. Not evaluated for numbers, abbreviations or URLs.
• Do not use it to impersonate real people. The voices are WAXAL Media Trust speakers, released under CC-BY-SA-4.0 for speech technology.

Training

1. Flash-DPO (the flash version): CosyVoice3 LM fine-tuned on the licensed set below (language embeddings, speaker token, adversarial speaker classifier, rate embedding), then DPO on 1,206 preference pairs.
2. Instruction SFT (3,000 steps): ~100.9k licensed clips labelled from the audio (articulation rate, pause rate, energy/F0 variation); each training item gets a natural-language instruction built from its labels in the CosyVoice3 system-prompt slot, plus explicit pause/energy control embeddings; 18% of items keep the default prompt, drawn from native-like clips.
3. DPO polish (134 steps, β 0.1, lr 1e-6, SFT weight 0.1): 700 prompts × 4 samples from the SFT model, 564 preference pairs; reward = CER + pace and pause-rate match to native + a clipped-ending penalty + a small UTMOS guard.

Inference: classifier-free guidance on the instruction (conditional = instruction + controls, unconditional = default prompt), g = 2.

What is in the repo

path	what it is	licence
checkpoints/pro/flash_instruct/	Pro v1.1 LM (best.pt, sha256 87c3a7e9…4cab3db), config.json, spk2id.json, DPO args.json, log.jsonl	CC-BY-SA-4.0
checkpoints/flash/flash_dpo/	Flash-DPO LM (best.pt, sha256 f59a6fc0…7e3a307)	CC-BY-SA-4.0
checkpoints/flash/diacritics_byt5/	Yoruba/Igbo diacritic restoration (fine-tune of google/byt5-small)	CC-BY-SA-4.0
prompts/	voice clips and transcripts (WAXAL)	CC-BY-SA-4.0
samples/	demo renders	CC-BY-SA-4.0
config.json	repo-level manifest (also lets the Hub count downloads)	Apache-2.0
settings.json	versions, voices, defaults	Apache-2.0
infer.py, flash_lm.py, flash_instruct.py, instruct_text.py, pause_text.py, cs_frontend.py, tagger.py, frontend/, code/	inference, model, instruction, text pause step, code-switch step, tagger, front-end, reference training code	Apache-2.0

Training data and attribution

Licensed data only. The weights are trained on, and the voice prompts taken from:

dataset	licence	attribution
WAXAL, Media Trust TTS subsets (Igbo, Yoruba, Hausa, Nigerian Pidgin); all voice prompts	CC-BY-SA-4.0	Google and Media Trust, WAXAL (arXiv:2602.02734)
Open.Bible Igbo	CC-BY-SA-4.0	Biblica, Inc. via open.bible
NaijaS2ST (Igbo)	CC-BY-4.0	McGill-NLP (arXiv:2604.16287)
IgboAPI	CC-BY-4.0	Nkọwa okwu / IgboAPI contributors
IgboSynCorp (+ pseudo-transcripts)	CC0-1.0	University of Ibadan and Afe Babalola University (Lacuna Fund)
FLEURS	CC-BY-4.0	Google (Conneau et al., 2022)
BibleTTS (Yoruba, Hausa)	CC-BY-SA-4.0	Meyer et al. (2022)
OpenSLR SLR70, SLR86	CC-BY-SA-4.0	Google
synthetic code-switch pairs (concatenated clips from the sets above)	follows the sources	after Zhang et al. (2019)
base model Fun-CosyVoice3-0.5B-2512	Apache-2.0	FunAudioLLM / Alibaba Tongyi Lab
optional Qwen3-32B (not bundled; code-switch step only)	Apache-2.0	Qwen team, Alibaba

DPO preference pairs are the model's own renders, scored automatically. Excluded on licence grounds: African Voices / WaZoBiaSpeech (cards ban TTS use), NaijaVoices (non-commercial), and non-commercial TTS weights.

Licence

• Model weights, voice prompts and samples (checkpoints/, prompts/, samples/): CC-BY-SA-4.0. Commercial use is allowed; credit Modular and the datasets above; derivatives of the weights must be shared under CC-BY-SA-4.0.
• Code (*.py, frontend/, code/, settings.json, config.json): Apache-2.0.
• Fun-CosyVoice3-0.5B-2512, byt5-small and Qwen3 are Apache-2.0; their notices apply to the parts they contribute.

Citation

@misc{dynamicaccentpro2026,
  title  = {Dynamic Accent Pro v1.1: prompt-controlled code-switching TTS for Nigerian languages},
  author = {Modular},
  year   = {2026},
  url    = {https://huggingface.co/Modularcomputing/dynamic-accent-pro-v1.1}
}