AMAImedia/NOESIS-Talker-Combo-LLM2-ASR2-BF16

NOESIS / AMAImedia

Released as part of the NOESIS Professional Multilingual Dubbing Automation Platform (framework: DHCF-FNO -- Deterministic Hybrid Control Framework for Frozen Neural Operators).

• Founder: Ilia Bolotnikov
• Organization: AMAImedia.com
• X (Twitter): @AMAImediacom
• LinkedIn: Ilia Bolotnikov
• Telegram: @AMAImediacom
• NOESIS version: v16.1
• Release date: 2026-06-11; r16 adapter update 2026-08-28

---

NOESIS-Talker-Combo (Darwin cross-modal 1.7B, 112-language tested route)

Production 1.7B Talker bundle for the NOESIS Voice Router-Swap. The frozen Combo backbone combines the Qwen3-TTS-1.7B Talker with a small cross-modal FFN blend from Qwen3-1.7B-LLM and Mega-ASR. The supplied per-family LoRA adapters are part of the NOESIS 112-language tested route; they remain separate and are selected by family without merging all adapters into one destructive checkpoint.

This directory is a model-weight and adapter bundle, not a standalone GUI application. In production it is loaded by the parent NOESIS router and its resident Qwen TTS worker. The router selects one language family, applies one adapter at the prescribed scale, synthesizes the current segment, and swaps the adapter or model only when the target language changes.

NOESIS provenance

Property	Value
Bundle	NOESIS-Talker-Combo-LLM2-ASR2-BF16
Parent bundle	NOESIS-3.5B-A0.5B-DUBBING-FILM/NOESIS-Qwen3-Omni-MoE-BF16
Upstream base	Qwen/Qwen3-TTS-12Hz-1.7B-Base, Qwen Team / Alibaba, Apache-2.0
NOESIS blend	Darwin cross-modal 1.7B merge: Talker + Combo + Qwen3-LLM + Qwen3-ASR + Qwen3-TTS Base branches; see the lineage table below
NOESIS role	Fallback and specialist 1.7B Talker route; OmniVoice 0.6B is primary generic clone route
Adapter training	FLEURS + Common Voice 25.0, family LoRA and GRPO/DPO post-pass
Main adapter rank	LoRA r=8, scale 0.32
Updated adapter rank	indic and iranian also have r=16 variants
Sealed rule	R-TALKER-112LANG-PER-FAMILY-SWAP-NOT-MERGE-2026-06-11
Maintainer	Ilia Bolotnikov / AMAImedia

Darwin / Qwen lineage

This bundle is the NOESIS Darwin cross-modal 1.7B Talker line. The model history has two distinct stages: first, the 1.7B speech/text branches were combined in the Darwin merge; second, the resulting Talker route was expanded with NOESIS training and family adapters. Language expansion did not mean merging all LoRA adapters into one checkpoint.

Stage	Component	Role and source
1	Qwen3-TTS Talker 1.7B	Speech-generating Talker backbone in the Qwen3-TTS family.
2	Qwen3-TTS Base 1.7B	Original TTS base scaffold: Qwen/Qwen3-TTS-12Hz-1.7B-Base.
3	Qwen3 LLM 1.7B	Text-language-model branch: Qwen/Qwen3-1.7B.
4	Qwen3 ASR 1.7B	Speech-understanding/ASR branch: Qwen/Qwen3-ASR-1.7B and its Transformers checkpoint Qwen/Qwen3-ASR-1.7B-hf.
5	Qwen/NOESIS Combo 1.7B	The intermediate Combo branch that combines the Talker, LLM and ASR-side information. It is a derived NOESIS branch, not a separate public upstream Qwen checkpoint; the published BF16 artifact is this repository, with the related public quantized Darwin artifact AMAImedia/Qwen3-1.7B-TTS-Cross-Darwin-NOESIS-AWQ-INT4.
6	Darwin merge	The cross-modal 1.7B Talker result used as the frozen backbone for the NOESIS Combo route. The public Darwin reference is FINAL-Bench/Darwin-TTS-1.7B-Cross, whose documented construction blends the Qwen3-TTS Base and Qwen3-1.7B LLM branches in weight space.
7	NOESIS language expansion	Family-specific LoRA/GRPO/DPO training and routing expanded the tested NOESIS surface to 112 languages. One family adapter is selected per request; adapters are never summed into a single destructive checkpoint.

The five 1.7B branches above must not be confused with five independent public Qwen repositories: the original Qwen Base, LLM and ASR checkpoints are public links; Talker/Combo are the speech and derived cross-modal branches represented by the Qwen3-TTS/Darwin lineage and this NOESIS artifact. The model card keeps this distinction explicit for reproducibility and attribution.

Language and test accounting

The canonical NOESIS claim for this bundle is 112 languages tested end-to-end (112/112). The 95-code family-adapter table below is a routing table for the documented family adapters, not the complete count of the NOESIS test surface. The legacy file named manifest_94.json is a partial adapter-sweep manifest and must not be interpreted as the total language coverage. The upstream Qwen3-TTS codec itself has 10 native language IDs; the remaining languages use the NOESIS family, sibling, or generic route selected by the production router.

What is in the bundle

NOESIS-Talker-Combo-LLM2-ASR2-BF16/
├── README.md
├── LICENSE
├── model.safetensors             # Combo 1.7B Talker base
├── config.json
├── generation_config.json
├── tokenizer_config.json
├── vocab.json / merges.txt
├── speech_tokenizer/              # Qwen3-TTS audio tokenizer assets
├── adapters/                      # one family adapter at a time
└── test_audio_perfamily/          # evaluation references and manifests

The deprecated attempt to sum all eleven adapters into one merged checkpoint is not the production model. Merging unrelated family adapters destroyed EOS and codec behavior and produced long non-speech outputs. Always load the Combo base and select exactly one family adapter when a non-native route is required.

Language → family adapter

Family	Adapter directory	Languages
germanic	adapters/NOESIS-Talker-Combo-germanic-LoRA-r8-BF16	en, de, nl, sv, da, nb, is, af
romance	adapters/…-romance-…	es, fr, it, pt, ro, ca, gl, oc
slavic	adapters/…-slavic-…	ru, uk, pl, cs, sk, sl, hr, sr, bg, mk, be
sinitic	adapters/…-sinitic-…	zh, yue
other_eurasian	adapters/…-other_eurasian-…	ja, ko, mn, hy, ka, fi, hu, et, lv, lt, el, eu
semitic	adapters/…-semitic-…	ar, he, mt, am, ti
indic	adapters/…-indic-…	hi, bn, mr, ta, te, pa, ur, gu, kn, ml, or, as, ne
turkic	adapters/…-turkic-…	tr, uz, kk, az, ky, tk, ug
iranian	adapters/…-iranian-…	fa, ps, tg, ckb, kmr
nigero_congo	adapters/…-nigero_congo-…	sw, ha, yo, zu, ig, xh, sn, ny, lg, om, so, wo, kam, ln, luo, rw
austronesian	adapters/…-austronesian-…	id, ms, vi, th, fil, jv, mi, ceb

The ten native Qwen3-TTS languages -- en, ru, zh, de, fr, es, ja, ko, it, and pt -- normally use plain Combo without a family adapter. Arabic (ar) is routed through the semitic family adapter in this bundle. Other languages select the matching family, sibling, or generic route. The production router performs this mapping automatically.

r16 adapter update

The indic and iranian families have r16 variants and should use them when the directory is present. The router falls back to r8 if an r16 directory is unavailable. The semitic family remains on r8. Several Brahmic Indic languages—especially te, pa, ml, or, and kn—showed EOS overgeneration in the current sweep; increasing rank alone does not fix that behavior. They require a dedicated training and EOS-fix pass rather than repeated re-generation.

Quality status

The complete NOESIS test program covered all 112 languages (112/112 tested). The dated per-family adapter sweep is a subset of that program: its legacy manifest_94.json records 86 scored checks plus six no-FLEURS skips and is not a total-coverage manifest. The full 112-language result is the canonical coverage claim; each route must still pass the normal NOESIS ASR, speaker-similarity, emotion, and timing gates before production acceptance.

Status	Guidance
Strong families	Romance, Slavic, Germanic, Austronesian, Other Eurasian, and Sinitic were the most consistent in the recorded sweep.
Low-resource / ASR-gap languages	am, as, ckb, ha, he, hy, ig, kam, ky, luo, nb, ps, so, ta, tg, yo need native listening checks because generic ASR scores can under-report quality.
EOS overgeneration	te, pa, ml, or, and kn can run to the token cap without a valid EOS and must not be accepted without a successful quality gate.
Production decision	Use the router and keep the 0.6B OmniVoice bundle as the primary generic clone engine; use Combo as the designated fallback or language route.

How to run in NOESIS production

There is intentionally no run.bat in this model directory. Do not launch the safetensors file directly. The supported route is the parent NOESIS runtime:

NOESIS-3.5B-A0.5B-DUBBING-FILM/
├── demo_server/_voice_swap_runner.py     # router / subprocess orchestration
├── NOESIS-Qwen3-Omni-MoE-BF16/
│   ├── _qwen_tts_worker.py              # resident JSON-IPC worker
│   └── NOESIS-Talker-Combo-LLM2-ASR2-BF16/
│       ├── model.safetensors
│       └── adapters/
└── dubbing/                              # timing and quality pipeline

The worker runs inside the frozen _venv_qwen_tts_457 environment. On Windows, its subprocess must inherit PYTHONUTF8=1 and PYTHONIOENCODING=utf-8; otherwise non-ASCII text can corrupt the JSON IPC stream.

A production request contains the target text, the reference audio or speaker embedding, the target language, the selected family adapter, and the adapter scale:

{
  "op": "clone",
  "text": "Welcome to the Sanctuary, Miss Amelia.",
  "target_lang": "en",
  "adapter": "germanic",
  "adapter_scale": 0.32
}

The exact worker schema is owned by the parent router and may add reference-audio, x-vector, duration, and cache fields. The adapter policy does not change: one family adapter, one scale, one active Talker route. Do not send all family adapters in one request.

Typical Windows environment

cd /d B:\Downloads\Portable\NOESIS-VC-ONE\models\llm\NOESIS-3.5B-A0.5B-DUBBING-FILM
set PYTHONUTF8=1
set PYTHONIOENCODING=utf-8
set NOESIS_TTS_PRIMARY=combo

REM The parent production server starts the resident worker and routes requests.
REM Start it using the bundle's normal demo_server launcher; do not run the
REM model.safetensors file as a Python script.

NOESIS_TTS_PRIMARY=combo is an explicit fallback selection. The default generic production route is OmniVoice (omni06); set the variable only when testing or intentionally routing through Combo.

Direct model-loading outline

For development and diagnostics, load the base and one adapter through the Qwen3-TTS/PEFT runtime used by the parent worker. The following is an outline of the required policy, not a replacement for the pinned worker implementation:

# Pseudocode: use the parent worker's pinned loader and tokenizer setup.
base = load_qwen3_tts("NOESIS-Talker-Combo-LLM2-ASR2-BF16")
base.load_lora("adapters/NOESIS-Talker-Combo-romance-LoRA-r8-BF16")
base.set_lora_scale(0.32)
waveform = base.generate_voice_clone(
    text="Bonjour, bienvenue au Sanctuaire.",
    reference_audio="clean_reference.wav",
    language="fr",
)
base.unload_lora()

The parent worker also handles CUDA placement, reference-text or x-vector requirements, output normalization, caching, and cleanup. Reimplementing those pieces outside the worker can create VRAM leaks or a Talker ICL failure.

Hardware and runtime notes

Property	Value
Official upstream model class	Qwen3-TTS Talker/Base 1.7B
NOESIS artifact class	Darwin/Combo cross-modal 1.7B derivative
Actual stored parameters in this derived checkpoint	1,928,677,440 (~1.929B)
Hugging Face display	2B params (automatic rounded display)
Base checkpoint	3.857 GB decimal (3.593 GiB) model.safetensors
Adapter size	Approximately 38.5 MB per r8 adapter and 77.0 MB per r16 adapter; full repository includes all adapter files
Full bundle size on disk	5.309 GB decimal (4.944 GiB) including base, speech tokenizer, adapters and test assets
Inference dtype	BF16 bundle / CUDA inference through the pinned worker
Generation time	Approximately 3–8 seconds for a short utterance in the recorded sweep
Family swap	Approximately 16 seconds when a base reload and LoRA merge are required
Runtime	Frozen _venv_qwen_tts_457, JSON IPC, one resident worker per GPU
Reference	Use a clean, single-speaker reference with transcript when ICL mode requires it

Failure handling

If a Talker clone job reports a misleading worker timeout, inspect the worker stderr first. A common underlying error is ValueError('ref_text required when x_vector_only_mode=False'). Provide the exact reference transcript or explicitly use x-vector-only mode when the selected route supports it. Do not diagnose this as a GPU timeout until the stderr log has been checked.

When a family adapter emits no EOS, very long non-speech, or a token-cap termination, reject the take and route to the designated OmniVoice or subtitle fallback. Re-running the same weights does not repair an under-trained adapter.

NOESIS sealed rules

• R-TALKER-112LANG-PER-FAMILY-SWAP-NOT-MERGE — load Combo plus one family adapter and swap by language; never sum all family adapters.
• R-VOICE-ROUTER-SWAP-NOT-MERGE — language specialists remain separate in the production router.
• R-ROUTER-LANG-SPLIT-1.7B-VS-0.6B — Combo is the 1.7B route for designated main languages and fallback cases; OmniVoice is the primary generic 0.6B route.
• R-TALKER-EOS-GATE — reject token-cap or no-EOS generations instead of passing non-speech downstream.
• R-REF-TEXT-REQUIRED-ICL — provide ref_text for ICL cloning unless x-vector-only mode is explicitly selected.
• R-APACHE-CLEAN — preserve the upstream Apache-2.0 license and NOESIS attribution.
• R-VENDORED-INTERNAL — this bundle is loaded from the parent NOESIS production tree, not treated as an unrelated standalone checkpoint.

References

License

Apache License 2.0. The Combo checkpoint derives from Qwen3-TTS-12Hz-1.7B-Base and the per-family adapters are NOESIS / AMAImedia LoRA additions. See LICENSE.