README.md

---
license: apache-2.0
library_name: transformers
pipeline_tag: automatic-speech-recognition
tags:
  - whisper
  - multi-speaker
  - speaker-diarization
  - meeting-transcription
  - asr
language:
  - en
  - zh
  - multilingual
---

# Tiron

*Released 21 July 2026. Updated 23 July 2026.*

> Watch the original launch video [here](https://youtu.be/pAOYxvSNNt0). Try via API [here](https://router.trelis.com/models).

Tiron is an open-weights multi-speaker meeting transcription model. It jointly transcribes **and** attributes speech to speakers in a single decoding pass: for each 30-second audio window it emits an inline transcript with `<|speakerN|>` turn markers (up to 8 speakers per window) and `<|t.tt|>` timestamps.

Tiron uses the Whisper large-v3 architecture with an extended token vocabulary (`<|speaker1|>` … `<|speaker8|>`, `<|nospeech|>`). It is a drop-in `WhisperForConditionalGeneration` checkpoint.

On whole-meeting benchmarks, Tiron outperforms leading commercial transcription APIs on every test set we evaluated, and trades leads with the best open research models. Tiron runs at ~43× real-time on a single GPU in 3–12 GB of VRAM, decoding chunks in parallel (see [Benchmarks](#benchmarks)).

For **whole meetings** (beyond a single 30s window), use the open-source harness at [TrelisResearch/tiron](https://github.com/TrelisResearch/tiron), which adds chunking, cross-window speaker linking (ECAPA voice embeddings), and SRT/VTT/JSON output.

## What's new (23 July 2026)

- **Updated checkpoint** with small performance gains (NOTSOFAR-1 37.55 → **36.23** pooled cpWER, AMI 35.24 → **34.68**; the 21 July release's numbers are kept in the table for comparison).
- **Expanded benchmarking**: added MOSS-Transcribe-Diarize and AssemblyAI `universal-3.5-pro` under the identical scoring harness.
- **Reproducibility**: the exact evaluation meetings are published with attribution, and minimal replication scripts live in the [harness repo](https://github.com/TrelisResearch/tiron) (links below).

## Benchmarks

Pooled corpus cpWER (lower is better) on held-out whole-meeting test sets, scored with identical references, normalization, and `<UNKNOWN/>` masking for every system:

![Tiron vs MOSS and AssemblyAI — pooled cpWER across AMI, ICSI, and NOTSOFAR-1 (lower is better)](tiron_benchmark.png)

| Test set | AssemblyAI u3-pro | AssemblyAI u3.5-pro | MOSS-TD 0.9B | **Tiron** | Tiron (21 Jul) |
|---|---:|---:|---:|---:|---:|
| AMI (4 meetings) | 39.49 | 39.29 | **28.61** | 34.68 | 35.24 |
| ICSI (3 meetings) | 34.64 | 30.50 | 21.84 | 21.24 | **20.91** |
| NOTSOFAR-1 (10 meetings) | 39.55 | 39.14 | **25.86** | 36.23 | 37.55 |
| **Macro (mean of corpora)** | 37.89 | 36.31 | **25.44** | 30.71 | 31.23 |

Tiron leads both AssemblyAI models on every corpus (−19% macro vs `universal-3-pro`, −15% vs `universal-3.5-pro`). [MOSS-Transcribe-Diarize](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) (Apache-2.0, ~2.5B incl. encoder) decodes the whole meeting in a single 128k context — it leads on AMI and NOTSOFAR-1, and the two are effectively tied on ICSI. Google Gemini 3.1 Pro is competitive on short clips but truncates or fails on longer meetings and cannot be scored across full corpora.

*MOSS numbers use a quality-preserving vLLM decode configuration. We found MOSS's scores are sensitive to serving configuration (an aggressive speed-oriented vLLM config cost it several points on NOTSOFAR-1), so we report its best-quality vLLM decode here.*

**Speed** (× real-time, whole-meeting inference over all 17 meetings; median [range]). Each system's speed is measured in the same configuration as its accuracy numbers above:

| System | ×RT median [range] | Notes |
|---|---|---|
| **Tiron** | **43× [8×–76×]** | single GPU, chunk-parallel, 3–12 GB VRAM |
| MOSS-TD (vLLM) | 3× [1×–8×] quality decode · ~41× speed-oriented | whole-meeting 128k context; accuracy above is the quality decode |

Tiron is chunk-based, so a meeting's 30-second windows decode in parallel on one GPU — that is where its speed comes from.
| AssemblyAI u3.5-pro | 34× [7×–94×] | cloud API round-trip (incl. upload/queue) |

Meetings evaluated (whole-meeting audio, far-field where applicable) — **published with references and attribution as [Trelis/tiron-eval-meetings](https://huggingface.co/datasets/Trelis/tiron-eval-meetings)**, with minimal replication scripts in [`eval/` of the harness repo](https://github.com/TrelisResearch/tiron/tree/main/eval):

- **[AMI](https://huggingface.co/datasets/Trelis/tiron-eval-meetings)** (CC BY 4.0, AMI consortium): `ES2004a`, `IS1009a`, `TS3003a`, `EN2002a`
- **[ICSI](https://huggingface.co/datasets/Trelis/tiron-eval-meetings)** (CC BY 4.0, ICSI): `Bmr013`, `Bmr018`, `Bro021`
- **[NOTSOFAR-1](https://huggingface.co/datasets/Trelis/tiron-eval-meetings)** (CC BY 4.0, Microsoft): `MTG_32040`, `MTG_32063`, `MTG_32072`, `MTG_32074`, `MTG_32092`, `MTG_32179`, `MTG_32185`, `MTG_32256`, `MTG_32257`, `MTG_32322`

Scoring notes: cpWER is concatenated-permutation WER over whole meetings (transcription and speaker-attribution errors both count). Each corpus figure is **pooled** — total errors ÷ total reference words across that corpus's meetings — and the macro is the mean of the three corpus figures. On NOTSOFAR-1, stretches the human annotators marked `<UNKNOWN/>` (unintelligible) are masked from both hypothesis and reference for every system, so no system is rewarded for staying silent there. AMI and ICSI are unaffected by this mask.

**Measurement uncertainty** — whole-meeting cpWER on small corpora is a noisy instrument for *every* system, and all numbers here are single decoding runs. Individual meetings can move by a few points between runs (speaker-count estimation flips, and in single-context decoders one divergent token early in the decode can cascade across the meeting); serving configuration can shift some systems' corpus figures by several points (see the MOSS note above). Corpus figures should be read as ±1 point, and cross-system gaps under ~2 points as ties. Expect small differences when reproducing.

## Output format

Per 30-second window the model emits speaker blocks with within-window timestamps:

```
<|speaker1|><|0.00|> Thanks everyone for joining.<|2.96|><|3.52|> Let's get started.<|4.80|><|speaker2|><|2.98|> Morning!<|3.40|>
```

Speaker indices are **local to the window** (first speaker to talk is `<|speaker1|>`). The harness links speakers across windows into stable meeting-level identities using ECAPA voice embeddings.
## Usage with transformers (single window, ≤30s)

```python
import torch
import soundfile as sf
from transformers import WhisperProcessor, WhisperForConditionalGeneration

repo = "Trelis/tiron"
processor = WhisperProcessor.from_pretrained(repo)
model = WhisperForConditionalGeneration.from_pretrained(
    repo, torch_dtype=torch.bfloat16
).to("cuda").eval()

# Tiron drives decoding itself — disable Whisper's default token suppression.
model.config.forced_decoder_ids = None
model.config.suppress_tokens = []
model.config.begin_suppress_tokens = []
gc = model.generation_config
gc.forced_decoder_ids = None
gc.language = None
gc.task = None
gc.suppress_tokens = None
gc.begin_suppress_tokens = None
if hasattr(gc, "no_timestamps_token_id"):
    delattr(gc, "no_timestamps_token_id")
gc.no_speech_threshold = None

tok = processor.tokenizer
audio, sr = sf.read("clip.wav", dtype="float32")  # 16 kHz mono, up to 30s
feats = processor.feature_extractor(
    audio, sampling_rate=16000, return_tensors="pt"
).input_features.to("cuda", torch.bfloat16)

prefix = [
    tok.convert_tokens_to_ids("<|startoftranscript|>"),
    tok.convert_tokens_to_ids("<|en|>"),  # or any Whisper language token
    tok.convert_tokens_to_ids("<|transcribe|>"),
]
with torch.no_grad():
    out = model.generate(
        input_features=feats,
        decoder_input_ids=torch.tensor([prefix], device="cuda"),
        max_new_tokens=444,
        do_sample=False,
        num_beams=1,
    )

# Render speaker + timestamp tokens inline. Whisper's built-in decoders show
# EITHER timestamps OR added tokens, not both, so walk the ids directly:
ts_begin = tok.convert_tokens_to_ids("<|notimestamps|>") + 1   # <|0.00|>
ts_end = tok.convert_tokens_to_ids("<|30.00|>")
skip = {tok.convert_tokens_to_ids(t) for t in
        ("<|startoftranscript|>", "<|en|>", "<|transcribe|>", "<|endoftext|>")}
parts, buf = [], []
def flush():
    if buf:
        parts.append(tok.decode(buf)); buf.clear()
for tid in out[0].tolist():
    if tid in skip:
        continue
    name = tok.convert_ids_to_tokens(tid)
    if name and name.startswith("<|speaker"):
        flush(); parts.append(name)
    elif ts_begin <= tid <= ts_end:
        flush(); parts.append(f"<|{(tid - ts_begin) * 0.02:.2f}|>")
    else:
        buf.append(tid)
flush()
print("".join(parts))  # <|speaker1|><|0.02|> ... <|2.38|><|speaker2|> ...
```

(Whisper's `decode(..., decode_with_timestamps=True)` renders timestamps but strips the `<|speakerN|>` tokens, and plain `decode(..., skip_special_tokens=False)` does the reverse — hence the small manual walk above. The [harness](https://github.com/TrelisResearch/tiron) does this for you, and also links speakers across windows.)

## Usage with the harness (whole meetings)

The [Tiron harness](https://github.com/TrelisResearch/tiron) runs the full meeting pipeline: 30s chunking with an onset guardrail, per-chunk decoding, ECAPA-based cross-chunk speaker linking (with an optional second staggered decode pass that calibrates the clustering threshold per meeting — on by default, as benchmarked above), and stable `SPEAKER_XX` labels.

```bash
git clone https://github.com/TrelisResearch/tiron
cd tiron && pip install -e .

tiron meeting.wav --output transcript.json          # JSON segments
tiron meeting.wav --format srt --output meeting.srt # subtitles
```

Python API:

```python
from tiron import TironEngine

engine = TironEngine("Trelis/tiron")          # cuda/mps/cpu auto-detected
result = engine.transcribe("meeting.wav", language="auto")

for seg in result["segments"]:
    print(f'[{seg["start"]:7.2f}–{seg["end"]:7.2f}] {seg["speaker"]}: {seg["text"]}')
```

Each segment is `{"speaker": "SPEAKER_00", "start": ..., "end": ..., "text": ...}` with meeting-global speaker labels and timestamps on the original file timeline.

The harness uses the same grammar-constrained decoding as Trelis' hosted serving (default on) and reproduces the benchmark configuration above (validated across the full test set, each corpus within ~0.2 pooled cpWER of the reference run).

## Limitations

- The model's native window is 30 seconds; whole-meeting quality depends on the harness' cross-window speaker linking.
- Up to 8 speakers per 30s window and 8 global speakers per meeting.
- Speaker labels are anonymous (`SPEAKER_00`, …); the model does not identify speakers by name or voice enrollment.
- Timestamps are decoded at 20ms resolution but are approximate, especially under heavy overlap.

## License

Apache 2.0.