Instructions to use marcosremar2/ucast-subtitle-compressor with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use marcosremar2/ucast-subtitle-compressor with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="marcosremar2/ucast-subtitle-compressor")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("marcosremar2/ucast-subtitle-compressor") model = AutoModelForTokenClassification.from_pretrained("marcosremar2/ucast-subtitle-compressor", device_map="auto") - Notebooks
- Google Colab
- Kaggle
ucast subtitle compressor (deletion only)
A small model that makes a live subtitle shorter by deleting words only. Give it 2–4 sentences of transcribed speech; it gives each word a keep probability, and a few fixed rules decide what goes. It never rewrites, so it cannot invent a number, a name or a word that was not said.
It was made for ucast.me live subtitles ("Resumida"): a shorter line is easier to
read on a projector while the speaker keeps talking. The app runs the ONNX file in onnx/ on the
viewer's own computer.
- Model: mBERT token classifier (
BertForTokenClassification, 12 layers, 2 labels: drop / keep), fine-tuned from Microsoft's LLMLingua-2 multilingual. Weights in fp16 (355 MB).onnx/compressor.onnx(279 MB) is the same model with the vocabulary cut to the Latin script (69,882 of 119,547 tokens; a word with a piece of another script reads as[UNK]). - Languages: Portuguese, French, Spanish, English. Trained on YouTube speech (YODAS2, CC BY 3.0).
- Default setting:
w = 0.6. It cuts about 20 % of the words. On 1,600 chunks from videos never used to train or to pick the setting, 1.3 % of chunks have a serious error (the meaning changed).
Resumo em português: modelo que encurta a legenda ao vivo só apagando palavras (nunca reescreve, então não inventa número nem nome). Com o ajuste padrão (
w = 0.6) corta ~20 % das palavras. Em 1.600 trechos de vídeos que o treino não viu, 1,3 % teve um erro grave segundo um juiz Qwen2.5-72B (limite superior de 95 %: 1,88 %). Treinado só com texto CC BY 3.0 (YODAS2).
Use
pip install torch transformers huggingface_hub
huggingface-cli download marcosremar2/ucast-subtitle-compressor --local-dir compressor
python compressor/compress.py "Bom, então, o que a gente viu no último trimestre foi que as vendas cresceram doze por cento." --lang pt
compress.py holds all of the inference code. It uses the model's keep probabilities,
common.decide (the rules) and common.render (writes the kept words back). You can also call it from Python:
from transformers import AutoModelForTokenClassification, AutoTokenizer
import sys; sys.path.insert(0, "compressor")
from compress import compress
tok = AutoTokenizer.from_pretrained("compressor")
model = AutoModelForTokenClassification.from_pretrained("compressor").float().eval()
print(compress(model, tok, "A reunião com a diretoria ficou para quinta-feira, dia vinte. "
"Quem não puder ir tem que avisar a Beatriz até terça. Ela vai mandar a pauta por e-mail.", "pt"))
# A reunião com a diretoria ficou para quinta-feira, dia vinte. Quem não puder ir tem que avisar a Beatriz até terça. Mandar pauta por e-mail.
The input is a pair: (context, text). The model was trained with up to 3 earlier subtitles as context.
With no context the first segment is "-". Context did not help at the same cut, so the default uses none.
The rules (common.decide)
The model's probabilities are only half of it. These rules keep the output readable and safe:
- These words always stay: numbers (digits or words, plus the unit after them), days, months, today and tomorrow, negations, names, acronyms, and the short word before one of them.
- A sentence goes whole only when nothing in it is protected, it is not a question, and its mean keep
probability is under
sentence_drop. - A word goes only when the model is sure (keep probability under
word_drop). - No fragments: a sentence left with under
min_share(0.5) of its words either goes whole, or stays whole if something in it is protected. - Repeats: a sentence said twice word for word keeps only its first copy.
rendermoves a sentence end that sat on a dropped word to the last kept word. A new sentence starts with a capital letter.
w sets word_drop = sentence_drop = w. A higher value cuts more.
How it was made
Data. Chunks of 2–4 sentences (25–70 words) from the subtitles of YODAS2 (the
*000subsets: YouTube videos their owners published under CC BY 3.0, with the owners' own subtitles). Only the text was read. Lines that are not speech (links, music, credits, sound tags) were left out, at most 6 chunks per video. Each video goes whole to one split, chosen by a hash of its id: train (2,100 per language), cal (400 per language, to pick the setting) and fresh (400 per language, to confirm it).Labels from a large model. On one H100, Qwen2.5-72B-Instruct (AWQ, served with vLLM) shortened each training chunk by deleting words, in two styles: light (keep most of it) and aggressive (cut more). An answer was kept only if it was the chunk with words removed: no word changed or moved (6,068 light and 6,536 aggressive answers of 8,400).
One model. The two styles were merged into one set of 5,451 chunks:
- the aggressive labels decide which whole sentences go;
- the light labels decide which words go inside the sentences that stay;
- protected words are always kept.
Fine-tuned from LLMLingua-2 multilingual for 4 epochs (lr 3e-5, batch 32). Every sub-token carries its word's label.
Grading the way subtitle quality is graded. Live-subtitle quality is scored with the NER model (Romero-Fresco; used by Ofcom, the BBC and ABNT NBR 15290):
- a cut that loses nothing is a correct edition (no penalty);
- minor: a dependent detail is lost;
- standard: a whole idea is lost;
- serious: the meaning changes.
The judge was Qwen2.5-72B, asked for the NER grade per chunk through vLLM guided JSON, so every chunk got a grade. On an earlier set with nothing cut it found 0 serious errors in 1,564 chunks.
Choosing the setting with a statistical guarantee. The goal was about 20 % cut with under 2 % serious errors. Settings were tested with Learn-Then-Test: a setting passes if P(Binomial(n, 0.02) ≤ serious) ≤ 0.1; the Clopper–Pearson 95 % upper bound is reported too. Settings were picked on cal and confirmed on fresh. w 0.65 passed on cal and failed on fresh, so the default is w 0.6.
Benchmark
Judge Qwen2.5-72B-AWQ, NER grades, every chunk graded. Cut is the share of words deleted. Aggregates
are in benchmark/results.json.
Fresh: 1,600 chunks from videos never used to train or pick the setting (400 per language)
| setting | cut | serious | rate | CP 95 % upper | LTT p | passes | standard | grammatical |
|---|---|---|---|---|---|---|---|---|
| w 0.6 (default) | 20.8 % | 21 | 1.31 % | 1.88 % | 0.025 | yes | 23.3 % | 96.1 % |
| w 0.65 | 23.5 % | 30 | 1.88 % | 2.53 % | 0.405 | no | 29.4 % | 93.9 % |
| w 0.67 | 24.6 % | 29 | 1.81 % | 2.46 % | 0.336 | no | 31.0 % | 93.8 % |
Serious errors per language at w 0.6: pt 4/400, es 5/400, fr 5/400, en 7/400 (1.75 %).
Cal: 1,600 chunks (the set the setting was picked on)
| setting | cut | serious | CP 95 % upper | LTT p | passes | standard | NER score |
|---|---|---|---|---|---|---|---|
| w 0.5 | 15.6 % | 12 | 1.21 % | 0.000 | yes | 16.3 % | 99.08 % |
| w 0.6 | 20.1 % | 19 | 1.74 % | 0.009 | yes | 22.4 % | 98.96 % |
| w 0.65 | 22.7 % | 21 | 1.88 % | 0.025 | yes | 27.3 % | 98.89 % |
| w 0.67 | 23.7 % | 23 | 2.03 % | 0.059 | yes | 29.4 % | 98.85 % |
| w 0.7 | 25.3 % | 31 | 2.61 % | 0.476 | no | 32.1 % | 98.78 % |
The NER pass mark for broadcast subtitles is 98 %; every setting here is above it.
12 hand-written items at w 0.6
Short sentences written for this project, with numbers, names, dates and negations
(benchmark/items.json, outputs in benchmark/items-w0.6.jsonl). Numbers, names, dates and negations
were kept. Shown as they came out, broken ones included:
| lang | output | kept |
|---|---|---|
| pt | O a gente viu no último trimestre foi as vendas cresceram doze por cento. E isso, isso foi principalmente por causa do mercado do Nordeste. Abriu três lojas novas em Recife e em Fortaleza. | 85 % — "O a gente viu", "foi as" are broken |
| pt | A reunião com a diretoria ficou para quinta-feira, dia vinte. Quem não puder ir tem que avisar a Beatriz até terça. Mandar pauta por e-mail. | 89 % — "Mandar pauta" lost its subject |
| pt | Usaram inteligência artificial no? | 20 % — bad: the question lost "Quantos" and "trabalho" |
| pt | O projeto não vai ser cancelado. Ele só vai atrasar uns dois meses, porque o fornecedor da China não entregou as peças. | 79 % |
| fr | Aujourd'hui on va parler du climat à Marseille. En dix ans, la température moyenne a augmenté de deux degrés. | 61 % |
| fr | Le budget de la mairie a été voté hier soir. Il prévoit quarante millions d'euros pour les écoles. | 82 % |
| fr | Je ne suis pas d'accord avec dire Monsieur Dubois. Le chômage a baissé de trois pour cent cette année. | 68 % — "d'accord avec dire" is broken |
| en | Our app now supports sixteen languages. Next month we're adding Japanese and Korean. | 50 % |
| en | The server went down at three in the morning. Nobody noticed until eight, when customers started calling. We lost about five hours of orders. | 100 % — nothing cut |
| en | Doctor Takahashi will not be joining us today. Her talk about Kubernetes moves to Friday afternoon. | 84 % |
| es | El proyecto empezó en dos mil diecinueve. Principio éramos solo cuatro personas. Hoy somos más de doscientas en seis países. | 74 % — "Principio éramos" lost "Al" |
| pt | Não desliguem o computador durante a atualização. Ela leva uns quinze minutos e reinicia sozinha no final. | 81 % |
Limits
- It loses ideas. At the default, 23 % of chunks lose a whole idea (a standard error). The setting meets the broadcast criterion (meaning rarely changes), not "no idea is ever lost". Deleting words alone hits a ceiling: professional subtitlers reach 20–40 % by rephrasing.
- Grammar sometimes breaks inside a sentence (about 4 % of chunks are judged ungrammatical), and a question can lose its question word (see the items above).
- It was graded by one LLM judge, not by human raters.
- YouTube subtitles written by the video owners are cleaner than live speech recognition. It has not been measured on raw ASR output.
Earlier version
The first version (commit e151c7fc) was trained on mTEDx and Europarl-ST and released under
CC BY-NC 4.0. This version replaces it, trained only on CC BY 3.0 text.
License and attribution
- The weights are released under CC BY 4.0.
- Training text: YODAS2 (Li et al., YODAS: YouTube-Oriented Dataset for Audio and Speech, 2023), subsets of YouTube videos published under CC BY 3.0 by their owners.
- The base model, microsoft/llmlingua-2-bert-base-multilingual-cased-meetingbank, is Apache-2.0.
- The labels and grades were produced with Qwen2.5-72B-Instruct — Built with Qwen.
- The code in this repo (
compress.py,common.py) is free to reuse. - Compute: labelling, training and grading ran on the Mésocentre Clermont Auvergne (Université Clermont Auvergne).
- Downloads last month
- 33