febilly/Hy-MT2-1.8B-StreamRevise-LoRA

Hy-MT2-1.8B-StreamRevise (LoRA)

A LoRA adapter for tencent/Hy-MT2-1.8B built for live subtitle translation, where the source text keeps changing as speech recognition runs.

Every time the ASR hypothesis updates, you hand the model its own previous translation of the sentence in progress. It decides whether to keep that text and extend it, or rewrite part of it because the meaning changed. The result is a subtitle line that grows smoothly instead of being re-translated from scratch and flickering on every update.

Want to just run it? Use the 4-bit build: Hy-MT2-1.8B-StreamRevise-GGUF (1.07 GB, runs on CPU).

中文简介：给实时字幕场景训的 LoRA。语音边说边识别，源文一直在变——每次更新时把模型上一版译文 一起喂回去，它自己决定是保留已经显示的部分继续往后接，还是因为意思变了而改写。效果是字幕平滑增长， 而不是每次重译导致整行闪烁。想直接用请下 GGUF 版。

---

Prompt format

Read this section before anything else. The adapter is trained on one specific prompt layout. A generic "translate this" instruction will work, but noticeably worse and much less stable.

First chunk of a new sentence (nothing to revise yet)

Hy-MT2's stock prompt, unchanged:

Translate the following text into {TARGET_LANGUAGE}. Note that you should only output the translated result without any additional explanation:

{CURRENT_SOURCE}

Every update after that

[Background Information]
Recent source utterances:
{PREVIOUS_SENTENCE_1}
{PREVIOUS_SENTENCE_2}
...

Previous version of the current source:
{PREVIOUS_SOURCE}

Previous translation of the current source:
{PREVIOUS_TRANSLATION}

When the source meaning has not changed, preserve the still-correct prefix of the previous translation whenever possible. When content is added or corrected, accuracy and completeness take priority.

Please translate the following text into {TARGET_LANGUAGE}, taking the provided background information into consideration.

[Source Text]
{CURRENT_SOURCE}

Rules that actually matter:

• The four background blocks are each optional, joined by a blank line, always in this order: recent utterances → previous source → previous translation → stability rule. If you have nothing for a block, drop the whole block including its label.
• Recent source utterances holds source sentences only — never the translations. That is how it was trained. Keep the last 10 at most.
• {TARGET_LANGUAGE} is a full English language name (Japanese, Traditional Chinese), not a code.
• Optionally state the source language too, which helps with CJK homographs. The instruction line becomes ...following text from {SOURCE_LANGUAGE} into {TARGET_LANGUAGE}, ..., and the cold-start prompt becomes Translate the following text from {SOURCE} into {TARGET}. ....

Chat template

One user turn. apply_chat_template handles it; against a raw completion endpoint the exact string is:

<｜hy_begin▁of▁sentence｜><｜hy_User｜>{PROMPT}<｜hy_Assistant｜>

No trailing newline. Generation stops on token 120020.

Copy-paste renderer

LANGUAGE_NAMES = {"zh": "Chinese", "en": "English", "ja": "Japanese", "ko": "Korean"}  # extend as needed

STABILITY_RULE = (
    "When the source meaning has not changed, preserve the still-correct prefix "
    "of the previous translation whenever possible. When content is added or "
    "corrected, accuracy and completeness take priority."
)

def render_prompt(*, target_language, current_source, history=(),
                  previous_source="", previous_translation="",
                  source_language=None, stability_rule=True):
    target = LANGUAGE_NAMES.get(target_language, target_language)
    source = LANGUAGE_NAMES.get(source_language, source_language)
    direction = f"from {source} into {target}" if source_language else f"into {target}"

    if not history and not previous_source and not previous_translation:
        return (
            f"Translate the following text {direction}. Note that you should only "
            f"output the translated result without any additional explanation:\n\n{current_source}"
        )

    blocks = []
    if history:
        blocks.append("Recent source utterances:\n" + "\n".join(history[-10:]))
    if previous_source:
        blocks.append("Previous version of the current source:\n" + previous_source)
    if previous_translation:
        blocks.append("Previous translation of the current source:\n" + previous_translation)
    if stability_rule:
        blocks.append(STABILITY_RULE)

    return (
        "[Background Information]\n" + "\n\n".join(blocks)
        + f"\n\nPlease translate the following text {direction}, taking the provided "
        "background information into consideration.\n\n[Source Text]\n" + current_source
    )

---

What a sentence looks like as it arrives

One utterance, three ASR updates (ja → zh). Actual model output, greedy:

step	current_source	previous_translation you pass in	output
1	その映画は	(none)	那部电影是
2	その映画はとても面白かった	那部电影是	那部电影非常有趣
3	その映画はとても面白かったですが、少し長かったです。	那部电影非常有趣	那部电影非常有趣，但有点长。

Both behaviours show up here. Step 2 revises: the dangling 是 from step 1 no longer fits once the predicate arrives, so it goes. Step 3 appends: the previous text is still correct, so all of it is kept and the new clause is added.

The step-3 prompt in full:

[Background Information]
Recent source utterances:
週末は何をしましたか？
友達と映画を見に行きました。

Previous version of the current source:
その映画はとても面白かった

Previous translation of the current source:
那部电影非常有趣

When the source meaning has not changed, preserve the still-correct prefix of the previous translation whenever possible. When content is added or corrected, accuracy and completeness take priority.

Please translate the following text into Chinese, taking the provided background information into consideration.

[Source Text]
その映画はとても面白かったですが、少し長かったです。

---

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "tencent/Hy-MT2-1.8B"
ADAPTER = "febilly/Hy-MT2-1.8B-StreamRevise-LoRA"

tokenizer = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    BASE, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)
model = PeftModel.from_pretrained(model, ADAPTER).eval()

def translate(prompt: str) -> str:
    text = tokenizer.apply_chat_template(
        [{"role": "user", "content": prompt}], tokenize=False, add_generation_prompt=True
    )
    inputs = tokenizer(text, return_tensors="pt").to(model.device)
    with torch.inference_mode():
        out = model.generate(
            **inputs, do_sample=False, max_new_tokens=256,
            eos_token_id=120020, pad_token_id=tokenizer.pad_token_id,
        )
    return tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()

print(translate(render_prompt(
    target_language="zh",
    current_source="その映画はとても面白かったですが、少し長かったです。",
    history=["週末は何をしましたか？", "友達と映画を見に行きました。"],
    previous_source="その映画はとても面白かった",
    previous_translation="那部电影非常有趣",
)))

Decoding: use greedy

do_sample=False (equivalently temperature=0). Not the base model's default of 0.7.

Two consecutive updates give the model almost the same prompt, and you want almost the same output back — that is the entire point of this adapter. Sampling injects differences that have nothing to do with the source changing, which shows up directly as subtitle flicker. Greedy also makes the whole thing reproducible and slightly faster. Every number below was measured greedy.

Driving it as a stream

Keep no state on the model side. The caller owns the revision chain and passes it in full every time:

history, prev_source, prev_translation = [], "", ""

for current_source in asr_updates():              # every time the ASR hypothesis changes
    out = translate(render_prompt(
        target_language="zh", current_source=current_source,
        history=history, previous_source=prev_source,
        previous_translation=prev_translation,
    ))
    display(out)                                  # may revise what is already on screen
    prev_source, prev_translation = current_source, out

    if utterance_finished:
        history.append(current_source)            # source only
        history = history[-10:]
        prev_source, prev_translation = "", ""    # next sentence starts cold

Because nothing persists between calls, requests can be retried, load-balanced or dropped without corrupting anything. Turn on prefix KV caching (cache_prompt in llama.cpp, or any HF cache) — consecutive updates share a long prefix, so the repeated background text costs almost nothing.

---

Numbers

Test set: 843 trajectories / 2,438 states (836 scored; 7 dropped for reasons unrelated to the model). bf16 = this adapter merged into the base model under HF transformers. Q4_K_M = the GGUF build under llama.cpp. Identical prompts, greedy decoding on both sides.

	bf16	Q4_K_M
state exact match	0.287	0.282
mean state similarity	0.814	0.801
final exact match	0.199	0.211
mean final similarity	0.820	0.811
characters erased per append	2.56	2.32
characters erased per final transition	3.01	3.05
prefix preserved (append transition)	0.798	0.824
prefix preserved (final transition)	0.917	0.925
revised when the reference revised	0.696	0.714
stayed put when the reference stayed put	0.922	0.945
empty-output rate	0.000	0.000

A "state" is one ASR update; "final" is the last state of an utterance. Erasure is the count of characters that have to be deleted off the end of the previous line to produce the new one — the standard way to quantify how much a live subtitle churns. Roughly 2.5 characters per update here.

The stability rows are the ones that describe what this adapter is for: it holds the existing prefix on ~80% of appends and ~92% of final transitions, while still changing the text when the reference translation genuinely changed ~70% of the time. Both halves matter — a model that never revises scores perfectly on stability and uselessly on accuracy.

Metric definitions are project-internal, so don't line these up directly against numbers in simultaneous-MT papers. Read the table as "4-bit costs you almost nothing".

By language group

The test set spans 107 directions, but coverage is very uneven and so is quality (bf16):

	zh↔en↔ja core	core ↔ other	other ↔ other
trajectories / states	627 / 1677	183 / 656	33 / 105
state exact match	0.321	0.215	0.200
mean state similarity	0.828	0.794	0.703
characters erased per append	2.60	2.07	4.64

Treat anything outside zh/en/ja as best-effort: the sample is thin and both accuracy and stability fall off.

---

Training

Base	tencent/Hy-MT2-1.8B (HunYuanDenseV1, 32 layers, hidden 2048)
Method	LoRA, r=32, alpha=64, dropout=0.05
Target modules	q_proj, k_proj, v_proj, o_proj
Trainable params	13,631,488 (~0.75%)
Schedule	2 epochs, 5,322 steps, batch size 8, lr 2e-4 with warmup → cosine decay, seed 42
Precision	bf16

How the training data was built

Training data is streaming-translation trajectories: for each utterance, a chain of partial ASR-style sources paired with the translation that should be on screen at that moment, so consecutive targets teach the model when to hold the prefix and when to rewrite. 85,121 samples, predominantly Chinese ↔ English ↔ Japanese in all directions, with a smaller amount of other languages mixed in.

The pipeline, in order:

1. Source pool. Conversational and subtitle-style dialogue corpora, cleaned and filtered into a pool of complete utterances with their surrounding context.
2. Causal teacher generation. A stronger LLM produces the target for every streaming state, one request per state, with no exposure to future source text. This is the part that matters: a teacher allowed to see the finished sentence would produce targets the student can never justify from what it actually has, and the model would learn to guess ahead instead of revise.
3. Realistic interim hypotheses. Rather than perturbing text mechanically, an LLM generates plausible partial and mis-recognized ASR states — homophones, word-boundary and ordering errors, false starts, deletions and repetitions, entity and number/time errors, writing-system slips.
4. Independent audit. Two separate judges, deliberately not the generator's own, check final-source quality and acoustic/decoder plausibility of each synthetic interim hypothesis. Both must pass or the sample is dropped.
5. SFT build from the surviving gated trajectories, mixed across three trigger regimes (revise on punctuation / on every update / never) plus targeted CJK homograph probes.

The dataset is not released.

---

Limitations

• Stability is a tendency, not a guarantee. The model learned to hold the prefix; nothing enforces it. A single update can, worst case, rewrite the whole line. If your UI can't tolerate that, only display the prefix that has been stable for N updates and hold back the tail.
• Prompt format matters a lot. Off-format prompts lose both quality and stability. Use the renderer above.
• Language coverage is uneven, and measurably so — see the per-group table above. zh/en/ja is the core; everything else is thin in the training mix and scores worse on both accuracy and stability.
• Sentence-scoped. The revision chain covers the current utterance only; cross-sentence context is the last 10 source lines. It will not go back and fix a sentence it already finalized.
• Greedy decoding assumed throughout.
• Very short or garbled input can produce hallucinated output, same as the base model.
• Inherits the base model's biases.

License

Apache 2.0, same as the base model tencent/Hy-MT2-1.8B.