Batched chat template silently drops all but the first sample

#19
by smartbearll - opened

Thank you for releasing the model. We found a batching bug in tokenization_hunyuan_image_3.py at revision 2ec2c78bee7d4b94157341fba86c4c2c7b1858b2.

In apply_general_template(batchify=True), prompt_list=[[]] contains only one entry. batch_gen_infer zips it with the per-sample arguments, so only the first sample is encoded.

Minimal reproduction, with no model weights:

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "tencent/HunyuanImage-3.0-Instruct",
    revision="2ec2c78bee7d4b94157341fba86c4c2c7b1858b2",
    trust_remote_code=True,
)
output = tokenizer.apply_chat_template(
    batch_prompt=["A cat.", "A boat on a lake.", "Hello."],
    mode="gen_text",
    sequence_template="instruct",
    cfg_factor=1,
)
print(output["output"].tokens.shape[0])  # 1, expected 3

The proposed fix is:

- prompt_list=[[]],
+ prompt_list=[[] for _ in message_list],

This lets the existing batch_gen_infer handle padding and CFG ordering. We verified the repair on CPU across text, image conditioning and image-generation inputs, including CFG factors 1, 2 and 3. UniRL currently applies a runtime workaround in PR #544, which can be removed once the checkpoint is fixed.

Sign up or log in to comment