Instructions to use tencent/HunyuanImage-3.0-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Inference
- Notebooks
- Google Colab
- Kaggle
Batched chat template silently drops all but the first sample
#19
by smartbearll - opened
Thank you for releasing the model. We found a batching bug in tokenization_hunyuan_image_3.py at revision 2ec2c78bee7d4b94157341fba86c4c2c7b1858b2.
In apply_general_template(batchify=True), prompt_list=[[]] contains only one entry. batch_gen_infer zips it with the per-sample arguments, so only the first sample is encoded.
Minimal reproduction, with no model weights:
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(
"tencent/HunyuanImage-3.0-Instruct",
revision="2ec2c78bee7d4b94157341fba86c4c2c7b1858b2",
trust_remote_code=True,
)
output = tokenizer.apply_chat_template(
batch_prompt=["A cat.", "A boat on a lake.", "Hello."],
mode="gen_text",
sequence_template="instruct",
cfg_factor=1,
)
print(output["output"].tokens.shape[0]) # 1, expected 3
The proposed fix is:
- prompt_list=[[]],
+ prompt_list=[[] for _ in message_list],
This lets the existing batch_gen_infer handle padding and CFG ordering. We verified the repair on CPU across text, image conditioning and image-generation inputs, including CFG factors 1, 2 and 3. UniRL currently applies a runtime workaround in PR #544, which can be removed once the checkpoint is fixed.