llm-c mtp test models - a inference-optimization Collection

inference-optimization 's Collections

Quantized-Drafters

llm-c mtp test models

llm-c mtp test models

updated 10 days ago

Toy-trained BF16/FP8 fixtures for PR #3225 on Transformers 5.17. Model cards document DSA and Inkling limitations.

inference-optimization/GLM-4.7-Flash-0.82B-MTP

Text Generation • 0.8B • Updated 17 days ago • 26

Note BF16 source; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/DeepSeek-V3-0.86B-MTP

Text Generation • 0.9B • Updated 17 days ago • 24

Note BF16 source; two-GPU quantization and vLLM MTP execution passed.
inference-optimization/GLM-5-0.88B-MTP

Text Generation • 0.9B • Updated 17 days ago • 24

Note BF16 source; backbone validated, MTP execution unverified.
inference-optimization/Inkling-0.92B-MTP

Image-Text-to-Text • 0.9B • Updated 11 days ago • 44

Note BF16 multimodal fixture; text backbone validated. Two-rank pipeline and MTP serving are not validated.