This quant needs fixing
#1
by trohrbaugh - opened
config.json's ignore is missing three entries present in the model card's creation snippet, and 86 weight_scale tensors exist for modules the recipe excludes, which trips assert key == "w2_weight" in vLLM's sink_experts loader.