ValueError: Expected shape (5120, 1280) but received shape (4096, 1280)
#2
by silverslash - opened
Getting this error when running the example:
ValueError: Expected shape (5120, 1280) but received shape (4096, 1280) for parameter model.layers.0.self_attn.q_proj.weight
How was this even possible? Was the quantization passed a wrong config? I wasted a 12 GB download...
It seems it need a mlx-lm version that's unreleased:
https://huggingface.co/mlx-community/Mistral-Nemo-Instruct-2407-4bit/discussions/2#669a7c9a49d1c43c4a42f702