Eval requests: G4 E4B ARA vs SOMA
#683
by MuXodious - opened
I looked into the issue with the previous request. It stemmed from the way that the latest version of Transformers save these models by pruning redundant tied embeddings, unlike the earlier version which Google used for release (v5.5.0). New QAT versions have the same pruning, for instance. Anyways, I re-exported the models with the Transformers v5.5.0. The vLLM issue should be resolved now.
https://huggingface.co/MuXodious/gemma-4-E4B-it-SOMPOA-heresy