Post
350
IBM Granite 4.1 series
New models came up, here is how they compare to models in the same size:
Brainwaves
Gemma-4
Qwen3.5
Right out of the gate, IBM delivered models with better starting metrics than both Gemma and Qwen. Training these should be fun :)
Here is the Nightmedia collection of Granite models
https://huggingface.co/collections/nightmedia/ibm-granite-41
-G
New models came up, here is how they compare to models in the same size:
Brainwaves
arc arc/e boolq hswag obkqa piqa wino
granite-4.1-30b
mxfp8 0.456,0.572,0.897,0.621,0.444,0.757,0.616
mxfp4 0.453,0.565,0.892,0.624,0.442,0.759,0.585
qx86-hi 0.451,0.568,0.897,0.636,0.440,0.763,0.598
granite-4.1-8b
mxfp8 0.486,0.666,0.875,0.636,0.450,0.766,0.631
granite-4.1-3b
mxfp8 0.406,0.581,0.821,0.484,0.434,0.712,0.559Gemma-4
quant arc arc/e boolq hswag obkqa piqa wino
gemma-4-E4B-it
mxfp8 0.480,0.656,0.797,0.608,0.400,0.755,0.665
mxfp4 0.455,0.607,0.851,0.585,0.402,0.744,0.651
gemma-4-E2B-it
mxfp8 0.376,0.464,0.743,0.490,0.378,0.709,0.622
mxfp4 0.380,0.451,0.762,0.494,0.374,0.699,0.594Qwen3.5
quant arc arc/e boolq hswag obkqa piqa wino
Qwen3.5-9B
mxfp8 0.417,0.458,0.623,0.634,0.338,0.737,0.639
mxfp4 0.419,0.472,0.622,0.634,0.352,0.739,0.644
Qwen3.5-4B
mxfp8 0.392,0.441,0.627,0.601,0.360,0.739,0.590
mxfp4 0.371,0.444,0.632,0.585,0.356,0.732,0.548Right out of the gate, IBM delivered models with better starting metrics than both Gemma and Qwen. Training these should be fun :)
Here is the Nightmedia collection of Granite models
https://huggingface.co/collections/nightmedia/ibm-granite-41
-G