ref/GRM_Score_Summary.csv

"Calculated as a weighted average across three categories, each contributing to an overall score",,,,,,4B,,,,2B ,,,Large,,
Category,Domain,Benchmark,Priority,Weight,,Nano-4B,Gemma-4-E4B,Qwen3.5-4B,,Gemma-4-E2B,Qwen3.5-2B,,Qwen3.5-9B,Nano-30B,Super-120B
"ROLEPLAY
(33%)
",Conversational breadth/coherence,MultiChallenge,P0,1.0,,35.3,36.2,49,,33,33.7,,54.5,43.2,55.2
,,RoleBench,P1,0.5,,17.66,20.04,16.21,,19.58,13.11,,16.78,50.2,57
,,RoleMRC,P0,1.0,,39.8,74,56.5,,72.9,43.7,,66.7,17.94,19.48
,,EQ-Bench v3,P1,0.5,,TBD,TBD,78.4,,TBD,54.3,,TBD,TBD,TBD
,,GRM- Coherence,P0,1.0,,27.6,89.7,41.4,,TBD,TBD,,TBD,TBD,TBD
,,GRM- Resp. Div,P0,1.0,,54,85,46,,TBD,TBD,,TBD,TBD,TBD
"ACTIONS
(33%)
",Multi-turn tool calling,BFCL v3,P0,1.0,,61.1,65.9,79.9,,60.9,65.7,,74.6,71.1,74.4
,,When2Call-MT,P0,1.0,,TBD,TBD,TBD,,TBD,TBD,,TBD,TBD,TBD
,,ToolSandbox,P0,1.0,,TBD,TBD,TBD,,TBD,TBD,,TBD,TBD,TBD
,,Tau2-Bench,P0,1.0,,29.2,42.2,79.9,,24.5,48.8,,79.1,49,61.2
,,BFCL v4,P1,0.5,,TBD,42.4,50.3,,40.2,43.6,,66.1,53.8,53.8
,,GRM- Tool Rec,P0,1.0,,44.1,64.7,41.2,,TBD,TBD,,TBD,TBD,TBD
"GENERAL
(33%)
",Input resilience,IHEval,P0,1.0,,75.6,68.6,84.6,,55.9,40.8,,85.6,72.4,76.3
,,RULER,P0,1.0,,91.2,TBD,95,,TBD,91.5,,95.7,95.3,96.8
,,LongMemEval,P1,0.5,,25.2,70.6,72.4,,57.2,32,,82.2,45.6,75.6
,,AgentIF,P1,0.5,,48.3,52.7,51.7,,52.5,46,,53.6,52.9,52.7
,,GRM - Promp Robu,P0,1.0,,TBD,TBD,TBD,,TBD,TBD,,TBD,TBD,TBD
,,GRM- State Adapt,P0,1.0,,TBD,TBD,TBD,,TBD,TBD,,TBD,TBD,TBD
,Input grounding,GaRAGe,P1,0.5,,TBD,TBD,TBD,,TBD,TBD,,TBD,TBD,TBD
,,RAGTruth,P0,1.0,,71.4,77.7,75.5,,67.7,58.4,,72.2,73,74.9
,Restriction adherence,StructEval-T,P1,0.5,,63.5,78.6,76.3,,80.3,54.4,,68.3,76.9,82.2
,,IFBench,P1,0.5,,60.1,43.3,59.2,,37.8,41.3,,64.5,71.5,72.6
,,GRM- Pesona Alig,P0,1.0,,78,92,39,,TBD,TBD,,TBD,TBD,TBD
,Common sense,GSM8K,P2,0.25,,TBD,85.9,89.7,,83,85.3,,93.9,92.3,TBD
,,HumanEval,P2,0.25,,TBD,75.2,75.9,,68.4,70.2,,84.8,78.1,79.4
,,MBPP,P2,0.25,,TBD,77.4,76.3,,68.1,68.3,,80.8,75.5,78.4
,,,GRM Score:,,,50.2,64.7,61.5,,50.0,49.9,,68.4,55.8,61.4