Param count in card (11,053,056) doesn't match config.json / weights (~48.7M)
I verified the parameter count against the actual weights and the numbers don't line up — happy to help fix the card.
What I found:
- The card claims 11,053,056 parameters.
config.json(n_layers=8, d_model=512, d_ff=2203, vocab=32000, GQA 2-kv, tied embeddings) computes to 48,706,048 parameters.- I parsed
model.ptdirectly: it containsembed (32000, 512)+ 8 blocks, each withq/o (512,512),k/v (128,512),gate/up (2203,512),down (512,2203)— i.e. exactly the config.json architecture, ~48.7M params.
So the weights match config.json (48.7M), and the card's 11,053,056 matches neither the config nor the weights.
One more thing: the card is internally inconsistent on depth — the body says "8 layers" but the config table a few lines down says n_layer = 12. The weights have 8 blocks, so 8 is the real number.
I'm happy to open a PR correcting the param count and the layer number — just say the word. (Separately, the file listing shows model.pt as ~21.9 MB but it actually downloads as ~194.8 MB, which is consistent with the 48.7M F32 weights — so the listing size also looks stale.)
Thx
Yeah, it would be helpful for a PR
Re-checked the card just now — it's already consistent, looks like you fixed it. It reads 48,706,048 params, 8 layers, and 194.8 MB of weights, all matching config.json and the weights I parsed.
For the record, 48,706,048 decomposes exactly as:
- embedding 32000×512 = 16,384,000
- 8 blocks × 4,040,192 = 32,321,536 (2×RMSNorm + q/o 512² + k/v 128×512 + SwiGLU 3×2203×512)
- final RMSNorm = 512
So there's nothing left for a PR to change — no PR needed on my end. If there's a specific remaining item you'd like one for, point at it and I'll open it. Otherwise this is resolved. Clean fix.
Could you put in your SLM collection
Please
Could you add this model to your clean SLM model collection for me please
Added to my verified-clean SLM collection — it's there now with the confirmed 48,706,048 param count noted.