Parameter count
#2
by LH-Tech-AI - opened
title
Here's what Claude said.
The file contains about 55.15 million parameters (55,151,433 exactly).
I got this from the data offsets rather than summing every shape by hand. Every tensor is stored as F32 (4 bytes per value), and the offsets are contiguous, so the last tensor, stem/w, ends at byte 220,605,732. Dividing by 4 gives 55,151,433 values.
I spot-checked the offsets against the shapes. For example, embedding/embedding is 8199 × 512 = 4,198,400 values, which matches its 16,791,552-byte span. The two engrams_*/embedding tables are 4 × 18432 × 128 = 9,437,184 values each, and they account for about 18.9 million of the total.
I pasted an earlier incorrect response, which was then hidden as low quality.
Ok
LH-Tech-AI changed discussion status to closed