aethertp/PicoLM-V4-85M
Text Generation • 85M • Updated
Aether
aethertp
AI & ML interests
None yet
Recent Activity
published a model 11 minutes ago
aethertp/PicoLM-V4-85M updated a model 42 minutes ago
aethertp/PicoLM-V4-85M updated a model 27 days ago
aethertp/PicoLM-V3-Pro-82M-InstructOrganizations
None yet
published a
model 11 minutes ago
updated a
model 42 minutes ago
updated 4
models 27 days ago
updated a
Space 28 days ago
published a
Space 28 days ago
published 2
models 28 days ago
Drop redundant untied lm_head.weight (Option 1) The checkpoint stored both tok_embeddings.weight and a separate lm_head.weight, each [24576, 576] = 14,155,776 params, even though config.json sets tie_word_embeddings: true. At runtime the head is tied to the embedding, so lm_head.weight is never read. This removes that one tensor. Verified: - 201 -> 200 tensors; stored elements 96,017,472 -> 81,861,696 (exactly the card's count) - file 192,054,704 -> 163,743,061 bytes (-28.3 MB) - loaded both versions with the repo's own PicoLMV2ForCausalLM: identical logits on fixed inputs (max |diff| = 0.0), same 81,861,696 unique params, head tied to embedding in both. Requested by aethertp in the discussion (Option 1).
#2 opened about 1 month ago
by 
Compactbot
Checkpoint stores an untied lm_head: 96.0M params on disk vs 81.86M on the card
4
#1 opened about 1 month ago
by Compactbot
Overcoming the ARC-Easy floor and vocabulary budget on an 80M footprint
❤️ 1
8
#1 opened about 2 months ago
by 
AndrewThompson1233
liked a
model about 1 month ago
published a
model about 1 month ago
updated a
model about 1 month ago
published a
model about 1 month ago
published a
model about 2 months ago
liked a
Space 4 months ago
liked a
model 8 months ago
