DeltonLabs/DCLM-Filtered-HQ
Updated • 884 • 3
DeltonLabs
community
AI & ML interests
None defined yet.
Recent Activity
Harley-ml
updated a
dataset about 11 hours ago
Harley-ml
updated a
dataset 2 days ago
Harley-ml
published 2
datasets 5 days ago
Post
131
Zero-v1.0-144M is currently training. So far, it has completed 26% of its training run, with about 98 billion tokens to go.
Current Val PPL: 13.705
Zero-v1.0 uses a custom architecture consisting of RMSNorm, RoPe, SwiGLU, Engram Conditional Memory, mHC, and XSA GQA.
Current Val PPL: 13.705
Zero-v1.0 uses a custom architecture consisting of RMSNorm, RoPe, SwiGLU, Engram Conditional Memory, mHC, and XSA GQA.