BEE-spoke-data/smol_llama-101M-GQA
Text Generation • 0.1B • Updated
• 1.95k • 32
LLM training
small-scale pretraining experiments of mine
-
-
BEE-spoke-data/smol_llama-220M-GQA
Text Generation • 0.2B • Updated • 2.43k • 13
BEE-spoke-data/smol_llama-220M-GQA-fineweb_edu
Text Generation • 0.2B • Updated • 8 • 1Note smol_llama-220M-GQA CPT on fineweb-edu for 10 billion tokens
-
BEE-spoke-data/smol_llama-81M-tied
Text Generation • 81.3M • Updated • 901 • 10 -
BEE-spoke-data/mega-ar-126m-4k
Text Generation • 0.1B • Updated • 770 • 4 -
BEE-spoke-data/verysmol_llama-v11-KIx2
Text Generation • 58.1M • Updated • 949 • 4 -
pszemraj/pythia-31m-KI_v1-2048-scratch
Text Generation • 30.5M • Updated • 803 -
BEE-spoke-data/bert-plus-L8-4096-v1.0
Fill-Mask • 88.1M • Updated • 4 -
BEE-spoke-data/mega-encoder-small-16k-v1
Fill-Mask • 0.1B • Updated • 6 • 4
BEE-spoke-data/NanoLlama-GQA-L10-A32_KV8-v13-KI
Text Generation • 0.2B • Updated • 8 • 2Note this is a mid-training checkpoint of what is now smol_llama-220M
-
pszemraj/jamba-900M-v0.13-KIx2
Text Generation • 0.9B • Updated • 17 • 4