Zhe-Zhang
/

GPT2-Small-Distilled

Model card Files Files and versions

GPT2-Small-Distilled / README.md

Zhe-Zhang's picture

Update README.md

052236a verified 5 months ago

|

history blame contribute delete

128 Bytes

· GPT2-small architecture

· Randomly initialized

· Distilled on BabyLM dataset (10M) using teacher model GPT2-large-BabyLM