Transformers grown layer-by-layer on frozen embeddings. Explores emergent capabilities with depth.
Andrey
Bochkov
AI & ML interests
None yet
Organizations
None yet
Transformers grown layer-by-layer on frozen embeddings. Explores emergent capabilities with depth.
Frozen embedding LMs (en/ru/zh) & their MoE fusion. Baselines: frozen vs unfrozen embedding ablation.
This collection features frozen, precomputed token embedding tensors designed for experimentation with semantic emergence in language models.