Smaller UL2 models
#8
by leshanbog - opened
Hi!
Thanks for sharing this 20B model with the community! From your paper it seems that smaller model also benefit from this kind of pretraining. Do you have any plans to release a UL2-base or something of that scale?