Post
1286
@juiceb0xc0de @ProCreations @Keeby-smilyai @MUK-IS-GOAT @Sbui503 @Fishtiks Hello Beta testers! I will release our new model in our beta testers organisation as soon as I can! Thanks for being part of our beta testers community!
Join the conversation
Join the community of Machine Learners and AI enthusiasts.
Sign Up
As always I cant wait to see what kind of hidden conflict your unpredictable and chaotic architecture has in store for me while configuring my mech-interp suite.
@juiceb0xc0de This time we have heterogenous MoE sizes as an experiment. To try and learn patterns smaller experts cannot. We also have shrunk the average size of experts to smaller to generalise better. It will take about 3 months as it stands!
In this post