README
🚀
AI & ML interests
Building Orion T2 arch on behalf of Smilyai-Labs 😀
Recent Activity
Bc-AI
updated a
Space 1 day ago
Post
91
Hello everyone! 👋
A small SmilyAI Labs update!
G1-MINI has now seen around 8B tokens during its current run, and pretraining is still going strong.
Our E1 (Efficiency-1) prototype has also reached 15B pretraining tokens. E1 has 1B total parameters while activating under 100M parameters per token. It features adaptive activation, meaning easier tokens can use less compute while harder tokens receive more.
We plan to open-source E1 ASAP! 🚀
We’re also excited to announce Project Prism, which will provide limited access to our upcoming Orion Flagship model, powered by our T2 architecture.
Note: T2 here refers to the architecture, not our T2 (Thinker-2) model.
Applications for Project Prism are available through the org page, with more details coming soon!
Finally, welcome @soyL061215 , who joined the Hugging Face org today! 🎉
Thanks to our existing members:
@smilyai-large-team @MUK-IS-GOAT @Keeby-smilyai @Bc-AI
— Bc-AI
SmilyAI Labs
A small SmilyAI Labs update!
G1-MINI has now seen around 8B tokens during its current run, and pretraining is still going strong.
Our E1 (Efficiency-1) prototype has also reached 15B pretraining tokens. E1 has 1B total parameters while activating under 100M parameters per token. It features adaptive activation, meaning easier tokens can use less compute while harder tokens receive more.
We plan to open-source E1 ASAP! 🚀
We’re also excited to announce Project Prism, which will provide limited access to our upcoming Orion Flagship model, powered by our T2 architecture.
Note: T2 here refers to the architecture, not our T2 (Thinker-2) model.
Applications for Project Prism are available through the org page, with more details coming soon!
Finally, welcome @soyL061215 , who joined the Hugging Face org today! 🎉
Thanks to our existing members:
@smilyai-large-team @MUK-IS-GOAT @Keeby-smilyai @Bc-AI
— Bc-AI
SmilyAI Labs
- 3 replies
Post
145
Hello Everyone!
I am happy to announce a few things.
1. G1-MINI
G1-MINI is now in pretraining and is training at a steady pace. Our current ETAs state completion and launch in about 15-20 days, somewhere near the end of September.
2. G1-NANO
G1-NANO is also being pretrained as we speak at a pace of over 400K tokens per second processing more than 10B tokens in 12 hours. This allows us to train extremely fast, and we will launch it somewhere around 15th September.
3. We have begun work on FrameShot, a dual image and video generation model at around 4B dense parameters. This is expected to launch around late December with no promised date.
- Bc-AI
I am happy to announce a few things.
1. G1-MINI
G1-MINI is now in pretraining and is training at a steady pace. Our current ETAs state completion and launch in about 15-20 days, somewhere near the end of September.
2. G1-NANO
G1-NANO is also being pretrained as we speak at a pace of over 400K tokens per second processing more than 10B tokens in 12 hours. This allows us to train extremely fast, and we will launch it somewhere around 15th September.
3. We have begun work on FrameShot, a dual image and video generation model at around 4B dense parameters. This is expected to launch around late December with no promised date.
- Bc-AI
Post
2640
Hello everyone!
I have 2 announcements today!
The first one is the launch of our new API platform! You can make a account and get 5 dollars free credits. No credits card needed because i have no idea how to set up a payment's thing. If you want more credits just email me at smilyai@outlook.com .
The platform currently features G1-Preview a preview of G1 and the older Mira-1-Large.
2nd announcement is we have started working on G1-MINI so expect a late October Ish launch
- Bc-AI on behalf of Smilyai-Labs
I have 2 announcements today!
The first one is the launch of our new API platform! You can make a account and get 5 dollars free credits. No credits card needed because i have no idea how to set up a payment's thing. If you want more credits just email me at smilyai@outlook.com .
The platform currently features G1-Preview a preview of G1 and the older Mira-1-Large.
2nd announcement is we have started working on G1-MINI so expect a late October Ish launch
- Bc-AI on behalf of Smilyai-Labs
- 2 replies
Post
134
Hey everyone,
I just wanna say sorry about the change to G1.
I know a lot of you were really looking forward to the original 20B MoE, and honestly, I was really excited about it too.
Unfortunately, the free compute credits I was using from ML Intern Explorers were removed by Hugging Face. That changed what I can realistically do with the original plan, so I've decided to move G1 over to a Qwen 3.8 27B base instead. Its not just another finetune though, I am inserting extra layers and putting it through my vigourous pipeline. Results will be open source.
I know that's probably disappointing, especially for the people who were specifically waiting for the 20B MoE. I'm genuinely sorry about that.
I really appreciate everyone who got excited about G1 in the first place. I didn't expect this change either, but I'm still really excited to see where G1 can go from here.
The original G1 codebase will stay open too. Its under my profile: Bc-AI/train-g1
Thanks for sticking with us
Thanks to our beta testers, you can become one in the beta testers organisation: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
— Bc on behalf of Smilyai Labs
I just wanna say sorry about the change to G1.
I know a lot of you were really looking forward to the original 20B MoE, and honestly, I was really excited about it too.
Unfortunately, the free compute credits I was using from ML Intern Explorers were removed by Hugging Face. That changed what I can realistically do with the original plan, so I've decided to move G1 over to a Qwen 3.8 27B base instead. Its not just another finetune though, I am inserting extra layers and putting it through my vigourous pipeline. Results will be open source.
I know that's probably disappointing, especially for the people who were specifically waiting for the 20B MoE. I'm genuinely sorry about that.
I really appreciate everyone who got excited about G1 in the first place. I didn't expect this change either, but I'm still really excited to see where G1 can go from here.
The original G1 codebase will stay open too. Its under my profile: Bc-AI/train-g1
Thanks for sticking with us
Thanks to our beta testers, you can become one in the beta testers organisation: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
— Bc on behalf of Smilyai Labs
- 2 replies
Post
127
SmilyAI Weekly Update
Hello everyone,
I have some unfortunate news to share with everyone. My earlier estimate for the launch of G1 in late October was inaccurate. We sincerely apologise for any inconvenience this may cause, but with our current compute resources, pretraining a 20B MoE model is not realistically possible within two months.
G1-MINI will also be postponed, but not for nearly as long — only by a few extra months.
This is disappointing, as I know I was excited to launch G1, and I know many people were also watching the model and looking forward to it.
However in my view, I would rather be honest about our limitations than be overly optimistic about something we currently cannot guarantee.
G1 is not cancelled but uh it will be postponed indefinitely until I have the resources needed to train it. This could be next month, or it could take years. For now, I don't want to give another estimated launch date until I know we have the resources to actually make it happen.
In the meantime, SmilyAI will continue developing AI and experimenting with new ideas, and we will provide updates as we go.
Thank you for sticking with us and supporting SmilyAI. We will continue working towards better models in the future.
Also, if you do have the hardware, to run it aka 8xH200s or better, my codebase is fully open under my very permissive license: i-have-no-idea-just-use-this. Basically, do whatever just mention me. Bc-AI/train-g1
— Bc-AI, on behalf of SmilyAI-Labs
Hello everyone,
I have some unfortunate news to share with everyone. My earlier estimate for the launch of G1 in late October was inaccurate. We sincerely apologise for any inconvenience this may cause, but with our current compute resources, pretraining a 20B MoE model is not realistically possible within two months.
G1-MINI will also be postponed, but not for nearly as long — only by a few extra months.
This is disappointing, as I know I was excited to launch G1, and I know many people were also watching the model and looking forward to it.
However in my view, I would rather be honest about our limitations than be overly optimistic about something we currently cannot guarantee.
G1 is not cancelled but uh it will be postponed indefinitely until I have the resources needed to train it. This could be next month, or it could take years. For now, I don't want to give another estimated launch date until I know we have the resources to actually make it happen.
In the meantime, SmilyAI will continue developing AI and experimenting with new ideas, and we will provide updates as we go.
Thank you for sticking with us and supporting SmilyAI. We will continue working towards better models in the future.
Also, if you do have the hardware, to run it aka 8xH200s or better, my codebase is fully open under my very permissive license: i-have-no-idea-just-use-this. Basically, do whatever just mention me. Bc-AI/train-g1
— Bc-AI, on behalf of SmilyAI-Labs
- 10 replies
Post
2518
Hello everyone!
Me and the team are working on G1-MINI and G1. Right now, G1-MINI is aimed at a launch in mid to late September, depending on how fast we fix the minor issues.
As for G1, it's looking like a late October to mid-November launch, based on current trajectory. If things go terribly wrong, we could postpone it to December, as we prefer to ship confidently, not ship a half-done dogs' breakfast of a model. 🤣
All dates could be changed at any moment, as we are high school students not full-time ML engineers 😅.
Other things to look out for is an overhaul of the UI and the information on my website. Thanks to my beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
Me and the team are working on G1-MINI and G1. Right now, G1-MINI is aimed at a launch in mid to late September, depending on how fast we fix the minor issues.
As for G1, it's looking like a late October to mid-November launch, based on current trajectory. If things go terribly wrong, we could postpone it to December, as we prefer to ship confidently, not ship a half-done dogs' breakfast of a model. 🤣
All dates could be changed at any moment, as we are high school students not full-time ML engineers 😅.
Other things to look out for is an overhaul of the UI and the information on my website. Thanks to my beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
- 3 replies
Post
3782
Hello everyone! A small update on things:
1. G1 series status. G1 is training nicely, and the loss is dropping nicely. The metrics are publicly available and i made a small space you can use to see the nice graphs: hugging-science/Loss-Plot-G1-Large
G1-MINI is a lot slower in converging for reasons unknown yet, but we are investigating it.
2. I have built a small chat app for open SLMs here: ml-intern-explorers/slm-arena
Feel free to add your models in a pull request!
That's all for now, early G1 versions will be available for beta testers soon. Thanks to our beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
1. G1 series status. G1 is training nicely, and the loss is dropping nicely. The metrics are publicly available and i made a small space you can use to see the nice graphs: hugging-science/Loss-Plot-G1-Large
G1-MINI is a lot slower in converging for reasons unknown yet, but we are investigating it.
2. I have built a small chat app for open SLMs here: ml-intern-explorers/slm-arena
Feel free to add your models in a pull request!
That's all for now, early G1 versions will be available for beta testers soon. Thanks to our beta testers: @guardamarcos @Timmy6767 @MUK-IS-GOAT @smilyai-large-team @Sbui503 @Banaxi-Tech @Bc-AI @atom77777 @Harley-ml @Datdanboi25 @Fishtiks @smartdigitalnetworks @vovaRL @EmetTheGolum @juiceb0xc0de @ProCreations
Post
2546
Building a 6.58B sparse MoE model from scratch on a single GPU.
Hey everyone! Today is day 1 of training Smilyai-Lab's new model I call G1-MINI. It's basically the smaller version of our planned model G1 which will be 20B and activate about 2B per token. MINI activates about 1.16B params per token and is currently training right now. If no errors spring up now, I'd say i can launch sometime around September 15th~ish. Thanks to our beta testers:
@guardamarcos
@ProCreations
@juiceb0xc0de
@Timmy6767
@Sbui503
@atom77777
@Fishtiks
@smartdigitalnetworks
@EmetTheGolum
@smilyai-large-team
@MUK-IS-GOAT
@Bc-AI
@Banaxi-Tech
@vovaRL
@Datdanboi25
Hey everyone! Today is day 1 of training Smilyai-Lab's new model I call G1-MINI. It's basically the smaller version of our planned model G1 which will be 20B and activate about 2B per token. MINI activates about 1.16B params per token and is currently training right now. If no errors spring up now, I'd say i can launch sometime around September 15th~ish. Thanks to our beta testers:
@guardamarcos
@ProCreations
@juiceb0xc0de
@Timmy6767
@Sbui503
@atom77777
@Fishtiks
@smartdigitalnetworks
@EmetTheGolum
@smilyai-large-team
@MUK-IS-GOAT
@Bc-AI
@Banaxi-Tech
@vovaRL
@Datdanboi25
- 9 replies
Post
2618
Smilyai News
Hello everyone! August has been a crazy month for us at Smilyai-Labs. We've been doing lots behind the scenes, so here's the latest 👇
1. MiniCoder
We are very close to releasing MiniCoder-1, our first-generation coding model designed for reasoning and coding. Our planned context window is 128K, but the earlier versions probably will not support that long! It's currently in the final stages of DPO so expect a release in early september.
Release: VERY SOON™🤣
2. Smilyai G1
So, the current plan is 20B parameter model total, with a MoE architecture, activating around 2B parameters per token. Its desgigned for maximum performance but keeping it runnable on consumer hardware. It's only a plan and i have no idea when me and the team can finish it. Expect a launch around the end of september to early october-ish. I have no guarantees so don't quote me on the launch date.
3. T1
Smilyai-T1 is another major model we are working on.
The goal for T1 is to take what we learnt from the countless architectural experiments and creating a powerful model designed for thinking. Think MiniCoder but reasons more and G1 but more capable. Its main goals are coding, math, reasoning and general capability.
4. Omni
We are also planning Omni, our first from scratch multimodal model. It will not launch this year as it will take a while. We are actively researching the best architecture for it and we will update progress as we go!
Thanks to our beta testers:
@guardamarcos
@ProCreations
@juiceb0xc0de
@Timmy6767
@Sbui503
@atom77777
@Fishtiks
@smartdigitalnetworks
@EmetTheGolum
@smilyai-large-team
@MUK-IS-GOAT
@Bc-AI
Thanks to my friends who work with me at lunchtimes (Smilyai-Labs team):
@MUK-IS-GOAT
@smilyai-large-team
August was wild. Let’s see what September brings. 🚀
— Bc-AI, on behalf of SmilyAI Labs
Hello everyone! August has been a crazy month for us at Smilyai-Labs. We've been doing lots behind the scenes, so here's the latest 👇
1. MiniCoder
We are very close to releasing MiniCoder-1, our first-generation coding model designed for reasoning and coding. Our planned context window is 128K, but the earlier versions probably will not support that long! It's currently in the final stages of DPO so expect a release in early september.
Release: VERY SOON™🤣
2. Smilyai G1
So, the current plan is 20B parameter model total, with a MoE architecture, activating around 2B parameters per token. Its desgigned for maximum performance but keeping it runnable on consumer hardware. It's only a plan and i have no idea when me and the team can finish it. Expect a launch around the end of september to early october-ish. I have no guarantees so don't quote me on the launch date.
3. T1
Smilyai-T1 is another major model we are working on.
The goal for T1 is to take what we learnt from the countless architectural experiments and creating a powerful model designed for thinking. Think MiniCoder but reasons more and G1 but more capable. Its main goals are coding, math, reasoning and general capability.
4. Omni
We are also planning Omni, our first from scratch multimodal model. It will not launch this year as it will take a while. We are actively researching the best architecture for it and we will update progress as we go!
Thanks to our beta testers:
@guardamarcos
@ProCreations
@juiceb0xc0de
@Timmy6767
@Sbui503
@atom77777
@Fishtiks
@smartdigitalnetworks
@EmetTheGolum
@smilyai-large-team
@MUK-IS-GOAT
@Bc-AI
Thanks to my friends who work with me at lunchtimes (Smilyai-Labs team):
@MUK-IS-GOAT
@smilyai-large-team
August was wild. Let’s see what September brings. 🚀
— Bc-AI, on behalf of SmilyAI Labs
Post
2028
Hello everyone! Happy to say that MiniCoder-1 is now in the instruction tuning phase. It is a 216M~ parameter model trained on 16B tokens. It ran on an RTX 6000 Pro Blackwell gpu for around 24~ hours. Now we will do SFT and launch as beta while we work on the final important DPO and RLHF phases. Our goal is a small extreemly fast on device coding assistant with CoT reasoning* baked in! - Bc-AI on behalf of the Smilyai-Labs team
*it is a small model so the reasoning quality wont be as good obviously!
*it is a small model so the reasoning quality wont be as good obviously!
- 2 replies
Post
3797
New update! We are currently training a few new models now! Our 3rd generation main LLM standard edition is in training right now. We are also training a new LLM line called Tiny Coder around 350~ish M params. Thanks to @Banaxi-Tech for inspiring the architecture with his Bananamind-2.1-unified test model. Thanks to our beta testers: @juiceb0xc0de @ProCreations @Sbui503 @Fishtiks @MUK-IS-GOAT
- 4 replies
Post
4157
@juiceb0xc0de @ProCreations @Keeby-smilyai @MUK-IS-GOAT @Sbui503 @Fishtiks Hello Beta testers! I will release our new model in our beta testers organisation as soon as I can! Thanks for being part of our beta testers community!
- 7 replies
Post
173
Please stand by, we will be providing the Nova-1 series with a major architectural and training update. Expect the New Nova-1-Standard release in late October to early November. - Regards, Bc-AI on behalf of Smilyai-Labs
Post
204
Hello Everyone! Bc-AI here from Smilyai-labs! Today we have done our latest update for CodVa-1-Small. It is very powerful for coding, and benchmark results will come soon. However, it is NOT good for other tasks, with high hallucination rates. We will perform RLHF and DPO very soon!
Post
171
Hello everyone! Today we announce our latest coding model, CodVa-1-Small! It is our most capable model to date for coding, which has completed pretraining and support multi-turn conversation! We will Instruction Tune it very soon! its at: Smilyai-labs/CodVa-1-Small
Post
132
I have begun training a new LLM on a Single RTX 6000 Pro Blackwell GPU on MoLab free notebooks. This model is a 10B parameter model designed for coding tasks named CodVa-Large. Please expect a launch in a few months! Meanwhile, our CodVa-Small model is wrapping up pretraining and will launch in the coming weeks. Nova-1-Standard is complete as is and we will launch Large very soon.
Post
518
we are going to release our latest NOVEL model next! it's called Nova-1-EXP and will be launched as private preview in the smilyai Laboratories BETA TESTERS organisation.
- 3 replies
Post
179
New Update on Nova-1 series status!!! So, after 3 days of fixing our dataset script, we finally have nova-1-standard in its final phases of instruction tuning hopefully. I genuinely do not know ha-ha. We are also doing a novel model named Nova-1-EXP with 5 novel components in the model, which we will announce when the time comes.