README
🦀
1
♾️ Thinking
Wop
wop
AI & ML interests
AI research
AGI
Recent Activity
updated a Space about 12 hours ago
bench-labs/README updated a dataset 1 day ago
bench-labs/slop-classification reacted to TobiasLogic's post with 🔥 1 day ago
We’ve been cooking something new at Bench Labs.
Introducing Cagliostro-v3, our new 146M parameter language model trained completely from scratch.
The run isn’t even finished yet.
At the current checkpoint:
• 146M parameters
• 72.7B / 75B tokens trained
• 26.27 Open SLM Index
• 43.80 ArithMark-3
• Trained on a single RTX 5090
• ~90K to 103K tokens/sec during training
• ~9 days for the full run
• Apache 2.0
For some context, SmolLM2-135M scores 27.13 on the same Index after being trained on roughly 2 trillion tokens.
Cagliostro-v3 is currently at 26.27 with only ~72.7B.
That’s around 27x fewer training tokens.
The model also currently Hold the number 3rd spot for ArithMark-3, scoring 43.80
This wasn’t achieved by just throwing more tokens at the model. A huge part of v3 has been figuring out architecture, data mixture, and training dynamics at this scale.
The model uses a custom 30-layer decoder architecture with grouped-query attention and cross-head subspace attenuation, SwiGLU, RMSNorm, RoPE, tied embeddings, and a warmup-stable-decay training schedule.
During cooldown we also substantially shifted the data mixture toward higher-quality synthetic textbook and mathematics data, with the mathematics share increasing from 10% to 28%.
And everything is open.
The repository contains the training history with checkpoints pushed roughly every 30 minutes, so you can inspect how the model evolved throughout training rather than only seeing the final weights.
This is still a pre-final checkpoint. We have roughly 2.3B tokens left and the learning-rate cooldown is still running.
So 26.27 isn’t the final number.
Really excited to see where the last part of the run lands.
Cagliostro-v3:
https://huggingface.co/bench-labs/cagliostro-v3
Built by Bench Labs.
Open SLM Leaderboard:
https://huggingface.co/spaces/AxiomicLabs/Open_SLM_Leaderboard
Organizations
updated a
Space about 12 hours ago
updated a
dataset 1 day ago
reacted to TobiasLogic's post with 🔥 1 day ago
Post
2676
We’ve been cooking something new at Bench Labs.
Introducing Cagliostro-v3, our new 146M parameter language model trained completely from scratch.
The run isn’t even finished yet.
At the current checkpoint:
• 146M parameters
• 72.7B / 75B tokens trained
• 26.27 Open SLM Index
• 43.80 ArithMark-3
• Trained on a single RTX 5090
• ~90K to 103K tokens/sec during training
• ~9 days for the full run
• Apache 2.0
For some context, SmolLM2-135M scores 27.13 on the same Index after being trained on roughly 2 trillion tokens.
Cagliostro-v3 is currently at 26.27 with only ~72.7B.
That’s around 27x fewer training tokens.
The model also currently Hold the number 3rd spot for ArithMark-3, scoring 43.80
This wasn’t achieved by just throwing more tokens at the model. A huge part of v3 has been figuring out architecture, data mixture, and training dynamics at this scale.
The model uses a custom 30-layer decoder architecture with grouped-query attention and cross-head subspace attenuation, SwiGLU, RMSNorm, RoPE, tied embeddings, and a warmup-stable-decay training schedule.
During cooldown we also substantially shifted the data mixture toward higher-quality synthetic textbook and mathematics data, with the mathematics share increasing from 10% to 28%.
And everything is open.
The repository contains the training history with checkpoints pushed roughly every 30 minutes, so you can inspect how the model evolved throughout training rather than only seeing the final weights.
This is still a pre-final checkpoint. We have roughly 2.3B tokens left and the learning-rate cooldown is still running.
So 26.27 isn’t the final number.
Really excited to see where the last part of the run lands.
Cagliostro-v3:
bench-labs/cagliostro-v3
Built by Bench Labs.
Open SLM Leaderboard:
AxiomicLabs/Open_SLM_Leaderboard
Introducing Cagliostro-v3, our new 146M parameter language model trained completely from scratch.
The run isn’t even finished yet.
At the current checkpoint:
• 146M parameters
• 72.7B / 75B tokens trained
• 26.27 Open SLM Index
• 43.80 ArithMark-3
• Trained on a single RTX 5090
• ~90K to 103K tokens/sec during training
• ~9 days for the full run
• Apache 2.0
For some context, SmolLM2-135M scores 27.13 on the same Index after being trained on roughly 2 trillion tokens.
Cagliostro-v3 is currently at 26.27 with only ~72.7B.
That’s around 27x fewer training tokens.
The model also currently Hold the number 3rd spot for ArithMark-3, scoring 43.80
This wasn’t achieved by just throwing more tokens at the model. A huge part of v3 has been figuring out architecture, data mixture, and training dynamics at this scale.
The model uses a custom 30-layer decoder architecture with grouped-query attention and cross-head subspace attenuation, SwiGLU, RMSNorm, RoPE, tied embeddings, and a warmup-stable-decay training schedule.
During cooldown we also substantially shifted the data mixture toward higher-quality synthetic textbook and mathematics data, with the mathematics share increasing from 10% to 28%.
And everything is open.
The repository contains the training history with checkpoints pushed roughly every 30 minutes, so you can inspect how the model evolved throughout training rather than only seeing the final weights.
This is still a pre-final checkpoint. We have roughly 2.3B tokens left and the learning-rate cooldown is still running.
So 26.27 isn’t the final number.
Really excited to see where the last part of the run lands.
Cagliostro-v3:
bench-labs/cagliostro-v3
Built by Bench Labs.
Open SLM Leaderboard:
AxiomicLabs/Open_SLM_Leaderboard
- 7 replies
liked a
model 1 day ago
liked a
model 4 days ago
liked a
model 5 days ago
reacted to RiverRider's post with 🔥 5 days ago
Post
109
Black Window — a chat model in your browser tab, on your hardware. A memory that stays on the device that opened the page.
https://blackwindow.xyz
Open the site, pick a model (about 0.6B to 8B), hit Load. The weights run in that tab, on that computer. After they load, the network can drop. The context window is a working set, auto-sized to that device, up to ~32K tokens.
Behind the window is the Weave. Every file, picture, recording, link, lookup, and reply is embedded as it arrives. Drop in audio and it is transcribed. Drop in an image and it is described. A question pulls the nearest passages back as notes. A long document is walked once so later questions can use the whole file, not the first pages.
Nothing leaves that tab unless you turn on live lookup or connect a rented GPU box, and the chat says so each time. Prompts can go to the box. Files and the Weave stay in the tab.
Console on that page: bw.ask, bw.search, bw.digest, bw.notes. A local relay exposes /v1/chat/completions on localhost so other tools on the same computer can talk to the tab. The tab polls the relay. That is the boundary.
Not a server with a policy. Your hardware, a window, a Load button.
If on mobile add to home-screen for best performance. If you break it lmk. It can serve a few hundred of you at a time before I have to buy a real server.
https://blackwindow.xyz
Open the site, pick a model (about 0.6B to 8B), hit Load. The weights run in that tab, on that computer. After they load, the network can drop. The context window is a working set, auto-sized to that device, up to ~32K tokens.
Behind the window is the Weave. Every file, picture, recording, link, lookup, and reply is embedded as it arrives. Drop in audio and it is transcribed. Drop in an image and it is described. A question pulls the nearest passages back as notes. A long document is walked once so later questions can use the whole file, not the first pages.
Nothing leaves that tab unless you turn on live lookup or connect a rented GPU box, and the chat says so each time. Prompts can go to the box. Files and the Weave stay in the tab.
Console on that page: bw.ask, bw.search, bw.digest, bw.notes. A local relay exposes /v1/chat/completions on localhost so other tools on the same computer can talk to the tab. The tab polls the relay. That is the boundary.
Not a server with a policy. Your hardware, a window, a Load button.
If on mobile add to home-screen for best performance. If you break it lmk. It can serve a few hundred of you at a time before I have to buy a real server.
- 5 replies
liked 2
Spaces 6 days ago
liked a
model 6 days ago
liked a
Space 6 days ago
liked a
dataset 7 days ago
liked a
model 10 days ago
liked a
model 11 days ago
liked a
model 12 days ago
liked a
Space 12 days ago
Will the qwen3.8-27b version be produced?
🚀➕ 28
15
#54 opened about 1 month ago
by 
cleverhack
reacted to branikita's post with 🔥🔥 17 days ago
Post
2223
SO-ARM 102 goes open source in the next few weeks.
What's new:
- A parallel gripper: the jaws stay parallel through the whole stroke instead of pivoting around the object as they close.
- PET-CF instead of PLA+ for a much stiffer frame.
- A topology-optimized structure.
- Wider joint rotation and folding range.
- STS3250 servos on the first shoulder joint.
Compared with the SO-ARM 101, that adds up to 2.5x the payload, roughly 2x better positioning accuracy, roughly 1.6x the movement speed, and about 36 mm more reach.
It runs on Hugging Face LeRobot, so the same tooling, training pipeline and tutorials for the SO-ARM 101 work on it from day one.
What's new:
- A parallel gripper: the jaws stay parallel through the whole stroke instead of pivoting around the object as they close.
- PET-CF instead of PLA+ for a much stiffer frame.
- A topology-optimized structure.
- Wider joint rotation and folding range.
- STS3250 servos on the first shoulder joint.
Compared with the SO-ARM 101, that adds up to 2.5x the payload, roughly 2x better positioning accuracy, roughly 1.6x the movement speed, and about 36 mm more reach.
It runs on Hugging Face LeRobot, so the same tooling, training pipeline and tutorials for the SO-ARM 101 work on it from day one.
- 1 reply
reacted to NitrAI's post with ❤️ 17 days ago
Post
1834
My (really ugly) thoughts on NVIDIA acquiring Hugging Face
I don’t care if you love me or hate me – something about one of the most open community efforts ever to achieve the tagline “The community building the future“ getting gobbled up by a company that arguably is the biggest hardware monopoly that has ever existed strikes me as deeply unsettling. I don’t like monopolies, and that is that. The whole appeal of HF for me personally was always having a neutral location where anyone could develop, deploy, and test a model on their silicon of choice without being pushed into a single “official“ proprietary infrastructure stack.
I am not going to pretend that I would believe NVIDIA “open and independent“ is ever going to happen – hell we have all heard the same lines dozens of times from every corporation that has ever uttered them before.
When the single biggest producer of compute also is one of the primary locations where all open weights live, it becomes very hard not to imagine where all of this is going to end up soon enough if we continue to let companies dictate the narrative. It might be the hyperbole but it is an absolute truth for me – open-sourced AI cannot be a slave to the whims of a trillion dollar company. It is high time we realize that open AI cannot live and breathe only on the goodwill of corporate entities.
I don’t care if you love me or hate me – something about one of the most open community efforts ever to achieve the tagline “The community building the future“ getting gobbled up by a company that arguably is the biggest hardware monopoly that has ever existed strikes me as deeply unsettling. I don’t like monopolies, and that is that. The whole appeal of HF for me personally was always having a neutral location where anyone could develop, deploy, and test a model on their silicon of choice without being pushed into a single “official“ proprietary infrastructure stack.
I am not going to pretend that I would believe NVIDIA “open and independent“ is ever going to happen – hell we have all heard the same lines dozens of times from every corporation that has ever uttered them before.
When the single biggest producer of compute also is one of the primary locations where all open weights live, it becomes very hard not to imagine where all of this is going to end up soon enough if we continue to let companies dictate the narrative. It might be the hyperbole but it is an absolute truth for me – open-sourced AI cannot be a slave to the whims of a trillion dollar company. It is high time we realize that open AI cannot live and breathe only on the goodwill of corporate entities.
- 11 replies