appvoid

/

picollama

Like 0

sure you have

may i ask where did you find benchmark contamination you claim?

1 sec

https://arxiv.org/pdf/2504.13161

https://arxiv.org/pdf/2504.13161

clearly says validation there, they validated because they were benchmaxxing smollm and nemotron datasets towards a mix that make those benchmarks do good (funny thing is smollm is part of fineweb which most open slm leaderboard models are fed on)

It's kind of a confusing language from them (Nvidia), grant you that. But if these models do good it's just because they went for the models that did good with the right mixture of smollm and nemotron not literally training, there's no point on making a paper on how we train on the benchmarks lol you are understimating scientists to a whole level

@appvoid thats still benchmaxxing on benchmark style data, idk ask dan he didnt allow this kind of stuff before

Hi guys, just wanted to make it clearthat we do allow climbmix for a couple of reasons:

1. while benchmarks were used to optimize the data mix (as they are for every model) they are not included in the mix.
2. climb not only improved the benchmarks they optimized for, it also improved entirely held out ones as well as generation.
   The modes that have been rejected in the past were rejected for training on sythetic datasets created off of arithmark 3, not for climb.