Hugging Face's logo
Activity Feed

AI & ML interests

None defined yet.

Recent Activity
akhaliq  submitted a paper 13 days ago
Image Generators are Generalist Vision Learners
ozayezerceli  authored a paper 13 days ago
RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models
Q-bert  authored a paper 15 days ago
Selectivity and Shape in the Design of Forward-Forward Goodness Functions
View all activity
Tonic 
posted an update 8 days ago
view post
Post
4000
🙋🏻‍♂️ Hey there folks,

since everyone liked my previous announcement post ( https://huggingface.co/posts/Tonic/338509028435394 ) so much , i'm back with more high quality proceedural datasets in the Geospacial domain for SFT training !

Check this one out :
NuTonic/sat-bbox-metadata-sft-v1

the goal is to be able to train vision models on multiple images for remote sensing analysis with one shot .

hope you like it ! 🚀
  • 2 replies
·
Tonic 
posted an update 13 days ago
view post
Post
3501
🙋🏻‍♂️ Hey there folks ,

I'm sharing huggingface's largest dataset of annotated statelite images today.

check it out here : NuTonic/sat-image-boundingbox-sft-full

I hope you like it , the idea is to be able to use this with small vision models 🚀
XuehangCang 
posted an update 15 days ago
tricktreat 
submitted 2 papers to Daily Papers 21 days ago
tricktreat 
authored 2 papers 22 days ago
appvoid 
posted an update 26 days ago
tricktreat 
authored 6 papers 29 days ago
Muennighoff 
submitted a paper to Daily Papers about 1 month ago
xianbao 
submitted a paper to Daily Papers about 2 months ago
codelion 
posted an update about 2 months ago
view post
Post
3307
Scaling Pedagogical Pre-training to 10 Billion Tokens

New blog post exploring what happens when you take optimal data mixing insights and scale up the data generation itself.

We built Sutra, a multi-stage framework for generating pedagogical pre-training data guided by a knowledge graph of ~2,000 concepts across 9 domains. The pipeline includes structured content generation, six-dimension quality evaluation, diversity management across 20 content styles, and a cleaning stage to prevent collapse.

The result is codelion/sutra-10B, a 10.2 billion token pedagogical dataset with rich metadata (domain, complexity, prerequisites, quality scores) on every entry.

We trained codelion/SmolLM2-70M on it for 3 full epochs (30.6B tokens) on a single A10 GPU in ~78 hours.

Key finding: perplexity kept improving across epochs, but benchmark gains plateaued fast. At 70M parameters, the model hits a representational ceiling that more data alone can't break through.

Full writeup with comparisons against 7 other datasets, detailed benchmark breakdowns, and connections to recent work on synthetic data scaling, curriculum learning, and data mixing laws: https://huggingface.co/blog/codelion/scaling-pedagogical-pretraining-10-billion-tokens

All datasets at multiple scales (10M, 100M, 1B, 10B) plus seed concepts and an SFT variant are in the Sutra Pedagogical Datasets collection.
  • 2 replies
·
GeorgeBredis 
authored a paper 2 months ago
GeorgeBredis 
submitted a paper to Daily Papers 2 months ago
appvoid 
posted an update 2 months ago
view post
Post
2516
Let's keep the momentum for small models. I just published dot. It's the first pretrained causal model that is trained on math/symbols rather than english. The goal is to get an agnostic fewshot meta learner that learns from reality itself instead of language.

It's already decent at some tasks, with next version coming in a few weeks.


appvoid/dot
  • 5 replies
·