Naman Vats's picture

Building on HF

Naman Vats PRO

namanvats

·

AI & ML interests

Make Open Source AI win

Recent Activity

upvoted a collection 14 days ago

reacted to anakin87's post with ❤️ 15 days ago

How LLM training with RL Environments works? It all starts with 𝗥𝗲𝗶𝗻𝗳𝗼𝗿𝗰𝗲𝗺𝗲𝗻𝘁 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝘄𝗶𝘁𝗵 𝗩𝗲𝗿𝗶𝗳𝗶𝗮𝗯𝗹𝗲 𝗥𝗲𝘄𝗮𝗿𝗱𝘀 - question asked - model generates reasoning + answer - answer checked against ground truth - reward drives RL training In this setup, the environment is simple: fixed questions and answers, rollout logic, reward(s) Consider a more complex tic-tac-toe env ❌⭕ It adds: - dynamic game generation/handling - tunable opponent skill - multi-turn interactions (envs can also include tools) --- What happens at training? We use 𝗚𝗿𝗼𝘂𝗽 𝗥𝗲𝗹𝗮𝘁𝗶𝘃𝗲 𝗣𝗼𝗹𝗶𝗰𝘆 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 with a tic-tac-toe env No critic model needed, the group is the baseline Simpler than PPO 1️⃣ Rollout generation: from the same board, model plays N games via sampling 2️⃣ Each game scored with deterministic rewards (win, format, ...) 3️⃣ Mean score computed across the group 4️⃣ Each rollout's advantage = its score minus the group mean 5️⃣ Model updated to favor trajectories above baseline 🔁 Repeat For a deep dive, check out 🌱 https://github.com/anakin87/llm-rl-environments-lil-course a free hands-on course on RL environments for LLMs

reacted to ajibawa-2023's post with 👍 16 days ago

Go-Code-Large Dataset: https://huggingface.co/datasets/ajibawa-2023/Go-Code-Large Go-Code-Large is a large-scale corpus of Go (Golang) programming language source code, comprising 316,427 code samples stored in .jsonl format. The dataset is designed to support research and development in large language model (LLM) pretraining, static analysis, cloud-native systems, and modern backend software engineering. By offering a focused and curated dataset for Go, this corpus enables experimentation in concurrent programming, distributed systems, and performance-oriented backend services—domains where Go is widely adopted. Go-Code-Large addresses the relative scarcity of large, language-specific datasets for Go, enabling targeted research into idiomatic Go patterns, concurrency primitives, and scalable system design.

View all activity

Organizations

liked 2 models 16 days ago

MiniMaxAI/MiniMax-M2.7

Text Generation • 229B • Updated 16 days ago • 582k • • 1.1k

Qwen/Qwen3.6-35B-A3B

Image-Text-to-Text • 36B • Updated 12 days ago • 2.88M • • 1.63k

liked a dataset 17 days ago

namanvats/harbor-goose-openhands-benchmark

Viewer • Updated 24 days ago • 246 • 131 • 3

liked a model 2 months ago

google/timesfm-2.5-200m-transformers

Time Series Forecasting • 0.2B • Updated 25 days ago • 188k • 79

liked a model over 1 year ago

sentence-transformers/all-MiniLM-L6-v2

Sentence Similarity • 22.7M • Updated Mar 6, 2025 • 240M • • 4.75k

liked a Space almost 2 years ago

OpenGPT 4o

GPT 4o like bot.

liked a model almost 2 years ago

meta-llama/Meta-Llama-3-70B-Instruct

Text Generation • 71B • Updated Jun 18, 2025 • 71k • • 1.51k

liked a model about 2 years ago

meta-llama/Meta-Llama-3-8B

Text Generation • 8B • Updated Sep 27, 2024 • 3.33M • • 6.53k