Hugging Face's logo
Krrish Agarwalla's picture
4 16

Krrish Agarwalla

kkkamur07
pritamdeka's profile picture abdullahguch's profile picture webxos's profile picture
Ā·
https://krrish-a07.github.io/

AI & ML interests

Anything DL

Recent Activity

liked a model 19 days ago
deepseek-ai/DeepSeek-V4-Pro
reacted to anakin87's post with ā¤ļø 21 days ago
How LLM training with RL Environments works? It all starts with š—„š—²š—¶š—»š—³š—¼š—暝—°š—²š—ŗš—²š—»š˜ š—Ÿš—²š—®š—暝—»š—¶š—»š—“ š˜„š—¶š˜š—µ š—©š—²š—暝—¶š—³š—¶š—®š—Æš—¹š—² š—„š—²š˜„š—®š—暝—±š˜€ - question asked - model generates reasoning + answer - answer checked against ground truth - reward drives RL training In this setup, the environment is simple: fixed questions and answers, rollout logic, reward(s) Consider a more complex tic-tac-toe env āŒā­• It adds: - dynamic game generation/handling - tunable opponent skill - multi-turn interactions (envs can also include tools) --- What happens at training? We use š—šš—暝—¼š˜‚š—½ š—„š—²š—¹š—®š˜š—¶š˜ƒš—² š—£š—¼š—¹š—¶š—°š˜† š—¢š—½š˜š—¶š—ŗš—¶š˜‡š—®š˜š—¶š—¼š—» with a tic-tac-toe env No critic model needed, the group is the baseline Simpler than PPO 1ļøāƒ£ Rollout generation: from the same board, model plays N games via sampling 2ļøāƒ£ Each game scored with deterministic rewards (win, format, ...) 3ļøāƒ£ Mean score computed across the group 4ļøāƒ£ Each rollout's advantage = its score minus the group mean 5ļøāƒ£ Model updated to favor trajectories above baseline šŸ” Repeat For a deep dive, check out šŸŒ± https://github.com/anakin87/llm-rl-environments-lil-course a free hands-on course on RL environments for LLMs
liked a model 2 months ago
nvidia/personaplex-7b-v1
View all activity

Organizations

None yet