iSDFT: Information-Proximal Self-Distillation for Continual Learning in LLMs
Paper • 2609.24646 • Published • 2
🔄 In a Training Loop
Ahmed Khaled Khamis
KickItLikeShika
AI & ML interests
NLP
Recent Activity
upvoted a paper 4 days ago
iSDFT: Information-Proximal Self-Distillation for Continual Learning in LLMs updated a collection 5 days ago
iSDFT: Info-Proximal Self-Distillation Fine-Tuning liked a model 9 days ago
Qwen/Qwen2.5-7B-InstructOrganizations
upvoted a paper 4 days ago
upvoted a collection 14 days ago
upvoted 4
papers 4 months ago
Robots Need More than VLA and World Models
Paper • 2606.06556 • Published • 31
MinT: Managed Infrastructure for Training and Serving Millions of LLMs
Paper • 2605.13779 • Published • 226
Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning
Paper • 2402.13669 • Published • 2
Heterogeneous Scientific Foundation Model Collaboration
Paper • 2604.27351 • Published • 175
upvoted 14
papers 5 months ago
UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
Paper • 2605.06597 • Published • 15
Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages
Paper • 2406.12739 • Published • 2
Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision
Paper • 2604.12002 • Published • 12
Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
Paper • 2604.02288 • Published • 32
Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
Paper • 2601.18734 • Published • 8
Self-Distillation Enables Continual Learning
Paper • 2601.19897 • Published • 41
Reinforcement Learning via Self-Distillation
Paper • 2601.20802 • Published • 51
Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?
Paper • 2603.24472 • Published • 58
Co-Evolving Policy Distillation
Paper • 2604.27083 • Published • 68
DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models
Paper • 2603.26164 • Published • 280
Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction
Paper • 2508.03613 • Published • 16
DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
Paper • 2504.21801 • Published • 7
Adam's Law: Textual Frequency Law on Large Language Models
Paper • 2604.02176 • Published • 109
MA-LoT: Multi-Agent Lean-based Long Chain-of-Thought Reasoning enhances Formal Theorem Proving
Paper • 2503.03205 • Published • 5