LLaMAX/GlotMAX-101-8B-LST
8B • Updated • 24 • 4
FeiYuan
FeYuan
AI & ML interests
Multilingual & Code Intelligence
Recent Activity
posted an update 9 days ago
Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models
Standard SFT usually ask:
How can we SFT an instruct model without behavioral drift?
We reverse the question:
Given a fixed drift budget, how should we SFT?
This predicts that changing *where* the model can move—not the supervision—can reverse fine-tuning failure.
On Qwen3-8B and Qwen3-14B, it does.
code: https://github.com/CONE-MT/DCO/tree/main/dco
models: https://huggingface.co/collections/LLaMAX/dcoOrganizations
liked 2
models 7 months ago
liked a
dataset 11 months ago
liked a
Space about 2 years ago