Update README.md

README.md

@@ -2,31 +2,30 @@
 license: mit
 language:
 - en
+tags:
+- code-llm
+- qwen
+- sft
+- dpo
 ---
+
 # Code-Centric-Align: A Post-Training Pipeline for Code LLMs
 
-[cite_start]This project presents a systematic study of the post-training engineering pipeline for code-specific large language models, using **Qwen2.5-Coder-7B** as the base model[cite: 2, 6, 17, 179]. [cite_start]It establishes a "diagnosable and iterative" framework covering the full lifecycle from data engineering to deployment[cite: 25, 28].
+This project presents a systematic study of the post-training engineering pipeline for code-specific large language models, using **Qwen2.5-Coder-7B** as the base model. It establishes a "diagnosable and iterative" framework covering the full lifecycle from data engineering to deployment.
 
 ## 🛠️ Core Workflow
-* [cite_start]**Data Engineering**: Implemented streaming collection, three-layer quality filtering, and MinHashLSH-based fuzzy deduplication[cite: 50, 52, 57, 180].
-* [cite_start]**Instruction Evolution**: Utilized DeepSeek APIs for Evol-Instruct difficulty enhancement and diversity expansion[cite: 62, 65, 180].
-* [cite_start]**Supervised Fine-Tuning (SFT)**: Applied QLoRA with a custom **Instruction Masking** strategy (QwenDataCollator) to ensure the model only learns from assistant responses[cite: 77, 113, 114, 181].
-* [cite_start]**Rejection Sampling (RFT)**: Developed a high-throughput engine using vLLM for 10-path sampling, verified through a multi-process safe execution sandbox[cite: 135, 137, 151, 181].
-* [cite_start]**Preference Alignment (DPO)**: Investigated Direct Preference Optimization, identifying critical failure modes such as length bias and low-quality negative samples[cite: 17, 138, 145].
-* [cite_start]**Quantization & Deployment**: Performed 4-bit activation-aware quantization (AutoAWQ) and deployed the model via a vLLM OpenAI-compatible API[cite: 146, 151, 182].
+* **Data Engineering**: Implemented streaming collection, three-layer quality filtering, and MinHashLSH-based fuzzy deduplication.
+* **Instruction Evolution**: Utilized DeepSeek APIs for Evol-Instruct difficulty enhancement and diversity expansion.
+* **Supervised Fine-Tuning (SFT)**: Applied QLoRA with a custom **Instruction Masking** strategy (QwenDataCollator) to ensure the model only learns from assistant responses.
+* **Rejection Sampling (RFT)**: Developed a high-throughput engine using vLLM for 10-path sampling, verified through a multi-process safe execution sandbox.
+* **Preference Alignment (DPO)**: Investigated Direct Preference Optimization, identifying critical failure modes such as length bias and low-quality negative samples.
+* **Quantization & Deployment**: Performed 4-bit activation-aware quantization (AutoAWQ) and deployed the model via a vLLM OpenAI-compatible API.
 
 ## 📈 Experimental Results (HumanEval Pass@1)
-[cite_start]The project tracked performance gains and losses across multiple iterations[cite: 17, 143, 179]:
-* [cite_start]**Base Model**: 0.628[cite: 17, 143].
-* [cite_start]**SFT v3 (Final)**: **0.671 (+6.8%)** — achieved through precise loss calculation and data cleaning[cite: 17, 143, 144].
-* [cite_start]**DPO Merged**: 0.280 — highlighting the extreme sensitivity of code models to preference data quality[cite: 17, 143, 145].
-
-## 💡 Key Engineering Insights
-* [cite_start]**Format Over Hyperparameters**: Correct data formatting (Instruction Masking and Packing strategies) was found to be as critical as learning rate tuning for code SFT[cite: 131, 133].
-* [cite_start]**Alignment Challenges**: Code DPO requires strict length balance and functional correctness verification (unit tests) rather than simple runtime checks[cite: 145, 150].
-
----
-[cite_start]**Author**: Kaige Shi [cite: 159]
-[cite_start]**Affiliation**: Dalian University of Technology / University of Science and Technology of China [cite: 166, 167]
+The project tracked performance gains and losses across multiple iterations:
+* **Base Model**: 0.628
+* **SFT v3 (released)**: **0.671 (+6.8%)** — achieved through precise loss calculation and data cleaning.
+* **DPO Merged**: < 0.628 — highlighting the extreme sensitivity of code models to preference data quality.
 
-Would you like me to generate a **Quick Start** guide in English to explain how to run the SFT or RFT scripts?
+⚠️ Status & Roadmap
+This project is actively under development. Currently, the DPO alignment exhibits performance regression (Pass@1 < 0.628) due to preference data sensitivity. We are investigating advanced filtering and reward modeling to resolve this. Optimized weights will be uploaded as soon as the alignment bottleneck is cleared.