Add metadata and link to code

README.md

@@ -1,9 +1,22 @@
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- qwen2
+- pretraining-science
+---
+
 ## Overview
 
+**daVinci-LLM-3B** is a 3B-parameter base language model presented in [daVinci-LLM: Towards the Science of Pretraining](https://huggingface.co/papers/2603.27164). This project aims to make the pretraining process a transparent and reproducible scientific endeavor. 
+
+We release not only the final weights but also training trajectories, intermediate checkpoints, data processing decisions, and 200+ ablation studies covering data quality, mixture design, training dynamics, and evaluation validity. 
 
+- **GitHub:** [GAIR-NLP/daVinci-LLM](https://github.com/GAIR-NLP/daVinci-LLM)
+- **Paper:** [arXiv:2603.27164](https://arxiv.org/abs/2603.27164)
+- **Dataset:** [davinci-llm-data](https://huggingface.co/datasets/SII-GAIR-NLP/davinci-llm-data)
 
-daVinci-LLM-3B is a 3B-parameter base language model aimed at making pretraining a transparent and reproducible scientific process. We release not only the final weights but also training trajectories, intermediate checkpoints, data processing decisions, and 200+ ablation studies covering data quality, mixture design, training dynamics, and evaluation validity. The model reaches an overall score of 51.72 across 19 benchmarks, approaching or matching larger 7B-scale models such as OLMo-3 7B.
-![score-daVinci-llm](score-daVinci-llm.png)
 The model follows a two-stage curriculum over ~8T tokens:
 - **Stage 1 (6T tokens):** broad pretraining over diverse web-scale corpora.
 - **Stage 2 (2T tokens):** structured QA and reasoning-heavy data to amplify math and code reasoning.
@@ -43,24 +56,21 @@ Major categories:
 - **Science/Math:** MegaMath, Nemotron-CC-Math, and Darwin-Science series (L3–L5).
 - **QA:** multi-source QA data with rejection sampling (L5).
 
-## Training Recipe (Summary)
-
-- **Stage 1:** 6T tokens with progressively adjusted mixtures (shifting weight from web text to code/science).
-- **Stage 2:** 2T tokens with structured QA (30% → 70%) for stronger reasoning and problem-solving.
+## Evaluation
 
-## Evaluation (Summary)
+The model reaches an overall average score of **51.72** across 19 benchmarks, matching or exceeding the performance of larger 7B-scale models like OLMo-3 7B.
 
-- **Benchmarks:** 19 tasks spanning General, Code, and Science/Math.
-- **Tooling:** lm-eval-harness.
-- **Result:** Overall average 51.72, comparable to OLMo-3 7B.
-
-## License
-
-Apache-2.0
+| Capability Dimension | daVinci-3B | OLMo-3 7B | LLaMA-3.2-3B | Qwen-2.5-3B |
+| -------------------- | ---------: | --------: | -----------: | ----------: |
+| **Overall Performance** | **51.72** | 51.65 | 37.58 | 51.44 |
+| General Knowledge | 52.96 | 55.13 | 51.08 | 55.16 |
+| Code Generation | 55.99 | 54.42 | 32.40 | 56.13 |
+| Scientific Reasoning | 48.30 | 45.98 | 22.45 | 44.65 |
+| MATH | 62.80 | 39.60 | 9.00 | 37.20 |
 
 ## Citation
 
-```
+```bibtex
 @misc{qin2026davincillmtowardssciencepretraining,
       title={daVinci-LLM:Towards the Science of Pretraining}, 
       author={Yiwei Qin and Yixiu Liu and Tiantian Mi and Muhang Xie and Zhen Huang and Weiye Si and Pengrui Lu and Siyuan Feng and Xia Wu and Liming Liu and Ye Luo and Jinlong Hou and Qipeng Guo and Yu Qiao and Pengfei Liu},
@@ -70,5 +80,4 @@ Apache-2.0
       primaryClass={cs.AI},
       url={https://arxiv.org/abs/2603.27164}, 
 }
-```
-
+```