Update README.md

README.md

@@ -1,24 +1,25 @@
 ---
 license: other
+base_model: deepseek-ai/deepseek-math-7b-base
 tags:
 - alignment-handbook
 - generated_from_trainer
-base_model: deepseek-ai/deepseek-math-7b-base
 datasets:
-- AI-MO/numina-problems-sft-v1.7-preproc
+- AI-MO/numina-dataset-v1.0-release-candidate-1-preproc
 model-index:
-- name: sft_deepseek-math-7b_aimo_v31.24
+- name: sft_deepseek-math-7b_aimo_v53.24
   results: []
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-# sft_deepseek-math-7b_aimo_v31.24
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/huggingface/h4/runs/8n1h8p0v)
+# sft_deepseek-math-7b_aimo_v53.24
 
-This model is a fine-tuned version of [deepseek-ai/deepseek-math-7b-base](https://huggingface.co/deepseek-ai/deepseek-math-7b-base) on the AI-MO/numina-problems-sft-v1.7-preproc dataset.
+This model is a fine-tuned version of [deepseek-ai/deepseek-math-7b-base](https://huggingface.co/deepseek-ai/deepseek-math-7b-base) on the AI-MO/numina-dataset-v1.0-release-candidate-1-preproc dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.4538
+- Loss: 0.4859
 
 ## Model description
 
@@ -47,20 +48,21 @@ The following hyperparameters were used during training:
 - total_eval_batch_size: 64
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
 - num_epochs: 3.0
 
 ### Training results
 
 | Training Loss | Epoch | Step  | Validation Loss |
 |:-------------:|:-----:|:-----:|:---------------:|
-| 0.4649        | 1.0   | 6954  | 0.4518          |
-| 0.4026        | 2.0   | 13908 | 0.4403          |
-| 0.3461        | 3.0   | 20862 | 0.4538          |
+| 0.4814        | 1.0   | 6920  | 0.4942          |
+| 0.4188        | 2.0   | 13840 | 0.4728          |
+| 0.3496        | 3.0   | 20760 | 0.4859          |
 
 
 ### Framework versions
 
-- Transformers 4.41.2
-- Pytorch 2.1.2+cu121
+- Transformers 4.42.3
+- Pytorch 2.3.0+cu121
 - Datasets 2.18.0
-- Tokenizers 0.19.1
+- Tokenizers 0.19.1