Update README.md

README.md

@@ -18,6 +18,86 @@ tags:
 This model was converted to GGUF format from [`agentica-org/DeepScaleR-1.5B-Preview`](https://huggingface.co/agentica-org/DeepScaleR-1.5B-Preview) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
 Refer to the [original model card](https://huggingface.co/agentica-org/DeepScaleR-1.5B-Preview) for more details on the model.
 
+---
+DeepScaleR-1.5B-Preview is a language model fine-tuned from 
+DeepSeek-R1-Distilled-Qwen-1.5B using distributed reinforcement learning
+ (RL) to scale up to long context lengths. The model achieves 43.1% 
+Pass@1 accuracy on AIME 2024, representing a 15% improvement over the 
+base model (28.8%) and surpassing OpenAI's O1-Preview performance with 
+just 1.5B parameters.
+
+
+
+	
+		
+	
+
+		Data
+	
+
+
+
+Our training dataset consists of approximately 40,000 unique problem-answer pairs compiled from:
+
+
+AIME problems (1984-2023)
+AMC problems (prior to 2023)
+Omni-MATH dataset
+Still dataset
+
+
+
+	
+		
+	
+
+		Training Recipe
+	
+
+
+
+We employ Deepseek's Group Relative Policy Optimization (GRPO), a simplified RL algorithm that extends PPO by:
+
+
+Normalizing advantage function over all samples generated from the same prompt.
+Applying KL divergence regularization on top of PPO's surrogate loss to prevent significant policy drift.
+
+
+Reward Function: Our reward function is simple but effective:
+
+
+1 for correct answers passing LaTeX/Sympy checks
+0 for incorrect or improperly formatted answers
+Note: No partial rewards (such as PRMs) or intermediate feedback.
+
+
+Iterative Context Lengthening: A key challenge in 
+scaling RL for reasoning is compute cost. Our approach trains models 
+with progressively longer contexts as the model improves, thus saving 
+monetary costs and end2end training time: 
+
+
+Initial 8K Context (0-1040 steps):
+22.9% -> 33% Pass@1 on AIME 2024
+Trained on 8 A100-80GB GPUs, BS= (Prompts) * (Samples/Prompt) = 128 * 8 = 1024
+
+
+Extended to 16K (steps 1040-1520):
+33% -> 43% Pass@1 on AIME 2024
+Trained on 32 A100-80GB GPUs, BS= (Prompts) * (Samples/Prompt) = 128 * 16 = 2048
+
+
+Further extended to 24K (step 1520+):
+38% -> 43% Pass@1 on AIME 2024
+Trained on 32 A100-80GB GPUs, BS= (Prompts) * (Samples/Prompt) = 128 * 16 = 2048
+Significant improvements within <200 steps
+
+
+
+
+A more detailed description of the training recipe can be found in our blog post.
+
+---
 ## Use with llama.cpp
 Install llama.cpp through brew (works on Mac and Linux)