Update README.md

README.md

@@ -3,197 +3,159 @@ library_name: transformers
 tags: []
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
+# Model Card for _Qwen2.5-0.5B-Instruct (Fine-Tuned on OpenR1-Math-220k, 2% subset)_
 
 ## Model Details
 
-### Model Description
+**Model Name**: Qwen2.5-0.5B-Instruct (GRPO Fine-Tuned)  
+**Model ID**: `_Qwen2.5-0.5B-R1subset_`  
+**License**: [Apache 2.0 / or whichever applies]  
+**Finetuned From**: [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct)  
+**Language(s)**: English (mathematical text)
 
-<!-- Provide a longer summary of what this model is. -->
+**Developed By**: Christian Cooper and collaborators  
+**Funding**: Self-sponsored  
+**Shared By**: Christian Cooper  
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+### Model Description
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+This model is a **Qwen2.5-0.5B** base LLM fine-tuned on a **2% subset** of the [OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k) dataset. I used **Group Relative Policy Optimization (GRPO)** from the `trl` library, guiding the model toward producing well-formatted chain-of-thought answers in:
 
-### Model Sources [optional]
+```
+<reasoning>
+  ...
+</reasoning>
+<answer>
+  ...
+</answer>
+```
 
-<!-- Provide the basic links for the model. -->
+It focuses on math reasoning tasks, learning to generate a step-by-step solution (`<reasoning>`) and a numeric or final textual answer (`<answer>`). We incorporate reward functions that encourage correct chain-of-thought structure, numeric answers, and correctness.
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+### Model Sources
+- **GitHub or Repo**: *[Pending]*  
+- **Paper/Demo**: *[Pending]*
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
+- **Math Problem Solving**: The model tries to reason through math word problems, providing step-by-step reasoning and a final answer.
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
+### Downstream Use
+- **Educational Tools**: Potentially used in tutoring or step-by-step solution generation.  
+- **Math Chatbots**: A math helper that can respond in a structured `<reasoning>/<answer>` format.
 
 ### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+- **High-Stakes Decisions**: Model is not guaranteed to be correct for advanced or critical math scenarios (finance, medical, engineering safety).  
+- **Non-English**: Primary training data is English math text, so reliability in other languages is minimal.
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
+- **Bias**: Although this is a math-focused dataset, any language model can exhibit unintended biases.  
+- **Risks**: The model may produce mathematically incorrect or incomplete solutions. The partial coverage (2% of the dataset) further limits accuracy.  
+- **Limitations**: 
+  - Only partially fine-tuned on 2% of the data, so correctness is not guaranteed.  
+  - The chain-of-thought is for interpretability but may still contain flawed reasoning or leaps.
 
-### Recommendations
+## How to Get Started
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+model_name = "HarleyCooper/Qwen.5B-OpenR1Math"  # Will keep the same name through all % iterations. 
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name).to("cuda")
 
-## How to Get Started with the Model
+prompt = """<reasoning>
+Question: It is known that in a convex $n$-gon ($n>3$) no three diagonals pass through the same point.
 
-Use the code below to get started with the model.
+Find the number of points (distinct from the vertices) of intersection of pairs of diagonals.
+</reasoning>
+<answer>
+"""
 
-[More Information Needed]
+inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
+outputs = model.generate(**inputs, max_new_tokens=2000)
+answer = tokenizer.decode(outputs[0])
+print(answer)
+```
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
+- **Dataset**: A 2% subsample (~4.4k problems) of [OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k).  
+- **Data Format**: Each sample has `problem`, `solution`, `answer`. We transform them into:
+  - `"prompt"`: A single string containing system instructions + the problem text.  
+  - `"answer"`: A string with `<reasoning>` + `<answer>` blocks.
 
 ### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
+- **Framework**: [TRL (v0.4+)](https://github.com/lvwerra/trl) with Group Relative Policy Optimization (GRPO).  
+- **Objective**: Reinforcement learning on chain-of-thought format, numeric correctness, and final-answer consistency.  
+- **Reward Functions**:
+  1. **`xmlcount_reward_func`**: Encourages `<reasoning>`/`<answer>` structure.  
+  2. **`soft_format_reward_func`**: Checks for `<reasoning>.*</reasoning><answer>.*</answer>` in any multiline arrangement.  
+  3. **`strict_format_reward_func`**: Strict multiline regex for exact formatting.  
+  4. **`int_reward_func`**: Partial reward if the final `<answer>` is purely numeric.  
+  5. **`correctness_reward_func`**: Binary reward if the final extracted answer exactly matches the known correct answer.
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
+- **Base Model**: Qwen2.5-0.5B  
+- **Learning Rate**: ~5e-6  
+- **Batch Size**: 1–2 (due to GPU constraints)  
+- **Optimizer**: AdamW (β1=0.9, β2=0.99)  
+- **Scheduler**: Cosine with warmup_ratio=0.1  
+- **Num Generations**: 16 (GRPO config)  
+- **Number of Training Epochs**: 1 epoch on 2% data  
+- **Hardware**: Single A100 40GB on Colab  
+- **Max Prompt Length**: 256 tokens  
+- **Max Completion Length**: 200 tokens  
+
+### Speeds, Sizes, Times
+- **Approx. Steps**: ~200–300 steps for 2% subset  
+- **Run Time**: Varies from ~1 to 2 hours on Colab A100
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
+### Testing Data
+- Currently trained + tested on the same subset (2%). Next step would be to evaluate on a withheld portion or the full set to measure true correctness.
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
+### Metrics
+- **Format Rewards**: `xmlcount`, `soft_format`, `strict_format`  
+- **Correctness**: Exact match final numeric/string answer  
+- **Partial Numeric**: `int_reward_func`
 
 ### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
+- The model shows a strong improvement in output format (70–80% format compliance) but relatively low exact numeric correctness. Additional epochs or a larger training fraction are needed for better correctness.
 
 ## Environmental Impact
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
+- **Hardware**: Single A100 40GB GPU in a Colab environment  
+- **Train Time**: ~1–2 hours on 2% data  
+- **Carbon Footprint**: Not measured exactly, but minimal compared to large-scale runs
 
-[More Information Needed]
+## Model Architecture & Objective
 
-#### Hardware
+- **Architecture**: Transformer-based causal language model (Qwen2.5-0.5B)  
+- **Objective**: RL-based chain-of-thought generation for math reasoning
 
-[More Information Needed]
+## Citation
 
-#### Software
+```
+@misc{cooperQwen2.5-0.5B,
+  title={Qwen2.5-0.5B Fine-Tuned on OpenR1 (2% subset)},
+  author={Christian H. Cooper.},
+  howpublished={\url{https://huggingface.co/Christian-cooper-us/Qwen2.5-0.5B-R1subset}},
+  year={2025},
+}
+```
 
-[More Information Needed]
+## Contact
+- Maintainers: Christian Cooper (GitHub: [@christian-cooper-us](https://huggingface.co/HarleyCooper)), others.
 
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
+---
 
-[More Information Needed]
+**Disclaimer**: This model is experimental, trained on only 2% of the dataset. It may produce inaccurate math solutions and is not suitable for high-stakes or time-sensitive deployments.