Update README.md

README.md

@@ -1,3 +1,124 @@
----
-license: mit
----
+---
+license: mit
+language: en
+base_model: meta-llama/Meta-Llama-3-8B-Instruct
+tags:
+- math
+- differential-equations
+- dpo
+- lbt
+- instruction-tuned
+---
+
+# LMT-tuning: Llama-3-8B Fine-tuned for Differential Equations
+
+This model is a fine-tuned version of `meta-llama/Meta-Llama-3-8B-Instruct`, specialized for solving university-level differential equations problems.
+
+The model was trained using the **Learning by Teaching (LbT)** paradigm combined with **Direct Preference Optimization (DPO)**. This approach aims to improve a "teacher" model's reasoning capabilities by having it teach a "student" model and learning from the student's performance.
+
+## Model Description
+
+The core idea of the training process was to create a high-quality preference dataset where the "better" response was not just more correct, but also a better piece of teaching material.
+
+The pipeline involved:
+1.  **Data Augmentation:** A raw corpus of ~1500 differential equations problems was flattened and structured into a training set (~1200 problems) and a test set (~300 problems).
+2.  **Teacher Generation:** The base Llama-3-8B model generated 32 step-by-step solutions (rationales) for each of the 1200 training problems.
+3.  **Student Examination (LbT Scoring):** For each of the ~39,000 generated rationales, a "student" model (also Llama-3-8B) was taught using that rationale as a one-shot example. The student then took a similarity-based exam, and its performance yielded an "LbT score" for the rationale.
+4.  **Preference Creation:** Rationales were scored based on a combination of correctness and their LbT score. High-scoring rationales were paired with low-scoring ones to create a preference dataset of `(prompt, chosen, rejected)` triplets.
+5.  **DPO Fine-tuning:** The base Llama-3-8B model was fine-tuned on this preference dataset using `trl`'s `DPOTrainer` and QLoRA.
+
+## Intended Use
+
+This model is primarily intended for:
+-   **Solving differential equations problems:** Providing step-by-step reasoning and a final answer.
+-   **Educational purposes:** Serving as a tool for students to check their work and understand problem-solving steps.
+-   **Research:** Acting as a baseline for further fine-tuning on specialized mathematical domains.
+
+**Note:** This is a specialist model. While it has been fine-tuned for differential equations, its capabilities on general-purpose chat or other reasoning tasks may have degraded.
+
+## How to Use
+
+You can use this model with the `transformers` library pipeline. It is crucial to use the Llama 3 chat template for best results.
+
+```python
+import torch
+from transformers import pipeline
+
+# Load the model and tokenizer
+pipe = pipeline(
+    "text-generation",
+    model="Sandesh-Zenteiq/LMT-tuning",
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+
+# Your differential equations problem
+problem = "Solve the initial value problem: y' - 2y = 0, with y(0) = 3."
+
+# This is the full instruction set the model was trained on
+instruction_text = (
+    "Your task is to answer the last question below. "
+    "Give step by step reasoning before you answer. "
+    "When you're ready to answer, please wrap your answer and conclude using the format\n"
+    "'''\n[[Final Answer]]:\n$ANSWER$\n'''\n\n\n\n"
+)
+exam_template = (
+    "[[Question]]:\n{question}\n\n"
+    "[[Solution]]:\nLet's think step by step.\n\n"
+)
+
+# Format the prompt using the Llama 3 chat template
+prompt = (
+    f"<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n"
+    f"{instruction_text}{exam_template.format(question=problem)}"
+    f"<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"
+)
+
+# Generate the response
+# The pipeline will handle the prompt and only show you the generated part
+response = pipe(
+    prompt,
+    max_new_tokens=1024,
+    do_sample=False, # Use do_sample=True for more creative answers
+    temperature=0.7,
+    top_p=0.9
+)
+
+# Extract and print the generated text
+# The pipeline returns a list of outputs
+generated_text = response['generated_text']
+# The generated text includes the prompt, so we can slice it to see only the model's answer
+assistant_response = generated_text[len(prompt):]
+print(assistant_response)
+
+Training Details
+
+Base Model: meta-llama/Meta-Llama-3-8B-Instruct
+
+Framework: trl.DPOTrainer with QLoRA
+
+Hardware: NVIDIA A6000 / H200 class GPUs
+
+Key Hyperparameters:
+
+learning_rate: 2e-5
+
+num_epochs: 1
+
+lora_r: 128
+
+lora_alpha: 256
+
+gradient_accumulation_steps: 16
+
+Evaluation
+
+The model was evaluated on a held-out test set of 305 differential equations problems that were not seen during training. The metric is Pass@1 accuracy.
+
+Model	Accuracy
+meta-llama/Llama-3-8B-Instruct (Base)	10.16%
+LMT-tuning (This Model)	16.07%
+
+This represents a +5.90 point absolute improvement and a ~58% relative improvement in performance on this specialized task.
+
+Model fine-tuned by Sandesh-Zenteiq. The methodology is based on the paper "Can LLMs Learn by Teaching for Better Reasoning?"```