Nishant9689 commited on
Commit
73d5919
·
verified ·
1 Parent(s): ba503ae

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +75 -3
README.md CHANGED
@@ -1,3 +1,75 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - math
6
+ - education
7
+ - llama-3
8
+ - peft
9
+ - lora
10
+ base_model: meta-llama/Llama-3.2-1B-Instruct
11
+ license: apache-2.0
12
+ ---
13
+
14
+ # NexusLLM-Math-1B-v1
15
+
16
+ ## Model Details
17
+ NexusLLM-Math-1B-v1 is a fine-tuned version of Llama 3.2 (1B parameters) optimized specifically for solving advanced high-school mathematics problems, with a focus on JEE Main and Advanced syllabus topics.
18
+
19
+ - **Developed by:** ZentithLLM
20
+ - **Model Type:** Causal Language Model (Fine-tuned with LoRA)
21
+ - **Language:** English
22
+ - **Base Model:** meta-llama/Llama-3.2-1B-Instruct
23
+ - **Precision:** FP16
24
+
25
+ ## Intended Use
26
+ This model is designed to act as an educational assistant for 11th-grade mathematics. It is trained to provide step-by-step reasoning and explanations for complex topics, rather than just outputting the final answer.
27
+
28
+ **Primary Topics Covered:**
29
+ - Binomial Theorem
30
+ - Geometry (Circle Theorems, cyclic quadrilaterals, tangents, etc.)
31
+
32
+ ## Training Data
33
+ The model was trained on a custom dataset of structured mathematics Q&A pairs. The dataset maps specific mathematical prompts to detailed completions, heavily utilizing an `explanation` field to teach the model the underlying mathematical logic and derivation steps.
34
+
35
+ ## Training Procedure
36
+ The model was fine-tuned using the standard Hugging Face `trl` and `peft` libraries on a single NVIDIA T4 GPU, utilizing strictly native FP16 precision to ensure mathematical gradient stability.
37
+
38
+ - **Training Framework:** Pure Hugging Face (No Unsloth/Quantization)
39
+ - **Method:** LoRA (Low-Rank Adaptation)
40
+ - **Rank (r):** 32
41
+ - **Alpha:** 32
42
+ - **Optimizer:** adamw_torch
43
+ - **Learning Rate:** 2e-4
44
+ - **Max Sequence Length:** 2048
45
+
46
+ ## How to Use
47
+ Because this model was trained on a specific dataset structure, you **must** wrap your prompts in the `### Instruction:` and `### Response:` format for it to output the correct mathematical explanations.
48
+
49
+ ```python
50
+ import torch
51
+ from transformers import AutoModelForCausalLM, AutoTokenizer
52
+
53
+ model_id = "ZentithLLM/NexusLLM-Math-1B-v1"
54
+
55
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
56
+ model = AutoModelForCausalLM.from_pretrained(
57
+ model_id,
58
+ torch_dtype=torch.float16,
59
+ device_map="auto"
60
+ )
61
+
62
+ question = "What is the general term in the expansion of (x+y)^n?"
63
+ formatted_prompt = f"### Instruction:\\n{question}\\n\\n### Response:\\n"
64
+
65
+ inputs = tokenizer(formatted_prompt, return_tensors="pt").to(model.device)
66
+
67
+ outputs = model.generate(
68
+ **inputs,
69
+ max_new_tokens=250,
70
+ temperature=0.3,
71
+ do_sample=True,
72
+ pad_token_id=tokenizer.eos_token_id
73
+ )
74
+
75
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))