Phase-Technologies commited on
Commit
73a8f60
·
verified ·
1 Parent(s): 2793272

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +126 -12
README.md CHANGED
@@ -1,21 +1,135 @@
1
  ---
2
- base_model: unsloth/qwen2.5-math-1.5b-bnb-4bit
3
- tags:
4
- - text-generation-inference
5
- - transformers
6
- - unsloth
7
- - qwen2
8
  license: apache-2.0
9
  language:
10
  - en
 
 
 
 
 
 
 
 
 
 
 
 
11
  ---
12
 
13
- # Uploaded finetuned model
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
 
15
- - **Developed by:** Phase-Technologies
16
- - **License:** apache-2.0
17
- - **Finetuned from model :** unsloth/qwen2.5-math-1.5b-bnb-4bit
 
18
 
19
- This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
 
 
 
 
 
 
 
 
 
20
 
21
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
1
  ---
 
 
 
 
 
 
2
  license: apache-2.0
3
  language:
4
  - en
5
+ pipeline_tag: text-generation
6
+ tags:
7
+ - unsloth
8
+ - qwen
9
+ - qwen2.5
10
+ - math
11
+ - reasoning
12
+ - alpaca
13
+ - pytorch
14
+ - custom-finetune
15
+ - lor-merged
16
+ base_model: unsloth/Qwen2.5-Math-1.5B
17
  ---
18
 
19
+ # Qwen2.5-Math-1.5B: Generalized & Merged (Phase-Technologies)
20
+
21
+ An ultra-lightweight, high-speed reasoning model heavily fine-tuned for graduate-level mathematical proofs while retaining full conversational generalization and fundamental safety alignments.
22
+
23
+ This model was fine-tuned using **Unsloth** on a standard Google Colab T4 GPU, utilizing Low-Rank Adaptation (LoRA) and subsequently merged into a full 16-bit standalone model for seamless deployment.
24
+
25
+ ## 🚀 Model Overview
26
+
27
+ Standard math-specific LLMs often suffer from **catastrophic forgetting**—when prompted with basic conversational queries, they either hallucinate lengthy pseudo-proofs or fail entirely.
28
+
29
+ This model was engineered to solve that problem. It bridges the gap between hyper-specialized mathematical reasoning and general instruction-following by utilizing a carefully balanced, dual-distribution training dataset.
30
+
31
+ * **Primary Capability:** Capable of outputting highly detailed, step-by-step proofs for advanced algebraic, topological, and geometric problems.
32
+ * **Secondary Capability:** Capable of standard, conversational instruction-following without format-forced hallucinations.
33
+ * **Hardware Optimized:** Designed to run at maximum throughput on low-VRAM environments (like the 16GB T4 GPU) using 4-bit quantization and Triton kernels.
34
+
35
+ ## 🏗 Architecture & Training Methodology
36
+
37
+ ### Base Model
38
+ * **Original Architecture:** [Qwen2.5-Math-1.5B](https://huggingface.co/Qwen/Qwen2.5-Math-1.5B)
39
+ * **Parameters:** 1.5 Billion
40
+ * **Precision:** Merged 16-bit (Fine-tuned in 4-bit via Unsloth)
41
+
42
+ ### The Dataset: Dual-Distribution Blending
43
+ To achieve generalization, the model was fine-tuned on a 50/50 blend of two distinct datasets, batched and streamed via high-throughput Parquet files:
44
+ 1. **Xerv-AI/GRAD (1.93k rows):** A synthetic dataset containing exceptionally long (average 8,000 characters) graduate and research-level mathematical proofs formatted in strict LaTeX.
45
+ 2. **yahma/alpaca-cleaned (2k rows):** A subset of the standard Alpaca dataset used to teach the model how to answer basic queries, roleplay, and recognize when *not* to use complex math.
46
+
47
+ ### Training Configuration
48
+ The fine-tuning process was executed via Supervised Fine-Tuning (SFT) targeting the attention mechanisms.
49
+ * **Target Modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
50
+ * **LoRA Rank (r):** 16
51
+ * **LoRA Alpha:** 16
52
+ * **Optimizer:** `adamw_8bit`
53
+ * **Learning Rate:** 2e-4
54
+ * **Effective Batch Size:** 8 (Batch size 2 with 4 Gradient Accumulation steps)
55
+
56
+ ## 🛡️ Safety & Alignment
57
+
58
+ Despite being fine-tuned on unfiltered mathematical and conversational data, **this model retains its original safety alignment**. Because only 1-2% of the parameters were updated via LoRA (and later merged), the base Qwen2.5 weights responsible for safety remain fully intact.
59
+
60
+ * **NSFW/18+ Prompts:** The model will actively refuse to generate explicit, illegal, or harmful content, relying on the RLHF and DPO safety guardrails instilled during its original pre-training phase.
61
+
62
+ ## 💻 Usage & Inference
63
+
64
+ The model is highly responsive to a strict Instruction/Response template. For best results, use a `repetition_penalty` of roughly 1.15 to prevent the model from infinitely looping through math steps on simpler problems.
65
+
66
+ ### Installation
67
+ ```bash
68
+ pip install unsloth transformers accelerate
69
+
70
+ ```
71
+ ### Python Inference Script
72
+ ```python
73
+ from unsloth import FastLanguageModel
74
+ import torch
75
+
76
+ # 1. Configuration
77
+ repo_name = "Phase-Technologies/qwen2.5-math-1.5b-generalized-merged"
78
+ max_seq_length = 2048
79
+
80
+ # 2. Load the fully merged model
81
+ # Loading in 4-bit is highly recommended for low-VRAM GPUs (like T4)
82
+ model, tokenizer = FastLanguageModel.from_pretrained(
83
+ model_name = repo_name,
84
+ max_seq_length = max_seq_length,
85
+ dtype = None,
86
+ load_in_4bit = True,
87
+ )
88
+
89
+ # 3. Switch to highly optimized inference mode
90
+ FastLanguageModel.for_inference(model)
91
+
92
+ # 4. Define the universal prompt template
93
+ universal_prompt = """### Instruction:
94
+ {}
95
+
96
+ ### Response:
97
+ {}"""
98
+
99
+ # 5. Prepare your query
100
+ query = "Provide a step-by-step proof finding the eigenvalues of the matrix [[2, 1], [1, 2]]."
101
+
102
+ inputs = tokenizer(
103
+ [universal_prompt.format(query, "")],
104
+ return_tensors = "pt"
105
+ ).to("cuda")
106
+
107
+ print("Generating response...")
108
+
109
+ # 6. Generate the output
110
+ outputs = model.generate(
111
+ **inputs,
112
+ max_new_tokens = 1024,
113
+ max_length = None, # Bypasses Hugging Face length warnings
114
+ use_cache = True,
115
+ repetition_penalty = 1.15, # Critical: prevents infinite generation loops
116
+ pad_token_id = tokenizer.eos_token_id
117
+ )
118
 
119
+ # 7. Decode and print the result
120
+ response = tokenizer.batch_decode(outputs, skip_special_tokens = True)[0]
121
+ print(f"\n{'='*50}\nOutput:\n{'='*50}")
122
+ print(response.split("### Response:\n")[-1])
123
 
124
+ ```
125
+ ## 📊 Limitations & Biases
126
+ * **Language:** The model is optimized exclusively for English.
127
+ * **Arithmetic Hallucinations:** While highly capable of symbolic logic and structured proofs, 1.5B parameter models can occasionally suffer from minor arithmetic errors (e.g., simple subtraction mistakes) deep within long proofs.
128
+ * **Prompt Sensitivity:** The model performs best when math queries explicitly ask for a "proof" or "step-by-step" breakdown in the instruction block.
129
+ ## 🤝 Acknowledgements
130
+ * **Alibaba Cloud:** For the phenomenal base Qwen2.5-Math architecture.
131
+ * **Unsloth AI:** For the Triton-optimized training kernels that made compiling this model possible on consumer hardware.
132
+ * **Xerv-AI:** For the GRAD synthetic dataset powering the advanced reasoning capabilities.
133
+ ```
134
 
135
+ ```