File size: 3,286 Bytes
de59a65 7a7dff6 b8e160b 7a7dff6 de59a65 b8e160b de59a65 462017f de59a65 462017f de59a65 462017f de59a65 462017f de59a65 7a7dff6 de59a65 7a7dff6 de59a65 462017f de59a65 462017f de59a65 462017f de59a65 7a7dff6 de59a65 7a7dff6 de59a65 75c6654 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 |
---
base_model: unsloth/gemma-2-2b-it-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- gemma2
- trl
- reasoning
- chain-of-thought
license: apache-2.0
language:
- en
datasets:
- Glint-Research/Fable-5-traces
pipeline_tag: text-generation
---
<div align="center">
<img src="https://cdn.gamma.app/euux6rah8m7qc94/7ae19a8ded5b494390354ae0f0c6dc43/original/image.png" alt="Mable-1 Banner" width="100%">
# Mable-0.5 🧠⚡
*A high-reasoning 2B language model fine-tuned on Gemma-2-2B using Fable-5 reasoning traces.*
</div>
---
## 📌 Model Overview
**Mable-0.5** is a fine-tuned variant of Google's **Gemma-2-2B-it**, trained with 3,500 curated reasoning traces from the **Fable-5** dataset. It specializes in step-by-step reasoning, structured chain-of-thought (CoT) breakdown, and execution-oriented decision making.
* **Developer:** Moonlink
* **Base Model:** `unsloth/gemma-2-2b-it-bnb-4bit`
* **Fine-Tuning Technique:** LoRA (Rank = 16, Alpha = 32)
* **Optimization:** Fine-tuned via [Unsloth](https://github.com/unslothai/unsloth)
---
## 🚀 Available Formats & Usage
This repository contains all 3 formats for maximum flexibility across deployment environments:
### 1. 🦙 GGUF Format (Local / Ollama / LM Studio)
Run `Mable-0.5` locally on CPU or Apple Silicon using the quantized `.gguf` file.
**Using Ollama:**
```bash
# Download and run the quantized GGUF directly from Hugging Face
ollama run hf.co/Moonlink/Mable-0.5:Q4_K_M
```
---
### 2. ⚡ LoRA Adapters (Transformers / Unsloth)
Attach the lightweight adapter weights to the base Gemma-2-2B model.
```python
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "Moonlink/Mable-0.5",
max_seq_length = 2048,
load_in_4bit = True,
)
FastLanguageModel.for_inference(model)
prompt = """<start_of_turn>user
How many r's are in the word strawberry?<end_of_turn>
<start_of_turn>model
THOUGHT:
"""
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
---
### 3. 📦 Merged 16-Bit Weights (vLLM / Pipeline Deployment)
Use the fully merged standalone model for production serving.
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Moonlink/Mable-0.5"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
device_map="auto"
)
```
---
## 📋 Prompt Format
`Mable-0.5` follows the Gemma chat template with explicit `THOUGHT:` and `ACTION:` structural blocks:
```text
<start_of_turn>user
{Your prompt here}<end_of_turn>
<start_of_turn>model
THOUGHT:
{Chain-of-thought reasoning steps}
ACTION:
{Final response or action}
<end_of_turn>
```
---
## 🛠️ Fine-Tuning Hyperparameters
* **Max Sequence Length:** 2,048 tokens
* **Optimizer:** AdamW 8-bit
* **Learning Rate:** 2e-4 (Linear decay)
* **Effective Batch Size:** 4 (Batch size = 1, Gradient Accumulation = 4)
* **Epochs/Steps:** 120 steps (~3,500 rows processed)
* **Precision:** Mixed FP16/BF16
## 🤗 If you benefit from any of our work in HuggingFace please give us a Like or Follow. |