README.md

---
base_model: unsloth/gemma-2-2b-it-bnb-4bit
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - gemma2
  - trl
  - reasoning
  - chain-of-thought
license: apache-2.0
language:
  - en
datasets:
  - Glint-Research/Fable-5-traces
pipeline_tag: text-generation
---

<div align="center">

<img src="https://cdn.gamma.app/euux6rah8m7qc94/7ae19a8ded5b494390354ae0f0c6dc43/original/image.png" alt="Mable-1 Banner" width="100%">

# Mable-0.5 🧠⚡

*A high-reasoning 2B language model fine-tuned on Gemma-2-2B using Fable-5 reasoning traces.*

</div>

---

## 📌 Model Overview

**Mable-0.5** is a fine-tuned variant of Google's **Gemma-2-2B-it**, trained with 3,500 curated reasoning traces from the **Fable-5** dataset. It specializes in step-by-step reasoning, structured chain-of-thought (CoT) breakdown, and execution-oriented decision making.

* **Developer:** Moonlink
* **Base Model:** `unsloth/gemma-2-2b-it-bnb-4bit`
* **Fine-Tuning Technique:** LoRA (Rank = 16, Alpha = 32)
* **Optimization:** Fine-tuned via [Unsloth](https://github.com/unslothai/unsloth)

---

## 🚀 Available Formats & Usage

This repository contains all 3 formats for maximum flexibility across deployment environments:

### 1. 🦙 GGUF Format (Local / Ollama / LM Studio)
Run `Mable-0.5` locally on CPU or Apple Silicon using the quantized `.gguf` file.

**Using Ollama:**
```bash
# Download and run the quantized GGUF directly from Hugging Face
ollama run hf.co/Moonlink/Mable-0.5:Q4_K_M
```

---

### 2. ⚡ LoRA Adapters (Transformers / Unsloth)

Attach the lightweight adapter weights to the base Gemma-2-2B model.

```python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "Moonlink/Mable-0.5",
    max_seq_length = 2048,
    load_in_4bit = True,
)
FastLanguageModel.for_inference(model)

prompt = """<start_of_turn>user
How many r's are in the word strawberry?<end_of_turn>
<start_of_turn>model
THOUGHT:
"""

inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

---

### 3. 📦 Merged 16-Bit Weights (vLLM / Pipeline Deployment)

Use the fully merged standalone model for production serving.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Moonlink/Mable-0.5"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto"
)
```

---

## 📋 Prompt Format

`Mable-0.5` follows the Gemma chat template with explicit `THOUGHT:` and `ACTION:` structural blocks:

```text
<start_of_turn>user
{Your prompt here}<end_of_turn>
<start_of_turn>model
THOUGHT:
{Chain-of-thought reasoning steps}

ACTION:
{Final response or action}
<end_of_turn>
```

---

## 🛠️ Fine-Tuning Hyperparameters

* **Max Sequence Length:** 2,048 tokens
* **Optimizer:** AdamW 8-bit
* **Learning Rate:** 2e-4 (Linear decay)
* **Effective Batch Size:** 4 (Batch size = 1, Gradient Accumulation = 4)
* **Epochs/Steps:** 120 steps (~3,500 rows processed)
* **Precision:** Mixed FP16/BF16



## 🤗 If you benefit from any of our work in HuggingFace please give us a Like or Follow.