README.md

---
base_model: unsloth/llama-3-8b-Instruct-bnb-4bit
library_name: peft
tags:
- llama-3
- reasoning
- chain-of-thought
- unsloth
- trl
- logic
license: llama3
---

# Llama 3 8B - Thinking V2

This model is a specialized LoRA fine-tune of `Meta-Llama-3-8B-Instruct` designed to enforce **Chain-of-Thought (CoT) reasoning** before providing a final answer. By utilizing specialized `<thinking>` tags, the model pauses to break down logic puzzles, coding problems, and tricky phrasing (like riddles and state-tracking) before generating its response.

## 🧠 Model Details
* **Base Model:** Meta Llama 3 8B Instruct
* **Fine-Tuning Method:** LoRA (Low-Rank Adaptation) via Unsloth
* **Dataset:** 475 hand-curated logic, math, and reasoning puzzles.
* **Epochs:** 3
* **Primary Goal:** To force "System 2" thinking, reducing hallucinations and impulsive errors on complex prompts.

## 🚀 How to Use

**CRITICAL:** To trigger the reasoning engine, your prompt **must** be formatted to anticipate the `<thinking>` tag. If you do not prompt the model correctly, it may bypass the reasoning phase and act like a standard Llama 3 model.

### Inference Code (Python / Transformers)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "eelixir/llama3-8b-thinking-v2"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

test_question = "A farmer has 17 sheep. All but 9 run away. How many sheep are left?"

# Note the intentional inclusion of <thinking>\n at the end to "prime" the reasoning!
prompt = f"<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n\n{test_question}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n<thinking>\n"

inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=1024, use_cache=True)

print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])