README.md

---
base_model: unsloth/meta-llama-3.1-8b-bnb-4bit
library_name: peft
pipeline_tag: text-generation
tags:
- base_model:adapter:unsloth/meta-llama-3.1-8b-bnb-4bit
- lora
- sft
- transformers
- trl
- unsloth
license: apache-2.0
datasets:
- sixfingerdev/turkish-qa-multi-dialog-dataset
language:
- tr
- en
- zh
---
# SixFinger-8B Adapter for LLaMA 3.1 8B  

This repository contains a **LoRA adapter** for the SixFinger-8B model.  
The adapter allows fine-tuned responses on top of the base model **```unsloth/llama-3.1-8b-bnb-4bit```** without modifying the base weights.  

---

## Overview

- **Base Model:** unsloth/llama-3.1-8b-bnb-4bit  
- **Adapter Type:** LoRA  
- **Quantization:** 4-bit (via bitsandbytes)  
- **Purpose:** Enhanced response generation for Turkish/English mixed datasets.  
- **Compatibility:** Use with Hugging Face Transformers + PEFT library.  

---

## Installation

Install required dependencies:

```!pip install transformers accelerate bitsandbytes peft```

Ensure you have a GPU with sufficient VRAM for 4-bit inference.  

---

## Loading the Model

1. **Load the Base Model**

```from transformers import AutoTokenizer, AutoModelForCausalLM```

```base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/llama-3.1-8b-bnb-4bit",
    device_map="auto"
)
```

2. **Load the Adapter**

'from peft import PeftModel'

'model = PeftModel.from_pretrained('
'    base_model,'
'    "sixfingerdev/SixFinger-8B"'
')'

3. **Load the Tokenizer**

'tokenizer = AutoTokenizer.from_pretrained("unsloth/llama-3.1-8b-bnb-4bit")'

---

## Example Usage

Generate text using the adapter:
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Base model
base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/llama-3.1-8b-bnb-4bit",
    device_map="auto"
)

# LoRA adapter
model = PeftModel.from_pretrained(base_model, "sixfingerdev/SixFinger-8B")

# Tokenizer
tokenizer = AutoTokenizer.from_pretrained("unsloth/llama-3.1-8b-bnb-4bit")

# Örnek text generation
prompt = "Soru: Yapay zeka nedir?\nCevap:"
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.7)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
---

## Notes

- The adapter does **not** modify the base model; it only applies LoRA weights on top.  
- 4-bit quantization significantly reduces VRAM usage. Ensure your GPU supports **bitsandbytes 4-bit operations**.  
- You can merge the adapter into the base model for easier deployment if needed.  

---

## References

- [PEFT (Parameter-Efficient Fine-Tuning)](https://huggingface.co/docs/peft/index)  
- [Transformers 4-bit Quantization](https://huggingface.co/docs/transformers/main/en/main_classes/quantization)  

---

## License

The adapter and its usage are provided under the terms specified in the repository.  
Ensure compliance with the **base model license** (Meta’s LLaMA).