---
base_model: unsloth/qwen3-8b-bnb-4bit
library_name: peft
license: apache-2.0
tags:
  - lora
  - sft
  - transformers
  - trl
  - unsloth
  - nba
  - sports-analysis
pipeline_tag: text-generation
model-index:
  - name: LeLM
    results: []
---

# LeLM - NBA Take Analysis Language Model

A LoRA fine-tuned adapter on top of [Qwen3-8B](https://huggingface.co/unsloth/qwen3-8b-bnb-4bit) for analyzing and fact-checking NBA takes using real statistics.

## Model Details

| Parameter | Value |
|---|---|
| Base model | Qwen3-8B (4-bit quantized via Unsloth) |
| Fine-tuning method | LoRA (Low-Rank Adaptation) |
| LoRA rank (r) | 64 |
| LoRA alpha | 128 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Training epochs | 3 |
| Total steps | 915 |
| Batch size | 2 |
| Final training loss | 0.288 |
| Eval loss (epoch 1) | 0.840 |
| Eval loss (epoch 2) | 0.755 |
| Eval loss (epoch 3) | 0.804 |

## Usage

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/qwen3-8b-bnb-4bit",
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "KenWuqianghao/LeLM")
tokenizer = AutoTokenizer.from_pretrained("KenWuqianghao/LeLM")

messages = [
    {"role": "user", "content": "Fact check this NBA take: LeBron is washed"}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Training

Trained with [TRL](https://github.com/huggingface/trl) SFT (Supervised Fine-Tuning) using [Unsloth](https://github.com/unslothai/unsloth) for efficient LoRA training.

### Framework Versions

- PEFT: 0.18.1
- TRL: 0.24.0
- Transformers: 4.57.6
- PyTorch: 2.10.0+cu128
- Datasets: 4.3.0
- Tokenizers: 0.22.2

## Part of LeGM-Lab

This model powers [LeGM-Lab](https://github.com/KenWuqianghao/LeGM-Lab), an LLM-powered NBA take analysis and roasting bot.