README.md

---
base_model:
- Qwen/Qwen2.5-7B-Instruct
library_name: transformers
pipeline_tag: text-generation
language:
- lua
license: apache-2.0
tags:
- fivem
- qbcore
- lua
- gta-v
- gta5
- game-development
- code-generation
- code
- qwen2
- qwen2.5
- qwen-coder
- fine-tuned
---

# Rooja

**Rooja** is a fine-tuned version of
**Qwen2.5-7B-Instruct**, specialized for **FiveM development,
Lua scripting, QBCore, and GTA V server development**.

Rooja is designed to act as a coding assistant for developers building
and maintaining FiveM resources.

## Model Details

| Property | Value |
|---|---|
| Model | Rooja |
| Base Model | Qwen/Qwen2.5-7B-Instruct |
| Parameters | ~7B |
| Fine-tuning Method | QLoRA |
| LoRA Rank | 64 |
| LoRA Alpha | 128 |
| LoRA Dropout | 0.05 |
| Quantization During Training | 4-bit NF4 |
| Double Quantization | Enabled |
| Maximum Training Sequence Length | 8192 |
| Training Epochs | 2 |
| Learning Rate | 1e-4 |
| Effective Batch Size | 16 |
| Optimizer | paged_adamw_8bit |
| Learning Rate Scheduler | cosine |
| Gradient Checkpointing | Enabled |

## Training Dataset

The training run contained:

- **5,969 total examples**
- **4,460 training examples**
- **1,509 validation examples**

The dataset was created for FiveM-oriented coding and development tasks.

## Training Results

Final training results:

| Metric | Result |
|---|---:|
| Final Training Loss | 0.4496 |
| Final Training Token Accuracy | ~91.6% |
| Final Validation Loss | 0.5274 |
| Final Validation Token Accuracy | ~88.0% |
| Epochs | 2 |

The training run completed successfully after 2 epochs.

## What Rooja Is Designed For

Rooja is intended to help with:

- FiveM Lua development
- QBCore scripting
- GTA V server development
- Client-side Lua
- Server-side Lua
- FiveM resources
- `fxmanifest.lua`
- QBCore events and callbacks
- Player and character systems
- Server/client communication
- Configuration files
- Debugging
- Code explanation
- Code generation
- Resource architecture
- FiveM development workflows

## Example Prompt

```text
Create a QBCore FiveM server-side command that gives
cash to another player.

Validate the target player and amount and make sure the
command cannot be abused with invalid values.
```

## Usage

### Transformers

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "lone17k/Rooja"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto"
)

messages = [
    {
        "role": "user",
        "content": "Create a basic FiveM QBCore server-side command."
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(
    text,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.2
)

response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True
)

print(response)
```

## Recommended Generation Settings

For code generation, a low temperature is recommended.

```text
temperature: 0.1 - 0.3
top_p: 0.8 - 0.95
```

For deterministic coding:

```text
temperature: 0.2
```

## Limitations

Rooja may generate incorrect, incomplete, or outdated FiveM and
QBCore APIs.

FiveM resources and frameworks can change over time. Generated code
should therefore be reviewed and tested before being deployed to a
production server.

Rooja should be treated as a coding assistant and not as an
authoritative source of FiveM documentation.

## Base Model

Rooja is based on:

**Qwen/Qwen/Qwen2.5-7B-Instruct**

The original Qwen/Qwen2.5-7B-Instruct model contains approximately
14.7B parameters and supports long-context usage. Its Hugging Face
model card currently identifies the model as Apache-2.0 licensed. 

For the original model and its license, see:

https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct

## Creator

Created and fine-tuned by **Lone17k**.

Hugging Face:

https://huggingface.co/lone17k

Model:

https://huggingface.co/lone17k/Rooja

## Disclaimer

This project is an independent fine-tune and is not affiliated with
Qwen, Alibaba Cloud, FiveM, or Rockstar Games.