README.md

---
license: mit
language:
- en
pipeline_tag: text-generation
tags:
- nlu
- intent-classification
- entity-extraction
- telegram-bot
- phi-3.5
- fine-tuned
base_model: microsoft/Phi-3.5-mini-instruct
---

# Telegram Bot NLU - Fine-tuned Phi-3.5-mini

Fine-tuned Phi-3.5-mini-instruct for intent classification and entity extraction in a Telegram bot.

## Model Details
- **Base Model:** microsoft/Phi-3.5-mini-instruct (3.8B parameters)
- **Fine-tuning Method:** LoRA (Low-Rank Adaptation)
- **Training Data:** 170 examples across 6 intents
- **Training Time:** 5 minutes on RTX 3090 Ti
- **Final Loss:** 10.4

## Intents
- check_weather
- monitor_chess
- goodbye
- help
- show_menu
- cancel

## Entities
- city
- url
- tournament_id
- player_snr
- round
- tournament

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "CyberSecify/telegram-bot-phi35mini-nlu-merged",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
    "CyberSecify/telegram-bot-phi35mini-nlu-merged",
    trust_remote_code=True
)

# Format prompt
system_msg = "You are an NLU assistant. Classify intents and extract entities. Return JSON."
user_msg = "what is the weather in london?"
prompt = f"<|system|>{system_msg}<|end|><|user|>{user_msg}<|end|><|assistant|>"

# Generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.1)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
```

## Training Details
- **Dataset:** 170 examples (Weather: 120, Chess: 12, Simple: 38)
- **Epochs:** 3
- **Batch Size:** 4 (effective 16 with gradient accumulation)
- **Learning Rate:** 2e-4
- **LoRA Config:** r=16, alpha=32, dropout=0.05
- **Trainable Parameters:** 3.1M (0.16% of total)