|
|
--- |
|
|
license: mit |
|
|
base_model: |
|
|
- microsoft/Phi-4-mini-instruct |
|
|
tags: |
|
|
- telecommunications |
|
|
- text-generation-inference |
|
|
--- |
|
|
|
|
|
# Phi-4-Telecom-Instruct |
|
|
|
|
|
**Base Model**: [`microsoft/Phi-4-mini-instruct`](https://huggingface.co/microsoft/Phi-4-mini-instruct) |
|
|
**License**: MIT |
|
|
|
|
|
## Overview |
|
|
|
|
|
**TSLAM-Mini** is a domain-adapted language model fine-tuned on **100,000 telecom-specific examples**, focused on real-time, industry-grade interactions. It extends the capabilities of the base Phi-4-mini-instruct model to understand and generate text relevant to telecom scenarios such as: |
|
|
|
|
|
- SME (Subject Matter Expert) and customer interactions |
|
|
- Network configuration and troubleshooting workflows |
|
|
- Device provisioning and service activation |
|
|
- Operational support dialogue |
|
|
- Knowledge retrieval for telecom documents |
|
|
|
|
|
## Key Features |
|
|
|
|
|
- **Telecom-Tuned**: Finetuned on domain-specific conversations, logs, and structured dialogues. |
|
|
- **Instruction-Following**: Retains Phi-4’s compact instruction-tuned behavior while adapting to industry-specific patterns. |
|
|
- **Real-Time Scenarios**: Performs well in use cases that require contextual understanding of real-world telecom operations. |
|
|
|
|
|
## Intended Use |
|
|
The areas TSLAM-Mini excels in are: |
|
|
|
|
|
- **Customer Support Agents** (AI copilots or chatbots) |
|
|
- **Network Operations Tools** that process commands or log queries |
|
|
- **Internal Assistants** for engineers and field technicians |
|
|
- **Telecom Knowledge Graphs & RAG Pipelines** |
|
|
|
|
|
## Model Details |
|
|
|
|
|
| Property | Value | |
|
|
|------------------|----------------------------------------| |
|
|
| Base Model | `microsoft/Phi-4-mini-instruct` | |
|
|
| Fine-tuning Data | 100k telecom domain examples | |
|
|
| Training Method | Supervised fine-tuning (SFT) | |
|
|
| License | MIT | |
|
|
|
|
|
|
|
|
## Benchmarks Results |
|
|
|
|
|
| **Benchmark** | **TSLAM-Mini-2B** | Phi-3.5-mini-Ins | Llama-3.2-3B-Ins | Mistral-3B | Qwen2.5-3B-Ins | Qwen2.5-7B-Ins | Mistral-8B-2410 | Llama-3.1-8B-Ins | Llama-3.1-Tulu-3-8B | Gemma2-9B-Ins | GPT-4o-mini-2024-07-18 | |
|
|
|-------------------------------|-----------------|------------------|------------------|------------|----------------|----------------|------------------|-------------------|----------------------|---------------|-------------------------| |
|
|
| **Popular aggregated benchmark** | | | | | | | | | | | | |
|
|
| Arena Hard | 32.8 | 34.4 | 17.0 | 26.9 | 32.0 | 55.5 | 37.3 | 25.7 | 42.7 | 43.7 | 53.7 | |
|
|
| BigBench Hard (0-shot, CoT) | 70.4 | 63.1 | 55.4 | 51.2 | 56.2 | 72.4 | 53.3 | 63.4 | 55.5 | 65.7 | 80.4 | |
|
|
| MMLU (5-shot) | 67.3 | 65.5 | 61.8 | 60.8 | 65.0 | 72.6 | 63.0 | 68.1 | 65.0 | 71.3 | 77.2 | |
|
|
| MMLU-Pro (0-shot, CoT) | 52.8 | 47.4 | 39.2 | 35.3 | 44.7 | 56.2 | 36.6 | 44.0 | 40.9 | 50.1 | 62.8 | |
|
|
| **Reasoning** | | | | | | | | | | | | |
|
|
| ARC Challenge (10-shot) | 83.7 | 84.6 | 76.1 | 80.3 | 82.6 | 90.1 | 82.7 | 83.1 | 79.4 | 89.8 | 93.5 | |
|
|
| BoolQ (2-shot) | 81.2 | 77.7 | 71.4 | 79.4 | 65.4 | 80.0 | 80.5 | 82.8 | 79.3 | 85.7 | 88.7 | |
|
|
| GPQA (0-shot, CoT) | 25.2 | 26.6 | 24.3 | 24.4 | 23.4 | 30.6 | 26.3 | 26.3 | 29.9 | 39.1 | 41.1 | |
|
|
| HellaSwag (5-shot) | 69.1 | 72.2 | 77.2 | 74.6 | 74.6 | 80.0 | 73.5 | 72.8 | 80.9 | 87.1 | 88.7 | |
|
|
| OpenBookQA (10-shot) | 79.2 | 81.2 | 72.6 | 79.8 | 79.3 | 82.6 | 80.2 | 84.8 | 79.8 | 90.0 | 90.0 | |
|
|
| PIQA (5-shot) | 77.6 | 78.2 | 68.2 | 73.2 | 72.6 | 76.2 | 81.2 | 83.2 | 78.3 | 83.7 | 88.7 | |
|
|
| Social IQA (5-shot) | 72.5 | 75.1 | 68.3 | 73.9 | 75.3 | 75.3 | 77.6 | 71.8 | 73.4 | 74.7 | 82.9 | |
|
|
| TruthfulQA (MC2) (10-shot) | 66.4 | 65.2 | 59.2 | 62.9 | 64.3 | 69.4 | 63.0 | 69.2 | 64.1 | 76.6 | 78.2 | |
|
|
| Winogrande (5-shot) | 67.0 | 72.2 | 53.2 | 59.8 | 63.3 | 71.1 | 63.1 | 64.7 | 65.4 | 74.0 | 76.9 | |
|
|
| **Multilingual** | | | | | | | | | | | | |
|
|
| Multilingual MMLU (5-shot) | 49.3 | 51.8 | 48.1 | 46.4 | 55.9 | 64.4 | 53.7 | 56.2 | 54.5 | 63.8 | 72.9 | |
|
|
| MGSM (0-shot, CoT) | 63.9 | 49.6 | 44.6 | 44.6 | 53.5 | 64.5 | 56.7 | 56.7 | 58.6 | 75.1 | 81.7 | |
|
|
| **Math** | | | | | | | | | | | | |
|
|
| GSM8K (8-shot, CoT) | 88.6 | 76.9 | 75.6 | 80.1 | 80.6 | 88.7 | 81.9 | 82.4 | 84.3 | 84.9 | 91.3 | |
|
|
| MATH (0-shot, CoT) | 64.0 | 49.8 | 46.7 | 41.8 | 61.7 | 60.4 | 41.6 | 47.6 | 46.1 | 51.3 | 70.2 | |
|
|
| **Telecom (domain-specific)** | **88.2** | 52.1 | 47.6 | 49.3 | 58.0 | 61.5 | 54.9 | 57.3 | 59.0 | 64.1 | 70.3 | |
|
|
| **Overall** | **63.5** | 60.5 | 56.2 | 56.9 | 60.1 | 67.9 | 60.2 | 62.3 | 60.9 | 65.0 | **75.5** | |
|
|
|
|
|
## Example |
|
|
|
|
|
```text |
|
|
**User**: How do I reconfigure a 5G core node remotely? |
|
|
|
|
|
**Model**: To reconfigure a 5G core node remotely, ensure you have SSH access enabled and the necessary configuration scripts preloaded. From your NOC terminal, run the secure update command with the node's IP and authentication key... |
|
|
``` |
|
|
|
|
|
## How to Use |
|
|
|
|
|
```python |
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
|
|
# Load tokenizer and model |
|
|
model_name = "NetoAISolutions/TSLAM-Mini" |
|
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
model = AutoModelForCausalLM.from_pretrained(model_name) |
|
|
|
|
|
# Define the input using the Phi-style chat template |
|
|
def build_prompt(user_input): |
|
|
system = "<|system|>\nYou are a helpful assistant.<|end|>\n" |
|
|
user = f"<|user|>\n{user_input}<|end|>\n" |
|
|
assistant = "<|assistant|>\n" # Start assistant response |
|
|
return system + user + assistant |
|
|
|
|
|
# Example input |
|
|
user_query = "How do I activate VoLTE on a user's device?" |
|
|
prompt = build_prompt(user_query) |
|
|
|
|
|
# Tokenize and generate |
|
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
outputs = model.generate( |
|
|
**inputs, |
|
|
max_new_tokens=200, |
|
|
temperature=0.7, |
|
|
top_p=0.9, |
|
|
do_sample=True, |
|
|
eos_token_id=tokenizer.convert_tokens_to_ids("<|end|>") |
|
|
) |
|
|
|
|
|
# Decode output |
|
|
response = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
|
print(response) |
|
|
|
|
|
``` |
|
|
|
|
|
## Acknowledgements |
|
|
|
|
|
- Built on top of Microsoft’s [Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct) |
|
|
- Data curation and tuning by [NetoAISolutions](https://netoai.ai/) |
|
|
|
|
|
--- |
