File size: 3,988 Bytes
4918845 04e95d6 4918845 04e95d6 4918845 04e95d6 4918845 04e95d6 4918845 04e95d6 4918845 04e95d6 4918845 04e95d6 4918845 04e95d6 4918845 04e95d6 4918845 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 | # CPU-Optimized Small Language Model (SLM)
## 🚀 Revolutionary CPU-First Conversational AI
This is a **blazing-fast, CPU-optimized Small Language Model** that achieves unprecedented speed and efficiency:
### ⚡ Performance Highlights
- **893 tokens/sec** on CPU (fast production speed)
- **3.7MB model size** (76.6% smaller than original)
- **3.7M parameters** (tiny but powerful)
- **Q&A specialized** (learned conversation patterns)
### 🎯 Training Speed
- **2.35 minutes** for fine-tuning (unheard of!)
- **28 minutes** for base training (4 epochs)
- **Total time:** ~30 minutes from scratch to production
### 🔧 Technical Specs
- **Architecture:** Transformer-lite with RMSNorm, SwiGLU, Rotary embeddings
- **Quantization:** 8-bit post-training quantization
- **Optimization:** CPU-first with memory mapping and efficient batching
- **Framework:** PyTorch (CPU optimized)
### 📱 Deployment Ready
- **Mobile-friendly:** 3.7MB fits in any mobile app
- **No GPU required:** Pure CPU inference
- **Fast startup:** Instant model loading
- **Low memory:** Minimal RAM requirements
## Usage
### Quick Start
```python
from huggingface_hub import hf_hub_download
import torch
import sys
sys.path.append('src') # Add your model code path
from model import create_model_from_config
from tokenizer import BPETokenizer
from quantize import QuantizedModel
# Download model files
model_path = hf_hub_download(repo_id="Rahulwale12/SLM", filename="pytorch_model.bin")
config_path = hf_hub_download(repo_id="Rahulwale12/SLM", filename="config.json")
tokenizer_path = hf_hub_download(repo_id="Rahulwale12/SLM", filename="tokenizer.json")
# Load config
import json
with open(config_path, 'r') as f:
config = json.load(f)
# Create model
model_config = {
'model': {
'vocab_size': config['vocab_size'],
'd_model': config['hidden_size'],
'n_layers': config['num_hidden_layers'],
'n_heads': config['num_attention_heads'],
'd_ff': config['intermediate_size'],
'seq_len': config['max_position_embeddings'],
'dropout': 0.1,
'use_rmsnorm': True,
'use_rotary': True,
'use_swiglu': True
}
}
model = create_model_from_config({'model': model_config['model']})
# Load quantized weights
checkpoint = torch.load(model_path, map_location='cpu')
quantized_model = QuantizedModel(model, checkpoint['quantization_bits'])
quantized_model.quantized_weights = checkpoint['quantized_weights']
quantized_model.scales = checkpoint['scales']
quantized_model.zeros = checkpoint['zeros']
quantized_model.dequantize_weights()
# Load tokenizer
tokenizer = BPETokenizer()
tokenizer.load(tokenizer_path)
# Generate text
prompt = "Question: How are you? Answer:"
input_ids = tokenizer.encode(prompt, add_special_tokens=True)
input_ids = torch.tensor([input_ids], dtype=torch.long)
model.eval()
with torch.no_grad():
for _ in range(20):
logits = model(input_ids)[0, -1, :]
next_token = torch.argmax(logits, dim=-1).unsqueeze(0)
input_ids = torch.cat([input_ids, next_token.unsqueeze(0)], dim=1)
response = tokenizer.decode(input_ids[0].tolist(), skip_special_tokens=True)
print(response)
```
### Complete Usage Guide
Run the comprehensive usage guide:
```bash
python usage_guide.py
```
## Model Details
- **Base Model:** Trained on conversational data
- **Fine-tuning:** Specialized for Q&A conversations
- **Quantization:** 8-bit for optimal speed/size balance
- **License:** MIT
## Performance Comparison
| Model | Speed (tokens/sec) | Size | Training Time |
|-------|-------------------|------|---------------|
| Base | 942 | 45.2MB | 28 min |
| **Fine-tuned** | **893** | **3.7MB** | **2.35 min** |
This model represents a breakthrough in CPU-optimized language models, making conversational AI accessible on any device without requiring specialized hardware.
|