DEPLOY_GUIDE.md

# DeepX v0.9 — Deployment Guide

## Package Deploy

Tất cả nằm trong 1 folder `deploy/`, không cần download thêm gì:

```
deploy/
├── deepx_v09.pt              # Model weights (1.7GB, float16)
├── tokenizer/                # Gemma tokenizer (~4MB)
│   ├── tokenizer.model
│   ├── tokenizer_config.json
│   └── special_tokens_map.json
├── config.py                 # Model config
├── modeling/                 # Model code
│   ├── __init__.py
│   ├── pipeline.py
│   ├── gdn2_attention.py
│   └── hyperloop.py
└── serve_embedding.py        # Server script
```

---

## Setup Server (RTX 3060 Ti 12GB)

```bash
# 1. Install dependencies
pip install torch>=2.0 transformers flask numpy
pip install triton fla  # FLA kernel (compile JIT lần đầu, mất ~10s)

# 2. Start server
python serve_embedding.py \
    --checkpoint deepx_v09.pt \
    --tokenizer tokenizer/ \
    --port 8080

# Server ready tại http://localhost:8080
```

Lần chạy đầu tiên Triton sẽ compile kernel (~10s). Sau đó cache lại, fast.

---

## API

### Embed texts
```bash
curl -X POST http://localhost:8080/embed \
  -H "Content-Type: application/json" \
  -d '{"texts": ["Mức phạt khi vượt đèn đỏ?", "Thủ tục đăng ký kinh doanh"], "normalize": true}'
```

Response:
```json
{
  "embeddings": [[0.012, -0.034, ...], [0.045, 0.021, ...]],
  "dim": 1536,
  "count": 2,
  "time_ms": 45.2
}
```

### Similarity search
```bash
curl -X POST http://localhost:8080/similarity \
  -H "Content-Type: application/json" \
  -d '{"query": "Mức phạt khi vượt đèn đỏ?", "documents": ["Điều 5. Phạt tiền...", "Điều 7. Quy định..."]}'
```

Response:
```json
{
  "scores": [0.85, 0.42],
  "time_ms": 52.1
}
```

### Health check
```bash
curl http://localhost:8080/health
```

---

## Python Client

```python
import requests

SERVER = "http://localhost:8080"

def embed(texts):
    r = requests.post(f"{SERVER}/embed", json={"texts": texts})
    return r.json()["embeddings"]

def search(query, documents):
    r = requests.post(f"{SERVER}/similarity", json={
        "query": query, "documents": documents
    })
    return r.json()["scores"]

# Sử dụng
embeddings = embed(["Mức phạt vượt đèn đỏ?"])
scores = search("Thủ tục đăng ký?", ["Doc 1...", "Doc 2..."])
```

---

## Hardware Requirements

| Component | Minimum | Recommended |
|---|---|---|
| GPU | RTX 3060 12GB | RTX 3060 Ti 12GB+ |
| RAM | 16GB | 32GB |
| Disk | 3GB | 5GB |
| CUDA | 11.8+ | 12.0+ |
| Python | 3.10+ | 3.11+ |

### VRAM Usage
- Model load: ~3.5GB
- Inference (batch=1, seq=2048): ~1.5GB
- Total peak: ~5GB → fits 12GB comfortably

---

## Performance (RTX 3060 Ti, float16)

| Seq Length | Batch=1 | Batch=8 | Batch=32 |
|---|---|---|---|
| 128 tokens | ~20ms | ~80ms | ~280ms |
| 512 tokens | ~50ms | ~200ms | ~700ms |
| 2048 tokens | ~150ms | ~600ms | ~2000ms |

Throughput: ~30-150 docs/sec depending on length.

---

## Model Info

| Property | Value |
|---|---|
| Architecture | GDN-2 (Gated DeltaNet-2) + Hyperloop |
| Total params | 889M |
| Embedding dim | 1536 |
| Max sequence | 2048 tokens (training) |
| Attention | O(n) linear (FLA Triton kernel) |
| Zalo Legal nDCG@10 | 0.7449 |
| Version | 0.9 |

---

## Troubleshooting

| Issue | Fix |
|---|---|
| CUDA OOM | Giảm batch: sửa MAX_BATCH=16 trong serve_embedding.py |
| Slow first request | Bình thường — Triton compile kernel lần đầu |
| `fla` import error | `pip install fla` hoặc `pip install flash-linear-attention` |
| Triton error | Đảm bảo CUDA toolkit cùng version với PyTorch |

---

*Version: 0.9 | Date: 2026-07-09*