DEPLOY_GUIDE.md

DeepX v0.9 — Deployment Guide

Package Deploy

Tất cả nằm trong 1 folder deploy/, không cần download thêm gì:

deploy/
├── deepx_v09.pt              # Model weights (1.7GB, float16)
├── tokenizer/                # Gemma tokenizer (~4MB)
│   ├── tokenizer.model
│   ├── tokenizer_config.json
│   └── special_tokens_map.json
├── config.py                 # Model config
├── modeling/                 # Model code
│   ├── __init__.py
│   ├── pipeline.py
│   ├── gdn2_attention.py
│   └── hyperloop.py
└── serve_embedding.py        # Server script

---

Setup Server (RTX 3060 Ti 12GB)

# 1. Install dependencies
pip install torch>=2.0 transformers flask numpy
pip install triton fla  # FLA kernel (compile JIT lần đầu, mất ~10s)

# 2. Start server
python serve_embedding.py \
    --checkpoint deepx_v09.pt \
    --tokenizer tokenizer/ \
    --port 8080

# Server ready tại http://localhost:8080

Lần chạy đầu tiên Triton sẽ compile kernel (~10s). Sau đó cache lại, fast.

---

API

Embed texts

curl -X POST http://localhost:8080/embed \
  -H "Content-Type: application/json" \
  -d '{"texts": ["Mức phạt khi vượt đèn đỏ?", "Thủ tục đăng ký kinh doanh"], "normalize": true}'

Response:

{
  "embeddings": [[0.012, -0.034, ...], [0.045, 0.021, ...]],
  "dim": 1536,
  "count": 2,
  "time_ms": 45.2
}

Similarity search

curl -X POST http://localhost:8080/similarity \
  -H "Content-Type: application/json" \
  -d '{"query": "Mức phạt khi vượt đèn đỏ?", "documents": ["Điều 5. Phạt tiền...", "Điều 7. Quy định..."]}'

Response:

{
  "scores": [0.85, 0.42],
  "time_ms": 52.1
}

Health check

curl http://localhost:8080/health

---

Python Client

import requests

SERVER = "http://localhost:8080"

def embed(texts):
    r = requests.post(f"{SERVER}/embed", json={"texts": texts})
    return r.json()["embeddings"]

def search(query, documents):
    r = requests.post(f"{SERVER}/similarity", json={
        "query": query, "documents": documents
    })
    return r.json()["scores"]

# Sử dụng
embeddings = embed(["Mức phạt vượt đèn đỏ?"])
scores = search("Thủ tục đăng ký?", ["Doc 1...", "Doc 2..."])

---

Hardware Requirements

Component	Minimum	Recommended
GPU	RTX 3060 12GB	RTX 3060 Ti 12GB+
RAM	16GB	32GB
Disk	3GB	5GB
CUDA	11.8+	12.0+
Python	3.10+	3.11+

VRAM Usage

• Model load: ~3.5GB
• Inference (batch=1, seq=2048): ~1.5GB
• Total peak: ~5GB → fits 12GB comfortably

---

Performance (RTX 3060 Ti, float16)

Seq Length	Batch=1	Batch=8	Batch=32
128 tokens	~20ms	~80ms	~280ms
512 tokens	~50ms	~200ms	~700ms
2048 tokens	~150ms	~600ms	~2000ms

Throughput: ~30-150 docs/sec depending on length.

---

Model Info

Property	Value
Architecture	GDN-2 (Gated DeltaNet-2) + Hyperloop
Total params	889M
Embedding dim	1536
Max sequence	2048 tokens (training)
Attention	O(n) linear (FLA Triton kernel)
Zalo Legal nDCG@10	0.7449
Version	0.9

---

Troubleshooting

Issue	Fix
CUDA OOM	Giảm batch: sửa MAX_BATCH=16 trong serve_embedding.py
Slow first request	Bình thường — Triton compile kernel lần đầu
fla import error	pip install fla hoặc pip install flash-linear-attention
Triton error	Đảm bảo CUDA toolkit cùng version với PyTorch

---

Version: 0.9 | Date: 2026-07-09