Upload TRAINING_DOCUMENTATION.md with huggingface_hub

TRAINING_DOCUMENTATION.md

@@ -418,14 +418,65 @@ hf upload Rayantion26/JINGSI /path/to/JINGSI_merged --repo-type model
 
 ### Step 4: Deploy with vLLM (4-bit on-the-fly) / 使用 vLLM 部署（即時 4-bit 量化）
 
+**⚠️ Important — CUDA Toolkit Required / 重要 — 需要 CUDA 工具包**
+
+**English:** vLLM requires a full CUDA Toolkit installation (with `nvcc` compiler) to serve this model. Without it, vLLM will fail with errors like `FileNotFoundError: 'ninja'` or `Could not find nvcc`. The pip-installed `nvidia-cuda-*` packages are NOT sufficient — you need the system-level CUDA Toolkit.
+
+**繁體中文：** vLLM 需要完整安裝 CUDA 工具包（包含 `nvcc` 編譯器）才能提供此模型。若未安裝，vLLM 會出現 `FileNotFoundError: 'ninja'` 或 `Could not find nvcc` 等錯誤。pip 安裝的 `nvidia-cuda-*` 套件不足夠 — 需要系統級別的 CUDA 工具包。
+
+```bash
+# Ubuntu — Install CUDA Toolkit 13.1
+sudo apt update
+sudo apt install -y cuda-toolkit-13-1
+
+# Set environment variables
+export CUDA_HOME=/usr/local/cuda-13.1
+export PATH=$CUDA_HOME/bin:$PATH
+
+# Verify nvcc
+nvcc --version
+
+# Install ninja (required by FlashInfer for JIT compilation)
+pip install ninja
+```
+
+**⚠️ Known Issue: k_norm Weights Error / 已知問題：k_norm 權重錯誤**
+
+**English:** When serving a fine-tuned Gemma 4 model with vLLM, you may encounter `ValueError: Following weights were not initialized from checkpoint: {'language_model.model.layers.X.self_attn.k_norm.weight'}`. This is a known bug (vLLM PR #41385, not yet released in v0.27.x). The workaround is to patch the weight validation check to skip `k_norm` weights.
+
+**繁體中文：** 在 vLLM 上提供微調後的 Gemma 4 模型時，可能會遇到 `ValueError: Following weights were not initialized from checkpoint` 錯誤，涉及 `k_norm` 權重。這是一個已知問題（vLLM PR #41385，尚未在 v0.27.x 中修復）。解決方法是修改權重驗證檢查以跳過 `k_norm` 權重。
+
+```python
+# Workaround: Patch vLLM default_loader.py
+# File: vllm/model_executor/model_loader/default_loader.py
+# In function track_weights_loading(), before the ValueError raise, add:
+#   weights_not_loaded = {w for w in weights_not_loaded if "k_norm" not in w}
+```
+
+**English:** If you also encounter FlashInfer JIT compilation errors (`Could not find nvcc`), ensure `CUDA_HOME` is set and `nvcc` is on your `PATH`. The Podman/Docker vLLM image already includes CUDA Toolkit and does not need this fix.
+
+**繁體中文：** 如果還遇到 FlashInfer JIT 編譯錯誤（`Could not find nvcc`），請確保已設定 `CUDA_HOME` 且 `nvcc` 在 `PATH` 中。Podman/Docker 的 vLLM 映像檔已包含 CUDA 工具包，不需要此修復。
+
+#### Direct vLLM Serve / 直接使用 vLLM
+
 ```bash
 # Direct vLLM serve (quantizes 16-bit → 4-bit on load)
+export CUDA_HOME=/usr/local/cuda-13.1
+export PATH=$CUDA_HOME/bin:$PATH
+
 vllm serve Rayantion26/JINGSI \
   --quantization bitsandbytes \
   --max-model-len 4096 \
   --host 0.0.0.0 --port 8000
+```
+
+#### Podman Container (Kubernetes-Ready) / Podman 容器（Kubernetes 就緒）
 
-# Podman container (Kubernetes-ready)
+**English:** The official vLLM Docker/Podman image already includes CUDA Toolkit, nvcc, and ninja — so no additional installation is needed when running in a container.
+
+**繁體中文：** 官方 vLLM Docker/Podman 映像檔已包含 CUDA 工具包、nvcc 和 ninja，因此在容器中運行時無需額外安裝。
+
+```bash
 podman run -d --name vllm_engine --gpus all -p 8000:8000 \
   vllm/vllm-openai:latest \
   --model Rayantion26/JINGSI \