christopher-kapic's picture
Upload folder using huggingface_hub
fdc6474 verified
|
Raw
History Blame Contribute Delete
966 Bytes

New-box setup

The vLLM changes now live in the fork github.com/jarrelscy/vllm-glm52-sm120 (branch glm52-sm120) — everything is committed, there are no patches to apply.

  1. bash code/run_rtx6000.sh [1m|500k|250k] — does everything (clones the fork, builds the venv with VLLM_USE_PRECOMPILED=1, installs flashinfer git-main for the SM120 sparse-MLA path, sets up the JIT toolchain, fetches weights, serves).
    • Or use the fork's Docker image (RTX PRO 6000): docker build -f Dockerfile.glm52-sm120 -t glm52-sm120 . then docker run --gpus all --ipc=host -p 8001:8001 -v <weights>:/models/1m -e PARALLEL=tp4-1m-mtp glm52-sm120 (see the Dockerfile header for all PARALLEL modes: pp4-1m, tp4-1m-mtp, tp2pp2, …).
  2. Verify with code/tools/verify_sm120.py against the golden bundle (upload it separately or re-capture via code/tools/capture_golden.py).
  3. Full runbook: code/PLAN.md; history: code/glm52-RUNLOG.md.