harrypart's picture
engine plugin (vllm + sglang)
697ef33 verified
Raw
History Blame Contribute Delete
757 Bytes
# SPDX-License-Identifier: Apache-2.0
"""sglang_glm5v: out-of-tree SGLang support for GLM-5.2-Vision.
MoonViT vision tower + trained PatchMerger projector + GLM-5.2 text backbone.
Registered through SGLang's shipped external-package hooks — no in-tree edits:
export SGLANG_EXTERNAL_MODEL_PACKAGE=sglang_glm5v # models (EntryClass)
export SGLANG_EXTERNAL_MM_PROCESSOR_PACKAGE=sglang_glm5v # image processor
export SGLANG_EXTERNAL_MM_MODEL_ARCH=Glm5vForConditionalGeneration
Those hooks do not cover one thing: SGLang gates its DeepSeek-sparse-attention
path on a hardcoded architecture list. Register the arch there with
python -m sglang_glm5v.patch
See patch.py for why that is a genuine gap rather than a workaround.
"""