pipeline_tag: image-text-to-text
license: mit
base_model: zai-org/GLM-4.6V
library_name: kerasformers
language:
- en
- zh
tags:
- keras
- kerasformers
- glm
- glm4v_moe
- multimodal
- vision
- image-text-to-text
- mixture-of-experts
- pytorch
- jax
- tf
Run GLM-4.5V with Keras 3: JAX, PyTorch, or TensorFlow
kerasformers/glm-4.6v
Pure-Keras 3 conversion of zai-org/GLM-4.6V for kerasformers. One implementation runs unmodified on TensorFlow / Torch / JAX. GLM-4.6V is a mixture-of-experts vision-language model (GLM-4V vision tower + GLM-4.5 MoE decoder) served as image + text -> text via Glm4vMoeProcessor; weights are stored in bfloat16, with the MoE router correction bias kept in float32 (matching the upstream mixed-precision checkpoint). See kf_config.json (weight_dtype + weight_dtype_overrides) for the exact layout.
For model details, license, and usage terms, see the upstream model card.
Paper: GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning (arXiv:2507.01006) · HF Papers
✨ Quick start
import os
os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow"
from PIL import Image
from kerasformers.models.glm4v_moe import Glm4vMoeConditionalGenerate, Glm4vMoeProcessor
model = Glm4vMoeConditionalGenerate.from_weights("kerasformers/glm-4.6v")
processor = Glm4vMoeProcessor.from_weights("kerasformers/glm-4.6v")
inputs = processor(conversation=[
{"role": "user", "content": [
{"type": "image", "image": Image.open("photo.jpg")},
{"type": "text", "text": "Describe this image in one sentence."},
]}
])
outputs = model.generate(**inputs, max_new_tokens=64)
print(processor.decode(outputs[0]))
Load any GLM variant the same way with from_weights("kerasformers/<variant>"). Browse them all in the GLM collection.
Special Thanks
A huge thank you to the Zhipu AI / THUDM team for creating and releasing the GLM models.
License: mit (per the upstream model card).