turbonext-corp/zai-glm5-5layers

GLM-5 (5-Layer Truncated Variant)

This repository contains a heavily truncated 5-layer variant of the original THUDM GLM-5 model.

It is created by keeping only the first 5 transformer layers and removing the remaining depth of the original model.

---

Important Notice

This is NOT the full GLM-5 model and is not functionally equivalent to the original checkpoint.

Key differences:

• Only the first 5 transformer layers are retained
• Significant reduction in model capacity and reasoning ability
• Outputs are degraded compared to the full model
• Intended for research and experimentation only

---

Purpose

This model is intended for:

• Research on model depth reduction
• Latency and memory profiling experiments
• Studying early-layer representations in LLMs
• Building lightweight experimental inference pipelines
• Distillation / student model initialization

It is not recommended for production use.

---

Model Details

• Base model: GLM-5
• Architecture: Transformer (MoE-based in original model)
• Layers kept: 5 (layers 0-4)
• Layers removed: all higher layers
• Vocabulary / tokenizer: unchanged from original

---

Files

This repository includes:

• config.json: modified to reflect 5 layers
• model.safetensors: truncated weights
• tokenizer files: identical to original GLM-5 tokenizer

---

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sofiadsg/zai-glm5-5layers"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True
)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
)

---

Limitations

Because most transformer layers are removed:

• Reasoning depth is heavily reduced
• Long-context coherence is degraded
• Instruction-following quality may be unstable
• Outputs may be repetitive or incomplete

This model should be treated as a structural research artifact, not a general-purpose LLM.

---

Origin

This model is derived from the original GLM-5 checkpoint released by zai.

No additional training was performed; this is a direct structural truncation of the base model.

---

License

This model inherits the license of the original GLM-5 release. Users must comply with the original models terms of use.

---

Recommendation

For better performance, consider:

• Using the full GLM-5 model when possible
• Or training a distilled 5-layer student model instead of truncation

---

Disclaimer

This repository is provided for research purposes only. No guarantees are made regarding correctness, stability, or performance.