Text Generation
Transformers
Safetensors
Korean
English
glm4_moe_lite
glm
Mixture of Experts
pruning
reap
expert-pruning
experimental
code-calibration
baseline
conversational
Instructions to use buugiiiiii/GLM-4.7-Flash-REAP-Code-48E with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use buugiiiiii/GLM-4.7-Flash-REAP-Code-48E with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="buugiiiiii/GLM-4.7-Flash-REAP-Code-48E") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("buugiiiiii/GLM-4.7-Flash-REAP-Code-48E") model = AutoModelForCausalLM.from_pretrained("buugiiiiii/GLM-4.7-Flash-REAP-Code-48E", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use buugiiiiii/GLM-4.7-Flash-REAP-Code-48E with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/buugiiiiii/GLM-4.7-Flash-REAP-Code-48E
- SGLang
How to use buugiiiiii/GLM-4.7-Flash-REAP-Code-48E with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use buugiiiiii/GLM-4.7-Flash-REAP-Code-48E with Docker Model Runner:
docker model run hf.co/buugiiiiii/GLM-4.7-Flash-REAP-Code-48E
| { | |
| "status": "PASS", | |
| "stage": "lossless safetensors checkpoint reshard validation", | |
| "original_shards": 48, | |
| "output_shards": 16, | |
| "tensors": 7495, | |
| "tensor_equality_checked": 7495, | |
| "tensor_values_identical": true, | |
| "total_tensor_bytes": 48548430720, | |
| "index_sha256": "7c1ac6991008cc774bec6562a910f90bc02947b15496acfc4f03d309c4733386", | |
| "shards": [ | |
| { | |
| "filename": "model-00001-of-00016.safetensors", | |
| "tensors": 273, | |
| "bytes": 3035883560, | |
| "sha256": "347e1e966e05955da3adb53b2eec474e50500d640ce3c424fe78a62b2526c0e4" | |
| }, | |
| { | |
| "filename": "model-00002-of-00016.safetensors", | |
| "tensors": 494, | |
| "bytes": 3031592368, | |
| "sha256": "1c705147d500c51516fe79e384c6be9be70a79c7bf4647ab645b956003b52384" | |
| }, | |
| { | |
| "filename": "model-00003-of-00016.safetensors", | |
| "tensors": 494, | |
| "bytes": 3031592360, | |
| "sha256": "74823b8acddac5b6dfb07e3bc941029d09c3474f5d75654d785515e43adb7ee9" | |
| }, | |
| { | |
| "filename": "model-00004-of-00016.safetensors", | |
| "tensors": 499, | |
| "bytes": 3037567928, | |
| "sha256": "296e87b41d35e12e1d1e7fc2f847bc643d78b08c89769f32524d501be73cb41e" | |
| }, | |
| { | |
| "filename": "model-00005-of-00016.safetensors", | |
| "tensors": 494, | |
| "bytes": 3031592848, | |
| "sha256": "f2187a7ec8b874ee511bbc182520fb69d84881de80ce78dea45b3357443a77aa" | |
| }, | |
| { | |
| "filename": "model-00006-of-00016.safetensors", | |
| "tensors": 494, | |
| "bytes": 3031592856, | |
| "sha256": "381226817ade2041556834f9e7fa513fccde9334fbebb69025c23f1ece537b5e" | |
| }, | |
| { | |
| "filename": "model-00007-of-00016.safetensors", | |
| "tensors": 495, | |
| "bytes": 3037884448, | |
| "sha256": "2ccb276c9c6ed3b8281eb91813040f861746693b884b23fe179295ea96cf766a" | |
| }, | |
| { | |
| "filename": "model-00008-of-00016.safetensors", | |
| "tensors": 494, | |
| "bytes": 3031592856, | |
| "sha256": "a1a09e2a579d53bad441f52ffef055244c666abc52d9c15da29e94c1bf290f74" | |
| }, | |
| { | |
| "filename": "model-00009-of-00016.safetensors", | |
| "tensors": 495, | |
| "bytes": 3037884432, | |
| "sha256": "fce41f48d52c8b2be4ce92609e9ba4d76df539672118bf096ffca5d00687cfd9" | |
| }, | |
| { | |
| "filename": "model-00010-of-00016.safetensors", | |
| "tensors": 494, | |
| "bytes": 3031592856, | |
| "sha256": "e38c471badaf501e967572cb383b347df9b378057a594bcb8bfa31dd4ada7b1e" | |
| }, | |
| { | |
| "filename": "model-00011-of-00016.safetensors", | |
| "tensors": 498, | |
| "bytes": 3030746656, | |
| "sha256": "d5b5a7cc1efa62f09e4d63a903256bf803df36e89ebcf283b215bf15dcb475a1" | |
| }, | |
| { | |
| "filename": "model-00012-of-00016.safetensors", | |
| "tensors": 495, | |
| "bytes": 3038414312, | |
| "sha256": "af96466440fa80dd2c24b6a8ff10ea21b40c472b3f0ad773673776e97ab20c36" | |
| }, | |
| { | |
| "filename": "model-00013-of-00016.safetensors", | |
| "tensors": 495, | |
| "bytes": 3037884448, | |
| "sha256": "089ca2caba198d86c5a5c927d5f0566ec19639d8ec54b1f9b1f1e64dfbda53c8" | |
| }, | |
| { | |
| "filename": "model-00014-of-00016.safetensors", | |
| "tensors": 494, | |
| "bytes": 3031592848, | |
| "sha256": "3b470dabda3dee1c486bcc639aa0678b7c0c8741b5cef5c038de262fcf5badf7" | |
| }, | |
| { | |
| "filename": "model-00015-of-00016.safetensors", | |
| "tensors": 417, | |
| "bytes": 3181525720, | |
| "sha256": "0e0b30c84bc3b05d4a3cec22663dfff02454ddd111e788b064a5ea342b793da5" | |
| }, | |
| { | |
| "filename": "model-00016-of-00016.safetensors", | |
| "tensors": 370, | |
| "bytes": 2890418256, | |
| "sha256": "200f87350730eb72fd3ad4d813fd9b441058c9bb928d600a70cd8299754d64a4" | |
| } | |
| ], | |
| "runtime_validation": { | |
| "status": "PASS", | |
| "model_class": "Glm4MoeLiteForCausalLM", | |
| "model_dtype": "torch.bfloat16", | |
| "trust_remote_code": false, | |
| "experts": 48, | |
| "generated_tokens": 63, | |
| "failure_count": 0, | |
| "failures": [] | |
| }, | |
| "failure_count": 0, | |
| "failures": [] | |
| } | |