Text Generation
Transformers
Safetensors
Korean
English
glm4_moe_lite
glm
Mixture of Experts
pruning
reap
expert-pruning
experimental
code-calibration
baseline
conversational
Instructions to use buugiiiiii/GLM-4.7-Flash-REAP-Code-48E with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use buugiiiiii/GLM-4.7-Flash-REAP-Code-48E with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="buugiiiiii/GLM-4.7-Flash-REAP-Code-48E") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("buugiiiiii/GLM-4.7-Flash-REAP-Code-48E") model = AutoModelForCausalLM.from_pretrained("buugiiiiii/GLM-4.7-Flash-REAP-Code-48E", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use buugiiiiii/GLM-4.7-Flash-REAP-Code-48E with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/buugiiiiii/GLM-4.7-Flash-REAP-Code-48E
- SGLang
How to use buugiiiiii/GLM-4.7-Flash-REAP-Code-48E with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "buugiiiiii/GLM-4.7-Flash-REAP-Code-48E", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use buugiiiiii/GLM-4.7-Flash-REAP-Code-48E with Docker Model Runner:
docker model run hf.co/buugiiiiii/GLM-4.7-Flash-REAP-Code-48E
File size: 3,847 Bytes
ff4c31a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 | {
"status": "PASS",
"stage": "lossless safetensors checkpoint reshard validation",
"original_shards": 48,
"output_shards": 16,
"tensors": 7495,
"tensor_equality_checked": 7495,
"tensor_values_identical": true,
"total_tensor_bytes": 48548430720,
"index_sha256": "7c1ac6991008cc774bec6562a910f90bc02947b15496acfc4f03d309c4733386",
"shards": [
{
"filename": "model-00001-of-00016.safetensors",
"tensors": 273,
"bytes": 3035883560,
"sha256": "347e1e966e05955da3adb53b2eec474e50500d640ce3c424fe78a62b2526c0e4"
},
{
"filename": "model-00002-of-00016.safetensors",
"tensors": 494,
"bytes": 3031592368,
"sha256": "1c705147d500c51516fe79e384c6be9be70a79c7bf4647ab645b956003b52384"
},
{
"filename": "model-00003-of-00016.safetensors",
"tensors": 494,
"bytes": 3031592360,
"sha256": "74823b8acddac5b6dfb07e3bc941029d09c3474f5d75654d785515e43adb7ee9"
},
{
"filename": "model-00004-of-00016.safetensors",
"tensors": 499,
"bytes": 3037567928,
"sha256": "296e87b41d35e12e1d1e7fc2f847bc643d78b08c89769f32524d501be73cb41e"
},
{
"filename": "model-00005-of-00016.safetensors",
"tensors": 494,
"bytes": 3031592848,
"sha256": "f2187a7ec8b874ee511bbc182520fb69d84881de80ce78dea45b3357443a77aa"
},
{
"filename": "model-00006-of-00016.safetensors",
"tensors": 494,
"bytes": 3031592856,
"sha256": "381226817ade2041556834f9e7fa513fccde9334fbebb69025c23f1ece537b5e"
},
{
"filename": "model-00007-of-00016.safetensors",
"tensors": 495,
"bytes": 3037884448,
"sha256": "2ccb276c9c6ed3b8281eb91813040f861746693b884b23fe179295ea96cf766a"
},
{
"filename": "model-00008-of-00016.safetensors",
"tensors": 494,
"bytes": 3031592856,
"sha256": "a1a09e2a579d53bad441f52ffef055244c666abc52d9c15da29e94c1bf290f74"
},
{
"filename": "model-00009-of-00016.safetensors",
"tensors": 495,
"bytes": 3037884432,
"sha256": "fce41f48d52c8b2be4ce92609e9ba4d76df539672118bf096ffca5d00687cfd9"
},
{
"filename": "model-00010-of-00016.safetensors",
"tensors": 494,
"bytes": 3031592856,
"sha256": "e38c471badaf501e967572cb383b347df9b378057a594bcb8bfa31dd4ada7b1e"
},
{
"filename": "model-00011-of-00016.safetensors",
"tensors": 498,
"bytes": 3030746656,
"sha256": "d5b5a7cc1efa62f09e4d63a903256bf803df36e89ebcf283b215bf15dcb475a1"
},
{
"filename": "model-00012-of-00016.safetensors",
"tensors": 495,
"bytes": 3038414312,
"sha256": "af96466440fa80dd2c24b6a8ff10ea21b40c472b3f0ad773673776e97ab20c36"
},
{
"filename": "model-00013-of-00016.safetensors",
"tensors": 495,
"bytes": 3037884448,
"sha256": "089ca2caba198d86c5a5c927d5f0566ec19639d8ec54b1f9b1f1e64dfbda53c8"
},
{
"filename": "model-00014-of-00016.safetensors",
"tensors": 494,
"bytes": 3031592848,
"sha256": "3b470dabda3dee1c486bcc639aa0678b7c0c8741b5cef5c038de262fcf5badf7"
},
{
"filename": "model-00015-of-00016.safetensors",
"tensors": 417,
"bytes": 3181525720,
"sha256": "0e0b30c84bc3b05d4a3cec22663dfff02454ddd111e788b064a5ea342b793da5"
},
{
"filename": "model-00016-of-00016.safetensors",
"tensors": 370,
"bytes": 2890418256,
"sha256": "200f87350730eb72fd3ad4d813fd9b441058c9bb928d600a70cd8299754d64a4"
}
],
"runtime_validation": {
"status": "PASS",
"model_class": "Glm4MoeLiteForCausalLM",
"model_dtype": "torch.bfloat16",
"trust_remote_code": false,
"experts": 48,
"generated_tokens": 63,
"failure_count": 0,
"failures": []
},
"failure_count": 0,
"failures": []
}
|