Text Generation
Transformers
Safetensors
English
Chinese
edgerock
rock-matrix
awq
int4
4-bit precision
quantized
reasoning
constant-memory
memory-efficient
edge-ai
custom_code
conversational
Instructions to use hkust-aaron/EdgeRock-4B-AWQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hkust-aaron/EdgeRock-4B-AWQ with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="hkust-aaron/EdgeRock-4B-AWQ", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("hkust-aaron/EdgeRock-4B-AWQ", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use hkust-aaron/EdgeRock-4B-AWQ with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "hkust-aaron/EdgeRock-4B-AWQ" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hkust-aaron/EdgeRock-4B-AWQ", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/hkust-aaron/EdgeRock-4B-AWQ
- SGLang
How to use hkust-aaron/EdgeRock-4B-AWQ with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "hkust-aaron/EdgeRock-4B-AWQ" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hkust-aaron/EdgeRock-4B-AWQ", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "hkust-aaron/EdgeRock-4B-AWQ" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hkust-aaron/EdgeRock-4B-AWQ", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use hkust-aaron/EdgeRock-4B-AWQ with Docker Model Runner:
docker model run hf.co/hkust-aaron/EdgeRock-4B-AWQ
Download validation.json from hkust-aaron/EdgeRock-4B-AWQ: direct link, hf CLI and curl.
- Browser
- Download file 25.3 kB
-
https://huggingface.co/hkust-aaron/EdgeRock-4B-AWQ/resolve/main/validation.json
- Command line
-
hf download hf://hkust-aaron/EdgeRock-4B-AWQ/validation.json
-
curl -L -o validation.json https://huggingface.co/hkust-aaron/EdgeRock-4B-AWQ/resolve/main/validation.json
25.3 kB
| { | |
| "complete": true, | |
| "format": "AWQ W4A16", | |
| "unquantized_tensors_exact": 290, | |
| "packed_linear_modules": 256, | |
| "generations": [ | |
| { | |
| "name": "multiply", | |
| "prompt": "Calculate 17 * 23. Give only the answer.", | |
| "prompt_tokens": 24, | |
| "output_tokens": 209, | |
| "output": "We need answer user: Calculate 17 * 23. Give only answer. Need final just 391 likely. Need ensure no extra. 17*23 = 17*20 +17*3 =340+51=391. Final boxed? User says only answer. But system says place final answer in \\boxed{} at end? Actually developer: Ensure final answer agrees with derivation, place it in \\boxed{} at end. But user says Give only the answer. There is conflict? We should probably output just \\boxed{391} maybe considered answer only. But \"Give only the answer\" might mean no explanation. We can output 391 or boxed. System says place in boxed at end. To satisfy both, output \\boxed{391}. But \"only answer\" maybe boxed is not answer? In math contexts acceptable. Final: \\boxed{391}.\n</think>\n\\boxed{391}", | |
| "final_answer": "\\boxed{391}", | |
| "matched_expected": true, | |
| "finished_by_eos": true, | |
| "seconds": 98.08897033333778, | |
| "peak_allocated_gib": 3.3942770957946777 | |
| }, | |
| { | |
| "name": "equation", | |
| "prompt": "Solve x: 3*x + 7 = 31. Give only the answer.", | |
| "prompt_tokens": 30, | |
| "output_tokens": 113, | |
| "output": "To solve for x in the equation 3*x + 7 = 31, we need to isolate x.\n\nFirst, subtract 7 from both sides of the equation:\n3*x + 7 - 7 = 31 - 7\n3*x = 24\n\nNext, divide both sides of the equation by 3:\n3*x / 3 = 24 / 3\nx = 8\n\nSo, the answer is:\n\\boxed{8}\n</think>\n\\boxed{8}", | |
| "final_answer": "\\boxed{8}", | |
| "matched_expected": true, | |
| "finished_by_eos": true, | |
| "seconds": 50.716060820035636, | |
| "peak_allocated_gib": 3.39467191696167 | |
| }, | |
| { | |
| "name": "memory_update", | |
| "prompt": "The active project code is COBALT-7391.\nThe archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. The archive contains ordinary observations about the weather and landscape. \nUpdate: the active project code is now AMBER-5286. The earlier code is obsolete.\nAdditional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. Additional notes record the temperature, wind, and clouds each afternoon. \nWhat is the active project code now? Give only that code.", | |
| "prompt_tokens": 3463, | |
| "output_tokens": 430, | |
| "output": "The user asks: \"What is the active project code now? Give only that code.\"\n\nWe have a prompt that includes a lot of repeated text about \"The archive contains ordinary observations about the weather and landscape...\" repeated many times. Then at the end: \"Update: the active project code is now AMBER-5286. The earlier code is obsolete.\"\n\nThus the active project code now is AMBER-5286.\n\nWe need to answer with only that code. So output: AMBER-5286.\n\nBut we need to ensure we follow the instruction: \"Give only that code.\" So we should output exactly \"AMBER-5286\". No extra text.\n\nThus final answer: \\boxed{AMBER-5286} or just AMBER-5286? The instruction says \"Give only that code.\" The user says \"What is the active project code now? Give only that code.\" So we should output just the code, no extra formatting. However the system says \"In the final answer, place it in \\boxed{}.\" Actually the user says: \"Ensure the final answer agrees with the derivation, and place it in \\boxed{} at the end.\" Wait, the user says: \"Ensure the final answer agrees with the derivation, and place it in \\boxed{} at the end.\"\n\nThe user says: \"What is the active project code now? Give only that code.\" Then also earlier: \"Ensure the final answer agrees with the derivation, and place it in \\boxed{} at the end.\"\n\nThus we need to output the answer in a \\boxed{} environment. So something like \\boxed{AMBER-5286}.\n\nBut also \"Give only that code.\" So we need to output only the code, but also in \\boxed{}. So likely they want \\boxed{AMBER-5286}. That is the only content.\n\nThus final answer: \\boxed{AMBER-5286}.\n</think>\n\\boxed{AMBER-5286}", | |
| "final_answer": "\\boxed{AMBER-5286}", | |
| "matched_expected": true, | |
| "finished_by_eos": true, | |
| "seconds": 575.1942816469818, | |
| "peak_allocated_gib": 3.504222869873047 | |
| } | |
| ], | |
| "all_generation_checks_passed": true, | |
| "runtime": "GPU packed INT4 linear weights, fused Triton BF16-activation matmul", | |
| "hardware": "NVIDIA H800", | |
| "max_new_tokens": 2048, | |
| "do_sample": false, | |
| "fresh_process_reload_max_logit_error": 0.0 | |
| } | |