Duplicate from xai-org/grok-2

Co-authored-by: Lianmin <lmzheng@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,39 @@
+xAI Community License Agreement
+Last Updated:  November 4, 2025
+
+1. Background and Definitions
+By downloading, accessing, displaying, publishing, modifying, or using the materials provided by X.AI LLC (“xAI”) that xAI designates are subject to this xAI Community License Agreement (“Materials”), you (“Licensee” or “you”) agree to the terms of this agreement (“Agreement”). If you accept this Agreement for or on behalf of an entity, you represent that you have the authority to bind that entity. xAI designates that Materials for Grok 2 consists of: (1) one or more machine learning models (including architecture and parameters); and (2) related artifacts (including associated data, documentation, and software) that are provided to you hereunder.  
+
+2. License Grant & Scope
+a. Permitted Uses: xAI grants you a non-exclusive, worldwide, revocable license to use, reproduce, distribute, and modify the Materials:
+	•	For non-commercial and research purposes; and
+	•	For commercial use solely if you and your affiliates abide by all of the guardrails provided in xAI's Acceptable Use Policy (https://x.ai/legal/acceptable-use-policy), including 1. Comply with the law, 2. Do not harm people or property, and 3. Respect guardrails and don't mislead.
+b. Restrictions:
+	•	You may not use the Materials, derivatives, or outputs (including generated data) to train, create, or improve any foundational, large language, or general-purpose AI models, except for modifications or fine-tuning of xAI Materials permitted under and in accordance with the terms of this Agreement.
+	•	No right to use xAI’s trademarks is granted, except as required for attribution (see below).
+
+3. Distribution & Attribution
+If you distribute the Materials, derivatives, or products/services incorporating them:
+	•	Include this Agreement and a notice stating: “This product includes materials licensed under the xAI Community License. All rights reserved. Copyright © xAI [applicable year].”
+	•	Prominently display “Powered by xAI” in related materials or interfaces.
+
+4. Ownership & Outputs
+xAI retains all rights in the Materials. This Agreement does not impose any restrictions or obligations with respect to any use, modification, or sharing of any outputs generated by using applicable model related Materials. If you provide feedback, suggestions, or ideas, you grant xAI a perpetual, worldwide, irrevocable, royalty-free license to use and incorporate that feedback without restriction.
+
+5. Acceptable Use
+You are responsible for implementing appropriate safety measures, including filters and human oversight, suitable for your use case. You must comply with xAI’s Acceptable Use Policy (AUP), as well as all applicable laws. You may not use the Materials for illegal, unlawful, harmful, or abusive activities.
+
+6. Warranty Disclaimer & Limitation of Liability
+THE MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, NONINFRINGEMENT, ACCURACY, OR THE ABSENCE OF LATENT OR OTHER DEFECTS OR ERRORS, WHETHER OR NOT DISCOVERABLE, ALL TO THE GREATEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW.
+YOU ARE SOLELY RESPONSIBLE FOR (1) CLEARING RIGHTS OF OTHER PERSONS THAT MAY APPLY TO THE MATERIALS OR ANY USE THEREOF, INCLUDING WITHOUT LIMITATION ANY PERSON'S COPYRIGHTS OR OTHER RIGHTS INCLUDED OR EMBODIED IN THE MATERIALS; (2) OBTAINING ANY NECESSARY CONSENTS, PERMISSIONS OR OTHER RIGHTS REQUIRED FOR ANY USE OF THE MATERIALS; OR (3) PERFORMING ANY DUE DILIGENCE OR UNDERTAKING ANY OTHER INVESTIGATIONS INTO THE MATERIALS OR ANYTHING INCORPORATED OR EMBODIED THEREIN.
+IN NO EVENT SHALL XAI BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT, OR OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION WITH THE MATERIALS, THE USE THEREOF, OR OTHER DEALINGS THEREIN. TO THE MAXIMUM EXTENT PERMITTED BY LAW, XAI WILL NOT BE LIABLE FOR ANY INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, OR FOR AGGREGATE LIABILITY EXCEEDING $100, REGARDLESS OF THE LEGAL THEORY.
+
+7. Termination
+This license terminates immediately upon your breach or if you at any point exceed the permitted uses. Upon termination, you must cease all use and delete all copies of the Materials and derivatives.
+Additionally, if you file, maintain, or voluntarily participate in a lawsuit against any person or entity alleging that the Materials, or any part thereof, directly or indirectly infringe any patent or copyright or violate any intellectual property, then your license under this Agreement shall immediately terminate. This does not apply to a lawsuit brought in response to a corresponding lawsuit first filed against you.
+
+8. Governing Law
+The laws of Texas govern this Agreement, and any dispute shall be resolved exclusively in the courts located in Tarrant County, Texas.
+
+9. Miscellaneous
+This Agreement is the entire agreement between the parties on this subject. Failure to enforce any provision is not a waiver. If any provision is unenforceable, the remainder remains in effect. xAI may assign this Agreement, including in connection with a merger or acquisition. Licensee may not assign this Agreement without xAI’s prior written consent. This Agreement creates no third-party beneficiaries. You must comply with all applicable export control, trade compliance, and sanctions laws.

README.md

@@ -0,0 +1,39 @@
+# Grok 2
+
+This repository contains the weights of Grok 2, a model trained and used at xAI in 2024.
+
+## Usage: Serving with SGLang
+
+- Download the weights. You can replace `/local/grok-2` with any other folder name you prefer.
+
+  ```
+  hf download xai-org/grok-2 --local-dir /local/grok-2
+  ```
+
+  You might encounter some errors during the download. Please retry until the download is successful.  
+  If the download succeeds, the folder should contain **42 files** and be approximately 500 GB.
+
+- Launch a server.
+
+  Install the latest SGLang inference engine (>= v0.5.1) from https://github.com/sgl-project/sglang/
+
+  Use the command below to launch an inference server. This checkpoint is TP=8, so you will need 8 GPUs (each with > 40GB of memory).
+  ```
+  python3 -m sglang.launch_server --model /local/grok-2 --tokenizer-path /local/grok-2/tokenizer.tok.json --tp 8 --quantization fp8 --attention-backend triton
+  ```
+
+- Send a request.
+
+  This is a post-trained model, so please use the correct [chat template](https://github.com/sgl-project/sglang/blob/97a38ee85ba62e268bde6388f1bf8edfe2ca9d76/python/sglang/srt/tokenizer/tiktoken_tokenizer.py#L106).
+
+  ```
+  python3 -m sglang.test.send_one --prompt "Human: What is your name?<|separator|>\n\nAssistant:"
+  ```
+
+  You should be able to see the model output its name, Grok.
+
+  Learn more about other ways to send requests [here](https://docs.sglang.ai/basic_usage/send_request.html).
+
+## License
+
+The weights are licensed under the [Grok 2 Community License Agreement](https://huggingface.co/xai-org/grok-2/blob/main/LICENSE).

config.json

@@ -0,0 +1,36 @@
+{
+  "architectures": [
+    "Grok1ForCausalLM"
+  ],
+  "embedding_multiplier_scale": 90.50966799187809,
+  "output_multiplier_scale": 0.5,
+  "vocab_size": 131072,
+  "hidden_size": 8192,
+  "intermediate_size": 32768,
+  "moe_intermediate_size": 16384,
+  "max_position_embeddings": 131072,
+  "num_experts_per_tok": 2,
+  "num_local_experts": 8,
+  "residual_moe": true,
+  "num_attention_heads": 64,
+  "num_key_value_heads": 8,
+  "num_hidden_layers": 64,
+  "head_dim": 128,
+  "rms_norm_eps": 1e-05,
+  "final_logit_softcapping": 50,
+  "attn_logit_softcapping": 30.0,
+  "router_logit_softcapping": 30.0,
+  "rope_theta": 208533496,
+  "attn_temperature_len": 1024,
+  "sliding_window_size": -1,
+  "global_attn_every_n": 1,
+  "model_type": "git",
+  "torch_dtype": "bfloat16",
+  "rope_type": "original",
+  "original_max_position_embeddings": 8192,
+  "scaling_factor": 16.0,
+  "extrapolation_factor": 1.0,
+  "attn_factor": 1.0,
+  "beta_fast": 8,
+  "beta_slow": 1
+}

pytorch_model-00000-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:315d2eb626636477b35a19c60b481ae43b70fc7b9d24645255ded00cecf2a6bb
+size 2147483760

pytorch_model-00001-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f8001b2d7baaee0bd5bce3ad41d117148c591e485dc8a838b8abccb650ef9bb
+size 2147483744

pytorch_model-00002-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e41eeadefc5da657bac925e752dab11672d7ccee729ed60df8c32df60c6fcd6e
+size 16472

pytorch_model-00003-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5117eeb28c1447403df924fa93c62a9ea09d2d6fa773e27367fc5e6687b10b99
+size 34359745872

pytorch_model-00004-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6921cc3380cfbd783a3b52bccf81bc5c7c645548f9f2f1b86279649edbba3cf0
+size 34359745872

pytorch_model-00005-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90bb1c9d09d31b58becb2cf13f1694166708a52cd97315185c6a7ddbcc64f1ca
+size 34359745744

pytorch_model-00006-TP-000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f5128445f49d89ed53463d6e454ef9f466ca65e2607a006761a558f9fd60754d
+size 17179936544

pytorch_model-00006-TP-001.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9f0869351c7a2baac725b2b2903dec79b9afb0f49f2004ce3a1f35763708e40
+size 17179936544

pytorch_model-00006-TP-002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:930f397a72ed323783e3f029bc1252beb9f2d2bce69610fa0f094e361547d1dc
+size 17179936544

pytorch_model-00006-TP-003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b266d9293692e69cc2decbf1914916cde492de8b485e7ad670447674e5170e49
+size 17179936544

pytorch_model-00006-TP-004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be0f2daffdc5a7c031ce685cc5bd89fe9058eedb14a107855ea45e272b7022c0
+size 17179936544

pytorch_model-00006-TP-005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4697b87056edc930ec836528985fb25a40040417226e47a9c483786b81b56a7d
+size 17179936544

pytorch_model-00006-TP-006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d572dc1910937e16be0017118daca90734af892ea019581584c738abd74975d
+size 17179936544

pytorch_model-00006-TP-007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cddca91357cc76894e1704d0bba02dcddeb7e063329f30198a26f3eec98f1ca1
+size 17179936544

pytorch_model-00007-TP-000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ca76131b44751d11ba6abafa0567939812c83af63ff3ed91771e5a03b28e395
+size 17179936544

pytorch_model-00007-TP-001.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6689603a91fec740d69a67512a7d6c3d4176a2068fcc88e08a79646c7bc9a4ec
+size 17179936544

pytorch_model-00007-TP-002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91c5af3bc69e126adf09cf066b5554fd55a804cb246e48c769725044bf6c2482
+size 17179936544

pytorch_model-00007-TP-003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1643a8a743dd135e3aa082cb732ade54c624e188e264ce2ca29818eca4579129
+size 17179936544

pytorch_model-00007-TP-004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1cce342c135799ea961f7c7335e1bd485190e7ed4511f1a4b25792af856b702b
+size 17179936544

pytorch_model-00007-TP-005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecb1e412e72be6acf5a94300d20b922440710dbc3ff655081e14fde52dead193
+size 17179936544

pytorch_model-00007-TP-006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff8c1dc7ba5222e7e88eeaa8c71f9bcc6c72abb2642cd53708e89c3bb29e9a5e
+size 17179936544

pytorch_model-00007-TP-007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d890e53bae6eb64606791a70ec4055a0909eb75d704bc505050f87a8f4304776
+size 17179936544

pytorch_model-00008-TP-000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f6023ef14f89644bba6dea2b3cd268ef90a603a724e58e6f25fa4b35e09da805
+size 17179936544

pytorch_model-00008-TP-001.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:adf26b682c3898af5f1ab6fe106579d29aea00ab661e8d164988b6e8093f6412
+size 17179936544

pytorch_model-00008-TP-002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8381e5ff848f33a2a715155aec5303309d1044bb105d0267bbbbd702e2ea4181
+size 17179936544

pytorch_model-00008-TP-003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38471f261ce898dc7d1266fbabbacd922248767d4e909a0f431d2adaf77c9d1f
+size 17179936544

pytorch_model-00008-TP-004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:22d33701fa88dff2ecadb388238a12c7a5d63c6de2dae120ae98ffacca109683
+size 17179936544

pytorch_model-00008-TP-005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:abdaa52c585fe182cd82918bde1ec3fa2b12033a1c9c11cebd753a2b3db88355
+size 17179936544

pytorch_model-00008-TP-006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f50b318ba532e0032bd6f25b59b54852029e6a026ef19852baa7851b6d5d7004
+size 17179936544

pytorch_model-00008-TP-007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7846ffc97ad222ca88bb38771a97409e8a80b62ce3573c096378a76f97c54578
+size 17179936544

pytorch_model-00009-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cebb4ddcdb6520cb50dabdb0a5568e83de081dbebb00208e2a44afd82fd3c352
+size 1073749240

pytorch_model-00010-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db9376f06a12092da29d028c2ebfe1206b1c85c50513d5138e7442f02ede635c
+size 8589942120

pytorch_model-00011-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1306ca738a956652a3d6bea2df5f3899d0c42dd96f22d75c62f72792f236d7a7
+size 8589942120

pytorch_model-00012-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88ca01f2876be5a7b268f0cd068a8e49bc9daae48426322bb420eacfcf308703
+size 1073749240

pytorch_model-00013-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:924822ee882d55ac56938342ca435bcd2c4e71a7116dba90e6301845ab19f509
+size 1055096

pytorch_model-00014-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:658fddf97d85edcd527861c7785d9d4a7801d09799819f9de85aa7b2d29ecf9e
+size 1055160

pytorch_model-00015-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc94f4dff86cad956d7eae017ff7219d55e2d902d45378fb5502729020376c30
+size 1055032

pytorch_model-00016-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63b443a0a63169c76e6342304583351a50a571ce45070f32650ec0da0f8de08b
+size 1055096

pytorch_model-00017-TP-common.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49021a8d7c657eedb8bb83de222fe9d8015cb1db2388ee2e02ad017b1d289f32
+size 8395888