Instructions to use mlx-community/Kimi-K2.7-Code-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mlx-community/Kimi-K2.7-Code-4bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Kimi-K2.7-Code-4bit")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use mlx-community/Kimi-K2.7-Code-4bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/Kimi-K2.7-Code-4bit"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mlx-community/Kimi-K2.7-Code-4bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use mlx-community/Kimi-K2.7-Code-4bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/Kimi-K2.7-Code-4bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mlx-community/Kimi-K2.7-Code-4bit

Run Hermes

hermes

OpenClaw new

How to use mlx-community/Kimi-K2.7-Code-4bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/Kimi-K2.7-Code-4bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mlx-community/Kimi-K2.7-Code-4bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MLX LM

How to use mlx-community/Kimi-K2.7-Code-4bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "mlx-community/Kimi-K2.7-Code-4bit"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "mlx-community/Kimi-K2.7-Code-4bit"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "mlx-community/Kimi-K2.7-Code-4bit",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

machiabeli commited on about 1 month ago

Commit

10df575

verified ·

1 Parent(s): 74dfecb

Add files using upload-large-folder tool

Browse files

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

model-00003-of-00180.safetensors +3 -0
model-00006-of-00180.safetensors +3 -0
model-00009-of-00180.safetensors +3 -0
model-00011-of-00180.safetensors +3 -0
model-00014-of-00180.safetensors +3 -0
model-00015-of-00180.safetensors +3 -0
model-00017-of-00180.safetensors +3 -0
model-00018-of-00180.safetensors +3 -0
model-00020-of-00180.safetensors +3 -0
model-00023-of-00180.safetensors +3 -0
model-00025-of-00180.safetensors +3 -0
model-00026-of-00180.safetensors +3 -0
model-00028-of-00180.safetensors +3 -0
model-00029-of-00180.safetensors +3 -0
model-00031-of-00180.safetensors +3 -0
model-00034-of-00180.safetensors +3 -0
model-00042-of-00180.safetensors +3 -0
model-00047-of-00180.safetensors +3 -0
model-00048-of-00180.safetensors +3 -0
model-00050-of-00180.safetensors +3 -0
model-00051-of-00180.safetensors +3 -0
model-00053-of-00180.safetensors +3 -0
model-00054-of-00180.safetensors +3 -0
model-00055-of-00180.safetensors +3 -0
model-00056-of-00180.safetensors +3 -0
model-00059-of-00180.safetensors +3 -0
model-00061-of-00180.safetensors +3 -0
model-00062-of-00180.safetensors +3 -0
model-00064-of-00180.safetensors +3 -0
model-00066-of-00180.safetensors +3 -0
model-00067-of-00180.safetensors +3 -0
model-00068-of-00180.safetensors +3 -0
model-00069-of-00180.safetensors +3 -0
model-00070-of-00180.safetensors +3 -0
model-00075-of-00180.safetensors +3 -0
model-00083-of-00180.safetensors +3 -0
model-00086-of-00180.safetensors +3 -0
model-00089-of-00180.safetensors +3 -0
model-00091-of-00180.safetensors +3 -0
model-00092-of-00180.safetensors +3 -0
model-00094-of-00180.safetensors +3 -0
model-00097-of-00180.safetensors +3 -0
model-00101-of-00180.safetensors +3 -0
model-00102-of-00180.safetensors +3 -0
model-00104-of-00180.safetensors +3 -0
model-00107-of-00180.safetensors +3 -0
model-00108-of-00180.safetensors +3 -0
model-00110-of-00180.safetensors +3 -0
model-00115-of-00180.safetensors +3 -0
model-00122-of-00180.safetensors +3 -0

model-00003-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:694e325ac18997580bc0f0dd15dd041e43cb8f9d67c4d88f72fe131e69f1fec1
+size 3610413297

model-00006-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6145708aea0875f1aea356afc3895bff14518118151b6254698ebb875263d7ac
+size 3610413289

model-00009-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:69f965a509260901cf2c954a7b7e60aac4d4a642b46c513289cc1f8d7f110ca8
+size 3610413257

model-00011-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a4748d708251db884eb7d299fe3694b7f21fb07ad57c872cbaec870afe0a6b59
+size 3523215810

model-00014-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f53b14bec6b7413c006d21801f49fcdc98a7f3f1ae54cb4659fc896fef03d48e
+size 3523215810

model-00015-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bfaffcaf91ba91cedc1fed4b6fbe250f47076bbd21b21145bea0b2139183d5f7
+size 3610413277

model-00017-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:421f8b4bf3b6d336669faaba0d0e7b023f63292e432e4f32c93c3d521c38c777
+size 3523215810

model-00018-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7f3938cf0227ac57917a487a83435a17bbbd484bfb29f003b5c40d05126676ee
+size 3610413281

model-00020-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cfa52b464b2e0180eb83227d9d3dbca3fbff13d06874ba722f86f433f9d0dc68
+size 3523215810

model-00023-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c690ae67f1146be9574476d8205dc99c4eb17fed1854aa7826ddd4bf2662e4a4
+size 3523215810

model-00025-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d7d686ec6c38be34281c61a82af280696c5bfaa2ccdfb11db6ca18c0f17d8fdd
+size 3523215816

model-00026-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:186f7363b23da26b6ddff751d65f603fadd20cd6d898cabe3cdbc9b4c9a2e0e0
+size 3523215810

model-00028-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9ae39be594530e086f236454b4d43e97b7fe2bac8267c4009e3b8c39405fe237
+size 3523215819

model-00029-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a723640e2e655b2568e6e82121c523a0bc5f15537d70dd0d7140bd0f21c64846
+size 3523215813

model-00031-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d772e76a37a077eea773a3b9bffcfcd7be981612927f265939b2a04851396d7b
+size 3523215819

model-00034-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e0b1dcb1e65f1ee177943cd087af227c51a82d40242f351cdcaf50d51b426b7c
+size 3523215819

model-00042-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f02540bcfca8037b7731c0bb44ec59a3c862d4371e9384183e356aa141ae06a0
+size 3610413299

model-00047-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ee745527683ab3f7bc8c130a4b0017b1b4bc4d6c5584504d1cff400a98edd9d0
+size 3523215813

model-00048-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2c20cf660046d2d1386f205db8f36e494a9c764d72cba2e338f8625390cef743
+size 3610413301

model-00050-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:62d97f5aa7ef1b6432fc93d9996eb3f21d338e4086b4eb472d5a6a201e495fc6
+size 3523215813

model-00051-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9f90fceae0faaae93b96c937f392f3bccbc77646897a54f356dc2249d20fc942
+size 3610413333

model-00053-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:93117d77d11439aae689fa0c0140678651d72f39367dc939b46c6c890726c792
+size 3523215813

model-00054-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9e0b526189c4e0364de7e2f6ae88c55d2cd13955fcbca14eeaaee3f7eca49840
+size 3610413319

model-00055-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:16658a4c6dcefe9468a9e47dd8b99f81c6e196775c6f18e8c177b12ab45d0337
+size 3523215819

model-00056-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6762341aad6edfc8b5f77e2035aaa1745f281ecb4abce9159e29a7289259162d
+size 3523215813

model-00059-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e12a318e3c448ff189088f0277f6a515cda41e092161db25be372b827131fb62
+size 3523215813

model-00061-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ae69b4978f5324b25a67cd4fa515957c254930b57677ac3d78f0b063048bcfb0
+size 3523215819

model-00062-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9a055aaa75e1908e6b64d2f04db8f59ca4151931a448be51d87f6bdbdc317c12
+size 3523215813

model-00064-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d7944fdb36978764ce138ac2d289bbfbf0b199ea515c2256e9803630a4060f36
+size 3523215819

model-00066-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b0487f1188d09ed0fbcd218fe6942a2ccdd6fa939654b3d30f01566766b7775a
+size 3610413269

model-00067-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:af5fcb0e95ad59376d8225b5b7862c6d2eba0d0f247a4b0e3ac926e333cf7b01
+size 3523215819

model-00068-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:433dff396c6a355816459a2e66bcdd821ab0f4b39c2028392dd919057761939b
+size 3523215813

model-00069-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0dcb8ebdd1a7b03faa7271c932a2d7484c479cb69ff7200a9509b686bc14a86b
+size 3610413327

model-00070-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8392b27c18431379151380565388b7bc0392abbf942315cf4acbd35f6848bfb3
+size 3523215819

model-00075-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6002ac4012c6a579e25c6e43876ae2194366b4fa6f9fcbf9fc4457b43cf10ad4
+size 3610413307

model-00083-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d4525ef59737cd8d3dc394c525ccc939503fa8f98e9f28460ff58bbc1b2f9d82
+size 3523215813

model-00086-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:228c8ecffdac54ac50f15bef19216d93e929958151b64107a2467eba49bba1af
+size 3523215813

model-00089-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:daa71151e98ad95f725748af47667195733760125cfce8ff229a913f442c7f83
+size 3523215813

model-00091-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bd0084d43a14c308e0990212801767dfb4a7d660092028a4d4d8f33cb929a148
+size 3523215819

model-00092-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4a055959cc9c563b8d5ab20e6391f8bdbb8b5df4379587983dadfa7855ff0df6
+size 3523215813

model-00094-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5d15460639a5f2e2f9c7cc835ed738578733cf11ff9597b2893854ac67797dad
+size 3523215819

model-00097-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:25d2d84488c72b1c7291dd9b34eb4aedfea285df677f3ed124e995f80677c6b5
+size 3523215819

model-00101-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c13a00f6daba63081814063df55107f91610762e6b5f65532b403807ee48886c
+size 3523215813

model-00102-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:589e57c81d27ac7c521b9f2287c4229ee1010339c9f4b17e4f057d3f47668f50
+size 3610413383

model-00104-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:834ea4a7b2f2815de31f74a910addea5689e77bb65a8a547bf5bac22cceb8484
+size 3523215813

model-00107-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:371dde8cedfce97c43e95ac1e332ab24d939330ac9e9fc36cc3792662488b3b4
+size 3523215813

model-00108-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bf94ac3551a7df90ab0b2547789cbeca7b184a272c9235d9c1492bb7b55f9d91
+size 3610413303

model-00110-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:052fcfd02a70cf9ab8dc661afe9c0105a3d8f633fa0d0de490a00b933ac56d6c
+size 3523215813

model-00115-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0423972ec4f3eaf514589abec394e5e7d0cb059b05eae4e961c6f7e59964b726
+size 3523215819

model-00122-of-00180.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fa98d717fc49a63268ea712d601bccef423738bd258b2fe6993cab08b69224bb
+size 3523215813