Instructions to use mlx-community/Inkling-mlx-2bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mlx-community/Inkling-mlx-2bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Inkling-mlx-2bit")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use mlx-community/Inkling-mlx-2bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/Inkling-mlx-2bit"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mlx-community/Inkling-mlx-2bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use mlx-community/Inkling-mlx-2bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/Inkling-mlx-2bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mlx-community/Inkling-mlx-2bit

Run Hermes

hermes

OpenClaw new

How to use mlx-community/Inkling-mlx-2bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/Inkling-mlx-2bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mlx-community/Inkling-mlx-2bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MLX LM

How to use mlx-community/Inkling-mlx-2bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "mlx-community/Inkling-mlx-2bit"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "mlx-community/Inkling-mlx-2bit"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "mlx-community/Inkling-mlx-2bit",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

huckiyang commited on about 9 hours ago

Commit

f0928e6

verified ·

1 Parent(s): eee7a92

Add files using upload-large-folder tool

Browse files

Files changed (23) hide show

.gitattributes +1 -0
model-00002-of-00065.safetensors +3 -0
model-00005-of-00065.safetensors +3 -0
model-00006-of-00065.safetensors +3 -0
model-00007-of-00065.safetensors +3 -0
model-00024-of-00065.safetensors +3 -0
model-00025-of-00065.safetensors +3 -0
model-00027-of-00065.safetensors +3 -0
model-00029-of-00065.safetensors +3 -0
model-00030-of-00065.safetensors +3 -0
model-00032-of-00065.safetensors +3 -0
model-00033-of-00065.safetensors +3 -0
model-00036-of-00065.safetensors +3 -0
model-00039-of-00065.safetensors +3 -0
model-00044-of-00065.safetensors +3 -0
model-00046-of-00065.safetensors +3 -0
model-00047-of-00065.safetensors +3 -0
model-00048-of-00065.safetensors +3 -0
model-00050-of-00065.safetensors +3 -0
model-00052-of-00065.safetensors +3 -0
model-00055-of-00065.safetensors +3 -0
model-00065-of-00065.safetensors +3 -0
tokenizer.json +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

model-00002-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3e5ee6e5e218822828a16acf5f8b7fc16c272c60a3d0618026d0413766041c20
+size 5011345331

model-00005-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3cd50f02f96d5c09925ce5f911e3904e01c84474554ae034feb72c3f7319babb
+size 5023928236

model-00006-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:122fad7d6c089b0515eb6a0dbd4a10fc04ea6fb35aecc5036a4368508a8d0c8b
+size 5023928248

model-00007-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bb1b8daa8c715ed5aed9538a7b067a425e070384c3a81651b4fea8a575e6d529
+size 5023928242

model-00024-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e9be1449947eef2c43a8851fb717f37a92a6abc7590f7c19728348c7bde2c15e
+size 5023928238

model-00025-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:25d2ec252dbf9003093ccef29fc12e6c0969ae8da938f9dac837eb4a29c6c5f3
+size 5023928230

model-00027-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b75893d95f98d5da7d5da6a138cf72abba35346c8c0ac01fd33d59ad0a4d9fa3
+size 5023928252

model-00029-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2db2f1a391f36e876c31ff643852ad727f78d12b9156ce08a7d70443b8364fab
+size 5011345326

model-00030-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3390e276404bdd800848c898406af48b615e20554e3d2005330be280186302fd
+size 5023928240

model-00032-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:02cc3992918a7ba77b0d08eaa9e6a7ee7aca6fe9d5190605b7796e92eb04e7a2
+size 5023928240

model-00033-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cf2ff45b0e38d033dec948daa37d2996f092edcaba97e00262232c35a4c33173
+size 5023928226

model-00036-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a2cdc6c5e3e3be1f4ec04cda39216d8466d90a52c6c19ff795928179979acf91
+size 5011345336

model-00039-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:54da4a66e00b6ec1b6b110e034be2176884f5e567693d4984ca221a0378e989e
+size 5023928242

model-00044-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5125c520e8234d37c56c776a14a0af50cb3b20a62e9abb783e4ff78ced274a5d
+size 5011345331

model-00046-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4f4bda32682032627f37357d0dbcd98fa45da6f73a6a5003550ac8b62be828e0
+size 5023928234

model-00047-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7b82e2c2355af32c3f875f1aaffe12d26f45abeec00d0847e8a1ea7b3652ad7a
+size 5023928234

model-00048-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fa1803534881b94a7d6970d9b73b14ff771a2a9436633356cd43b108a92d7ec8
+size 5011345313

model-00050-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ac836f6068ad427b2f446de96d243497b7d20cf91978e25e9021d652b3f78a55
+size 5023928244

model-00052-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:32810f09d5e0a95e924cbac94664c98ae670cae1a8fccb7c175653a3a85c91ce
+size 5023928246

model-00055-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bb6e5379378146112f63ef37992e8993441047312da1cf1a8e7eedfe652a49f1
+size 5011345318

model-00065-of-00065.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b753c4c6765710c29b6fb88b1fa01f8cada45e08f2dfe99ea28ccb8b6a8ebd64
+size 7267580610

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9fb6333a7db8fe5da90728e741e4a3ee4ac2ae12c5dd4958cc6f31688787d3c2
+size 27875797