Text Generation
MLX
Safetensors
English
qwen3
apple-silicon
4bit
quantized
qlora
agentic
coding
reasoning
thinking
claude
conversational
4-bit precision
Instructions to use AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Add Parable banner and match the series card format
Browse files
README.md
CHANGED
|
@@ -1,59 +1,80 @@
|
|
| 1 |
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
base_model: AnkitAI/Parable-Qwen3-4B-Claude-Fable-5
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
tags:
|
| 5 |
- mlx
|
| 6 |
- apple-silicon
|
| 7 |
-
-
|
|
|
|
|
|
|
| 8 |
- agentic
|
| 9 |
- coding
|
| 10 |
-
-
|
| 11 |
-
|
| 12 |
-
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
-
# Parable-Qwen3-4B-Claude-Fable-5
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
-
Apple Silicon build of
|
| 18 |
-
a Qwen3-4B fine-tuned on execution-verified agent traces.
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
##
|
| 23 |
|
| 24 |
```bash
|
| 25 |
pip install mlx-lm
|
|
|
|
|
|
|
|
|
|
| 26 |
mlx_lm.generate --model AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit \
|
| 27 |
-
--prompt "Write a Python function that retries an HTTP
|
| 28 |
```
|
| 29 |
|
| 30 |
-
Or
|
| 31 |
|
| 32 |
```python
|
| 33 |
from mlx_lm import load, generate
|
|
|
|
| 34 |
model, tokenizer = load("AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit")
|
| 35 |
-
|
|
|
|
|
|
|
| 36 |
```
|
| 37 |
|
| 38 |
-
##
|
| 39 |
|
| 40 |
-
|
| 41 |
-
it is v3.1: LoRA on agent traces plus a replay mix, completion-only loss, two
|
| 42 |
-
seeds souped, then merged into the base at scale 0.6 to limit drift.
|
| 43 |
|
| 44 |
-
Measured on the 4B, base against tuned in one session on one harness:
|
| 45 |
|
| 46 |
| | base | v3.1 |
|
| 47 |
|---|---|---|
|
| 48 |
| HumanEval+ | 0.616 | **0.683** |
|
| 49 |
| MBPP+ | 0.603 | **0.638** |
|
| 50 |
|
| 51 |
-
Those
|
| 52 |
-
some accuracy; they are the ceiling, not a promise for this build.
|
| 53 |
|
| 54 |
## Other formats
|
| 55 |
|
| 56 |
-
|
| 57 |
-
-
|
|
|
|
|
|
|
|
|
|
| 58 |
|
| 59 |
-
Apache-2.0,
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
base_model: AnkitAI/Parable-Qwen3-4B-Claude-Fable-5
|
| 3 |
+
base_model_relation: quantized
|
| 4 |
+
datasets:
|
| 5 |
+
- Glint-Research/Fable-5-traces
|
| 6 |
+
- Roman1111111/gpt5.5-terminal
|
| 7 |
+
license: apache-2.0
|
| 8 |
+
language:
|
| 9 |
+
- en
|
| 10 |
+
pipeline_tag: text-generation
|
| 11 |
+
library_name: mlx
|
| 12 |
tags:
|
| 13 |
- mlx
|
| 14 |
- apple-silicon
|
| 15 |
+
- 4bit
|
| 16 |
+
- quantized
|
| 17 |
+
- qlora
|
| 18 |
- agentic
|
| 19 |
- coding
|
| 20 |
+
- reasoning
|
| 21 |
+
- thinking
|
| 22 |
+
- claude
|
| 23 |
+
- qwen3
|
| 24 |
---
|
| 25 |
|
| 26 |
+
# Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit
|
| 27 |
+
|
| 28 |
+
<picture>
|
| 29 |
+
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/ankit-aglawe/parable-assets/main/parable_header_dark.png">
|
| 30 |
+
<img alt="Parable" src="https://raw.githubusercontent.com/ankit-aglawe/parable-assets/main/parable_header.png">
|
| 31 |
+
</picture>
|
| 32 |
|
| 33 |
+
**Apple Silicon build of Parable-Qwen3-4B: 2.1 GB at 4.501 bits per weight, running natively on MLX with no llama.cpp in the way.**
|
|
|
|
| 34 |
|
| 35 |
+
A 4-bit MLX quantisation of [AnkitAI/Parable-Qwen3-4B-Claude-Fable-5](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5), a Qwen3-4B fine-tune trained on real multi-step agent sessions: planning, tool use, and `<think>` reasoning captured from actual Claude Fable 5 and GPT-5.5 agent work, not synthetic Q&A. Fits comfortably on any M-series Mac.
|
| 36 |
|
| 37 |
+
## Usage
|
| 38 |
|
| 39 |
```bash
|
| 40 |
pip install mlx-lm
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
```bash
|
| 44 |
mlx_lm.generate --model AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit \
|
| 45 |
+
--prompt "Write a Python function that retries an HTTP request with exponential backoff."
|
| 46 |
```
|
| 47 |
|
| 48 |
+
Or from Python:
|
| 49 |
|
| 50 |
```python
|
| 51 |
from mlx_lm import load, generate
|
| 52 |
+
|
| 53 |
model, tokenizer = load("AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit")
|
| 54 |
+
messages = [{"role": "user", "content": "Write a Python function that retries an HTTP request with exponential backoff."}]
|
| 55 |
+
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
|
| 56 |
+
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))
|
| 57 |
```
|
| 58 |
|
| 59 |
+
## Recipe
|
| 60 |
|
| 61 |
+
v3.1: LoRA on agent traces with a replay mix to limit forgetting, completion-only loss so the model trains on answers rather than prompts, two seeds souped, then merged into the base at scale 0.6 to bound drift from the original weights.
|
|
|
|
|
|
|
| 62 |
|
| 63 |
+
Measured on the full-precision 4B, base against tuned, in one session on one harness:
|
| 64 |
|
| 65 |
| | base | v3.1 |
|
| 66 |
|---|---|---|
|
| 67 |
| HumanEval+ | 0.616 | **0.683** |
|
| 68 |
| MBPP+ | 0.603 | **0.638** |
|
| 69 |
|
| 70 |
+
Those are the full-precision numbers. Quantising to 4 bits costs accuracy that this table does not measure, so treat them as the ceiling for this build rather than a claim about it.
|
|
|
|
| 71 |
|
| 72 |
## Other formats
|
| 73 |
|
| 74 |
+
| format | repo | for |
|
| 75 |
+
|---|---|---|
|
| 76 |
+
| GGUF | [Parable-Qwen3-4B-Claude-Fable-5-GGUF](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-GGUF) | llama.cpp, LM Studio, Ollama |
|
| 77 |
+
| MLX 8-bit | [Parable-Qwen3-4B-Claude-Fable-5-MLX-8bit](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-8bit) | Apple Silicon, closer to source |
|
| 78 |
+
| safetensors | [Parable-Qwen3-4B-Claude-Fable-5](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5) | transformers |
|
| 79 |
|
| 80 |
+
Apache-2.0, inherited from the base model.
|