AnkitAI commited on
Commit
7861455
·
verified ·
1 Parent(s): 856dea4

Add Parable banner and match the series card format

Browse files
Files changed (1) hide show
  1. README.md +44 -23
README.md CHANGED
@@ -1,59 +1,80 @@
1
  ---
2
- license: apache-2.0
3
  base_model: AnkitAI/Parable-Qwen3-4B-Claude-Fable-5
 
 
 
 
 
 
 
 
 
4
  tags:
5
  - mlx
6
  - apple-silicon
7
- - qwen3
 
 
8
  - agentic
9
  - coding
10
- - lora
11
- library_name: mlx
12
- pipeline_tag: text-generation
 
13
  ---
14
 
15
- # Parable-Qwen3-4B-Claude-Fable-5MLX 4-bit
 
 
 
 
 
16
 
17
- Apple Silicon build of [Parable-Qwen3-4B-Claude-Fable-5](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5),
18
- a Qwen3-4B fine-tuned on execution-verified agent traces.
19
 
20
- **2.1 GB, 4.501 bits per weight.** Runs on any M-series Mac with room to spare.
21
 
22
- ## Use it
23
 
24
  ```bash
25
  pip install mlx-lm
 
 
 
26
  mlx_lm.generate --model AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit \
27
- --prompt "Write a Python function that retries an HTTP call with backoff."
28
  ```
29
 
30
- Or in Python:
31
 
32
  ```python
33
  from mlx_lm import load, generate
 
34
  model, tokenizer = load("AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit")
35
- print(generate(model, tokenizer, prompt="...", max_tokens=512))
 
 
36
  ```
37
 
38
- ## What it is
39
 
40
- Same weights as the source model, quantised to 4-bit for MLX. The recipe behind
41
- it is v3.1: LoRA on agent traces plus a replay mix, completion-only loss, two
42
- seeds souped, then merged into the base at scale 0.6 to limit drift.
43
 
44
- Measured on the 4B, base against tuned in one session on one harness:
45
 
46
  | | base | v3.1 |
47
  |---|---|---|
48
  | HumanEval+ | 0.616 | **0.683** |
49
  | MBPP+ | 0.603 | **0.638** |
50
 
51
- Those numbers are from the full-precision model. Quantisation to 4 bits costs
52
- some accuracy; they are the ceiling, not a promise for this build.
53
 
54
  ## Other formats
55
 
56
- - [GGUF](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-GGUF) llama.cpp, LM Studio, Ollama
57
- - [safetensors](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5) — transformers
 
 
 
58
 
59
- Apache-2.0, same as the base.
 
1
  ---
 
2
  base_model: AnkitAI/Parable-Qwen3-4B-Claude-Fable-5
3
+ base_model_relation: quantized
4
+ datasets:
5
+ - Glint-Research/Fable-5-traces
6
+ - Roman1111111/gpt5.5-terminal
7
+ license: apache-2.0
8
+ language:
9
+ - en
10
+ pipeline_tag: text-generation
11
+ library_name: mlx
12
  tags:
13
  - mlx
14
  - apple-silicon
15
+ - 4bit
16
+ - quantized
17
+ - qlora
18
  - agentic
19
  - coding
20
+ - reasoning
21
+ - thinking
22
+ - claude
23
+ - qwen3
24
  ---
25
 
26
+ # Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit
27
+
28
+ <picture>
29
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/ankit-aglawe/parable-assets/main/parable_header_dark.png">
30
+ <img alt="Parable" src="https://raw.githubusercontent.com/ankit-aglawe/parable-assets/main/parable_header.png">
31
+ </picture>
32
 
33
+ **Apple Silicon build of Parable-Qwen3-4B: 2.1 GB at 4.501 bits per weight, running natively on MLX with no llama.cpp in the way.**
 
34
 
35
+ A 4-bit MLX quantisation of [AnkitAI/Parable-Qwen3-4B-Claude-Fable-5](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5), a Qwen3-4B fine-tune trained on real multi-step agent sessions: planning, tool use, and `<think>` reasoning captured from actual Claude Fable 5 and GPT-5.5 agent work, not synthetic Q&A. Fits comfortably on any M-series Mac.
36
 
37
+ ## Usage
38
 
39
  ```bash
40
  pip install mlx-lm
41
+ ```
42
+
43
+ ```bash
44
  mlx_lm.generate --model AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit \
45
+ --prompt "Write a Python function that retries an HTTP request with exponential backoff."
46
  ```
47
 
48
+ Or from Python:
49
 
50
  ```python
51
  from mlx_lm import load, generate
52
+
53
  model, tokenizer = load("AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-4bit")
54
+ messages = [{"role": "user", "content": "Write a Python function that retries an HTTP request with exponential backoff."}]
55
+ prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
56
+ print(generate(model, tokenizer, prompt=prompt, max_tokens=512))
57
  ```
58
 
59
+ ## Recipe
60
 
61
+ v3.1: LoRA on agent traces with a replay mix to limit forgetting, completion-only loss so the model trains on answers rather than prompts, two seeds souped, then merged into the base at scale 0.6 to bound drift from the original weights.
 
 
62
 
63
+ Measured on the full-precision 4B, base against tuned, in one session on one harness:
64
 
65
  | | base | v3.1 |
66
  |---|---|---|
67
  | HumanEval+ | 0.616 | **0.683** |
68
  | MBPP+ | 0.603 | **0.638** |
69
 
70
+ Those are the full-precision numbers. Quantising to 4 bits costs accuracy that this table does not measure, so treat them as the ceiling for this build rather than a claim about it.
 
71
 
72
  ## Other formats
73
 
74
+ | format | repo | for |
75
+ |---|---|---|
76
+ | GGUF | [Parable-Qwen3-4B-Claude-Fable-5-GGUF](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-GGUF) | llama.cpp, LM Studio, Ollama |
77
+ | MLX 8-bit | [Parable-Qwen3-4B-Claude-Fable-5-MLX-8bit](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5-MLX-8bit) | Apple Silicon, closer to source |
78
+ | safetensors | [Parable-Qwen3-4B-Claude-Fable-5](https://huggingface.co/AnkitAI/Parable-Qwen3-4B-Claude-Fable-5) | transformers |
79
 
80
+ Apache-2.0, inherited from the base model.