malgamves commited on
Commit
f15971e
·
verified ·
1 Parent(s): 326914e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -23
README.md CHANGED
@@ -17,15 +17,15 @@ tags:
17
 
18
  # Peripheral (Qwen3-8B)
19
 
20
- Peripheral is a fine-tuned 8B **knowledge-management agent (KMA)** that manages filesystem-based knowledge bases. Given a query it routes to the right files, decides whether a recent change is safe to serve, and classifies what that change is. It runs locally, costs nothing per call, and beats frontier models on the knowledge-management-specific tasks while being roughly 5x faster and 5x cheaper.
21
 
22
  This is the model behind the write-up *[Not all Context is Knowledge](https://madalitso.me/notes/not-all-context-is-knowledge)*.
23
 
24
- > **The honest framing:** this is a *specialist*, not a better frontier model. It was trained on ~7,000 task examples and beats a zero-shot generalist on its own turf. It transfers partially to unseen domains and gets weaker the further you go from what it saw. See "Limitations" before you deploy it.
25
 
26
  ## What it does
27
 
28
- It is the judgement layer on top of a markdown knowledge base. It performs three read-path tasks, each selected by a tag in the system prompt:
29
 
30
  | Task | Tag | Question it answers | Output keys |
31
  |------|-----|---------------------|-------------|
@@ -33,13 +33,11 @@ It is the judgement layer on top of a markdown knowledge base. It performs three
33
  | Routing | `[ROUTE]` | Which files answer this query? | `selected_files`, `reasoning`, `confidence` |
34
  | Gate | `[GATE]` | Serve, annotate, or block this content? | `gate_decision` (`serve`/`annotate`/`block`), `reasoning`, `risk_level`, `confidence` |
35
 
36
- Output is always a single JSON object.
37
-
38
- A fourth write-path task, `[ORGANIZE]` (deciding where new knowledge belongs), is scaffolded in the repo but **not trained in this model** — don't rely on it.
39
 
40
  ## Results (KMA-Bench, 226 cases, % correct)
41
 
42
- Averaged over multiple runs across three knowledge bases (French wiki, ClickHouse docs, PostHog handbook; 14 cases from real git commits).
43
 
44
  | Model | Diff Eval | Routing | Gate | Overall | Latency |
45
  |-------|-----------|---------|------|---------|---------|
@@ -48,34 +46,34 @@ Averaged over multiple runs across three knowledge bases (French wiki, ClickHous
48
  | **Peripheral (this model)** | **71%** | **88%** | **96%** | **81%** | **~760ms** |
49
  | Claude Sonnet 4.6 (zero-shot) | 60% | 96% | 83% | 76% | ~4,100ms |
50
 
51
- \* base model can't reliably emit valid JSON; fine-tuning carried it from 30% to 81%.
52
 
53
- A smaller 10-case quick benchmark is in the repo README. Numbers there differ slightly because the sample is different.
54
 
55
- **Data note.** ClickHouse and PostHog knowledge bases were used only for read-only generalization *testing* the released model was trained on French content only and contains none of theirs. ClickHouse documentation © ClickHouse, used under CC BY-NC-SA 4.0; the PostHog handbook © PostHog Inc., used under MIT.
56
 
57
  ## How to use
58
 
59
  ### LM Studio (recommended)
60
 
61
- 1. In LM Studio, search `malgamves/peripheral-8b` and download the `Q4_K_M` GGUF (runs in ~4.8GB of VRAM).
62
  2. Load the model.
63
- 3. Give the model the task instruction for what you want `[EVAL]`, `[ROUTE]`, or `[GATE]` (full text under "Prompt format" below) followed by your structured input. The `[TAG]` is what selects the task; you can put the instruction in LM Studio's System Prompt field or at the top of your message.
64
  4. Send it. You get a single JSON object back.
65
 
66
- Keep temperature low (~0.1); the model is trained to emit short, deterministic JSON.
67
 
68
  ### llama.cpp
69
 
70
- Download the `Q4_K_M` GGUF and run it with your usual llama.cpp setup. Same prompt format as above.
71
 
72
  ### Ollama (optional)
73
 
74
- A `Modelfile` is in the repo if you prefer Ollama: `ollama create` it (the `[EVAL]` prompt is baked in as the default system message) and `ollama run`. LM Studio is the recommended path.
75
 
76
  ## Prompt format
77
 
78
- Each task is a tagged instruction plus a structured input. The `[TAG]` is the operative signal that selects the task; whether you place the instruction in the system field or inline with your input, keep the format close to the examples below the model is sensitive to it.
79
 
80
  **System prompts**
81
 
@@ -128,19 +126,19 @@ Evaluate this change: accept, reject, or partial.
128
  | Epochs | 3 |
129
  | Export | Q4_K_M GGUF |
130
 
131
- Training data was generated locally at zero API cost. The data-gen and fine-tune-prep scripts are in the [repo](https://github.com/malgamves/peripheral). Both the `Q4_K_M` GGUF and the LoRA adapter are published here.
132
 
133
  ## Limitations
134
 
135
- - **Specialist, not generalist.** Trained on French-grammar knowledge management. It transfers to other domains but accuracy drops with distance: diff eval is 78% on French (trained), 70% on ClickHouse docs, 68% on PostHog handbook.
136
- - **Routing degrades at scale.** 95% at ~20 files, ~80% at ~86 files. More candidate files means more confusion.
137
- - **Quality gate, not a truth gate.** It catches corruption, bad grammar, and structural problems. It does **not** catch fabricated-but-plausible content (no traceable source). That needs external verification.
138
- - **Format-sensitive.** Off-format prompts degrade output. Use the prompts above.
139
- - **Evaluation bias.** Test labels and training labels share one annotator; inter-annotator agreement is not yet measured. Treat the headline numbers as directional.
140
 
141
  ## License
142
 
143
- This GGUF is a merged fine-tune of Qwen3-8B and is released under **Apache-2.0** (Qwen3's license), with attribution to Qwen. The training and inference code is **MIT**.
144
 
145
  ## Citation
146
 
 
17
 
18
  # Peripheral (Qwen3-8B)
19
 
20
+ Peripheral is a fine-tuned 8B **knowledge-management agent (KMA)** that manages filesystem-based knowledge bases. Given a query it routes to the right files, decides whether a recent change is safe to serve, and classifies what that change actually is. It runs locally at zero cost per call, and on the knowledge-management tasks it was built for it beats frontier models while being roughly five times faster and five times cheaper.
21
 
22
  This is the model behind the write-up *[Not all Context is Knowledge](https://madalitso.me/notes/not-all-context-is-knowledge)*.
23
 
24
+ > Peripheral is a specialist rather than a better frontier model. It was trained on about 7,000 task examples and beats a zero-shot generalist on its own turf, but it transfers only partially to unseen domains and grows less reliable the further it gets from what it saw in training. Read the Limitations before you deploy it.
25
 
26
  ## What it does
27
 
28
+ Peripheral is the judgement layer on top of a markdown knowledge base, and it performs three read-path tasks, each selected by a tag in the system prompt.
29
 
30
  | Task | Tag | Question it answers | Output keys |
31
  |------|-----|---------------------|-------------|
 
33
  | Routing | `[ROUTE]` | Which files answer this query? | `selected_files`, `reasoning`, `confidence` |
34
  | Gate | `[GATE]` | Serve, annotate, or block this content? | `gate_decision` (`serve`/`annotate`/`block`), `reasoning`, `risk_level`, `confidence` |
35
 
36
+ The output is always a single JSON object. There is a fourth write-path task, `[ORGANIZE]`, for deciding where new knowledge belongs, but it is only scaffolded in the repository and was not trained into this model, so you should not rely on it.
 
 
37
 
38
  ## Results (KMA-Bench, 226 cases, % correct)
39
 
40
+ Averaged over multiple runs across three knowledge bases (French wiki, ClickHouse docs, and the PostHog handbook, with 14 cases drawn from real git commits).
41
 
42
  | Model | Diff Eval | Routing | Gate | Overall | Latency |
43
  |-------|-----------|---------|------|---------|---------|
 
46
  | **Peripheral (this model)** | **71%** | **88%** | **96%** | **81%** | **~760ms** |
47
  | Claude Sonnet 4.6 (zero-shot) | 60% | 96% | 83% | 76% | ~4,100ms |
48
 
49
+ \* The base model cannot reliably emit valid JSON, and fine-tuning is what carried it from 30% to 81%.
50
 
51
+ A smaller 10-case quick benchmark is in the repo README; its numbers differ slightly because the sample is different.
52
 
53
+ **Data note.** The ClickHouse and PostHog knowledge bases were used only for read-only generalization testing, and the released model was trained on French content alone, so it contains none of their text. The ClickHouse documentation is © ClickHouse, used under CC BY-NC-SA 4.0, and the PostHog handbook is © PostHog Inc., used under MIT.
54
 
55
  ## How to use
56
 
57
  ### LM Studio (recommended)
58
 
59
+ 1. In LM Studio, search `malgamves/peripheral-8b` and download the `Q4_K_M` GGUF (it runs in about 4.8GB of VRAM).
60
  2. Load the model.
61
+ 3. Give the model the task instruction for what you want, one of `[EVAL]`, `[ROUTE]`, or `[GATE]` (the full text is under "Prompt format" below), followed by your structured input. The `[TAG]` is what selects the task, and you can put the instruction in LM Studio's System Prompt field or at the top of your message.
62
  4. Send it. You get a single JSON object back.
63
 
64
+ Keep the temperature low, around 0.1, since the model is trained to emit short, deterministic JSON.
65
 
66
  ### llama.cpp
67
 
68
+ Download the `Q4_K_M` GGUF and run it with your usual llama.cpp setup, using the same prompt format as above.
69
 
70
  ### Ollama (optional)
71
 
72
+ A `Modelfile` is in the repository if you prefer Ollama: `ollama create` it, with the `[EVAL]` prompt baked in as the default system message, and then `ollama run`. LM Studio is still the recommended path.
73
 
74
  ## Prompt format
75
 
76
+ Each task is a tagged instruction followed by a structured input. The `[TAG]` is the operative signal that selects the task, and whether you place the instruction in the system field or inline with your input, you should keep the format close to the examples below, because the model is sensitive to it.
77
 
78
  **System prompts**
79
 
 
126
  | Epochs | 3 |
127
  | Export | Q4_K_M GGUF |
128
 
129
+ The training data was generated locally at zero API cost, and the data-generation and fine-tune-prep scripts are in the [repository](https://github.com/malgamves/peripheral). Both the `Q4_K_M` GGUF and the LoRA adapter are published here.
130
 
131
  ## Limitations
132
 
133
+ - **It is a specialist, not a generalist.** Peripheral was trained on French-grammar knowledge management, and while it transfers to other domains, accuracy falls off with distance: diff evaluation drops from 78% on the French content it was trained on to 70% on ClickHouse docs and 68% on the PostHog handbook.
134
+ - **Routing degrades as the knowledge base grows.** It routes correctly about 95% of the time at roughly 20 files but slips to around 80% at 86 files, because more candidate files give the model more room to confuse itself.
135
+ - **It is a quality gate, not a truth gate.** It reliably catches corruption, broken grammar, and structural problems, but it does not catch content that is fabricated yet plausible and has no traceable source, which is a problem that needs external verification rather than diff analysis.
136
+ - **It is sensitive to prompt format.** Prompts that stray from the formats above degrade the output, so it is worth matching them closely.
137
+ - **The evaluation carries the bias of a single annotator.** The test labels and the training labels come from the same person, inter-annotator agreement has not been measured, and the headline numbers are best read as directional rather than definitive.
138
 
139
  ## License
140
 
141
+ This GGUF is a merged fine-tune of Qwen3-8B and is released under **Apache-2.0**, Qwen3's own license, with attribution to Qwen. The training and inference code is **MIT**.
142
 
143
  ## Citation
144