DominuZ commited on
Commit
bca3cd8
·
verified ·
1 Parent(s): 4c94989

Card: SEO (H1 com keywords, model-index, Ollama primeiro, tabela de arquivos, navegacao do projeto)

Browse files
Files changed (1) hide show
  1. README.md +115 -34
README.md CHANGED
@@ -5,30 +5,105 @@ language:
5
  - pt
6
  base_model: Qwen/Qwen3-0.6B-Base
7
  pipeline_tag: text-generation
 
 
 
8
  tags:
9
  - jq
10
  - json
 
 
11
  - code-generation
12
  - text-to-code
13
  - gguf
14
  - qwen3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  ---
16
 
17
- # jq-coder-0.6B
18
 
19
- **Translates natural-language requests (English + Brazilian Portuguese) into executable
20
- `jq` filters 100% offline.** For the JSON you can't send to a cloud API: production
21
- payloads, logs with PII, anything under NDA.
22
 
23
- Every existing NL→jq tool (llm-jq, gpt-jq, jiq) ships your JSON to a hosted LLM. This
24
- is a 0.6B full fine-tune of Qwen3-0.6B-Base that runs on your own machine — CPU-only is
25
- fine and answers in a fraction of a second. To our knowledge it is the first dedicated
26
- NL→jq model published on Hugging Face, and it ships with the first benchmark of the
27
- niche, [**jq-bench**](https://huggingface.co/datasets/DominuZ/jq-bench).
 
28
 
29
- ## Quickstart
 
 
 
30
 
31
- With [llama.cpp](https://github.com/ggml-org/llama.cpp):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
 
33
  ```bash
34
  llama-server -m jq-coder-v14-release-Q8_0.gguf --port 8091 -ngl 99
@@ -46,13 +121,19 @@ curl -s http://127.0.0.1:8091/v1/chat/completions -d '{
46
  }'
47
  ```
48
 
49
- Or with [Ollama](https://ollama.com), straight from this repo:
50
 
51
- ```bash
52
- ollama run hf.co/DominuZ/jq-coder-0.6B:Q8_0
53
- ```
54
 
55
- Or use the [**Standalone CLI Tool + Code**](https://github.com/EdelmarSchneider/jq-coder-cli)
 
 
 
 
 
 
 
 
56
 
57
  ## Prompt format
58
 
@@ -86,6 +167,10 @@ the generated filter with real jq. Input JSON:
86
  The model conditions on the JSON sample, so the same request over a differently shaped
87
  document correctly yields a different filter.
88
 
 
 
 
 
89
  ## How it was trained
90
 
91
  **Reverse generation with ground truth by execution** — the training data was never
@@ -107,15 +192,13 @@ written by an LLM guessing jq:
107
  Result: **36,879 verified pairs**, bilingual EN/PT-BR 50/50, full fine-tune (not LoRA)
108
  of Qwen3-0.6B-Base.
109
 
110
- ## Evaluation — jq-bench
111
 
112
- Both metrics are computed **by execution**: the generated filter runs against held-out
113
  JSON documents and the canonical output (`jq -cS`) is diffed against gold. No
114
  LLM-as-judge anywhere.
115
 
116
- The human slice of **jq-bench** is 30 real StackOverflow tasks, hand-curated,
117
- independent of the training grammar, never seen in training (CC BY-SA 4.0, per-item
118
- attribution).
119
 
120
  | Artifact (v14) | human slice, strict | human slice, task-solved¹ | in-distribution (400) |
121
  |---|---|---|---|
@@ -125,22 +208,20 @@ attribution).
125
  ¹ *strict* = byte-identical to gold after canonicalization; *task-solved* additionally
126
  accepts equivalent output shapes (stream vs. array wrapper, etc.).
127
 
 
 
 
 
 
 
 
 
 
 
128
  Two internal diagnostics of ~200 realistic probes each (execution-verified, authored
129
  independently of the training pipeline) track progress across data iterations: the v14
130
  iteration scores **63.7%** on the older set (v13: 52.5%; v12: 31.4%) and **50.0%** on a
131
- fresh, fully independent set (v13: 32.0%). The v14 data iteration targeted missing
132
- *constructs* — `any`/`all`, filtered aggregations, `if-then-else` derived fields,
133
- mid-level element paths (`.features[].properties.name`), string splitting — after the
134
- v13 iteration had fixed the dominant envelope/root-shape failures. The human slice
135
- remains the canonical metric.
136
-
137
- ## Quantization
138
-
139
- **Q8_0 matches f16 on every metric** and is the recommended build (~640 MB).
140
-
141
- Q4_K_M was measured and **rejected**: it preserves in-distribution accuracy but breaks
142
- exactly the hard compositions. Small models do not survive aggressive quantization
143
- unharmed — we measured instead of assuming, and we don't publish what failed the gate.
144
 
145
  ## Honest limitations
146
 
 
5
  - pt
6
  base_model: Qwen/Qwen3-0.6B-Base
7
  pipeline_tag: text-generation
8
+ library_name: gguf
9
+ inference: false
10
+ quantized_by: Edelmar Schneider
11
  tags:
12
  - jq
13
  - json
14
+ - text-to-jq
15
+ - natural-language-to-code
16
  - code-generation
17
  - text-to-code
18
  - gguf
19
  - qwen3
20
+ - ollama
21
+ - llama.cpp
22
+ - offline
23
+ - privacy
24
+ - structured-data
25
+ - portuguese
26
+ metrics:
27
+ - pass@1
28
+ model-index:
29
+ - name: jq-coder-0.6B
30
+ results:
31
+ - task:
32
+ type: text-generation
33
+ name: Natural language to jq
34
+ dataset:
35
+ type: DominuZ/jq-bench
36
+ name: jq-bench (human slice, 30 real StackOverflow tasks)
37
+ split: test
38
+ metrics:
39
+ - type: pass@1
40
+ name: pass@1 strict (execution-verified)
41
+ value: 0.333
42
+ verified: false
43
+ - type: pass@1
44
+ name: pass@1 task-solved (execution-verified)
45
+ value: 0.367
46
+ verified: false
47
+ - task:
48
+ type: text-generation
49
+ name: Natural language to jq
50
+ dataset:
51
+ type: gauthierpiarrette/nl2jq-bench
52
+ name: nl2jq-bench v1.0.0 (external, frozen, 400 items)
53
+ split: test
54
+ metrics:
55
+ - type: pass@1
56
+ name: pass@1 (execution-verified, one-shot)
57
+ value: 0.305
58
+ verified: false
59
  ---
60
 
61
+ # jq-coder-0.6B — Natural Language to jq, 100% Offline (GGUF)
62
 
63
+ *Part of the **jq-coder** project [all artifacts](https://huggingface.co/collections/DominuZ/jq-coder-natural-language-to-jq-offline-6a582d4af58d35f838b45d80) ·
64
+ [`jqc` CLI (prebuilt binaries)](https://github.com/EdelmarSchneider/jq-coder-cli) ·
65
+ [jq-bench benchmark](https://huggingface.co/datasets/DominuZ/jq-bench)*
66
 
67
+ **Convert natural language to jq filters without your JSON ever leaving your machine.**
68
+ jq-coder is a 0.6B full fine-tune of
69
+ Qwen3-0.6B-Base that turns a plain-language request plus a JSON sample into an
70
+ executable [`jq`](https://jqlang.org/) filter, on CPU, in a fraction of a second. An
71
+ offline jq filter generator, built for the JSON you can't send to a cloud API:
72
+ production payloads, logs with PII, anything under NDA.
73
 
74
+ To our knowledge it is the **first NL→jq model with GGUF builds on Hugging Face**. It
75
+ ships with [**jq-bench**](https://huggingface.co/datasets/DominuZ/jq-bench), an
76
+ execution-verified benchmark built from real StackOverflow questions — and, as a bonus,
77
+ it also understands requests in Brazilian Portuguese (the only NL→jq model that does).
78
 
79
+ ## Model overview
80
+
81
+ | | |
82
+ |---|---|
83
+ | Parameters | 0.6B (full fine-tune of Qwen3-0.6B-Base) |
84
+ | Recommended file | `jq-coder-v14-release-Q8_0.gguf` (~640 MB) |
85
+ | Runs on | CPU-only is fine; any llama.cpp-compatible runtime |
86
+ | Context | 4k used in practice (request + JSON sample) |
87
+ | Languages | English + Brazilian Portuguese |
88
+ | Output | one executable jq filter, nothing else |
89
+
90
+ ## Quickstart — 10 seconds with Ollama
91
+
92
+ ```bash
93
+ ollama run hf.co/DominuZ/jq-coder-0.6B:Q8_0
94
+ ```
95
+
96
+ ## Or the `jqc` CLI — one binary, batteries included
97
+
98
+ Prebuilt binaries for Windows, Linux and macOS (Apple Silicon) on the
99
+ [Releases page](https://github.com/EdelmarSchneider/jq-coder-cli/releases). It embeds
100
+ llama.cpp and runs the generated filter for you; the model downloads on first use.
101
+
102
+ ```bash
103
+ jqc "get the id of every order" orders.json
104
+ ```
105
+
106
+ ## Or llama.cpp / LM Studio
107
 
108
  ```bash
109
  llama-server -m jq-coder-v14-release-Q8_0.gguf --port 8091 -ngl 99
 
121
  }'
122
  ```
123
 
124
+ In [LM Studio](https://lmstudio.ai/), open this repo via "Use this model".
125
 
126
+ ## Files
 
 
127
 
128
+ | File | Size | Verdict |
129
+ |---|---|---|
130
+ | [jq-coder-v14-release-Q8_0.gguf](https://huggingface.co/DominuZ/jq-coder-0.6B/resolve/main/jq-coder-v14-release-Q8_0.gguf) | ~640 MB | **Recommended** — matches f16 on every metric (measured, not assumed) |
131
+ | [jq-coder-v14-release-f16.gguf](https://huggingface.co/DominuZ/jq-coder-0.6B/resolve/main/jq-coder-v14-release-f16.gguf) | ~1.2 GB | Reference precision |
132
+ | jq-coder-v13-release-{Q8_0,f16}.gguf | | Legacy (previous release, kept for pinned revisions) |
133
+
134
+ Q4_K_M was measured and **rejected**: it preserves in-distribution accuracy but breaks
135
+ exactly the hard compositions. Small models do not survive aggressive quantization
136
+ unharmed — we measured instead of assuming, and we don't publish what failed the gate.
137
 
138
  ## Prompt format
139
 
 
167
  The model conditions on the JSON sample, so the same request over a differently shaped
168
  document correctly yields a different filter.
169
 
170
+ **Bonus — Brazilian Portuguese**: the model was trained bilingually (EN/PT-BR 50/50),
171
+ so requests like "some o total de todos os pedidos" work out of the box, as shown above.
172
+ English remains the primary interface and documentation language.
173
+
174
  ## How it was trained
175
 
176
  **Reverse generation with ground truth by execution** — the training data was never
 
192
  Result: **36,879 verified pairs**, bilingual EN/PT-BR 50/50, full fine-tune (not LoRA)
193
  of Qwen3-0.6B-Base.
194
 
195
+ ## Evaluation
196
 
197
+ All metrics are computed **by execution**: the generated filter runs against held-out
198
  JSON documents and the canonical output (`jq -cS`) is diffed against gold. No
199
  LLM-as-judge anywhere.
200
 
201
+ ### jq-bench (ours — human slice of 30 real StackOverflow tasks)
 
 
202
 
203
  | Artifact (v14) | human slice, strict | human slice, task-solved¹ | in-distribution (400) |
204
  |---|---|---|---|
 
208
  ¹ *strict* = byte-identical to gold after canonicalization; *task-solved* additionally
209
  accepts equivalent output shapes (stream vs. array wrapper, etc.).
210
 
211
+ ### nl2jq-bench (external, frozen, 400 items)
212
+
213
+ On [nl2jq-bench v1.0.0](https://huggingface.co/datasets/gauthierpiarrette/nl2jq-bench)
214
+ — an independent frozen benchmark by another author, evaluated one-shot with greedy
215
+ decoding per its protocol — v14 scores **pass@1 0.305, valid@1 0.778** (tiers: T1 0.53
216
+ · T2 0.38 · T3 0.28 · T4 0.24 · T5 0.083). The T5 gap is honest and diagnosed: that
217
+ tier is built from constructs (`reduce`, `foreach`, recursive `..`, update-assignment)
218
+ that our v14 training grammar does not cover yet — they are the target of the next data
219
+ iteration.
220
+
221
  Two internal diagnostics of ~200 realistic probes each (execution-verified, authored
222
  independently of the training pipeline) track progress across data iterations: the v14
223
  iteration scores **63.7%** on the older set (v13: 52.5%; v12: 31.4%) and **50.0%** on a
224
+ fresh, fully independent set (v13: 32.0%). The human slice remains the canonical metric.
 
 
 
 
 
 
 
 
 
 
 
 
225
 
226
  ## Honest limitations
227