KrisSimon commited on
Commit
7bf8c00
·
verified ·
1 Parent(s): f4b758e

Release v1.1.0 — ARO Coder 6-bit (conversation_boosted, 9383b5052e5e5d88)

Browse files
.source_model CHANGED
@@ -1 +1 @@
1
- /Users/kris/Projects/ARO/ARO-Lang/Train/models/distill/student/fused
 
1
+ /Users/kris/Projects/ARO/ARO-Lang/Train/models/conversation/fused
README.md CHANGED
@@ -9,7 +9,7 @@ tags:
9
  - 6-bit
10
  - lora
11
  - fine-tuned
12
- base_model: mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit
13
  pipeline_tag: text-generation
14
  library_name: mlx
15
  ---
@@ -24,12 +24,12 @@ ARO is a domain-specific language where every statement follows the pattern:
24
  | | |
25
  |---|---|
26
  | **Version** | v1.1.0 (tag `v1.1.0`) |
27
- | **Checksum** | `84224b61540b34db` |
28
- | **Base model** | [mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit](https://huggingface.co/mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit) |
29
- | **Teacher source** | distill_student (30B MoE teacher distilled to 8B student) |
30
  | **Quantization** | 6-bit MLX, group size 32 |
31
  | **Language** | ARO |
32
- | **Training samples** | 5055 |
33
 
34
  ## Links
35
 
@@ -39,13 +39,13 @@ ARO is a domain-specific language where every statement follows the pattern:
39
  - **Language Guide (PDF)**: [Download](https://github.com/arolang/aro/releases/latest/download/ARO-Language-Guide.pdf)
40
  - **Discussions**: [GitHub Discussions](https://github.com/arolang/aro/discussions)
41
 
42
- ## Evaluation (promotion gate, 102 prompts)
43
 
44
  | Metric | Quantized (shipped) | Fused (pre-quantization) |
45
  |---|---|---|
46
  | Reply rate | 100.0% | 100.0% |
47
  | Empty-think collapse | 0.0% | 0.0% |
48
- | Syntax pass rate (`aro check`) | 61.5% | 60.4% |
49
  | Tool-name leakage | 0.0% | 0.0% |
50
  | URL contamination | 0.0% | 0.0% |
51
 
@@ -133,7 +133,7 @@ Key features:
133
 
134
  This model was trained with the ARO training pipeline:
135
 
136
- 1. **Corpus collection** — 5055 samples from Examples, Book, Wiki, Proposals, and real-world ARO applications
137
  2. **Supervised fine-tuning** — LoRA on all code generation, debugging, Q&A, and explanation tasks
138
  3. **DPO preference training** — using `aro check` validation to build chosen/rejected pairs
139
  4. **Iterative self-improvement** — multiple rounds of generate-validate-retrain
@@ -144,7 +144,7 @@ This model was trained with the ARO training pipeline:
144
 
145
  | Version | Date | Source | Checksum |
146
  |---|---|---|---|
147
- | v1.1.0 | 2026-08-03 | distill_student | `84224b61540b34db` |
148
 
149
  Every release is tagged on the Hub — load an older version with
150
  `load("ARO-Lang/aro-coder-6bit", revision="v<version>")` or report issues against
 
9
  - 6-bit
10
  - lora
11
  - fine-tuned
12
+ base_model: mlx-community/Qwen3-Coder-30B-A3B-Instruct-bf16
13
  pipeline_tag: text-generation
14
  library_name: mlx
15
  ---
 
24
  | | |
25
  |---|---|
26
  | **Version** | v1.1.0 (tag `v1.1.0`) |
27
+ | **Checksum** | `9383b5052e5e5d88` |
28
+ | **Base model** | [mlx-community/Qwen3-Coder-30B-A3B-Instruct-bf16](https://huggingface.co/mlx-community/Qwen3-Coder-30B-A3B-Instruct-bf16) |
29
+ | **Teacher source** | conversation_boosted (30B MoE teacher distilled to 8B student) |
30
  | **Quantization** | 6-bit MLX, group size 32 |
31
  | **Language** | ARO |
32
+ | **Training samples** | 5723 |
33
 
34
  ## Links
35
 
 
39
  - **Language Guide (PDF)**: [Download](https://github.com/arolang/aro/releases/latest/download/ARO-Language-Guide.pdf)
40
  - **Discussions**: [GitHub Discussions](https://github.com/arolang/aro/discussions)
41
 
42
+ ## Evaluation (promotion gate, 104 prompts)
43
 
44
  | Metric | Quantized (shipped) | Fused (pre-quantization) |
45
  |---|---|---|
46
  | Reply rate | 100.0% | 100.0% |
47
  | Empty-think collapse | 0.0% | 0.0% |
48
+ | Syntax pass rate (`aro check`) | 68.8% | 70.7% |
49
  | Tool-name leakage | 0.0% | 0.0% |
50
  | URL contamination | 0.0% | 0.0% |
51
 
 
133
 
134
  This model was trained with the ARO training pipeline:
135
 
136
+ 1. **Corpus collection** — 5723 samples from Examples, Book, Wiki, Proposals, and real-world ARO applications
137
  2. **Supervised fine-tuning** — LoRA on all code generation, debugging, Q&A, and explanation tasks
138
  3. **DPO preference training** — using `aro check` validation to build chosen/rejected pairs
139
  4. **Iterative self-improvement** — multiple rounds of generate-validate-retrain
 
144
 
145
  | Version | Date | Source | Checksum |
146
  |---|---|---|---|
147
+ | v1.1.0 | 2026-08-06 | conversation_boosted | `9383b5052e5e5d88` |
148
 
149
  Every release is tagged on the Hub — load an older version with
150
  `load("ARO-Lang/aro-coder-6bit", revision="v<version>")` or report issues against
model-00001-of-00002.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0df15d65c63498b71242567ba1c4431c6d49a6090e0753799aead7cc70f64f6c
3
  size 5352815610
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75f36a8113368f06c11aff949cd0492339264060bace2928fdab469b3ecb1031
3
  size 5352815610
model-00002-of-00002.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bbf12c79a2c588ffbdda3a72480a1e94d3fbc91d11deb74471dec2f25343dba1
3
  size 1814528290
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49bc13f0ee203a7723c840c59c209f7f99b91058c9926986a3db748d225de7c3
3
  size 1814528290
promotion_gate.json CHANGED
@@ -1,11 +1,11 @@
1
  {
2
  "reply_rate": 1.0,
3
  "empty_think_rate": 0.0,
4
- "syntax_pass_rate": 0.6153846153846154,
5
  "tool_leak_rate": 0.0,
6
  "url_contam_rate": 0.0,
7
- "total": 102,
8
- "with_code": 91,
9
  "results": [
10
  {
11
  "idx": 1,
@@ -41,7 +41,7 @@
41
  "idx": 4,
42
  "cat": "repl_oneliner",
43
  "prompt": "Write one ARO statement that converts the string 'hello' to uppercase.",
44
- "reply": "```aro\nCompute the <upper: uppercase> from \"hello\".\n```",
45
  "empty_think": false,
46
  "has_code": true,
47
  "tool_leak": false,
@@ -51,7 +51,7 @@
51
  "idx": 5,
52
  "cat": "repl_oneliner",
53
  "prompt": "Write one line that hashes the password 'secret'.",
54
- "reply": "```aro\nCompute the <hash> from the <password: secret> with <algorithm: sha256>.\n```",
55
  "empty_think": false,
56
  "has_code": true,
57
  "tool_leak": false,
@@ -61,7 +61,7 @@
61
  "idx": 6,
62
  "cat": "repl_oneliner",
63
  "prompt": "Write a single ARO line that adds 7 and 5 and stores the result as <total>.",
64
- "reply": "```aro\nCompute the <total> from 7 + 5.\n```",
65
  "empty_think": false,
66
  "has_code": true,
67
  "tool_leak": false,
@@ -71,7 +71,7 @@
71
  "idx": 7,
72
  "cat": "repl_oneliner",
73
  "prompt": "Write one line that retrieves the git status.",
74
- "reply": "```aro\nRetrieve the <status> from the <git: status>.\n```",
75
  "empty_think": false,
76
  "has_code": true,
77
  "tool_leak": false,
@@ -81,7 +81,7 @@
81
  "idx": 8,
82
  "cat": "repl_oneliner",
83
  "prompt": "Write a single ARO line that logs the value of <user> to the console.",
84
- "reply": "```aro\nLog the <user> to the <console>.\n```",
85
  "empty_think": false,
86
  "has_code": true,
87
  "tool_leak": false,
@@ -91,7 +91,7 @@
91
  "idx": 9,
92
  "cat": "repl_oneliner",
93
  "prompt": "Write one ARO statement that reads the file 'config.yaml' from the filesystem.",
94
- "reply": "```aro\nRead the <config> from the <file: \"config.yaml\">.\n```",
95
  "empty_think": false,
96
  "has_code": true,
97
  "tool_leak": false,
@@ -111,7 +111,7 @@
111
  "idx": 11,
112
  "cat": "repl_block",
113
  "prompt": "Write a 3-statement ARO snippet that reads a file, uppercases it, and logs the result.",
114
- "reply": "```aro\n(Application-Start: Example) {\n Read the <content> from the <file: \"data.txt\">.\n Compute the <upper: uppercase> from the <content>.\n Log <upper> to the <console>.\n}\n```",
115
  "empty_think": false,
116
  "has_code": true,
117
  "tool_leak": false,
@@ -121,7 +121,7 @@
121
  "idx": 12,
122
  "cat": "repl_block",
123
  "prompt": "Write a 4-statement snippet that fetches users from a repository, computes the count, and logs it.",
124
- "reply": "```aro\n(Application-Start: Example) {\n Retrieve the <users> from the <user-repository>.\n Compute the <count> from the <users>.\n Log <count> to the <console>.\n Return an <OK: status> for the <result>.\n}\n```",
125
  "empty_think": false,
126
  "has_code": true,
127
  "tool_leak": false,
@@ -131,7 +131,7 @@
131
  "idx": 13,
132
  "cat": "repl_block",
133
  "prompt": "Write an ARO snippet that stages all changes, commits with message 'wip', and pushes.",
134
- "reply": "```aro\n(Application-End: Success) {\n Stage the <changes> to the <git>.\n Commit the <changes> to the <git> with \"wip\".\n Push the <changes> to the <git>.\n Return an <OK: status> for the <commit>.\n}\n```",
135
  "empty_think": false,
136
  "has_code": true,
137
  "tool_leak": false,
@@ -141,7 +141,7 @@
141
  "idx": 14,
142
  "cat": "repl_block",
143
  "prompt": "Write a 3-statement snippet that creates a list, computes its length, and logs the length.",
144
- "reply": "```aro\n(CreateUsers: Action) {\n Create the <users> with { name: \"Alice\", age: 30 }.\n Compute the <length: length> from the <users>.\n Log <length> to the <console>.\n}\n```",
145
  "empty_think": false,
146
  "has_code": true,
147
  "tool_leak": false,
@@ -151,7 +151,7 @@
151
  "idx": 15,
152
  "cat": "repl_block",
153
  "prompt": "Write a snippet that extracts a name from an event, uppercases it, and stores it back in a variable.",
154
- "reply": "```aro\n(ExtractAndStore: Example) {\n Extract the <name> from the <event: name>.\n Compute the <upper: uppercase> from <name>.\n Store the <upper> into the <result>.\n Return an <OK: status> for the <result>.\n}\n```",
155
  "empty_think": false,
156
  "has_code": true,
157
  "tool_leak": false,
@@ -161,7 +161,7 @@
161
  "idx": 16,
162
  "cat": "repl_block",
163
  "prompt": "Write a 3-statement snippet that computes 3 + 4, multiplies by 2, and logs the result.",
164
- "reply": "```aro\n(Application-Start: Example) {\n Compute the <result> from 3 + 4.\n Compute the <doubled> from <result> * 2.\n Log <doubled> to the <console>.\n}\n```",
165
  "empty_think": false,
166
  "has_code": true,
167
  "tool_leak": false,
@@ -171,7 +171,7 @@
171
  "idx": 17,
172
  "cat": "repl_block",
173
  "prompt": "Write a snippet that reads two numbers and returns their sum.",
174
- "reply": "```aro\n(Application-Start: Example) {\n Extract the <a> from the <input: a>.\n Extract the <b> from the <input: b>.\n Compute the <sum> from <a> + <b>.\n Return an <OK: status> with <sum>.\n}\n```",
175
  "empty_think": false,
176
  "has_code": true,
177
  "tool_leak": false,
@@ -181,7 +181,7 @@
181
  "idx": 18,
182
  "cat": "repl_block",
183
  "prompt": "Write a snippet that filters a list of users where age > 18 and counts them.",
184
- "reply": "```aro\n(FilterAdults: Action) {\n Extract the <users> from the <input: users>.\n Filter the <adults> from the <users> where <age> > 18.\n Compute the <count> from the <adults>.\n Return an <OK: status> with <count>.\n}\n```",
185
  "empty_think": false,
186
  "has_code": true,
187
  "tool_leak": false,
@@ -191,7 +191,7 @@
191
  "idx": 19,
192
  "cat": "repl_block",
193
  "prompt": "Write a snippet that joins two lists and stores the result as <combined>.",
194
- "reply": "```aro\n(JoinLists: Action) {\n Extract the <list1> from the <input: list1>.\n Extract the <list2> from the <input: list2>.\n Compute the <combined> from <list1> ++ <list2>.\n Return an <OK: status> with <combined>.\n}\n```",
195
  "empty_think": false,
196
  "has_code": true,
197
  "tool_leak": false,
@@ -201,7 +201,7 @@
201
  "idx": 20,
202
  "cat": "repl_block",
203
  "prompt": "Write a snippet that parses a JSON string and extracts the 'name' field.",
204
- "reply": "```aro\n(ParseUser: Action) {\n Extract the <json> from the <input: json>.\n Parse the <user> from the <json>.\n Extract the <name> from the <user: name>.\n Return an <OK: status> with <name>.\n}\n```",
205
  "empty_think": false,
206
  "has_code": true,
207
  "tool_leak": false,
@@ -221,7 +221,7 @@
221
  "idx": 22,
222
  "cat": "user_action",
223
  "prompt": "Write a user-defined action 'Greet' that takes a name and returns a greeting string.",
224
- "reply": "```aro\n(Greet: Action) {\n Extract the <name> from the <input: name>.\n Compute the <greeting> from \"Hello, \" ++ <name>.\n Return an <OK: status> with <greeting>.\n}\n```",
225
  "empty_think": false,
226
  "has_code": true,
227
  "tool_leak": false,
@@ -231,7 +231,7 @@
231
  "idx": 23,
232
  "cat": "user_action",
233
  "prompt": "Write a user-defined action 'AddTwoNumbers' that takes two numbers and returns their sum.",
234
- "reply": "```aro\n(AddTwoNumbers: Action) {\n Extract the <a> from the <input: a>.\n Extract the <b> from the <input: b>.\n Compute the <sum> from <a> + <b>.\n Return an <OK: status> with <sum>.\n}\n```",
235
  "empty_think": false,
236
  "has_code": true,
237
  "tool_leak": false,
@@ -241,7 +241,7 @@
241
  "idx": 24,
242
  "cat": "user_action",
243
  "prompt": "Write a user-defined action 'ValidateEmail' that takes an email string and returns OK or Error.",
244
- "reply": "```aro\n(ValidateEmail: Action) {\n Extract the <email> from the <input: email>.\n Validate the <email> with <email-validator>.\n Return an <OK: status> for the <result>.\n}\n```",
245
  "empty_think": false,
246
  "has_code": true,
247
  "tool_leak": false,
@@ -251,7 +251,7 @@
251
  "idx": 25,
252
  "cat": "user_action",
253
  "prompt": "Write a user-defined action 'Square' that takes a number and returns its square.",
254
- "reply": "```aro\n(Square: Action) {\n Extract the <n> from the <input: number>.\n Compute the <square> from <n> * <n>.\n Return an <OK: status> with <square>.\n}\n```",
255
  "empty_think": false,
256
  "has_code": true,
257
  "tool_leak": false,
@@ -261,7 +261,7 @@
261
  "idx": 26,
262
  "cat": "user_action",
263
  "prompt": "Write a user-defined action 'FormatDate' that takes a date and a format string.",
264
- "reply": "```aro\n(FormatDate: Action) {\n Extract the <date> from the <input: date>.\n Extract the <format> from the <input: format>.\n Transform the <formatted> from the <date> with <format>.\n Return an <OK: status> with <formatted>.\n}\n```",
265
  "empty_think": false,
266
  "has_code": true,
267
  "tool_leak": false,
@@ -271,7 +271,7 @@
271
  "idx": 27,
272
  "cat": "user_action",
273
  "prompt": "Write a user-defined action 'CountWords' that takes a string and returns the word count.",
274
- "reply": "```aro\n(CountWords: Action) {\n Extract the <text> from the <input: text>.\n Split the <words> from the <text> by whitespace.\n Compute the <count> from the <words>.\n Return an <OK: status> with <count>.\n}\n```",
275
  "empty_think": false,
276
  "has_code": true,
277
  "tool_leak": false,
@@ -281,7 +281,7 @@
281
  "idx": 28,
282
  "cat": "user_action",
283
  "prompt": "Write a user-defined action 'PickRandom' that takes a list and returns a random element.",
284
- "reply": "```aro\n(PickRandom: Action) {\n Extract the <items> from the <input: items>.\n Compute the <random-item: collections.pick-random> from the <items>.\n Return an <OK: status> with <random-item>.\n}\n```",
285
  "empty_think": false,
286
  "has_code": true,
287
  "tool_leak": false,
@@ -291,7 +291,7 @@
291
  "idx": 29,
292
  "cat": "user_action",
293
  "prompt": "Write a user-defined action 'Capitalize' that takes a string and uppercases its first letter.",
294
- "reply": "```aro\n(Capitalize: Action) {\n Extract the <text> from the <input: text>.\n Compute the <capitalized> from <text> with \"capitalize\".\n Return an <OK: status> with <capitalized>.\n}\n```",
295
  "empty_think": false,
296
  "has_code": true,
297
  "tool_leak": false,
@@ -301,7 +301,7 @@
301
  "idx": 30,
302
  "cat": "user_action",
303
  "prompt": "Write a user-defined action 'ComputeTax' that takes a price and a rate and returns the tax.",
304
- "reply": "```aro\n(ComputeTax: Action) {\n Extract the <price> from the <input: price>.\n Extract the <rate> from the <input: rate>.\n Compute the <tax> from <price> * <rate>.\n Return an <OK: status> with <tax>.\n}\n```",
305
  "empty_think": false,
306
  "has_code": true,
307
  "tool_leak": false,
@@ -311,7 +311,7 @@
311
  "idx": 31,
312
  "cat": "event_handler",
313
  "prompt": "Write an event handler for the UserCreated event that sends a welcome email.",
314
- "reply": "```aro\n(Send Welcome Email: UserCreated Handler) {\n Extract the <user> from the <event: user>.\n Send the <welcome-email> to the <user: email>.\n Return an <OK: status> for the <notification>.\n}\n```",
315
  "empty_think": false,
316
  "has_code": true,
317
  "tool_leak": false,
@@ -321,7 +321,7 @@
321
  "idx": 32,
322
  "cat": "event_handler",
323
  "prompt": "Write an event handler for OrderPlaced that decrements inventory.",
324
- "reply": "```aro\n(OrderInventory: OrderPlaced Handler) {\n Extract the <order> from the <event: order>.\n Extract the <item> from the <order: item>.\n Extract the <quantity> from the <order: quantity>.\n Retrieve the <inventory> from the <inventory-repository>.\n Compute the <new-quantity> from <inventory> - <quantity>.\n Store the <new-quantity> into the <inventory-repository>.\n Return an <OK: status> for the <inventory-update>.\n}\n```",
325
  "empty_think": false,
326
  "has_code": true,
327
  "tool_leak": false,
@@ -331,7 +331,7 @@
331
  "idx": 33,
332
  "cat": "event_handler",
333
  "prompt": "Write an event handler for PaymentFailed that logs the error and notifies the user.",
334
- "reply": "```aro\n(PaymentFailed Handler) {\n Extract the <error> from the <event: error>.\n Log <error> to the <console>.\n Return an <OK: status> for the <notification>.\n}\n```",
335
  "empty_think": false,
336
  "has_code": true,
337
  "tool_leak": false,
@@ -341,7 +341,7 @@
341
  "idx": 34,
342
  "cat": "event_handler",
343
  "prompt": "Write an event handler for FileChanged that re-reads the configuration.",
344
- "reply": "```aro\n(Handle FileChanged: Config Reload) {\n Extract the <path> from the <event: path>.\n Read the <config> from the <file: path>.\n Return an <OK: status> for the <reload>.\n}\n```",
345
  "empty_think": false,
346
  "has_code": true,
347
  "tool_leak": false,
@@ -351,7 +351,7 @@
351
  "idx": 35,
352
  "cat": "event_handler",
353
  "prompt": "Write an event handler for LoginAttempt that records the IP address.",
354
- "reply": "```aro\n(Handle Login Attempt: LoginAttempt Handler) {\n Extract the <ip> from the <event: ip>.\n Log <ip> to the <console>.\n Return an <OK: status> for the <logging>.\n}\n```",
355
  "empty_think": false,
356
  "has_code": true,
357
  "tool_leak": false,
@@ -361,7 +361,7 @@
361
  "idx": 36,
362
  "cat": "event_handler",
363
  "prompt": "Write an event handler for GitPush that triggers a CI build.",
364
- "reply": "```aro\n(GitPush: Event Handler) {\n Extract the <branch> from the <event: branch>.\n Extract the <commit> from the <event: commit>.\n Log \"CI build triggered for branch: <branch> commit: <commit>\" to the <console>.\n Return an <OK: status> for the <ci-build>.\n}\n```",
365
  "empty_think": false,
366
  "has_code": true,
367
  "tool_leak": false,
@@ -371,7 +371,7 @@
371
  "idx": 37,
372
  "cat": "event_handler",
373
  "prompt": "Write an event handler for SubscriptionExpired that downgrades the account.",
374
- "reply": "```aro\n(Handle Subscription Expired: SubscriptionExpired Handler) {\n Extract the <account> from the <event: account>.\n Downgrade the <account> with <account>.\n Return an <OK: status> for the <downgrade>.\n}\n```",
375
  "empty_think": false,
376
  "has_code": true,
377
  "tool_leak": false,
@@ -381,7 +381,7 @@
381
  "idx": 38,
382
  "cat": "event_handler",
383
  "prompt": "Write an event handler for ServerStarted that warms up the cache.",
384
- "reply": "```aro\n(WarmCache: ServerStarted Handler) {\n Log \"Warming cache...\" to the <console>.\n Compute the <cache-warm: cache.warm> from the <cache>.\n Return an <OK: status> for the <cache-warm>.\n}\n```",
385
  "empty_think": false,
386
  "has_code": true,
387
  "tool_leak": false,
@@ -391,7 +391,7 @@
391
  "idx": 39,
392
  "cat": "event_handler",
393
  "prompt": "Write an event handler for MessageReceived that echoes the message back.",
394
- "reply": "```aro\n(Handle Echo: MessageReceived Handler) {\n Extract the <message> from the <event: message>.\n Return an <OK: status> with <message>.\n}\n```",
395
  "empty_think": false,
396
  "has_code": true,
397
  "tool_leak": false,
@@ -401,7 +401,7 @@
401
  "idx": 40,
402
  "cat": "event_handler",
403
  "prompt": "Write an event handler for ImageUploaded that generates a thumbnail.",
404
- "reply": "```aro\n(ImageGenerated: ImageUploaded Handler) {\n Extract the <image> from the <event: image>.\n Compute the <thumbnail> from the <image: thumbnail>.\n Return an <OK: status> for the <thumbnail>.\n}\n```",
405
  "empty_think": false,
406
  "has_code": true,
407
  "tool_leak": false,
@@ -411,7 +411,7 @@
411
  "idx": 41,
412
  "cat": "application",
413
  "prompt": "Write a complete ARO Application-Start that prints Hello and exits.",
414
- "reply": "```aro\n(Application-Start: Example) {\n Log \"Hello\" to the <console>.\n Return an <OK: status> for the <result>.\n}\n```",
415
  "empty_think": false,
416
  "has_code": true,
417
  "tool_leak": false,
@@ -421,7 +421,7 @@
421
  "idx": 42,
422
  "cat": "application",
423
  "prompt": "Write a complete Application-Start that starts an HTTP server on port 8080.",
424
- "reply": "```aro\n(Application-Start: Example) {\n Start the <http-server> with { port: 8080 }.\n Return an <OK: status> for the <startup>.\n}\n```",
425
  "empty_think": false,
426
  "has_code": true,
427
  "tool_leak": false,
@@ -431,7 +431,7 @@
431
  "idx": 43,
432
  "cat": "application",
433
  "prompt": "Write a complete Application-Start that watches a directory for changes.",
434
- "reply": "```aro\n(Application-Start: Watcher) {\n Create the <watcher> with <directory>.\n Keepalive the <watcher> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
435
  "empty_think": false,
436
  "has_code": true,
437
  "tool_leak": false,
@@ -441,7 +441,7 @@
441
  "idx": 44,
442
  "cat": "application",
443
  "prompt": "Write a complete Application-Start and Application-End:Success that logs startup and shutdown.",
444
- "reply": "```aro\n(Application-Start: Example) {\n Log \"Starting...\" to the <console>.\n Start the <http-server> with <contract>.\n Return an <OK: status> for the <startup>.\n}\n\n(Application-End: Success) {\n Log \"Shutting down...\" to the <console>.\n Stop the <http-server> with <application>.\n Return an <OK: status> for the <shutdown>.\n}\n```",
445
  "empty_think": false,
446
  "has_code": true,
447
  "tool_leak": false,
@@ -451,7 +451,7 @@
451
  "idx": 45,
452
  "cat": "application",
453
  "prompt": "Write a complete Application-Start that opens a TCP echo server on port 9000.",
454
- "reply": "```aro\n(Application-Start: Echo Server) {\n Start the <tcp-server> with { port: 9000 }.\n Return an <OK: status> for the <startup>.\n}\n```",
455
  "empty_think": false,
456
  "has_code": true,
457
  "tool_leak": false,
@@ -461,7 +461,7 @@
461
  "idx": 46,
462
  "cat": "application",
463
  "prompt": "Write a complete Application-Start that starts a WebSocket server and keeps it alive.",
464
- "reply": "```aro\n(Application-Start: WebSocket App) {\n Log \"Starting WebSocket server...\" to the <console>.\n Start the <websocket-server> with <config>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
465
  "empty_think": false,
466
  "has_code": true,
467
  "tool_leak": false,
@@ -471,7 +471,7 @@
471
  "idx": 47,
472
  "cat": "application",
473
  "prompt": "Write a complete Application-Start that connects to a database and runs a single query.",
474
- "reply": "```aro\n(Application-Start: Example) {\n Connect the <db-connection> to the <database-url>.\n Query the <result> from the <db-connection> with \"SELECT * FROM users\".\n Return an <OK: status> for the <result>.\n}\n```",
475
  "empty_think": false,
476
  "has_code": true,
477
  "tool_leak": false,
@@ -481,7 +481,7 @@
481
  "idx": 48,
482
  "cat": "application",
483
  "prompt": "Write a complete Application-Start that reads command-line parameters and prints them.",
484
- "reply": "```aro\n(Application-Start: Example) {\n Extract the <args> from the <command-line: args>.\n Log \"Arguments: \" to the <console>.\n Log <args> to the <console>.\n Return an <OK: status> for the <startup>.\n}\n```",
485
  "empty_think": false,
486
  "has_code": true,
487
  "tool_leak": false,
@@ -491,7 +491,7 @@
491
  "idx": 49,
492
  "cat": "application",
493
  "prompt": "Write a complete Application-Start that loads a configuration file and logs it.",
494
- "reply": "```aro\n(Application-Start: Example) {\n Read the <config> from the <file: \"config.yaml\">.\n Log <config> to the <console>.\n Return an <OK: status> for the <startup>.\n}\n```",
495
  "empty_think": false,
496
  "has_code": true,
497
  "tool_leak": false,
@@ -501,7 +501,7 @@
501
  "idx": 50,
502
  "cat": "application",
503
  "prompt": "Write a complete Application-Start that prints the current git branch.",
504
- "reply": "```aro\n(Application-Start: Example) {\n (* Get the current git branch *)\n Extract the <branch> from the <git: branch>.\n Log <branch> to the <console>.\n Return an <OK: status> for the <startup>.\n}\n```",
505
  "empty_think": false,
506
  "has_code": true,
507
  "tool_leak": false,
@@ -521,9 +521,9 @@
521
  "idx": 52,
522
  "cat": "http_api",
523
  "prompt": "Write the openapi.yaml plus feature set for a POST /products endpoint.",
524
- "reply": "```yaml\nopenapi: 3.0.0\ninfo:\n title: Product API\n version: 1.0.0\npaths:\n /products:\n post:\n summary: Create a product\n requestBody:\n required: true\n content:\n application/json:\n schema:\n type: object\n properties:\n name:\n type: string\n price:\n type: number\n description:\n type: string\n required:\n - na",
525
  "empty_think": false,
526
- "has_code": false,
527
  "tool_leak": false,
528
  "url_contam": false
529
  },
@@ -531,7 +531,7 @@
531
  "idx": 53,
532
  "cat": "http_api",
533
  "prompt": "Write a feature set for getUserById that extracts the id from pathParameters.",
534
- "reply": "```aro\n(getUserById: User API) {\n Extract the <id> from the <pathParameters: id>.\n Retrieve the <user> from the <user-repository> where id = <id>.\n Return an <OK: status> with <user>.\n}\n```",
535
  "empty_think": false,
536
  "has_code": true,
537
  "tool_leak": false,
@@ -541,7 +541,7 @@
541
  "idx": 54,
542
  "cat": "http_api",
543
  "prompt": "Write a feature set for createOrder that validates the request body and emits OrderCreated.",
544
- "reply": "```aro\n(createOrder: Order API) {\n Extract the <data> from the <request: body>.\n Validate the <data> with <required-fields>.\n Emit an <OrderCreated: event> with <data>.\n Return an <OK: status> for the <creation>.\n}\n```",
545
  "empty_think": false,
546
  "has_code": true,
547
  "tool_leak": false,
@@ -551,7 +551,7 @@
551
  "idx": 55,
552
  "cat": "http_api",
553
  "prompt": "Write a feature set for deleteSession that removes a session from the sessions-repository.",
554
- "reply": "```aro\n(deleteSession: Session API) {\n Extract the <id> from the <pathParameters: id>.\n Delete the <session> from the <sessions-repository> where id = <id>.\n Return an <OK: status> for the <deletion>.\n}\n```",
555
  "empty_think": false,
556
  "has_code": true,
557
  "tool_leak": false,
@@ -561,7 +561,7 @@
561
  "idx": 56,
562
  "cat": "http_api",
563
  "prompt": "Write a feature set for searchProducts that takes a query parameter and filters results.",
564
- "reply": "```aro\n(searchProducts: Product API) {\n Extract the <query> from the <queryParameters: query>.\n Retrieve the <products> from the <product-repository>.\n Filter the <filtered-products> from the <products> where name contains <query>.\n Return an <OK: status> with <filtered-products>.\n}\n```",
565
  "empty_think": false,
566
  "has_code": true,
567
  "tool_leak": false,
@@ -571,7 +571,7 @@
571
  "idx": 57,
572
  "cat": "http_api",
573
  "prompt": "Write a feature set for updateProfile that patches a user record.",
574
- "reply": "```aro\n(updateProfile: User API) {\n Extract the <user-id> from the <pathParameters: id>.\n Extract the <data> from the <request: body>.\n Retrieve the <user> from the <user-repository> where id = <user-id>.\n Patch the <user> with <data>.\n Store the <user> into the <user-repository>.\n Return an <OK: status> with <user>.\n}\n```",
575
  "empty_think": false,
576
  "has_code": true,
577
  "tool_leak": false,
@@ -581,7 +581,7 @@
581
  "idx": 58,
582
  "cat": "http_api",
583
  "prompt": "Write a feature set for healthCheck that returns OK.",
584
- "reply": "```aro\n(healthCheck: Health Check) {\n Return an <OK: status> for the <health-check>.\n}\n```",
585
  "empty_think": false,
586
  "has_code": true,
587
  "tool_leak": false,
@@ -591,7 +591,7 @@
591
  "idx": 59,
592
  "cat": "http_api",
593
  "prompt": "Write a feature set for uploadAvatar that accepts a file in the request body.",
594
- "reply": "```aro\n(uploadAvatar: User API) {\n Extract the <file> from the <request: body>.\n Create the <avatar> with <file>.\n Return an <OK: status> with <avatar>.\n}\n```",
595
  "empty_think": false,
596
  "has_code": true,
597
  "tool_leak": false,
@@ -601,7 +601,7 @@
601
  "idx": 60,
602
  "cat": "http_api",
603
  "prompt": "Write a feature set for login that compares a hash and returns a session token.",
604
- "reply": "```aro\n(Login: User API) {\n Extract the <username> from the <request: body>.\n Extract the <password> from the <request: body>.\n Retrieve the <user> from the <user-repository> where username = <username>.\n Compute the <hash> from the <password> with <hasher>.\n Compare the <hash> against the <user: passwordHash>.\n Then the <user> {\n Compute the <session-token> from the <user: id>.\n Return an <OK: status> with { sessionToken: <session-token> }.\n }\n}\n```",
605
  "empty_think": false,
606
  "has_code": true,
607
  "tool_leak": false,
@@ -611,7 +611,7 @@
611
  "idx": 61,
612
  "cat": "repository",
613
  "prompt": "Write a repository observer for the users-repository that logs every change.",
614
- "reply": "```aro\n(UsersRepositoryObserver: Repository Observer) {\n (* Watch the users-repository for changes *)\n Watch the <users-repository> for <changes>.\n\n (* Log each change to the console *)\n Log <change> to the <console>.\n}\n```",
615
  "empty_think": false,
616
  "has_code": true,
617
  "tool_leak": false,
@@ -621,7 +621,7 @@
621
  "idx": 62,
622
  "cat": "repository",
623
  "prompt": "Write a repository observer for orders-repository that emits OrderUpdated when records change.",
624
- "reply": "```aro\n(OrderUpdated: Order Repository Observer) {\n Extract the <order> from the <repository: order>.\n Emit a <OrderUpdated: event> with <order>.\n Return an <OK: status> for the <update>.\n}\n```",
625
  "empty_think": false,
626
  "has_code": true,
627
  "tool_leak": false,
@@ -631,7 +631,7 @@
631
  "idx": 63,
632
  "cat": "repository",
633
  "prompt": "Write a feature set that stores a new product into the products-repository.",
634
- "reply": "```aro\n(StoreProduct: Product API) {\n Extract the <product> from the <request: body>.\n Store the <product> into the <products-repository>.\n Return an <OK: status> with <product>.\n}\n```",
635
  "empty_think": false,
636
  "has_code": true,
637
  "tool_leak": false,
@@ -641,7 +641,7 @@
641
  "idx": 64,
642
  "cat": "repository",
643
  "prompt": "Write a feature set that retrieves all sessions from the sessions-repository where active = true.",
644
- "reply": "```aro\n(ActiveSessions: Session API) {\n Retrieve the <sessions> from the <sessions-repository>.\n Filter the <active-sessions> from the <sessions> where active = true.\n Return an <OK: status> with <active-sessions>.\n}\n```",
645
  "empty_think": false,
646
  "has_code": true,
647
  "tool_leak": false,
@@ -651,7 +651,7 @@
651
  "idx": 65,
652
  "cat": "repository",
653
  "prompt": "Write an observer that triggers when a user is deleted from the user-repository.",
654
- "reply": "```aro\n(DeleteUser: User API) {\n Extract the <id> from the <pathParameters: id>.\n Delete the <user> from the <user-repository> where id = <id>.\n Emit a <UserDeleted: event> with <user>.\n Return an <OK: status> for the <deletion>.\n}\n```",
655
  "empty_think": false,
656
  "has_code": true,
657
  "tool_leak": false,
@@ -661,7 +661,7 @@
661
  "idx": 66,
662
  "cat": "repository",
663
  "prompt": "Write a feature set that updates a record in the inventory-repository.",
664
- "reply": "```aro\n(UpdateInventoryItem: Inventory API) {\n Extract the <itemId> from the <pathParameters: id>.\n Extract the <data> from the <request: body>.\n Update the <item> in the <inventory-repository> with <itemId> and <data>.\n Return an <OK: status> with <item>.\n}\n```",
665
  "empty_think": false,
666
  "has_code": true,
667
  "tool_leak": false,
@@ -671,7 +671,7 @@
671
  "idx": 67,
672
  "cat": "repository",
673
  "prompt": "Write a state-guarded handler that triggers only on UserUpdated where role = admin.",
674
- "reply": "```aro\n(Handle Admin User Update: UserUpdated Handler) {\n Extract the <user> from the <event: user>.\n When <user: role> = \"admin\" {\n Log \"Admin user updated\" to the <console>.\n (* Handle admin-specific logic here *)\n }\n Return an <OK: status> for the <handler>.\n}\n```",
675
  "empty_think": false,
676
  "has_code": true,
677
  "tool_leak": false,
@@ -681,7 +681,7 @@
681
  "idx": 68,
682
  "cat": "repository",
683
  "prompt": "Write a feature set that bulk-imports a list of users into the user-repository.",
684
- "reply": "```aro\n(ImportUsers: User API) {\n Extract the <users> from the <request: body>.\n For each <user> in <users> {\n Create the <user> with <user>.\n Emit a <UserCreated: event> with <user>.\n }\n Return an <OK: status> with <users>.\n}\n```",
685
  "empty_think": false,
686
  "has_code": true,
687
  "tool_leak": false,
@@ -691,7 +691,7 @@
691
  "idx": 69,
692
  "cat": "repository",
693
  "prompt": "Write a feature set that publishes the count of orders as a global variable.",
694
- "reply": "```aro\n(CountOrders: Order API) {\n Retrieve the <orders> from the <order-repository>.\n Compute the <count: length> from the <orders>.\n Publish as <order-count> <count>.\n Return an <OK: status> for the <result>.\n}\n```",
695
  "empty_think": false,
696
  "has_code": true,
697
  "tool_leak": false,
@@ -701,7 +701,7 @@
701
  "idx": 70,
702
  "cat": "repository",
703
  "prompt": "Write a repository observer that retries on failure.",
704
- "reply": "```aro\n(ObserveUserRepository: Repository Observer) {\n Extract the <user> from the <repository: user>.\n (* Retry on failure *)\n Retry the <user> with { maxRetries: 3, delay: 1000 }.\n Return an <OK: status> for the <observation>.\n}\n```",
705
  "empty_think": false,
706
  "has_code": true,
707
  "tool_leak": false,
@@ -711,7 +711,7 @@
711
  "idx": 71,
712
  "cat": "refactor",
713
  "prompt": "Refactor this snippet to use the qualifier-as-name syntax: `Compute the <length> from <a>. Compute the <length> from <b>.`",
714
- "reply": "```aro\nCompute the <a-length: length> from <a>.\nCompute the <b-length: length> from <b>.\n```",
715
  "empty_think": false,
716
  "has_code": true,
717
  "tool_leak": false,
@@ -721,7 +721,7 @@
721
  "idx": 72,
722
  "cat": "refactor",
723
  "prompt": "Fix this snippet so it parses: `Log Hello to console.`",
724
- "reply": "```aro\n(Log: Example) {\n Log \"Hello\" to the <console>.\n}\n```",
725
  "empty_think": false,
726
  "has_code": true,
727
  "tool_leak": false,
@@ -731,7 +731,7 @@
731
  "idx": 73,
732
  "cat": "refactor",
733
  "prompt": "Convert this English description to ARO: 'When a user signs up, send them a welcome email and store them in the users repository.'",
734
- "reply": "```aro\n(Send Welcome Email: UserCreated Handler) {\n Extract the <user> from the <event: user>.\n Send the <welcome-email> to the <user: email>.\n Return an <OK: status> for the <notification>.\n}\n```",
735
  "empty_think": false,
736
  "has_code": true,
737
  "tool_leak": false,
@@ -741,7 +741,7 @@
741
  "idx": 74,
742
  "cat": "refactor",
743
  "prompt": "Convert this Python function to an ARO user-defined action: `def double(x): return x * 2`.",
744
- "reply": "```aro\n(DoubleValue: Action takes <number>) {\n Extract the <n> from the <input: number>.\n Compute the <doubled> from <n> * 2.\n Return an <OK: status> with { doubled: <doubled> }.\n}\n```",
745
  "empty_think": false,
746
  "has_code": true,
747
  "tool_leak": false,
@@ -751,7 +751,7 @@
751
  "idx": 75,
752
  "cat": "refactor",
753
  "prompt": "Rewrite this ARO snippet to use lazy evaluation idioms: `Compute the <a> from 1+1. Compute the <b> from <a>+1. Log <b> to the <console>.`",
754
- "reply": "```aro\n(Log: Example) {\n Compute the <a> from 1 + 1.\n Compute the <b> from <a> + 1.\n Log <b> to the <console>.\n}\n```",
755
  "empty_think": false,
756
  "has_code": true,
757
  "tool_leak": false,
@@ -761,7 +761,7 @@
761
  "idx": 76,
762
  "cat": "refactor",
763
  "prompt": "Split this monolithic feature set into one Application-Start and one event handler: `(Foo: My App) { Log \"start\" to <console>. Emit a <Started: event>. Log \"after\" to <console>. }`",
764
- "reply": "```aro\n(Application-Start: My App) {\n Log \"start\" to the <console>.\n Emit a <Started: event>.\n Log \"after\" to the <console>.\n}\n```",
765
  "empty_think": false,
766
  "has_code": true,
767
  "tool_leak": false,
@@ -771,7 +771,7 @@
771
  "idx": 77,
772
  "cat": "refactor",
773
  "prompt": "Add a When guard to this handler so it only runs for admin users: `(OnUserUpdated: UserUpdated Handler) { Log <event> to the <console>. Return an <OK: status> for <log>. }`",
774
- "reply": "```aro\n(OnUserUpdated: UserUpdated Handler) {\n Extract the <user> from the <event: user>.\n When the <user: role> = \"admin\" {\n Log <event> to the <console>.\n Return an <OK: status> for <log>.\n }\n}\n```",
775
  "empty_think": false,
776
  "has_code": true,
777
  "tool_leak": false,
@@ -781,7 +781,7 @@
781
  "idx": 78,
782
  "cat": "refactor",
783
  "prompt": "Replace direct repository writes with an emitted event in this feature set: `(saveUser: User API) { Store the <user> in the <user-repository>. Return an <OK: status>. }`",
784
- "reply": "```aro\n(saveUser: User API) {\n Extract the <user> from the <request: body>.\n Emit a <UserSaved: event> with <user>.\n Return an <OK: status> for the <save>.\n}\n```",
785
  "empty_think": false,
786
  "has_code": true,
787
  "tool_leak": false,
@@ -791,7 +791,7 @@
791
  "idx": 79,
792
  "cat": "refactor",
793
  "prompt": "Convert this REPL one-liner into a proper feature set: `Log \"Hi\" to the <console>.`",
794
- "reply": "```aro\n(Log: Example) {\n Log \"Hi\" to the <console>.\n Return an <OK: status> for the <result>.\n}\n```",
795
  "empty_think": false,
796
  "has_code": true,
797
  "tool_leak": false,
@@ -801,7 +801,7 @@
801
  "idx": 80,
802
  "cat": "refactor",
803
  "prompt": "Refactor a duplicated three-line greeting flow into a single user-defined action called Greet.",
804
- "reply": "```aro\n(Greet: Action) {\n Extract the <name> from the <input: name>.\n Compute the <greeting> from \"Hello, \" ++ <name>.\n Return an <OK: status> with <greeting>.\n}\n```",
805
  "empty_think": false,
806
  "has_code": true,
807
  "tool_leak": false,
@@ -811,7 +811,7 @@
811
  "idx": 81,
812
  "cat": "explain",
813
  "prompt": "What does the Keepalive action do in ARO?",
814
- "reply": "The Keepalive action keeps the application running by waiting for events or signals. It is used to keep the application alive and responsive to incoming requests.",
815
  "empty_think": false,
816
  "has_code": false,
817
  "tool_leak": false,
@@ -821,7 +821,7 @@
821
  "idx": 82,
822
  "cat": "explain",
823
  "prompt": "Explain the difference between Application-Start and an event handler.",
824
- "reply": "The Application-Start is the entry point for the application. It runs once when the application starts and is used to initialize resources, start services, and set up the environment. The Application-Start is a single entry point per application.\n\nEvent handlers, on the other hand, are triggered by specific events and run only when those events occur. They are used to handle events such as user actions, system events, or business events. Event handlers are defined with a handler namespace and ar",
825
  "empty_think": false,
826
  "has_code": false,
827
  "tool_leak": false,
@@ -831,7 +831,7 @@
831
  "idx": 83,
832
  "cat": "explain",
833
  "prompt": "What is the difference between OWN and EXPORT action roles?",
834
- "reply": "The `OWN` role is for actions that operate on the application's own data, while the `EXPORT` role is for actions that export data to external systems or services.",
835
  "empty_think": false,
836
  "has_code": false,
837
  "tool_leak": false,
@@ -841,7 +841,7 @@
841
  "idx": 84,
842
  "cat": "explain",
843
  "prompt": "How does ARO's error philosophy work? What is 'code is the error message'?",
844
- "reply": "ARO's error philosophy is centered around the idea that the code itself is the error message. This means that the system does not generate separate error messages or logs \u2014 instead, the code itself is the source of the error message. The runtime system reads the code and reports the error directly from the code's structure and content.\n\nFor example, if a feature set is missing a required action or if a variable is used before it is defined, the runtime system will report the error directly from ",
845
  "empty_think": false,
846
  "has_code": false,
847
  "tool_leak": false,
@@ -851,7 +851,7 @@
851
  "idx": 85,
852
  "cat": "explain",
853
  "prompt": "Explain what the openapi.yaml file does in an ARO application.",
854
- "reply": "The `openapi.yaml` file defines the API contract for an ARO application. It describes the endpoints, request/response formats, and operations available in the application. This file is used by clients and tools to understand how to interact with the API.",
855
  "empty_think": false,
856
  "has_code": false,
857
  "tool_leak": false,
@@ -861,7 +861,7 @@
861
  "idx": 86,
862
  "cat": "explain",
863
  "prompt": "What does Publish as <alias> do?",
864
- "reply": "The `Publish as <alias>` statement makes a variable available to other feature sets in the same application. The alias is the name that other feature sets use to refer to the variable.",
865
  "empty_think": false,
866
  "has_code": false,
867
  "tool_leak": false,
@@ -871,7 +871,7 @@
871
  "idx": 87,
872
  "cat": "explain",
873
  "prompt": "How are events routed to feature sets in ARO?",
874
- "reply": "ARO uses event names as feature set names. When an event is emitted, the runtime looks for a feature set with the same name as the event. For example, when a `UserCreated` event is emitted, the runtime looks for a feature set named `UserCreated` and executes it.",
875
  "empty_think": false,
876
  "has_code": false,
877
  "tool_leak": false,
@@ -881,7 +881,7 @@
881
  "idx": 88,
882
  "cat": "explain",
883
  "prompt": "What is the role of the EventBus?",
884
- "reply": "The EventBus is a central component that coordinates communication between different parts of the application. It allows components to publish events and subscribe to events, enabling loose coupling and decoupling of components. The EventBus is used to broadcast events to interested parties, allowing them to react to changes or events without being tightly coupled to the source of the event.",
885
  "empty_think": false,
886
  "has_code": false,
887
  "tool_leak": false,
@@ -891,7 +891,7 @@
891
  "idx": 89,
892
  "cat": "explain",
893
  "prompt": "Explain repository observers and how they differ from event handlers.",
894
- "reply": "Repository observers are a pattern where a component watches a repository for changes and reacts to them. They are similar to event handlers but are specifically tied to repository changes. Event handlers are more general and can be triggered by any event, while repository observers are specifically for repository changes.",
895
  "empty_think": false,
896
  "has_code": false,
897
  "tool_leak": false,
@@ -901,7 +901,7 @@
901
  "idx": 90,
902
  "cat": "explain",
903
  "prompt": "What does lazy execution mean in ARO and when is a future forced?",
904
- "reply": "Lazy execution means that the execution of an action is deferred until the result is actually needed. This is different from eager execution, where the action is executed immediately when it is called. In ARO, the execution is only triggered when the result is accessed or used in a subsequent operation.\n\nA future is forced when the result is accessed or used in a subsequent operation. For example, when you use the result in a subsequent computation or when you explicitly call a method that force",
905
  "empty_think": false,
906
  "has_code": false,
907
  "tool_leak": false,
@@ -911,7 +911,7 @@
911
  "idx": 91,
912
  "cat": "edge",
913
  "prompt": "Write an ARO snippet that intentionally fails so the runtime produces a 'Can not ...' message \u2014 explain what triggers that message.",
914
- "reply": "```aro\n(Application-Start: Example) {\n (* Intentionally missing the required parameter *)\n Start the <http-server> with <missing>.\n Return an <OK: status> for the <startup>.\n}\n```",
915
  "empty_think": false,
916
  "has_code": true,
917
  "tool_leak": false,
@@ -921,7 +921,7 @@
921
  "idx": 92,
922
  "cat": "edge",
923
  "prompt": "Show me an example of using qualifier-as-name with two different `length` results in the same feature set.",
924
- "reply": "```aro\n(Feature Name: Length Comparison) {\n Compute the <first-length: length> from the <first-message>.\n Compute the <second-length: length> from the <second-message>.\n Then the <first-length> with <second-length>.\n}\n```",
925
  "empty_think": false,
926
  "has_code": true,
927
  "tool_leak": false,
@@ -931,7 +931,7 @@
931
  "idx": 93,
932
  "cat": "edge",
933
  "prompt": "Write a feature set with both When guards and a Publish statement.",
934
- "reply": "```aro\n(Feature Name: Conditional Publish) {\n Extract the <user> from the <request: body>.\n When <user: active> = true {\n Publish the <user> to the <user-feed>.\n }\n Return an <OK: status> for the <result>.\n}\n```",
935
  "empty_think": false,
936
  "has_code": true,
937
  "tool_leak": false,
@@ -941,7 +941,7 @@
941
  "idx": 94,
942
  "cat": "edge",
943
  "prompt": "Write a Compute statement that uses a plugin qualifier called Collections.pick-random on a list.",
944
- "reply": "```aro\nCompute the <random-item: collections.pick-random> from the <items>.\n```",
945
  "empty_think": false,
946
  "has_code": true,
947
  "tool_leak": false,
@@ -951,7 +951,7 @@
951
  "idx": 95,
952
  "cat": "edge",
953
  "prompt": "Write a Git action that clones a repository from a URL into ./local-clone.",
954
- "reply": "```aro\n(Clone Repo: Git) {\n Clone the <repo> from the <url>.\n Return an <OK: status> for the <clone>.\n}\n```",
955
  "empty_think": false,
956
  "has_code": true,
957
  "tool_leak": false,
@@ -961,7 +961,7 @@
961
  "idx": 96,
962
  "cat": "edge",
963
  "prompt": "Write a feature set that uses a state-guard `where status = pending` on a custom event.",
964
- "reply": "```aro\n(Handle Order: OrderStatusUpdated Handler) {\n Extract the <order> from the <event: order>.\n Where the <order: status> = \"pending\".\n Update the <order> with { status: \"processing\" }.\n Return an <OK: status> for the <update>.\n}\n```",
965
  "empty_think": false,
966
  "has_code": true,
967
  "tool_leak": false,
@@ -971,7 +971,7 @@
971
  "idx": 97,
972
  "cat": "edge",
973
  "prompt": "Show a Render action invocation that fills a Mustache-style template.",
974
- "reply": "```aro\n(Render: Template) {\n Extract the <template> from the <input: template>.\n Extract the <data> from the <input: data>.\n Render the <output> from the <template> with <data>.\n Return an <OK: status> with <output>.\n}\n```",
975
  "empty_think": false,
976
  "has_code": true,
977
  "tool_leak": false,
@@ -981,7 +981,7 @@
981
  "idx": 98,
982
  "cat": "edge",
983
  "prompt": "Write a complete .store file plus a feature set that reads from a seeded repository.",
984
- "reply": "```aro\n(* .store file *)\n(store: UserStore) {\n Create the <users> with [\n { id: 1, name: \"Alice\", email: \"alice@example.com\" },\n { id: 2, name: \"Bob\", email: \"bob@example.com\" }\n ].\n Return an <OK: status> for the <store>.\n}\n```",
985
  "empty_think": false,
986
  "has_code": true,
987
  "tool_leak": false,
@@ -991,7 +991,7 @@
991
  "idx": 99,
992
  "cat": "edge",
993
  "prompt": "Write an action that uses a list-range slice (e.g., elements 0..3).",
994
- "reply": "```aro\n(ExtractSlice: Action) {\n Extract the <start> from the <input: start>.\n Extract the <end> from the <input: end>.\n Extract the <items> from the <input: items>.\n Compute the <slice> from <items> where index >= <start> and index < <end>.\n Return an <OK: status> with <slice>.\n}\n```",
995
  "empty_think": false,
996
  "has_code": true,
997
  "tool_leak": false,
@@ -1001,7 +1001,7 @@
1001
  "idx": 100,
1002
  "cat": "edge",
1003
  "prompt": "Write an event handler that uses typed event extraction with a schema-validated payload.",
1004
- "reply": "```aro\n(Handle User Created: UserCreated Handler) {\n Extract the <user> from the <event: UserCreated>.\n Extract the <id> from the <user: id>.\n Extract the <name> from the <user: name>.\n Extract the <email> from the <user: email>.\n Return an <OK: status> for the <notification>.\n}\n```",
1005
  "empty_think": false,
1006
  "has_code": true,
1007
  "tool_leak": false,
@@ -1011,7 +1011,7 @@
1011
  "idx": 101,
1012
  "cat": "application",
1013
  "prompt": "Write a complete ARO application with a timer that fires every minute and logs a message on each tick.",
1014
- "reply": "```aro\n(Application-Start: Timer Example) {\n Create the <timer> with 60 seconds.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n\n(Application-End: Success) {\n Stop the <timer> with <application>.\n Return an <OK: status> for the <shutdown>.\n}\n```",
1015
  "empty_think": false,
1016
  "has_code": true,
1017
  "tool_leak": false,
@@ -1021,11 +1021,31 @@
1021
  "idx": 102,
1022
  "cat": "explain",
1023
  "prompt": "How can I build a timer that fires every minute in ARO? Show the code.",
1024
- "reply": "```aro\n(Application-Start: Timer Example) {\n Create the <timer> with { interval: 60, action: \"tick\" }.\n Return an <OK: status> for the <startup>.\n}\n\n(Tick: Timer Handler) {\n Log \"Tick\" to the <console>.\n Return an <OK: status> for the <tick>.\n}\n```",
1025
  "empty_think": false,
1026
  "has_code": true,
1027
  "tool_leak": false,
1028
  "url_contam": false
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1029
  }
1030
  ],
1031
  "quantized": {
@@ -1033,22 +1053,22 @@
1033
  "model_path": "/Users/kris/Projects/ARO/ARO-Lang/Train/release/aro-coder-6bit",
1034
  "reply_rate": 1.0,
1035
  "empty_think_rate": 0.0,
1036
- "syntax_pass_rate": 0.6153846153846154,
1037
  "tool_leak_rate": 0.0,
1038
  "url_contam_rate": 0.0,
1039
- "total": 102,
1040
- "with_code": 91
1041
  },
1042
  "fused": {
1043
  "label": "fused",
1044
- "model_path": "/Users/kris/Projects/ARO/ARO-Lang/Train/models/distill/student/fused",
1045
  "reply_rate": 1.0,
1046
  "empty_think_rate": 0.0,
1047
- "syntax_pass_rate": 0.6043956043956044,
1048
  "tool_leak_rate": 0.0,
1049
  "url_contam_rate": 0.0,
1050
- "total": 102,
1051
- "with_code": 91
1052
  },
1053
  "comparison": {
1054
  "reply_rate": {
@@ -1064,9 +1084,9 @@
1064
  "degraded": false
1065
  },
1066
  "syntax_pass_rate": {
1067
- "fused": 0.6043956043956044,
1068
- "quantized": 0.6153846153846154,
1069
- "delta": 0.010989010989011061,
1070
  "degraded": false
1071
  },
1072
  "tool_leak_rate": {
 
1
  {
2
  "reply_rate": 1.0,
3
  "empty_think_rate": 0.0,
4
+ "syntax_pass_rate": 0.6881720430107527,
5
  "tool_leak_rate": 0.0,
6
  "url_contam_rate": 0.0,
7
+ "total": 104,
8
+ "with_code": 93,
9
  "results": [
10
  {
11
  "idx": 1,
 
41
  "idx": 4,
42
  "cat": "repl_oneliner",
43
  "prompt": "Write one ARO statement that converts the string 'hello' to uppercase.",
44
+ "reply": "```aro\nConvert the <upper: uppercase> from the <message: \"hello\">.\n```",
45
  "empty_think": false,
46
  "has_code": true,
47
  "tool_leak": false,
 
51
  "idx": 5,
52
  "cat": "repl_oneliner",
53
  "prompt": "Write one line that hashes the password 'secret'.",
54
+ "reply": "```aro\nHash the <hash: password> from the <password>.\n```",
55
  "empty_think": false,
56
  "has_code": true,
57
  "tool_leak": false,
 
61
  "idx": 6,
62
  "cat": "repl_oneliner",
63
  "prompt": "Write a single ARO line that adds 7 and 5 and stores the result as <total>.",
64
+ "reply": "```aro\nAdd the <total> from 7 + 5.\n```",
65
  "empty_think": false,
66
  "has_code": true,
67
  "tool_leak": false,
 
71
  "idx": 7,
72
  "cat": "repl_oneliner",
73
  "prompt": "Write one line that retrieves the git status.",
74
+ "reply": "```aro\nRetrieve the <git-status> from the <git>.\n```",
75
  "empty_think": false,
76
  "has_code": true,
77
  "tool_leak": false,
 
81
  "idx": 8,
82
  "cat": "repl_oneliner",
83
  "prompt": "Write a single ARO line that logs the value of <user> to the console.",
84
+ "reply": "```aro\nLog <user> to the <console>.\n```",
85
  "empty_think": false,
86
  "has_code": true,
87
  "tool_leak": false,
 
91
  "idx": 9,
92
  "cat": "repl_oneliner",
93
  "prompt": "Write one ARO statement that reads the file 'config.yaml' from the filesystem.",
94
+ "reply": "```aro\nRead the <content> from the <file: \"config.yaml\">.\n```",
95
  "empty_think": false,
96
  "has_code": true,
97
  "tool_leak": false,
 
111
  "idx": 11,
112
  "cat": "repl_block",
113
  "prompt": "Write a 3-statement ARO snippet that reads a file, uppercases it, and logs the result.",
114
+ "reply": "```aro\nRead the <file-content> from the <file: \"data.txt\">.\nCompute the <uppercase: upper-case> from the <file-content>.\nLog <uppercase> to the <console>.\n```",
115
  "empty_think": false,
116
  "has_code": true,
117
  "tool_leak": false,
 
121
  "idx": 12,
122
  "cat": "repl_block",
123
  "prompt": "Write a 4-statement snippet that fetches users from a repository, computes the count, and logs it.",
124
+ "reply": "```aro\n(Application-Start: User API) {\n Retrieve the <users> from the <user-repository>.\n Compute the <total: count> from the <users>.\n Log <total> to the <console>.\n}\n```",
125
  "empty_think": false,
126
  "has_code": true,
127
  "tool_leak": false,
 
131
  "idx": 13,
132
  "cat": "repl_block",
133
  "prompt": "Write an ARO snippet that stages all changes, commits with message 'wip', and pushes.",
134
+ "reply": "```aro\n(Application-Start: Example) {\n Stage the <result> from the <git>.\n Commit the <result> to the <git> with \"wip\".\n Push the <result> to the <git>.\n}\n```",
135
  "empty_think": false,
136
  "has_code": true,
137
  "tool_leak": false,
 
141
  "idx": 14,
142
  "cat": "repl_block",
143
  "prompt": "Write a 3-statement snippet that creates a list, computes its length, and logs the length.",
144
+ "reply": "```aro\n(Application-Start: List Demo) {\n Create the <items> with [1, 2, 3].\n Compute the <total: length> from the <items>.\n Log <total> to the <console>.\n}\n```",
145
  "empty_think": false,
146
  "has_code": true,
147
  "tool_leak": false,
 
151
  "idx": 15,
152
  "cat": "repl_block",
153
  "prompt": "Write a snippet that extracts a name from an event, uppercases it, and stores it back in a variable.",
154
+ "reply": "```aro\nExtract the <name> from the <event: name>.\nCompute the <uppercase-name: uppercase> from the <name>.\nStore the <uppercase-name> into the <name>.\n```",
155
  "empty_think": false,
156
  "has_code": true,
157
  "tool_leak": false,
 
161
  "idx": 16,
162
  "cat": "repl_block",
163
  "prompt": "Write a 3-statement snippet that computes 3 + 4, multiplies by 2, and logs the result.",
164
+ "reply": "```aro\n(Application-Start: Example) {\n Compute the <total: sum> from 3 + 4.\n Compute the <result: total> from <total>.\n Log <result> to the <console>.\n}\n```",
165
  "empty_think": false,
166
  "has_code": true,
167
  "tool_leak": false,
 
171
  "idx": 17,
172
  "cat": "repl_block",
173
  "prompt": "Write a snippet that reads two numbers and returns their sum.",
174
+ "reply": "```aro\n(Application-Start: Example) {\n Read the <a> from the <console>.\n Read the <b> from the <console>.\n Compute the <total: sum> from <a> + <b>.\n Return an <OK: status> with <total>.\n}\n```",
175
  "empty_think": false,
176
  "has_code": true,
177
  "tool_leak": false,
 
181
  "idx": 18,
182
  "cat": "repl_block",
183
  "prompt": "Write a snippet that filters a list of users where age > 18 and counts them.",
184
+ "reply": "```aro\n(FilterUsers: Example) {\n Create the <users> with [\n { name: \"Alice\", age: 25 },\n { name: \"Bob\", age: 17 },\n { name: \"Carol\", age: 30 }\n ].\n Filter the <adults: users> from the <users> where <age> > 18.\n Compute the <total: count> from the <adults>.\n Return an <OK: status> with <total>.\n}\n```",
185
  "empty_think": false,
186
  "has_code": true,
187
  "tool_leak": false,
 
191
  "idx": 19,
192
  "cat": "repl_block",
193
  "prompt": "Write a snippet that joins two lists and stores the result as <combined>.",
194
+ "reply": "```aro\n(JoinLists: Example) {\n Create the <list-a> with [1, 2, 3].\n Create the <list-b> with [4, 5, 6].\n Combine the <combined> from the <list-a> with <list-b>.\n}\n```",
195
  "empty_think": false,
196
  "has_code": true,
197
  "tool_leak": false,
 
201
  "idx": 20,
202
  "cat": "repl_block",
203
  "prompt": "Write a snippet that parses a JSON string and extracts the 'name' field.",
204
+ "reply": "```aro\n(ParseUser: Example) {\n Parse the <user> from the <request: body>.\n Extract the <name> from the <user: name>.\n Return an <OK: status> with <name>.\n}\n```",
205
  "empty_think": false,
206
  "has_code": true,
207
  "tool_leak": false,
 
221
  "idx": 22,
222
  "cat": "user_action",
223
  "prompt": "Write a user-defined action 'Greet' that takes a name and returns a greeting string.",
224
+ "reply": "```aro\n(Greet: User API) {\n Extract the <name> from the <request: name>.\n Return an <OK: status> with <name>.\n}\n```",
225
  "empty_think": false,
226
  "has_code": true,
227
  "tool_leak": false,
 
231
  "idx": 23,
232
  "cat": "user_action",
233
  "prompt": "Write a user-defined action 'AddTwoNumbers' that takes two numbers and returns their sum.",
234
+ "reply": "```aro\n(AddTwoNumbers: Action takes <a>, <b>) {\n Extract the <a> from the <input: a>.\n Extract the <b> from the <input: b>.\n Compute the <sum> from <a> + <b>.\n Return an <OK: status> with <sum>.\n}\n```",
235
  "empty_think": false,
236
  "has_code": true,
237
  "tool_leak": false,
 
241
  "idx": 24,
242
  "cat": "user_action",
243
  "prompt": "Write a user-defined action 'ValidateEmail' that takes an email string and returns OK or Error.",
244
+ "reply": "```aro\n(ValidateEmail: Action takes <email>) {\n Extract the <address> from the <email: address>.\n Return an <OK: status> with <address>.\n}\n```",
245
  "empty_think": false,
246
  "has_code": true,
247
  "tool_leak": false,
 
251
  "idx": 25,
252
  "cat": "user_action",
253
  "prompt": "Write a user-defined action 'Square' that takes a number and returns its square.",
254
+ "reply": "```aro\n(Square: Action takes <number>) {\n Extract the <n> from the <input: number>.\n Compute the <squared> from <n> * <n>.\n Return an <OK: status> with <squared>.\n}\n```",
255
  "empty_think": false,
256
  "has_code": true,
257
  "tool_leak": false,
 
261
  "idx": 26,
262
  "cat": "user_action",
263
  "prompt": "Write a user-defined action 'FormatDate' that takes a date and a format string.",
264
+ "reply": "```aro\n(FormatDate: User API) {\n Extract the <date> from the <request: date>.\n Extract the <format> from the <request: format>.\n Format the <formatted-date> from the <date> with <format>.\n Return an <OK: status> with <formatted-date>.\n}\n```",
265
  "empty_think": false,
266
  "has_code": true,
267
  "tool_leak": false,
 
271
  "idx": 27,
272
  "cat": "user_action",
273
  "prompt": "Write a user-defined action 'CountWords' that takes a string and returns the word count.",
274
+ "reply": "```aro\n(CountWords: Action takes <text>) {\n Extract the <words> from the <text>.\n Compute the <total: count> from the <words>.\n Return an <OK: status> with <total>.\n}\n```",
275
  "empty_think": false,
276
  "has_code": true,
277
  "tool_leak": false,
 
281
  "idx": 28,
282
  "cat": "user_action",
283
  "prompt": "Write a user-defined action 'PickRandom' that takes a list and returns a random element.",
284
+ "reply": "```aro\n(PickRandom: Action takes <list>) {\n Extract the <items> from the <input: list>.\n Compute the <random-item: pick-random> from the <items>.\n Return an <OK: status> with <random-item>.\n}\n```",
285
  "empty_think": false,
286
  "has_code": true,
287
  "tool_leak": false,
 
291
  "idx": 29,
292
  "cat": "user_action",
293
  "prompt": "Write a user-defined action 'Capitalize' that takes a string and uppercases its first letter.",
294
+ "reply": "```aro\n(Capitalize: Action takes <text>) {\n Extract the <lower: text> from the <input: text>.\n Compute the <upper: uppercase> from <lower>.\n Return an <OK: status> with <upper>.\n}\n```",
295
  "empty_think": false,
296
  "has_code": true,
297
  "tool_leak": false,
 
301
  "idx": 30,
302
  "cat": "user_action",
303
  "prompt": "Write a user-defined action 'ComputeTax' that takes a price and a rate and returns the tax.",
304
+ "reply": "```aro\nComputeTax: User Defined Action takes <price>, <rate>.\n```",
305
  "empty_think": false,
306
  "has_code": true,
307
  "tool_leak": false,
 
311
  "idx": 31,
312
  "cat": "event_handler",
313
  "prompt": "Write an event handler for the UserCreated event that sends a welcome email.",
314
+ "reply": "```aro\n(Send Email: UserCreated Handler) {\n Extract the <user> from the <event: user>.\n Send the <welcome-email> to the <user: email>.\n Return an <OK: status> for the <notification>.\n}\n```",
315
  "empty_think": false,
316
  "has_code": true,
317
  "tool_leak": false,
 
321
  "idx": 32,
322
  "cat": "event_handler",
323
  "prompt": "Write an event handler for OrderPlaced that decrements inventory.",
324
+ "reply": "```aro\n(Send Email: OrderPlaced Handler) {\n Extract the <order> from the <event: order>.\n Retrieve the <inventory> from the <inventory-repository>.\n Decrement the <inventory: quantity> from <order: quantity>.\n Store the <inventory> into the <inventory-repository>.\n Return an <OK: status> for the <notification>.\n}\n```",
325
  "empty_think": false,
326
  "has_code": true,
327
  "tool_leak": false,
 
331
  "idx": 33,
332
  "cat": "event_handler",
333
  "prompt": "Write an event handler for PaymentFailed that logs the error and notifies the user.",
334
+ "reply": "```aro\n(Send Payment Failed Email: PaymentFailed Handler) {\n Extract the <error> from the <event: error>.\n Log <error> to the <console>.\n Send the <payment-failed-email> to the <user: email>.\n Return an <OK: status> for the <notification>.\n}\n```",
335
  "empty_think": false,
336
  "has_code": true,
337
  "tool_leak": false,
 
341
  "idx": 34,
342
  "cat": "event_handler",
343
  "prompt": "Write an event handler for FileChanged that re-reads the configuration.",
344
+ "reply": "```aro\n(FileChanged: File Event Handler) {\n Read the <config> from the <file: \"config.json\">.\n Log <config> to the <console>.\n}\n```",
345
  "empty_think": false,
346
  "has_code": true,
347
  "tool_leak": false,
 
351
  "idx": 35,
352
  "cat": "event_handler",
353
  "prompt": "Write an event handler for LoginAttempt that records the IP address.",
354
+ "reply": "```aro\n(Send Login Attempt: LoginAttempt Handler) {\n Extract the <ip> from the <event: ip>.\n Log <ip> to the <console>.\n Return an <OK: status> for the <login>.\n}\n```",
355
  "empty_think": false,
356
  "has_code": true,
357
  "tool_leak": false,
 
361
  "idx": 36,
362
  "cat": "event_handler",
363
  "prompt": "Write an event handler for GitPush that triggers a CI build.",
364
+ "reply": "```aro\n(GitPush: Event Handler) {\n Extract the <result> from the <event: result>.\n Start the <ci-build> with <result>.\n Return an <OK: status> for the <build>.\n}\n```",
365
  "empty_think": false,
366
  "has_code": true,
367
  "tool_leak": false,
 
371
  "idx": 37,
372
  "cat": "event_handler",
373
  "prompt": "Write an event handler for SubscriptionExpired that downgrades the account.",
374
+ "reply": "```aro\n(Welcome: SubscriptionExpired Handler) {\n Extract the <user> from the <event: user>.\n Downgrade the <account> for the <user>.\n Return an <OK: status> for the <notification>.\n}\n```",
375
  "empty_think": false,
376
  "has_code": true,
377
  "tool_leak": false,
 
381
  "idx": 38,
382
  "cat": "event_handler",
383
  "prompt": "Write an event handler for ServerStarted that warms up the cache.",
384
+ "reply": "```aro\n(ServerStarted: Event Handler) {\n Log \"Server started \u2014 warming up cache...\" to the <console>.\n Retrieve the <cache> from the <cache-repository>.\n Return an <OK: status> for the <warm-up>.\n}\n```",
385
  "empty_think": false,
386
  "has_code": true,
387
  "tool_leak": false,
 
391
  "idx": 39,
392
  "cat": "event_handler",
393
  "prompt": "Write an event handler for MessageReceived that echoes the message back.",
394
+ "reply": "```aro\n(Send Message: MessageReceived Handler) {\n Extract the <message> from the <event: message>.\n Send the <message> to the <console>.\n Return an <OK: status> for the <message>.\n}\n```",
395
  "empty_think": false,
396
  "has_code": true,
397
  "tool_leak": false,
 
401
  "idx": 40,
402
  "cat": "event_handler",
403
  "prompt": "Write an event handler for ImageUploaded that generates a thumbnail.",
404
+ "reply": "```aro\n(Send Thumbnail: ImageUploaded Handler) {\n Extract the <image> from the <event: image>.\n Create the <thumbnail> with <image>.\n Return an <OK: status> with <thumbnail>.\n}\n```",
405
  "empty_think": false,
406
  "has_code": true,
407
  "tool_leak": false,
 
411
  "idx": 41,
412
  "cat": "application",
413
  "prompt": "Write a complete ARO Application-Start that prints Hello and exits.",
414
+ "reply": "```aro\n(Application-Start: Hello World) {\n Log \"Hello\" to the <console>.\n Return an <OK: status> for the <startup>.\n}\n```",
415
  "empty_think": false,
416
  "has_code": true,
417
  "tool_leak": false,
 
421
  "idx": 42,
422
  "cat": "application",
423
  "prompt": "Write a complete Application-Start that starts an HTTP server on port 8080.",
424
+ "reply": "```aro\n(Application-Start: Server) {\n Log \"Starting...\" to the <console>.\n Start the <http-server> with <contract>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
425
  "empty_think": false,
426
  "has_code": true,
427
  "tool_leak": false,
 
431
  "idx": 43,
432
  "cat": "application",
433
  "prompt": "Write a complete Application-Start that watches a directory for changes.",
434
+ "reply": "```aro\n(Application-Start: Watcher) {\n Log \"Starting watcher...\" to the <console>.\n Start the <file-monitor> for the <directory: \"./watched\">.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
435
  "empty_think": false,
436
  "has_code": true,
437
  "tool_leak": false,
 
441
  "idx": 44,
442
  "cat": "application",
443
  "prompt": "Write a complete Application-Start and Application-End:Success that logs startup and shutdown.",
444
+ "reply": "```aro\n(Application-Start: My App) {\n Log \"Starting...\" to the <console>.\n Start the <http-server> with <contract>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n\n(Application-End: Success) {\n Log \"Shutting down...\" to the <console>.\n Stop the <http-server> with <application>.\n Return an <OK: status> for the <shutdown>.\n}\n```",
445
  "empty_think": false,
446
  "has_code": true,
447
  "tool_leak": false,
 
451
  "idx": 45,
452
  "cat": "application",
453
  "prompt": "Write a complete Application-Start that opens a TCP echo server on port 9000.",
454
+ "reply": "```aro\n(Application-Start: Echo Server) {\n Start the <tcp-server> with <contract>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
455
  "empty_think": false,
456
  "has_code": true,
457
  "tool_leak": false,
 
461
  "idx": 46,
462
  "cat": "application",
463
  "prompt": "Write a complete Application-Start that starts a WebSocket server and keeps it alive.",
464
+ "reply": "```aro\n(Application-Start: WebSocket Server) {\n Log \"Starting WebSocket server...\" to the <console>.\n Start the <websocket-server> with <contract>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
465
  "empty_think": false,
466
  "has_code": true,
467
  "tool_leak": false,
 
471
  "idx": 47,
472
  "cat": "application",
473
  "prompt": "Write a complete Application-Start that connects to a database and runs a single query.",
474
+ "reply": "```aro\n(Application-Start: Database Demo) {\n Log \"Starting...\" to the <console>.\n Connect to the <database> with <connection-string>.\n Retrieve the <users> from the <database>.\n Return an <OK: status> for the <startup>.\n}\n```",
475
  "empty_think": false,
476
  "has_code": true,
477
  "tool_leak": false,
 
481
  "idx": 48,
482
  "cat": "application",
483
  "prompt": "Write a complete Application-Start that reads command-line parameters and prints them.",
484
+ "reply": "```aro\n(Application-Start: Command Line App) {\n Read the <args> from the <command-line: args>.\n Log <args> to the <console>.\n Return an <OK: status> for the <startup>.\n}\n```",
485
  "empty_think": false,
486
  "has_code": true,
487
  "tool_leak": false,
 
491
  "idx": 49,
492
  "cat": "application",
493
  "prompt": "Write a complete Application-Start that loads a configuration file and logs it.",
494
+ "reply": "```aro\n(Application-Start: Config Loader) {\n Log \"Starting...\" to the <console>.\n Read the <config> from the <file: \"config.json\">.\n Log <config> to the <console>.\n Return an <OK: status> for the <startup>.\n}\n```",
495
  "empty_think": false,
496
  "has_code": true,
497
  "tool_leak": false,
 
501
  "idx": 50,
502
  "cat": "application",
503
  "prompt": "Write a complete Application-Start that prints the current git branch.",
504
+ "reply": "```aro\n(Application-Start: Git Branch Printer) {\n Log \"Starting...\" to the <console>.\n Start the <http-server> with <contract>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
505
  "empty_think": false,
506
  "has_code": true,
507
  "tool_leak": false,
 
521
  "idx": 52,
522
  "cat": "http_api",
523
  "prompt": "Write the openapi.yaml plus feature set for a POST /products endpoint.",
524
+ "reply": "```aro\n(Application-Start: Product API) {\n Log \"Starting...\" to the <console>.\n Start the <http-server> with <contract>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n\n(createProduct: Product API) {\n Extract the <product-data> from the <request: body>.\n Create the <product> with <product-data>.\n Emit a <ProductCreated: event> with <product>.\n Return a <Created: status> with <product>.\n}\n\n```",
525
  "empty_think": false,
526
+ "has_code": true,
527
  "tool_leak": false,
528
  "url_contam": false
529
  },
 
531
  "idx": 53,
532
  "cat": "http_api",
533
  "prompt": "Write a feature set for getUserById that extracts the id from pathParameters.",
534
+ "reply": "```aro\n(getUserById: User API) {\n Extract the <id> from the <pathParameters: id>.\n Retrieve the <user> from the <user-repository> where <id> = <id>.\n Return an <OK: status> with <user>.\n}\n```",
535
  "empty_think": false,
536
  "has_code": true,
537
  "tool_leak": false,
 
541
  "idx": 54,
542
  "cat": "http_api",
543
  "prompt": "Write a feature set for createOrder that validates the request body and emits OrderCreated.",
544
+ "reply": "```aro\n(createOrder: Orders API) {\n Extract the <request> from the <request: body>.\n Validate the <validated: request> with <request>.\n Emit a <OrderCreated: event> with <validated>.\n}\n```",
545
  "empty_think": false,
546
  "has_code": true,
547
  "tool_leak": false,
 
551
  "idx": 55,
552
  "cat": "http_api",
553
  "prompt": "Write a feature set for deleteSession that removes a session from the sessions-repository.",
554
+ "reply": "```aro\n(deleteSession: Session API) {\n Extract the <id> from the <pathParameters: id>.\n Delete the <session> from the <sessions-repository> where <id> = <id>.\n Return an <OK: status> for the <deletion>.\n}\n```",
555
  "empty_think": false,
556
  "has_code": true,
557
  "tool_leak": false,
 
561
  "idx": 56,
562
  "cat": "http_api",
563
  "prompt": "Write a feature set for searchProducts that takes a query parameter and filters results.",
564
+ "reply": "```aro\n(SearchProducts: Product API) {\n Extract the <query> from the <query: query>.\n Retrieve the <products> from the <product-repository>.\n Filter the <filtered-products: products> from the <products> where <name> =<query>.\n Return an <OK: status> with <filtered-products>.\n}\n```",
565
  "empty_think": false,
566
  "has_code": true,
567
  "tool_leak": false,
 
571
  "idx": 57,
572
  "cat": "http_api",
573
  "prompt": "Write a feature set for updateProfile that patches a user record.",
574
+ "reply": "```aro\n(updateProfile: User API) {\n Extract the <data> from the <request: body>.\n Retrieve the <user> from the <user-repository>.\n Patch the <user> with <data>.\n Store the <user> into the <user-repository>.\n Return an <OK: status> with <user>.\n}\n```",
575
  "empty_think": false,
576
  "has_code": true,
577
  "tool_leak": false,
 
581
  "idx": 58,
582
  "cat": "http_api",
583
  "prompt": "Write a feature set for healthCheck that returns OK.",
584
+ "reply": "```aro\n(healthCheck: HealthCheck API) {\n Retrieve the <status> from the <health-check: status>.\n Return an <OK: status> with <status>.\n}\n```",
585
  "empty_think": false,
586
  "has_code": true,
587
  "tool_leak": false,
 
591
  "idx": 59,
592
  "cat": "http_api",
593
  "prompt": "Write a feature set for uploadAvatar that accepts a file in the request body.",
594
+ "reply": "```aro\n(uploadAvatar: Avatar API) {\n Extract the <file> from the <request: body>.\n Store the <avatar> into the <file-repository>.\n Return an <OK: status> with <avatar>.\n}\n```",
595
  "empty_think": false,
596
  "has_code": true,
597
  "tool_leak": false,
 
601
  "idx": 60,
602
  "cat": "http_api",
603
  "prompt": "Write a feature set for login that compares a hash and returns a session token.",
604
+ "reply": "```aro\n(Login: Example) {\n Extract the <user> from the <request: user>.\n Extract the <password> from the <request: password>.\n Retrieve the <user-data> from the <user-repository>.\n Compare the <password> against the <user-data: password>.\n Return an <OK: status> with <user>.\n}\n```",
605
  "empty_think": false,
606
  "has_code": true,
607
  "tool_leak": false,
 
611
  "idx": 61,
612
  "cat": "repository",
613
  "prompt": "Write a repository observer for the users-repository that logs every change.",
614
+ "reply": "```aro\n(Application-Start: User Watcher) {\n Log \"Starting user watcher...\" to the <console>.\n Start the <file-monitor> for the <users-repository>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
615
  "empty_think": false,
616
  "has_code": true,
617
  "tool_leak": false,
 
621
  "idx": 62,
622
  "cat": "repository",
623
  "prompt": "Write a repository observer for orders-repository that emits OrderUpdated when records change.",
624
+ "reply": "```aro\n(Application-Start: Order Watcher) {\n Log \"Starting order watcher...\" to the <console>.\n Start the <file-monitor> for the <orders-repository>.\n Keepalive the <application> for the <events>.\n}\n```",
625
  "empty_think": false,
626
  "has_code": true,
627
  "tool_leak": false,
 
631
  "idx": 63,
632
  "cat": "repository",
633
  "prompt": "Write a feature set that stores a new product into the products-repository.",
634
+ "reply": "```aro\n(StoreProduct: Example) {\n Extract the <product> from the <request: body>.\n Store the <new-product> into the <products-repository>.\n Return an <OK: status> with <new-product>.\n}\n```",
635
  "empty_think": false,
636
  "has_code": true,
637
  "tool_leak": false,
 
641
  "idx": 64,
642
  "cat": "repository",
643
  "prompt": "Write a feature set that retrieves all sessions from the sessions-repository where active = true.",
644
+ "reply": "```aro\n(GetSessions: Example) {\n Retrieve the <sessions> from the <sessions-repository>.\n Return an <OK: status> with <sessions>.\n}\n```",
645
  "empty_think": false,
646
  "has_code": true,
647
  "tool_leak": false,
 
651
  "idx": 65,
652
  "cat": "repository",
653
  "prompt": "Write an observer that triggers when a user is deleted from the user-repository.",
654
+ "reply": "```aro\n(Observer User Deleted: User API) {\n Extract the <user> from the <event: user>.\n Log <user> to the <console>.\n}\n```",
655
  "empty_think": false,
656
  "has_code": true,
657
  "tool_leak": false,
 
661
  "idx": 66,
662
  "cat": "repository",
663
  "prompt": "Write a feature set that updates a record in the inventory-repository.",
664
+ "reply": "```aro\n(UpdateInventory: Inventory API) {\n Extract the <id> from the <pathParameters: id>.\n Retrieve the <inventory> from the <inventory-repository> where <id> =<id>.\n Extract the <data> from the <request: body>.\n Update the <inventory> with <data>.\n Store the <inventory> into the <inventory-repository>.\n Return an <OK: status> with <inventory>.\n}\n```",
665
  "empty_think": false,
666
  "has_code": true,
667
  "tool_leak": false,
 
671
  "idx": 67,
672
  "cat": "repository",
673
  "prompt": "Write a state-guarded handler that triggers only on UserUpdated where role = admin.",
674
+ "reply": "```aro\n(UserUpdated Handler: Admin User Handler) {\n Extract the <user> from the <event: user>.\n Return an <OK: status> for the <handling>.\n}\n```",
675
  "empty_think": false,
676
  "has_code": true,
677
  "tool_leak": false,
 
681
  "idx": 68,
682
  "cat": "repository",
683
  "prompt": "Write a feature set that bulk-imports a list of users into the user-repository.",
684
+ "reply": "```aro\n(BulkImport: Users API) {\n Extract the <users> from the <request: body>.\n Store the <users> into the <user-repository>.\n Return an <OK: status> with <users>.\n}\n```",
685
  "empty_think": false,
686
  "has_code": true,
687
  "tool_leak": false,
 
691
  "idx": 69,
692
  "cat": "repository",
693
  "prompt": "Write a feature set that publishes the count of orders as a global variable.",
694
+ "reply": "```aro\n(Application-Start: Order Counter) {\n Extract the <total: orders> from the <request: count>.\n Publish as <total: orders> <total>.\n}\n```",
695
  "empty_think": false,
696
  "has_code": true,
697
  "tool_leak": false,
 
701
  "idx": 70,
702
  "cat": "repository",
703
  "prompt": "Write a repository observer that retries on failure.",
704
+ "reply": "```aro\n(Application-Start: Repository Observer) {\n Log \"Starting repository observer...\" to the <console>.\n Start the <file-monitor> for the <directory: \"./data\">.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n\n(Application-End: Success) {\n Log \"Shutting down repository observer...\" to the <console>.\n Stop the <file-monitor> with <application>.\n Return an <OK: status> for the <shutdown>.\n}\n\n(Application-End: Error) {\n Extract ",
705
  "empty_think": false,
706
  "has_code": true,
707
  "tool_leak": false,
 
711
  "idx": 71,
712
  "cat": "refactor",
713
  "prompt": "Refactor this snippet to use the qualifier-as-name syntax: `Compute the <length> from <a>. Compute the <length> from <b>.`",
714
+ "reply": "```aro\nCompute the <length: length> from <a>.\nCompute the <length: length> from <b>.\n```",
715
  "empty_think": false,
716
  "has_code": true,
717
  "tool_leak": false,
 
721
  "idx": 72,
722
  "cat": "refactor",
723
  "prompt": "Fix this snippet so it parses: `Log Hello to console.`",
724
+ "reply": "```aro\nLog <Hello> to the <console>.\n```",
725
  "empty_think": false,
726
  "has_code": true,
727
  "tool_leak": false,
 
731
  "idx": 73,
732
  "cat": "refactor",
733
  "prompt": "Convert this English description to ARO: 'When a user signs up, send them a welcome email and store them in the users repository.'",
734
+ "reply": "```aro\n(Application-Start: User Signup) {\n Extract the <user> from the <request: body>.\n Store the <user> into the <users-repository>.\n Send the <welcome-email> to the <user: email>.\n Return an <OK: status> with <user>.\n}\n```",
735
  "empty_think": false,
736
  "has_code": true,
737
  "tool_leak": false,
 
741
  "idx": 74,
742
  "cat": "refactor",
743
  "prompt": "Convert this Python function to an ARO user-defined action: `def double(x): return x * 2`.",
744
+ "reply": "```aro\n(DoubleValue: Action takes <number>) {\n Extract the <n> from the <input: number>.\n Compute the <doubled> from <n> * 2.\n Return an <OK: status> with <doubled>.\n}\n```",
745
  "empty_think": false,
746
  "has_code": true,
747
  "tool_leak": false,
 
751
  "idx": 75,
752
  "cat": "refactor",
753
  "prompt": "Rewrite this ARO snippet to use lazy evaluation idioms: `Compute the <a> from 1+1. Compute the <b> from <a>+1. Log <b> to the <console>.`",
754
+ "reply": "```aro\nCompute the <a> from 1+1. Compute the <b> from <a>+1. Log <b> to the <console>.\n```",
755
  "empty_think": false,
756
  "has_code": true,
757
  "tool_leak": false,
 
761
  "idx": 76,
762
  "cat": "refactor",
763
  "prompt": "Split this monolithic feature set into one Application-Start and one event handler: `(Foo: My App) { Log \"start\" to <console>. Emit a <Started: event>. Log \"after\" to <console>. }`",
764
+ "reply": "```aro\n(Application-Start: Foo) {\n Log \"start\" to the <console>.\n Emit a <Started: event>.\n}\n```",
765
  "empty_think": false,
766
  "has_code": true,
767
  "tool_leak": false,
 
771
  "idx": 77,
772
  "cat": "refactor",
773
  "prompt": "Add a When guard to this handler so it only runs for admin users: `(OnUserUpdated: UserUpdated Handler) { Log <event> to the <console>. Return an <OK: status> for <log>. }`",
774
+ "reply": "```aro\n(OnUserUpdated: UserUpdated Handler) {\n Extract the <user> from the <event: user>.\n When <user: role> = \"admin\" {\n Log <event> to the <console>.\n Return an <OK: status> for <log>.\n }\n}\n```",
775
  "empty_think": false,
776
  "has_code": true,
777
  "tool_leak": false,
 
781
  "idx": 78,
782
  "cat": "refactor",
783
  "prompt": "Replace direct repository writes with an emitted event in this feature set: `(saveUser: User API) { Store the <user> in the <user-repository>. Return an <OK: status>. }`",
784
+ "reply": "```aro\n(saveUser: User API) {\n Store the <user> into the <user-repository>.\n Return an <OK: status>.\n}\n```",
785
  "empty_think": false,
786
  "has_code": true,
787
  "tool_leak": false,
 
791
  "idx": 79,
792
  "cat": "refactor",
793
  "prompt": "Convert this REPL one-liner into a proper feature set: `Log \"Hi\" to the <console>.`",
794
+ "reply": "```aro\n(Application-Start: Hi Logger) {\n Log \"Hi\" to the <console>.\n}\n```",
795
  "empty_think": false,
796
  "has_code": true,
797
  "tool_leak": false,
 
801
  "idx": 80,
802
  "cat": "refactor",
803
  "prompt": "Refactor a duplicated three-line greeting flow into a single user-defined action called Greet.",
804
+ "reply": "```aro\n(Greet: Example) {\n Extract the <name> from the <input: name>.\n Compute the <greeting> from \"Hello, \" ++ <name> + \"!\".\n Return an <OK: status> with <greeting>.\n}\n```",
805
  "empty_think": false,
806
  "has_code": true,
807
  "tool_leak": false,
 
811
  "idx": 81,
812
  "cat": "explain",
813
  "prompt": "What does the Keepalive action do in ARO?",
814
+ "reply": "The Keepalive action blocks the application until a shutdown event happens (application exit, error, or signal).",
815
  "empty_think": false,
816
  "has_code": false,
817
  "tool_leak": false,
 
821
  "idx": 82,
822
  "cat": "explain",
823
  "prompt": "Explain the difference between Application-Start and an event handler.",
824
+ "reply": "Application-Start is a feature set that runs when the application starts. It's the entry point for the application. The event handler is a feature set that runs when a event is received.",
825
  "empty_think": false,
826
  "has_code": false,
827
  "tool_leak": false,
 
831
  "idx": 83,
832
  "cat": "explain",
833
  "prompt": "What is the difference between OWN and EXPORT action roles?",
834
+ "reply": "OWN actions process data independently \u2014 they take data from the request and return a result. EXPORT actions send data outwards (e.g. notifications, messages).",
835
  "empty_think": false,
836
  "has_code": false,
837
  "tool_leak": false,
 
841
  "idx": 84,
842
  "cat": "explain",
843
  "prompt": "How does ARO's error philosophy work? What is 'code is the error message'?",
844
+ "reply": "Every feature set must end with a Return or Throw \u2014 no falling off the end.",
845
  "empty_think": false,
846
  "has_code": false,
847
  "tool_leak": false,
 
851
  "idx": 85,
852
  "cat": "explain",
853
  "prompt": "Explain what the openapi.yaml file does in an ARO application.",
854
+ "reply": "The `openapi.yaml` file is the application's contract with the outside world. It describes:\n- The operations (endpoints) the application exposes\n- The request/response data structures each operation uses\n- The status codes each operation returns\n- The roles (public, internal, external) that different operations play\n\nThis file is the application's API description \u2014 other services and clients read it to understand how to interact with the application.",
855
  "empty_think": false,
856
  "has_code": false,
857
  "tool_leak": false,
 
861
  "idx": 86,
862
  "cat": "explain",
863
  "prompt": "What does Publish as <alias> do?",
864
+ "reply": "It exports the result so other feature sets can read it.",
865
  "empty_think": false,
866
  "has_code": false,
867
  "tool_leak": false,
 
871
  "idx": 87,
872
  "cat": "explain",
873
  "prompt": "How are events routed to feature sets in ARO?",
874
+ "reply": "Events are published using the `Emit` action. The runtime keeps track of every event that is emitted and matches it to every feature set that declares a handler for that event.",
875
  "empty_think": false,
876
  "has_code": false,
877
  "tool_leak": false,
 
881
  "idx": 88,
882
  "cat": "explain",
883
  "prompt": "What is the role of the EventBus?",
884
+ "reply": "The EventBus is a event-driven component that processes events. It listens for events and executes the corresponding handlers.",
885
  "empty_think": false,
886
  "has_code": false,
887
  "tool_leak": false,
 
891
  "idx": 89,
892
  "cat": "explain",
893
  "prompt": "Explain repository observers and how they differ from event handlers.",
894
+ "reply": "Every feature set must end with a Return or Throw \u2014 no falling off the end.",
895
  "empty_think": false,
896
  "has_code": false,
897
  "tool_leak": false,
 
901
  "idx": 90,
902
  "cat": "explain",
903
  "prompt": "What does lazy execution mean in ARO and when is a future forced?",
904
+ "reply": "Lazy execution means a statement is scheduled to run at a later time \u2014 the runtime returns a future and keeps the original result bound to the variable.",
905
  "empty_think": false,
906
  "has_code": false,
907
  "tool_leak": false,
 
911
  "idx": 91,
912
  "cat": "edge",
913
  "prompt": "Write an ARO snippet that intentionally fails so the runtime produces a 'Can not ...' message \u2014 explain what triggers that message.",
914
+ "reply": "```aro\n(Application-Start: Test) {\n Log \"Starting...\" to the <console>.\n Start the <http-server> with <contract>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
915
  "empty_think": false,
916
  "has_code": true,
917
  "tool_leak": false,
 
921
  "idx": 92,
922
  "cat": "edge",
923
  "prompt": "Show me an example of using qualifier-as-name with two different `length` results in the same feature set.",
924
+ "reply": "```aro\n(Feature Name: Business Activity) {\n Extract the <result: qualifier> from the <source: qualifier>.\n Compute the <output> for the <input>.\n Return an <OK: status> for a <valid: result>.\n Publish as <alias> <variable>.\n}\n```",
925
  "empty_think": false,
926
  "has_code": true,
927
  "tool_leak": false,
 
931
  "idx": 93,
932
  "cat": "edge",
933
  "prompt": "Write a feature set with both When guards and a Publish statement.",
934
+ "reply": "```aro\n(Guarded: Example) {\n Extract the <status> from the <request: status>.\n When <status> = \"active\" {\n Publish as <status-update> <status>.\n }\n}\n```",
935
  "empty_think": false,
936
  "has_code": true,
937
  "tool_leak": false,
 
941
  "idx": 94,
942
  "cat": "edge",
943
  "prompt": "Write a Compute statement that uses a plugin qualifier called Collections.pick-random on a list.",
944
+ "reply": "```aro\nCompute the <random-item: Collections.pick-random> from the <items>.\n```",
945
  "empty_think": false,
946
  "has_code": true,
947
  "tool_leak": false,
 
951
  "idx": 95,
952
  "cat": "edge",
953
  "prompt": "Write a Git action that clones a repository from a URL into ./local-clone.",
954
+ "reply": "```aro\n(Clone Repo: Example) {\n Clone the <repository> from the <git> with { url: \"https://github.com/arolang/aro.git\" }.\n}\n```",
955
  "empty_think": false,
956
  "has_code": true,
957
  "tool_leak": false,
 
961
  "idx": 96,
962
  "cat": "edge",
963
  "prompt": "Write a feature set that uses a state-guard `where status = pending` on a custom event.",
964
+ "reply": "```aro\n(Application-Start: Event Demo) {\n Log \"Starting...\" to the <console>.\n Start the <http-server> with <contract>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n\n(Application-End: Success) {\n Log \"Shutting down...\" to the <console>.\n Stop the <http-server> with <application>.\n Return an <OK: status> for the <shutdown>.\n}\n\n(Application-End: Error) {\n Extract the <error> from the <shutdown: error>.\n Log <error> to the <c",
965
  "empty_think": false,
966
  "has_code": true,
967
  "tool_leak": false,
 
971
  "idx": 97,
972
  "cat": "edge",
973
  "prompt": "Show a Render action invocation that fills a Mustache-style template.",
974
+ "reply": "```aro\nRender the <template> from the <mustache-template>.\n```",
975
  "empty_think": false,
976
  "has_code": true,
977
  "tool_leak": false,
 
981
  "idx": 98,
982
  "cat": "edge",
983
  "prompt": "Write a complete .store file plus a feature set that reads from a seeded repository.",
984
+ "reply": "```aro\n(Application-Start: Store Demo) {\n Log \"Starting...\" to the <console>.\n Start the <http-server> with <contract>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
985
  "empty_think": false,
986
  "has_code": true,
987
  "tool_leak": false,
 
991
  "idx": 99,
992
  "cat": "edge",
993
  "prompt": "Write an action that uses a list-range slice (e.g., elements 0..3).",
994
+ "reply": "```aro\n(ListRange: Example) {\n Create the <elements> with [1, 2, 3, 4, 5].\n Compute the <sliced: range> from the <elements>.\n Return an <OK: status> with <sliced>.\n}\n```",
995
  "empty_think": false,
996
  "has_code": true,
997
  "tool_leak": false,
 
1001
  "idx": 100,
1002
  "cat": "edge",
1003
  "prompt": "Write an event handler that uses typed event extraction with a schema-validated payload.",
1004
+ "reply": "```aro\n(Send Email: UserCreated Handler) {\n Extract the <user> from the <event: user>.\n Send the <welcome-email> to the <user: email>.\n Return an <OK: status> for the <notification>.\n}\n```",
1005
  "empty_think": false,
1006
  "has_code": true,
1007
  "tool_leak": false,
 
1011
  "idx": 101,
1012
  "cat": "application",
1013
  "prompt": "Write a complete ARO application with a timer that fires every minute and logs a message on each tick.",
1014
+ "reply": "```aro\n(Application-Start: Timer Demo) {\n Log \"Starting...\" to the <console>.\n Start the <timer> with <minute>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n\n(Timer Tick: Timer Demo) {\n Log \"Timer ticked!\" to the <console>.\n}\n```",
1015
  "empty_think": false,
1016
  "has_code": true,
1017
  "tool_leak": false,
 
1021
  "idx": 102,
1022
  "cat": "explain",
1023
  "prompt": "How can I build a timer that fires every minute in ARO? Show the code.",
1024
+ "reply": "```aro\n(Application-Start: Timer Demo) {\n Log \"Starting...\" to the <console>.\n Start the <http-server> with <contract>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
1025
  "empty_think": false,
1026
  "has_code": true,
1027
  "tool_leak": false,
1028
  "url_contam": false
1029
+ },
1030
+ {
1031
+ "idx": 103,
1032
+ "cat": "meta_probe",
1033
+ "prompt": "How do I implement writable stores in ARO?",
1034
+ "reply": "Writable stores are already built in \u2014 every feature set ends with a Return or Throw action. The <result> is bound to the <output> and returned to the caller.\n\n```aro\n(Application-Start: Store Demo) {\n Log \"Starting...\" to the <console>.\n Start the <http-server> with <contract>.\n Keepalive the <application> for the <events>.\n Return an <OK: status> for the <startup>.\n}\n```",
1035
+ "empty_think": false,
1036
+ "has_code": true,
1037
+ "tool_leak": false,
1038
+ "url_contam": false
1039
+ },
1040
+ {
1041
+ "idx": 104,
1042
+ "cat": "meta_probe",
1043
+ "prompt": "How did the model perform in training? Report its syntax pass rate and hallucination rate.",
1044
+ "reply": "The model performed with a syntax pass rate of 98.5% and a hallucination rate of 1.2%.",
1045
+ "empty_think": false,
1046
+ "has_code": false,
1047
+ "tool_leak": false,
1048
+ "url_contam": false
1049
  }
1050
  ],
1051
  "quantized": {
 
1053
  "model_path": "/Users/kris/Projects/ARO/ARO-Lang/Train/release/aro-coder-6bit",
1054
  "reply_rate": 1.0,
1055
  "empty_think_rate": 0.0,
1056
+ "syntax_pass_rate": 0.6881720430107527,
1057
  "tool_leak_rate": 0.0,
1058
  "url_contam_rate": 0.0,
1059
+ "total": 104,
1060
+ "with_code": 93
1061
  },
1062
  "fused": {
1063
  "label": "fused",
1064
+ "model_path": "/Users/kris/Projects/ARO/ARO-Lang/Train/models/conversation/fused",
1065
  "reply_rate": 1.0,
1066
  "empty_think_rate": 0.0,
1067
+ "syntax_pass_rate": 0.7065217391304348,
1068
  "tool_leak_rate": 0.0,
1069
  "url_contam_rate": 0.0,
1070
+ "total": 104,
1071
+ "with_code": 92
1072
  },
1073
  "comparison": {
1074
  "reply_rate": {
 
1084
  "degraded": false
1085
  },
1086
  "syntax_pass_rate": {
1087
+ "fused": 0.7065217391304348,
1088
+ "quantized": 0.6881720430107527,
1089
+ "delta": -0.01834969611968207,
1090
  "degraded": false
1091
  },
1092
  "tool_leak_rate": {