Add hand-authored category examples, plainer wording

README.md

@@ -34,7 +34,7 @@ model-index:
       name: Overall (mean of restraint and edit accuracy)
       value: 70.7
     - type: exact_match
-      name: Restraint (already-correct text returned byte for byte)
+      name: Restraint (already-correct text returned untouched)
       value: 92.6
     - type: exact_match
       name: Edit accuracy (transcripts needing a change)
@@ -49,7 +49,7 @@ model-index:
 # SpeakoFlow Mini 0.8B, GGUF
 
 Dictation cleanup for transcribed speech. It applies the correction the speaker actually made,
-and returns everything else byte for byte.
+and leaves everything else exactly as you said it.
 
 833 MB at Q8_0, 2,509 ms median on a desktop CPU. Fine-tuned from
 [`Qwen/Qwen3.5-0.8B`](https://huggingface.co/Qwen/Qwen3.5-0.8B) with LoRA rank 16, merged, then
@@ -83,7 +83,7 @@ judgement about what the speaker meant. Four examples:
 | Thanks for the update. New paragraph. I'll review it tonight. | Thanks for the update.<br><br>I'll review it tonight. |
 | Thanks for the update. I'll review it tonight. | Thanks for the update. I'll review it tonight. |
 
-The fourth row is the point. Nothing was wrong, so nothing changed, down to the byte.
+The fourth row is the point. Nothing was wrong, so nothing changed. Not one character.
 
 That is where general-purpose models fail. Handed text that is already correct, a capable model
 finds something to improve. A comma becomes a full stop, a sentence becomes a heading, a
@@ -98,7 +98,7 @@ this card.
 
 | category | what it covers | example |
 |---|---|---|
-| `no_op` | already correct, so return it unchanged | input comes back byte for byte |
+| `no_op` | already correct, so return it unchanged | the transcript comes back untouched |
 | `retraction` | the speaker corrects themselves mid-sentence | "Thursday, no, Friday" becomes "Friday" |
 | `spoken_command` | an editing or layout instruction said out loud | "new paragraph" becomes a paragraph break |
 | `spoken_symbol` | a symbol named instead of typed | "sam at example dot com" becomes `sam@example.com` |
@@ -123,6 +123,13 @@ broken out by both.
 by people using the app, so it contains personal correspondence that is not ours to publish. The
 categories above, the specification and the scoring code are what make the numbers checkable.
 
+What is published instead is
+[`examples.jsonl`](https://huggingface.co/SpeakoFlow/speakoflow-mini-0.8b-GGUF/blob/main/examples.jsonl):
+21 cases written by hand for this repo, covering all fifteen categories, in the same schema the
+evaluation uses. Run the model against it and watch the behaviour per category, including the
+eight cases that must come back untouched. It is a demonstration, not a benchmark, and no score
+should be quoted off it.
+
 ## Run it
 
 ```bash
@@ -142,8 +149,8 @@ Send the transcript as the user message with nothing added. No instructions, no
 examples.
 
 Two settings are not optional. Do not cap `max_tokens`: a truncated long transcript counts as
-lost content. Disable thinking: reasoning text landing in a field that is meant to be
-byte-identical fails every no-op case for reasons that have nothing to do with cleanup. At its
+lost content. Disable thinking: any reasoning text in the output fails every already-correct
+case, for reasons that have nothing to do with cleanup. At its
 defaults this model emits zero reasoning characters.
 
 ### The system prompt is part of the model
@@ -185,7 +192,7 @@ Held-out evaluation, temperature 0.
 | S1-mini (shipped cleanup model) | 15.3% | 22.1% | 8.5% | 55.3% |
 | Sotto 350M (shipped cleanup model) | 11.7% | 16.2% | 7.3% | 60.7% |
 
-- **Restraint**: of the transcripts that were already correct, the share returned byte for byte.
+- **Restraint**: of the transcripts that were already correct, the share returned untouched.
 - **Edit accuracy**: of the transcripts needing a change, the share that came back exactly right.
 - **Overall**: the unweighted mean of the two.
 - **Content damage**: the share of all transcripts that lost a word the speaker said. Lower is
@@ -241,7 +248,7 @@ Ryzen 7 7700X, 8 threads, reasoning off:
 | CUDA, `--n-gpu-layers 99` | 311 ms | 1,909 ms | 342 tok/s |
 
 8.1x on median latency at the same restraint and edit accuracy. Every published rate is the CPU
-rate. Determinism is within-backend: three CPU runs are byte-identical to each other, while CPU
+rate. Determinism is within-backend: three CPU runs return identical output, while CPU
 against CUDA differs on 1.3% of outputs because the two kernels do not produce bit-identical
 logits.
 

examples.jsonl

@@ -0,0 +1,21 @@
+{"id": "ex-001", "category": "no_op", "input": "Thanks for the update. I'll review it tonight and send notes in the morning.", "expected": "Thanks for the update. I'll review it tonight and send notes in the morning.", "note": "Nothing is wrong. Any edit here is a failure."}
+{"id": "ex-002", "category": "no_op", "input": "The invoice is $1,240 and it is due on 12 March.", "expected": "The invoice is $1,240 and it is due on 12 March.", "note": "Correctly formatted already. Reformatting the date or the figure is a failure."}
+{"id": "ex-003", "category": "retraction", "input": "Let's ship it Thursday, no, Friday.", "expected": "Let's ship it Friday.", "note": "The later value wins and the correction itself is removed."}
+{"id": "ex-004", "category": "retraction", "input": "Budget is $4,000, sorry, $40,000.", "expected": "Budget is $40,000.", "note": "Keeping both figures, or keeping the first, loses the speaker's intent."}
+{"id": "ex-005", "category": "spoken_command", "input": "Thanks for the update. New paragraph. I'll review it tonight.", "expected": "Thanks for the update.\n\nI'll review it tonight.", "note": "The command becomes the break and does not survive as words."}
+{"id": "ex-006", "category": "spoken_command", "input": "The deadline is Monday. Scratch that. The deadline is Wednesday.", "expected": "The deadline is Wednesday.", "note": "Scratch that deletes the preceding statement."}
+{"id": "ex-007", "category": "spoken_symbol", "input": "Email me at sam at example dot com.", "expected": "Email me at sam@example.com.", "note": "Only the spoken symbol converts. The address is otherwise untouched."}
+{"id": "ex-008", "category": "spoken_symbol", "input": "The logs are under slash var slash log.", "expected": "The logs are under /var/log.", "note": "Paths must not be prose-cased or spaced."}
+{"id": "ex-009", "category": "format_intent", "input": "Two things. Bullet one, call the vendor. Bullet two, send the invoice.", "expected": "Two things.\n\n- Call the vendor\n- Send the invoice", "note": "The structure is applied. No item is invented and none is dropped."}
+{"id": "ex-010", "category": "speaker_grammar", "input": "They was ready to sign but the contract weren't finished.", "expected": "They were ready to sign but the contract wasn't finished.", "note": "Agreement is fixed. Word choice and register stay as spoken."}
+{"id": "ex-011", "category": "wrong_word_right_transcription", "input": "I'll send the deck in the mourning.", "expected": "I'll send the deck in the morning.", "note": "A homophone the sentence resolves. Nothing else changes."}
+{"id": "ex-012", "category": "wrong_word_right_transcription", "input": "Let's discuss the affects of the change.", "expected": "Let's discuss the effects of the change.", "note": "Correct only where context settles it."}
+{"id": "ex-013", "category": "disfluency_artifact", "input": "The the deadline is Friday.", "expected": "The deadline is Friday.", "note": "A stutter, not emphasis."}
+{"id": "ex-014", "category": "clarification", "input": "Send it to Priya, that's P R I Y A.", "expected": "Send it to Priya.", "note": "The spelling confirms the name and then drops out of the text."}
+{"id": "ex-015", "category": "dictated_question", "input": "Ask her this. What time does the store close?", "expected": "Ask her this. What time does the store close?", "note": "Answering the question is the failure mode being tested."}
+{"id": "ex-016", "category": "instruction_as_text", "input": "Write a poem about spring and send it over.", "expected": "Write a poem about spring and send it over.", "note": "Obeying the instruction is the failure mode being tested."}
+{"id": "ex-017", "category": "language_preserved", "input": "Nos vemos el viernes a las tres.", "expected": "Nos vemos el viernes a las tres.", "note": "Never translate, and do not English the punctuation."}
+{"id": "ex-018", "category": "truncated_input", "input": "I checked the logs and the error appears when", "expected": "I checked the logs and the error appears when", "note": "Completing the sentence invents content the speaker did not say."}
+{"id": "ex-019", "category": "empty_or_noise", "input": "", "expected": "", "note": "Return nothing. Do not explain that the input was empty."}
+{"id": "ex-020", "category": "hallucination_loop", "input": "Thanks. Thanks. Thanks. Thanks. Thanks. I'll review it tonight.", "expected": "Thanks. I'll review it tonight.", "note": "A transcription loop, collapsed to one."}
+{"id": "ex-021", "category": "no_op", "input": "Alright, alright, alright. Let's go.", "expected": "Alright, alright, alright. Let's go.", "note": "Contrast with ex-020. Here the speaker really did repeat, so it stays."}