Describe model architecture, usage and measured evaluation in public model card

README.md

@@ -3,1770 +3,94 @@ license: cc-by-nc-4.0
 language: [en, fr, nl, de, pl, el, ja]
 base_model: jhu-clsp/mmBERT-base
 datasets: [Antix5/tabular-errors-v1]
-tags: [tabular-data, masked-lm, token-classification, experimental]
+tags: [tabular-data, masked-lm, token-classification, data-cleaning]
 ---
-# TabFix shared-encoder detector and masked corrector
+# TabFix
 
-Experimental budget-limited mmBERT-base joint fine-tuning checkpoint.
+**Schema-aware detection and correction for multilingual tabular data.** TabFix combines deterministic column checks with a shared mmBERT encoder to identify errors and generate cell replacements. It uses table context and the original value to repair cells while preserving valid data.
 
-One shared encoder, 2 residual category BIO/insertion outputs (`text.encoding`, `text.spelling`), and the pretrained MLM vocabulary head. Declared schema rules detect deterministic violations before neural classification. Correction generates a complete cell inside `<replacement>` beside the visible `<original>` in the same cell. Valid copies are training targets too. The MLM head emits replacement tokens followed by `END_EDIT` and visible padding tokens.
-Completed optimizer updates: **3600**. This checkpoint is a research baseline, not a validated automatic cleaning system.
+Supports English, French, Dutch, German, Polish, Greek, and Japanese.
 
-## End-to-end test: 76 held-out tables
+## How it works
 
-The frozen full CSV pipeline repaired **48/67 faulty cells**, with **0/8,600 valid cells harmed**. It applied one incorrect replacement to an already faulty cell: Japanese mojibake corresponding to aluminium was changed to cardboard despite low pseudo-perplexity. Thus low perplexity is not a correctness guarantee. All 18 explicit hard-negative cases were preserved.
+1. **Detect:** schema rules check supported formats, allowed values, required fields, and structured relationships. Neural detection handles residual spelling and encoding errors.
+2. **Correct:** a masked-language-model head generates a replacement using the original cell, its column schema, and neighboring table rows.
+3. **Accept or abstain:** supported schema checks and conditional pseudo-perplexity filter candidates before the CSV runner applies changes.
 
-The panel covers 18 business categories and seven languages, including open-vocabulary residual cases. Detection and perplexity thresholds were held fixed from validation; gold locations and values were never supplied to inference. This is a representative synthetic panel, not the entire test split.
+The original value remains visible during generation:
 
-On the same 55 cases evaluated previously: **30/45 repairs and zero valid cells harmed**, compared with 23/45 repairs and 55 valid cells harmed in the earlier pipeline. This is a whole-pipeline comparison. Remaining gaps include unsupported deterministic contracts, ambiguous relational repairs, missing values and linked field swaps.
+```xml
+<cell column="Date"><original>2026/09/20</original><replacement>[MASK]…</replacement></cell>
+```
+
+Replacement length is variable: an `END_EDIT` token terminates the generated value. Valid-copy examples teach the corrector to leave correct values unchanged.
+
+## Model specification
 
-See [findings](evaluation/v2/FINDINGS.md), [summary](evaluation/v2/summary.json) and [visual report](evaluation/v2/review.html). Evaluated model revision: `913f3efbf4f0557b1ad5ec1693730d8b7535ac58`; weights are unchanged by this documentation update.
+| Property | Value |
+| --- | --- |
+| Backbone | [jhu-clsp/mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base) |
+| Parameters | 307.8 million |
+| Neural detection categories | `text.encoding`, `text.spelling` |
+| Detection heads | Per-category BIO classification and insertion detection |
+| Correction head | Shared encoder with an MLM vocabulary head |
+| Decoding | Iterative masked-token completion of a whole-cell replacement |
+| Confidence filter | Conditional pseudo-perplexity; threshold 1.0092596407844958 |
 
-## Use and resume
+The pipeline retains 18 business error categories for rules, reporting, and category controls. These are distinct from the two neural detection categories. Deterministic correction directly handles supported unambiguous whitespace, case, and boolean normalization; detecting a rule violation does not always determine its repair.
 
-The custom model must be loaded through the accompanying self-contained `train_tabfix.py`, not `AutoModelForMaskedLM` directly. Its weights include detection heads and have a custom key layout. `load_checkpoint(path, device)` returns the model and tokenizer; `Consumer`, `predict_edits` and `generate_repair` implement input alignment and suggestions. The command `uv run train_tabfix.py --resume Antix5/tabfix-preview --verify-only` verifies checksums and loads the model. Resume training with `--resume ... --repo-id ... --train-seconds 1200`; hardware timeouts must be set separately.
+## End-to-end evaluation
 
-The recoverable state includes model, tokenizer, optimizer and random generators. `training_state.pt` uses PyTorch serialization: load only from a trusted checkpoint you control. No API keys are saved. The original training script and resolved package pins are embedded in the repository.
+Evaluated on **76 held-out test tables**, covering seven languages and all 18 business error categories. The complete CSV pipeline processes every cell; reference answers and error locations are withheld from inference. Acceptance thresholds were fixed using validation before this evaluation.
 
-## Scope
+| Measure | Result |
+| --- | ---: |
+| Faulty cells repaired exactly | **48 / 67 (71.6%)** |
+| Correct changes among applied changes | **48 / 49 (98.0%)** |
+| Valid cells changed incorrectly | **0 / 8,600** |
+| Faulty cells left unchanged | 18 / 67 |
+| Faulty cells changed to an incorrect value | 1 / 67 |
+| Explicit hard-negative tables preserved | 18 / 18 |
 
-Canonical labels refer to decoded Unicode cell coordinates. Prediction is token-granular. Disabled categories are filtered before edits; overlapping proposals and linked structural repairs require abstention/group handling. Suggestions are not automatically applied. Missing values need not have recoverable intended contents. Real-source unlabelled cells are excluded from negative supervision.
+Of the 49 applied changes, seven were deterministic and 42 were generated by the model. These results measure the combined pipeline, rather than the neural model alone.
 
-Dataset revision: `602b8a454ca448713d8a5658650d4feaad8d579c`. Train/validation formats are held out; test data was not consumed. This dataset is mostly synthetic. The detector/correction task weight is `ln(2)/ln(256000) = 0.055661`; valid-copy correction examples are sampled with probability 0.30. The run records periodic held-out validation losses for both tasks in `validation_history`, then a final validation report. These are development diagnostics, not deployment precision or end-to-end repair accuracy. Conditional pseudo-perplexity is scored with each response token masked in turn, original visible; an acceptance threshold is fitted on the reported validation sample. Its sample size and observed errors limit its reliability. No latency guarantee is claimed. See `metrics.json` for sample sizes, loss, operating-point diagnostics and exact-repair outcomes.
+This is a small, deliberately balanced diagnostic panel, not a prevalence-weighted benchmark or an evaluation of the entire test split. Category and language subgroups are small. The zero observed changes to valid cells is a sample result, not a guarantee.
 
-## Validation results
+See the [evaluation summary](evaluation/v2/summary.json) for category breakdowns and the [case selection](evaluation/v2/selection.json) for reproducibility.
 
-Selected step: 3600. Stop reason: early_stopping.
+## Using the model
 
-```json
-{
-  "sample_policy": "seeded language x positive/clean round-robin validation sample",
-  "validation_examples": 28,
-  "detection_loss": 0.01834406687046502,
-  "correction_examples": 28,
-  "correction_loss": 0.03511501088584217,
-  "token_category_confusion_tp_fp_fn_tn": {
-    "0.5": [
-      6,
-      0,
-      0,
-      3504
-    ],
-    "0.8": [
-      6,
-      0,
-      0,
-      3504
-    ],
-    "0.9": [
-      6,
-      0,
-      0,
-      3504
-    ],
-    "0.95": [
-      6,
-      0,
-      0,
-      3504
-    ],
-    "0.99": [
-      6,
-      0,
-      0,
-      3504
-    ]
-  },
-  "gold_cell_repairs": [
-    {
-      "id": "d55a35e0ae58a7236ecca68f66ba9e9d44f64eb7ad3ef4287c5da4b1eea735eb",
-      "category": "copy",
-      "target": "kg",
-      "prediction": "kg",
-      "exact": true,
-      "perplexity": 1.0000185370089685
-    },
-    {
-      "id": "16692fd42ba9001286947671f8c57463e75a68e6a083756675cddf256ea3258c",
-      "category": "text.invisible",
-      "target": "26,03",
-      "prediction": "26,03",
-      "exact": true,
-      "perplexity": 1.000349410993324
-    },
-    {
-      "id": "3b0fb29968d32172d06df90b66dad0b4e23113bc76e26edc5f41f3272ee7631f",
-      "category": "copy",
-      "target": "9",
-      "prediction": "9",
-      "exact": true,
-      "perplexity": 1.0015306218806486
-    },
-    {
-      "id": "8775404f74728fc546c80f1711f1113bb7dedae1edc1c7aa03225027ab2dc058",
-      "category": "copy",
-      "target": "132",
-      "prediction": "132",
-      "exact": true,
-      "perplexity": 1.0974386629257948
-    },
-    {
-      "id": "309d7716a2b94d9261e3d10a29d7207bd07f2067a322c5e4fa7e6749d40ada2d",
-      "category": "copy",
-      "target": "137",
-      "prediction": "137",
-      "exact": true,
-      "perplexity": 1.1483397911536126
-    },
-    {
-      "id": "98659accb59c446ec81ba33b755c40624b42bdb7e5afc0ec9cecb6f3ed32361e",
-      "category": "format.case",
-      "target": "RC-000024",
-      "prediction": "RC-000024",
-      "exact": true,
-      "perplexity": 1.0008599332774821
-    },
-    {
-      "id": "15cd8ffbca06b9b35abd39c981e5d77543f9bc765b0bb3d142235be8a28f70a0",
-      "category": "copy",
-      "target": "RC-000039",
-      "prediction": "RC-000039",
-      "exact": true,
-      "perplexity": 1.0011891985193202
-    },
-    {
-      "id": "c3b152c04a5b1170a1aefcb463d0bb00c36fae800aaee751c606fd781c0ebe1e",
-      "category": "text.spelling",
-      "target": "Échange linguistique",
-      "prediction": "Échange linguistique",
-      "exact": true,
-      "perplexity": 1.000352948138701
-    },
-    {
-      "id": "a2017e565e0328f1e33660cb03501ac8919f431b4b5e974ee3a15e4968753a60",
-      "category": "copy",
-      "target": "河川拠点から東倉庫",
-      "prediction": "河川拠点から東倉庫",
-      "exact": true,
-      "perplexity": 1.000208767100492
-    },
-    {
-      "id": "0182b13075f4332b76b2233f3bae4c52326057cadf0ea31a03d915316f4534e4",
-      "category": "schema.type",
-      "target": "52.00",
-      "prediction": "52.00",
-      "exact": true,
-      "perplexity": 1.0000143843922722
-    },
-    {
-      "id": "24847e0afe2df215f39333f839a4eea1053be89484268fc87b71cbb2c1cf74b9",
-      "category": "copy",
-      "target": "105",
-      "prediction": "104",
-      "exact": false,
-      "perplexity": 1.0284562214621615
-    },
-    {
-      "id": "a1ea4dda06275d912c5a857e04714fd9a4d59d489431ba70f2209151b6e6a551",
-      "category": "text.invisible",
-      "target": "4",
-      "prediction": "4",
-      "exact": true,
-      "perplexity": 1.000054537353746
-    },
-    {
-      "id": "f9f6ad70c2fcd6edf84505032104aac29631806ade8bd905eb6d223547fefa12",
-      "category": "copy",
-      "target": "Konto warsztatu",
-      "prediction": "Konto warsztatu",
-      "exact": true,
-      "perplexity": 1.0000307319084698
-    },
-    {
-      "id": "bb8ad536e68892ad5c3dc00cce1fe5722110ffcdb34f471aa5669db8c5fa3b1c",
-      "category": "schema.type",
-      "target": "4,19",
-      "prediction": "4,19",
-      "exact": true,
-      "perplexity": 1.0000092029202488
-    },
-    {
-      "id": "7bc66172b9d0ef4c9d1ceec538d042a1fc9ea35aefd96297310e52b79f5cd64b",
-      "category": "copy",
-      "target": "Salatsamen",
-      "prediction": "Salatsamen",
-      "exact": true,
-      "perplexity": 1.000030695377152
-    },
-    {
-      "id": "d491893e4413819baf49f50c46c954065aa70f6355e550929acfe64ce9639fd5",
-      "category": "text.spelling",
-      "target": "Erbsensamen",
-      "prediction": "Erbsensamen",
-      "exact": true,
-      "perplexity": 1.0000587926331692
-    },
-    {
-      "id": "d288e998948e5e51e89175237dc4ecdf3aa5d05fccac1e905307b288bab7e114",
-      "category": "copy",
-      "target": "13",
-      "prediction": "13",
-      "exact": true,
-      "perplexity": 1.0006689992697302
-    },
-    {
-      "id": "3186c2b31f593fab93254c01992c49379e669b73b5e57de33cd0f723e9cc399f",
-      "category": "text.spelling",
-      "target": "Σπόροι αρακά",
-      "prediction": "Σπόροι αρακά",
-      "exact": true,
-      "perplexity": 1.0001046579537995
-    },
-    {
-      "id": "b9f2f18d19b82a3f37b76e8b7f67bf1618289d5b5bce4567a016dd7c3f5c867d",
-      "category": "copy",
-      "target": "2026-02-08",
-      "prediction": "2026-02-08",
-      "exact": true,
-      "perplexity": 1.0044930397206753
-    },
-    {
-      "id": "18d9c8ef520ab257fd319d50d94ed5bbfc14e83ffc234cdbdd6fc5a6a2026c0b-open",
-      "category": "text.spelling",
-      "target": "Bean seeds",
-      "prediction": "Bean seeds",
-      "exact": true,
-      "perplexity": 1.0037579276335722
-    },
-    {
-      "id": "342f02a3a4f225b5002c9fd250afd4b1dc791884bd5c75249a63c2f10301ac0d-open-clean",
-      "category": "copy",
-      "target": "172",
-      "prediction": "172",
-      "exact": true,
-      "perplexity": 1.0092596407844958
-    },
-    {
-      "id": "7f16d555e641b36102a4b9c2cec0d3a1951522420473c5f8988026d326fbb9b9-open",
-      "category": "text.encoding",
-      "target": "Luma 東京",
-      "prediction": "Luma 東京",
-      "exact": true,
-      "perplexity": 1.0000134706127768
-    },
-    {
-      "id": "1ec1e5aabd33e35963b76c5559bada0a21757e70f3f8cbb46d12a428d955f415",
-      "category": "copy",
-      "target": "修理講座",
-      "prediction": "修理講座",
-      "exact": true,
-      "perplexity": 1.000008384385619
-    },
-    {
-      "id": "6dde22c81981d5d4fce090931f998ad995c3d3db0b5f8d8ce24a41b3c473e19d",
-      "category": "format.case",
-      "target": "RC-000048",
-      "prediction": "RC-000048",
-      "exact": true,
-      "perplexity": 1.00041237012412
-    },
-    {
-      "id": "77867c4faa620f772ecc9f148d9601d178311ca1ca653238fe509bc283043f5c",
-      "category": "copy",
-      "target": "  \"Controleer de bijgevoegde kaart voordat Slazaden wordt verplaatst; post 5 behandelt dit dossier.\"\n26  ",
-      "prediction": "  \"Controleer de bijgevoegde kaart voordat Slazaden wordt verplaatst; post 5 behandelt dit dossier.\"\n26  ",
-      "exact": true,
-      "perplexity": 1.0400289977685477
-    },
-    {
-      "id": "b35415ea92e9b8153e01186a486f9aa92bcb200e944a530e139838664215e79c",
-      "category": "format.number",
-      "target": "82,54",
-      "prediction": "82,54",
-      "exact": true,
-      "perplexity": 1.000831890641031
-    },
-    {
-      "id": "91f076346d5d8984a52a7977b6ffad67c2081d1ec611f6165835759299f2cb83",
-      "category": "copy",
-      "target": "4",
-      "prediction": "4",
-      "exact": true,
-      "perplexity": 1.0000392788962549
-    },
-    {
-      "id": "508c51bafcd810febf19f11c7464262a48f93ec76fe56b581dcee8917997c77d-open",
-      "category": "text.spelling",
-      "target": "Warsztat naprawczy",
-      "prediction": "Warsztat naprawczy",
-      "exact": true,
-      "perplexity": 1.0002412024957317
-    }
-  ],
-  "perplexity_calibration": {
-    "threshold": 1.0092596407844958,
-    "scored": 28,
-    "accepted": 24,
-    "observed_errors_at_threshold": 0,
-    "policy": "largest validation prefix with zero observed exact-match errors; not a statistical guarantee"
-  },
-  "limitations": "Token diagnostics, not cell/span calibration; limited gold-span repair sample; test untouched."
-}
+Download the repository, install the pinned runtime dependencies, and run the included CSV interface:
+
+```bash
+hf download Antix5/tabfix-preview --local-dir tabfix
+uv venv --python 3.12
+uv pip install torch==2.13.0 transformers==5.16.1 huggingface-hub==1.31.0 \
+  pyarrow==25.0.1 safetensors==0.8.0 tokenizers==0.23.2
+uv run --no-project tabfix/test_csv.py input.csv \
+  --checkpoint tabfix --schema schema.xml --output output
 ```
 
-## Licensing
+The runner produces a corrected CSV, an audit of proposed and applied changes, and an HTML review. Supply the table's column rules in the XML schema; see [evaluated examples](tree/main/evaluation/v2/cases) for input CSVs and matching schemas. No clean reference is required for inference.
 
-TabFix fine-tuning contributions are CC BY-NC 4.0; commercial use requires the owner's permission. The underlying mmBERT material remains under its MIT terms and attribution, attributed in BASE_MODEL_CARD.md for pretrained checkpoints. Dataset source terms and USDA attribution remain documented in the dataset card.
+The checkpoint has custom detection heads. Load it through the bundled `train_tabfix.py` rather than directly through `AutoModelForMaskedLM`. Its `load_checkpoint(path, device)` function returns the model, tokenizer, and manifest and verifies checkpoint checksums.
 
-## Training controls
+## Training
 
-The learning rate warms up for 100 updates, then follows cosine decay to 10% of its peak over a fixed 5000-update horizon. A fixed, stratified validation panel of up to 28 records is evaluated before training and every 300 updates or 600 seconds. Selection minimizes detection loss plus 0.055661 times correction loss (equal averages of fully and partially masked views). Training stops after 4 checks without an improvement of at least 0.001. Final publication restores the best complete checkpoint; recoverable latest and best revisions are retained as tags in this repository. These controls reduce risk; they do not guarantee generalization. The test split is not used for checkpoint selection.
+The model was jointly fine-tuned from mmBERT-base on **27,948 training records** from [tabular-errors-v1](https://huggingface.co/datasets/Antix5/tabular-errors-v1), a predominantly synthetic corpus. The dataset revision is `602b8a454ca448713d8a5658650d4feaad8d579c`.
 
-## Recorded run results
+The objective combines detection loss with correction loss weighted by `ln(2) / ln(256000) ≈ 0.055661`. Valid-copy tasks comprise 30% of correction sampling. Training uses learning-rate warmup, cosine decay, periodic held-out validation, and early stopping. Checkpoint selection uses a fixed 28-record validation panel; the published weights are from update 3,600. The test split is excluded from checkpoint selection.
 
-```json
-{
-  "tiny_smoke": false,
-  "dataset_revision": "602b8a454ca448713d8a5658650d4feaad8d579c",
-  "train_records": 27948,
-  "correction_eligible_records": 27805,
-  "correction_request_version": 4,
-  "batch_size": 4,
-  "start_step": 0,
-  "device": "cuda",
-  "parameters": 307794699,
-  "validation_history": [
-    {
-      "step": 0,
-      "detection_loss": 1.5472150828157152,
-      "correction_loss": 6.555269168309259,
-      "correction_full_mask_loss": 7.438174559519841,
-      "correction_partial_mask_loss": 5.672363777098676,
-      "selection_score": 1.9120902847241386,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        1,
-        234,
-        69
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 300,
-      "detection_loss": 0.06014835045139729,
-      "correction_loss": 0.6461185749559752,
-      "correction_full_mask_loss": 0.7436635444509403,
-      "correction_partial_mask_loss": 0.5485736054610103,
-      "selection_score": 0.09611218953119663,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        69,
-        0,
-        1
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 600,
-      "detection_loss": 0.0051202295918691175,
-      "correction_loss": 0.5691586527824746,
-      "correction_full_mask_loss": 0.6588184935826575,
-      "correction_partial_mask_loss": 0.47949881198229166,
-      "selection_score": 0.03680037468124225,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        69,
-        0,
-        1
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 900,
-      "detection_loss": 0.0011060185287793242,
-      "correction_loss": 0.5639237430226995,
-      "correction_full_mask_loss": 0.6801767638384123,
-      "correction_partial_mask_loss": 0.4476707222069868,
-      "selection_score": 0.03249478141758556,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        70,
-        0,
-        0
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 1200,
-      "detection_loss": 0.05877711018732563,
-      "correction_loss": 0.5446714895918796,
-      "correction_full_mask_loss": 0.6555208008509138,
-      "correction_partial_mask_loss": 0.4338221783328453,
-      "selection_score": 0.08909426645290995,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        69,
-        0,
-        1
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 1500,
-      "detection_loss": 0.0011259028613039845,
-      "correction_loss": 0.5327788614510452,
-      "correction_full_mask_loss": 0.6256758404323102,
-      "correction_partial_mask_loss": 0.43988188246978027,
-      "selection_score": 0.030781099261824835,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        70,
-        0,
-        0
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 1800,
-      "detection_loss": 0.01747405317801817,
-      "correction_loss": 0.5289730659742592,
-      "correction_full_mask_loss": 0.6006959225230205,
-      "correction_partial_mask_loss": 0.45725020942549793,
-      "selection_score": 0.04691741382361263,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        69,
-        0,
-        1
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 2100,
-      "detection_loss": 0.012940295429871185,
-      "correction_loss": 0.49840453000904117,
-      "correction_full_mask_loss": 0.5877918274218246,
-      "correction_partial_mask_loss": 0.4090172325962577,
-      "selection_score": 0.04068216976788957,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        69,
-        0,
-        1
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 2400,
-      "detection_loss": 0.0006650434716805519,
-      "correction_loss": 0.3986833988373917,
-      "correction_full_mask_loss": 0.4582971079234295,
-      "correction_partial_mask_loss": 0.3390696897513539,
-      "selection_score": 0.022856303954406,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        70,
-        0,
-        0
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 2700,
-      "detection_loss": 0.005633368319130179,
-      "correction_loss": 0.517131781694795,
-      "correction_full_mask_loss": 0.6256272318727008,
-      "correction_partial_mask_loss": 0.4086363315168893,
-      "selection_score": 0.03441762696885055,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        69,
-        0,
-        1
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 3000,
-      "detection_loss": 0.0001724759085846017,
-      "correction_loss": 0.5180134402396173,
-      "correction_full_mask_loss": 0.6309543849382897,
-      "correction_partial_mask_loss": 0.4050724955409449,
-      "selection_score": 0.02900580887261559,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        70,
-        0,
-        0
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 3300,
-      "detection_loss": 0.007816463095001122,
-      "correction_loss": 0.4535625426051334,
-      "correction_full_mask_loss": 0.5581409413015745,
-      "correction_partial_mask_loss": 0.3489841439086923,
-      "selection_score": 0.03306237139594587,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        69,
-        0,
-        1
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 3600,
-      "detection_loss": 3.5733819637633655e-05,
-      "correction_loss": 0.3458413546207081,
-      "correction_full_mask_loss": 0.4218773429547932,
-      "correction_partial_mask_loss": 0.26980536628662305,
-      "selection_score": 0.019285734217113347,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        70,
-        0,
-        0
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 3900,
-      "detection_loss": 2.3894811834322483e-05,
-      "correction_loss": 0.3426973419940286,
-      "correction_full_mask_loss": 0.4295682630474158,
-      "correction_partial_mask_loss": 0.25582642094064145,
-      "selection_score": 0.01909889518833329,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        70,
-        0,
-        0
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 4200,
-      "detection_loss": 0.0059675526542507995,
-      "correction_loss": 0.3332285964459076,
-      "correction_full_mask_loss": 0.43784401744393076,
-      "correction_partial_mask_loss": 0.22861317544788443,
-      "selection_score": 0.024515509769064567,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        69,
-        0,
-        1
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 4500,
-      "detection_loss": 0.0035310874307177592,
-      "correction_loss": 0.3053514478717175,
-      "correction_full_mask_loss": 0.4151939125343653,
-      "correction_partial_mask_loss": 0.19550898320906968,
-      "selection_score": 0.02052736452241171,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        69,
-        0,
-        1
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    },
-    {
-      "step": 4800,
-      "detection_loss": 0.0016974807663327088,
-      "correction_loss": 0.336344982334732,
-      "correction_full_mask_loss": 0.4563249572901957,
-      "correction_partial_mask_loss": 0.2163650073792683,
-      "selection_score": 0.02041890016030147,
-      "examples": 28,
-      "correction_edits": 52,
-      "correction_category_counts": {
-        "copy": 10,
-        "text.spelling": 7,
-        "format.boolean": 7,
-        "format.date": 3,
-        "text.encoding": 5,
-        "schema.category": 3,
-        "schema.identifier": 4,
-        "schema.type": 2,
-        "consistency.dependency": 6,
-        "consistency.temporal": 3,
-        "consistency.unit": 1,
-        "format.case": 1
-      },
-      "token_and_gap_tp_fp_fn_at_0.9": [
-        69,
-        0,
-        1
-      ],
-      "sample_ids": [
-        "7cc2b2069f314821114a2a2b09543c680923083980692c6a95b3371b316390d9-open-clean",
-        "2c30950b5459afe6967673e8b23c222adab49967bf80e72af0ef1d4fb96296ca-open",
-        "bd4bba00c41c73249e4fd9fe97876df42dbaefbe6aabeec4430d2acb63926318-open",
-        "0121e99febc488b49db6e39b07ef71255f5a94d9ef0292f7a34a705e59271bbe-open",
-        "4d74e65f621bbb1dfc2848cdb8aefbd32befa5042d8482c3340de2d9833c06ec-open",
-        "d120e4d54be4c9692441a06d9cc4bf1db02bd205c6a317a1a7ac4548c09666cc-open",
-        "d6b7a9c8fc0552cfcf0d26ce0ce1be2663d5aa86d80527fc77c4ccc02a036984-open",
-        "05f04c447b2d5e98f170add5fcdde123cb8db18946eb5e21ea8573fdfdd66ffe-open",
-        "b68fd4277af10beef7958e8d2f6b31805d635db888e60b38ae2306bf138ea584-open",
-        "c2710a557bc21c7f5d4c38d580e9d54042923e32d22848cc3f025caa864aa51c-open",
-        "da8d1a36641b16437fd72e9db23a60fbb6f185acdceec0f217491c232d3602b5-open-clean",
-        "55e92ca80f64845e208d3cda2f1e8fc3cb620ca57c66fce22d9bb9a86321af4a-open",
-        "53820fc3b14267390c15dfad0a80f83c9d0ed59c95e40c96c2283e4f3b167e09-open",
-        "d5f74ad40dca51caabef5cfcfc1845e3b40fcd71608304e4036786b6e27bfad6-open",
-        "6481cc1b3a6148ca5e8df1783aa3c15bd97b4224ea44ec304e2448bd613c3a07",
-        "8a1d338e2b1a352eac2c0834432588239c231b43df14e68402999151d62a70d8",
-        "04434b82c7685003521e12204c5af32606d402dd454be5d270c2b39c5a3c2b55",
-        "7aeaf4538d564e7bfc4a20cac6b97e42655170cd4ac40c98db0d24ff056ba388",
-        "9d7d44d25b2bd975829249dc3cd673c9a972f47ad8b1fa1bf0206f527d27934c",
-        "b01405d247c32e01b05e977fe963ca8ab440e1559f1c7f8adf0a99928d29b997",
-        "89f4e9132a30351d7836acfa80b56de26924f82d1b2403e48f1c265b4053ab99",
-        "fbe61f1cc2096a02443b68b5dbf6da5694e8cb60ec5fb445ca5bc29fc147b0e9",
-        "4dee644acbc9859035ebc82ce0d74ac45065b429f74ec833d9ae5dda3daf02a0",
-        "86e7d7c56f4e30ae4e63da4327d92c7a9ff2e18d47611dcba9e794b11a222f93",
-        "d515d8316e03100e37bb3be520d62c30463d7d30a235e0921c7ff2d88921c714",
-        "b4a926b6d1c343e53f4fd04dc327fb0fd86d3f4532b964fea22340ef73ca68de",
-        "3e8dff1e9e6db8d09c388b141194864e22c347550db958abeae4500015fa8ecb",
-        "522ef97370ddbad950ea41bb8000b46488a7db7a05af8ee602117f2c0e2c400f"
-      ]
-    }
-  ],
-  "selection": {
-    "best_score": 0.019285734217113347,
-    "best_step": 3600,
-    "bad_checks": 0
-  },
-  "peak_gpu_bytes": 9911241728,
-  "step": 3600,
-  "learning_rate": 2.0746295961673375e-06,
-  "last_detection_loss": 8.09200355433859e-06,
-  "last_correction_loss": 2.6780951884575188e-05,
-  "training_elapsed_seconds": 8136.125818869001,
-  "exclusions": {},
-  "mean_detection_loss": 4.638133941625711e-06,
-  "mean_correction_loss": 0.03080463317783142,
-  "loss_interval_updates": 10,
-  "stop_reason": "early_stopping",
-  "last_training_step": 4800,
-  "selected_step": 3600,
-  "selection_at_stop": {
-    "best_score": 0.019285734217113347,
-    "best_step": 3600,
-    "bad_checks": 4
-  },
-  "validation": {
-    "sample_policy": "seeded language x positive/clean round-robin validation sample",
-    "validation_examples": 28,
-    "detection_loss": 0.01834406687046502,
-    "correction_examples": 28,
-    "correction_loss": 0.03511501088584217,
-    "token_category_confusion_tp_fp_fn_tn": {
-      "0.5": [
-        6,
-        0,
-        0,
-        3504
-      ],
-      "0.8": [
-        6,
-        0,
-        0,
-        3504
-      ],
-      "0.9": [
-        6,
-        0,
-        0,
-        3504
-      ],
-      "0.95": [
-        6,
-        0,
-        0,
-        3504
-      ],
-      "0.99": [
-        6,
-        0,
-        0,
-        3504
-      ]
-    },
-    "gold_cell_repairs": [
-      {
-        "id": "d55a35e0ae58a7236ecca68f66ba9e9d44f64eb7ad3ef4287c5da4b1eea735eb",
-        "category": "copy",
-        "target": "kg",
-        "prediction": "kg",
-        "exact": true,
-        "perplexity": 1.0000185370089685
-      },
-      {
-        "id": "16692fd42ba9001286947671f8c57463e75a68e6a083756675cddf256ea3258c",
-        "category": "text.invisible",
-        "target": "26,03",
-        "prediction": "26,03",
-        "exact": true,
-        "perplexity": 1.000349410993324
-      },
-      {
-        "id": "3b0fb29968d32172d06df90b66dad0b4e23113bc76e26edc5f41f3272ee7631f",
-        "category": "copy",
-        "target": "9",
-        "prediction": "9",
-        "exact": true,
-        "perplexity": 1.0015306218806486
-      },
-      {
-        "id": "8775404f74728fc546c80f1711f1113bb7dedae1edc1c7aa03225027ab2dc058",
-        "category": "copy",
-        "target": "132",
-        "prediction": "132",
-        "exact": true,
-        "perplexity": 1.0974386629257948
-      },
-      {
-        "id": "309d7716a2b94d9261e3d10a29d7207bd07f2067a322c5e4fa7e6749d40ada2d",
-        "category": "copy",
-        "target": "137",
-        "prediction": "137",
-        "exact": true,
-        "perplexity": 1.1483397911536126
-      },
-      {
-        "id": "98659accb59c446ec81ba33b755c40624b42bdb7e5afc0ec9cecb6f3ed32361e",
-        "category": "format.case",
-        "target": "RC-000024",
-        "prediction": "RC-000024",
-        "exact": true,
-        "perplexity": 1.0008599332774821
-      },
-      {
-        "id": "15cd8ffbca06b9b35abd39c981e5d77543f9bc765b0bb3d142235be8a28f70a0",
-        "category": "copy",
-        "target": "RC-000039",
-        "prediction": "RC-000039",
-        "exact": true,
-        "perplexity": 1.0011891985193202
-      },
-      {
-        "id": "c3b152c04a5b1170a1aefcb463d0bb00c36fae800aaee751c606fd781c0ebe1e",
-        "category": "text.spelling",
-        "target": "Échange linguistique",
-        "prediction": "Échange linguistique",
-        "exact": true,
-        "perplexity": 1.000352948138701
-      },
-      {
-        "id": "a2017e565e0328f1e33660cb03501ac8919f431b4b5e974ee3a15e4968753a60",
-        "category": "copy",
-        "target": "河川拠点から東倉庫",
-        "prediction": "河川拠点から東倉庫",
-        "exact": true,
-        "perplexity": 1.000208767100492
-      },
-      {
-        "id": "0182b13075f4332b76b2233f3bae4c52326057cadf0ea31a03d915316f4534e4",
-        "category": "schema.type",
-        "target": "52.00",
-        "prediction": "52.00",
-        "exact": true,
-        "perplexity": 1.0000143843922722
-      },
-      {
-        "id": "24847e0afe2df215f39333f839a4eea1053be89484268fc87b71cbb2c1cf74b9",
-        "category": "copy",
-        "target": "105",
-        "prediction": "104",
-        "exact": false,
-        "perplexity": 1.0284562214621615
-      },
-      {
-        "id": "a1ea4dda06275d912c5a857e04714fd9a4d59d489431ba70f2209151b6e6a551",
-        "category": "text.invisible",
-        "target": "4",
-        "prediction": "4",
-        "exact": true,
-        "perplexity": 1.000054537353746
-      },
-      {
-        "id": "f9f6ad70c2fcd6edf84505032104aac29631806ade8bd905eb6d223547fefa12",
-        "category": "copy",
-        "target": "Konto warsztatu",
-        "prediction": "Konto warsztatu",
-        "exact": true,
-        "perplexity": 1.0000307319084698
-      },
-      {
-        "id": "bb8ad536e68892ad5c3dc00cce1fe5722110ffcdb34f471aa5669db8c5fa3b1c",
-        "category": "schema.type",
-        "target": "4,19",
-        "prediction": "4,19",
-        "exact": true,
-        "perplexity": 1.0000092029202488
-      },
-      {
-        "id": "7bc66172b9d0ef4c9d1ceec538d042a1fc9ea35aefd96297310e52b79f5cd64b",
-        "category": "copy",
-        "target": "Salatsamen",
-        "prediction": "Salatsamen",
-        "exact": true,
-        "perplexity": 1.000030695377152
-      },
-      {
-        "id": "d491893e4413819baf49f50c46c954065aa70f6355e550929acfe64ce9639fd5",
-        "category": "text.spelling",
-        "target": "Erbsensamen",
-        "prediction": "Erbsensamen",
-        "exact": true,
-        "perplexity": 1.0000587926331692
-      },
-      {
-        "id": "d288e998948e5e51e89175237dc4ecdf3aa5d05fccac1e905307b288bab7e114",
-        "category": "copy",
-        "target": "13",
-        "prediction": "13",
-        "exact": true,
-        "perplexity": 1.0006689992697302
-      },
-      {
-        "id": "3186c2b31f593fab93254c01992c49379e669b73b5e57de33cd0f723e9cc399f",
-        "category": "text.spelling",
-        "target": "Σπόροι αρακά",
-        "prediction": "Σπόροι αρακά",
-        "exact": true,
-        "perplexity": 1.0001046579537995
-      },
-      {
-        "id": "b9f2f18d19b82a3f37b76e8b7f67bf1618289d5b5bce4567a016dd7c3f5c867d",
-        "category": "copy",
-        "target": "2026-02-08",
-        "prediction": "2026-02-08",
-        "exact": true,
-        "perplexity": 1.0044930397206753
-      },
-      {
-        "id": "18d9c8ef520ab257fd319d50d94ed5bbfc14e83ffc234cdbdd6fc5a6a2026c0b-open",
-        "category": "text.spelling",
-        "target": "Bean seeds",
-        "prediction": "Bean seeds",
-        "exact": true,
-        "perplexity": 1.0037579276335722
-      },
-      {
-        "id": "342f02a3a4f225b5002c9fd250afd4b1dc791884bd5c75249a63c2f10301ac0d-open-clean",
-        "category": "copy",
-        "target": "172",
-        "prediction": "172",
-        "exact": true,
-        "perplexity": 1.0092596407844958
-      },
-      {
-        "id": "7f16d555e641b36102a4b9c2cec0d3a1951522420473c5f8988026d326fbb9b9-open",
-        "category": "text.encoding",
-        "target": "Luma 東京",
-        "prediction": "Luma 東京",
-        "exact": true,
-        "perplexity": 1.0000134706127768
-      },
-      {
-        "id": "1ec1e5aabd33e35963b76c5559bada0a21757e70f3f8cbb46d12a428d955f415",
-        "category": "copy",
-        "target": "修理講座",
-        "prediction": "修理講座",
-        "exact": true,
-        "perplexity": 1.000008384385619
-      },
-      {
-        "id": "6dde22c81981d5d4fce090931f998ad995c3d3db0b5f8d8ce24a41b3c473e19d",
-        "category": "format.case",
-        "target": "RC-000048",
-        "prediction": "RC-000048",
-        "exact": true,
-        "perplexity": 1.00041237012412
-      },
-      {
-        "id": "77867c4faa620f772ecc9f148d9601d178311ca1ca653238fe509bc283043f5c",
-        "category": "copy",
-        "target": "  \"Controleer de bijgevoegde kaart voordat Slazaden wordt verplaatst; post 5 behandelt dit dossier.\"\n26  ",
-        "prediction": "  \"Controleer de bijgevoegde kaart voordat Slazaden wordt verplaatst; post 5 behandelt dit dossier.\"\n26  ",
-        "exact": true,
-        "perplexity": 1.0400289977685477
-      },
-      {
-        "id": "b35415ea92e9b8153e01186a486f9aa92bcb200e944a530e139838664215e79c",
-        "category": "format.number",
-        "target": "82,54",
-        "prediction": "82,54",
-        "exact": true,
-        "perplexity": 1.000831890641031
-      },
-      {
-        "id": "91f076346d5d8984a52a7977b6ffad67c2081d1ec611f6165835759299f2cb83",
-        "category": "copy",
-        "target": "4",
-        "prediction": "4",
-        "exact": true,
-        "perplexity": 1.0000392788962549
-      },
-      {
-        "id": "508c51bafcd810febf19f11c7464262a48f93ec76fe56b581dcee8917997c77d-open",
-        "category": "text.spelling",
-        "target": "Warsztat naprawczy",
-        "prediction": "Warsztat naprawczy",
-        "exact": true,
-        "perplexity": 1.0002412024957317
-      }
-    ],
-    "perplexity_calibration": {
-      "threshold": 1.0092596407844958,
-      "scored": 28,
-      "accepted": 24,
-      "observed_errors_at_threshold": 0,
-      "policy": "largest validation prefix with zero observed exact-match errors; not a statistical guarantee"
-    },
-    "prediction_sample": [
-      {
-        "id": "d55a35e0ae58a7236ecca68f66ba9e9d44f64eb7ad3ef4287c5da4b1eea735eb",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "16692fd42ba9001286947671f8c57463e75a68e6a083756675cddf256ea3258c",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "3b0fb29968d32172d06df90b66dad0b4e23113bc76e26edc5f41f3272ee7631f",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "8775404f74728fc546c80f1711f1113bb7dedae1edc1c7aa03225027ab2dc058",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "309d7716a2b94d9261e3d10a29d7207bd07f2067a322c5e4fa7e6749d40ada2d",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "98659accb59c446ec81ba33b755c40624b42bdb7e5afc0ec9cecb6f3ed32361e",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "15cd8ffbca06b9b35abd39c981e5d77543f9bc765b0bb3d142235be8a28f70a0",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "c3b152c04a5b1170a1aefcb463d0bb00c36fae800aaee751c606fd781c0ebe1e",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "a2017e565e0328f1e33660cb03501ac8919f431b4b5e974ee3a15e4968753a60",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "0182b13075f4332b76b2233f3bae4c52326057cadf0ea31a03d915316f4534e4",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "24847e0afe2df215f39333f839a4eea1053be89484268fc87b71cbb2c1cf74b9",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "a1ea4dda06275d912c5a857e04714fd9a4d59d489431ba70f2209151b6e6a551",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "f9f6ad70c2fcd6edf84505032104aac29631806ade8bd905eb6d223547fefa12",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "bb8ad536e68892ad5c3dc00cce1fe5722110ffcdb34f471aa5669db8c5fa3b1c",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "7bc66172b9d0ef4c9d1ceec538d042a1fc9ea35aefd96297310e52b79f5cd64b",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "d491893e4413819baf49f50c46c954065aa70f6355e550929acfe64ce9639fd5",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "d288e998948e5e51e89175237dc4ecdf3aa5d05fccac1e905307b288bab7e114",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "3186c2b31f593fab93254c01992c49379e669b73b5e57de33cd0f723e9cc399f",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "b9f2f18d19b82a3f37b76e8b7f67bf1618289d5b5bce4567a016dd7c3f5c867d",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "18d9c8ef520ab257fd319d50d94ed5bbfc14e83ffc234cdbdd6fc5a6a2026c0b-open",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "342f02a3a4f225b5002c9fd250afd4b1dc791884bd5c75249a63c2f10301ac0d-open-clean",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "7f16d555e641b36102a4b9c2cec0d3a1951522420473c5f8988026d326fbb9b9-open",
-        "predictions_at_0.9": [
-          {
-            "row": 30,
-            "column": 13,
-            "start": 4,
-            "end": 11,
-            "replacement": "",
-            "category": "text.encoding"
-          }
-        ]
-      },
-      {
-        "id": "1ec1e5aabd33e35963b76c5559bada0a21757e70f3f8cbb46d12a428d955f415",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "6dde22c81981d5d4fce090931f998ad995c3d3db0b5f8d8ce24a41b3c473e19d",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "77867c4faa620f772ecc9f148d9601d178311ca1ca653238fe509bc283043f5c",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "b35415ea92e9b8153e01186a486f9aa92bcb200e944a530e139838664215e79c",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "91f076346d5d8984a52a7977b6ffad67c2081d1ec611f6165835759299f2cb83",
-        "predictions_at_0.9": []
-      },
-      {
-        "id": "508c51bafcd810febf19f11c7464262a48f93ec76fe56b581dcee8917997c77d-open",
-        "predictions_at_0.9": [
-          {
-            "row": 16,
-            "column": 2,
-            "start": 17,
-            "end": 17,
-            "replacement": "",
-            "category": "text.spelling"
-          }
-        ]
-      }
-    ],
-    "limitations": "Token diagnostics, not cell/span calibration; limited gold-span repair sample; test untouched."
-  }
-}
-```
+Detailed losses and validation measurements are available in [metrics.json](metrics.json). Targeted correction validation and end-to-end CSV evaluation measure different tasks.
+
+## Limitations
+
+- Relational inconsistencies may identify an affected group without identifying which value is wrong. Missing contents and swapped values can remain unresolved.
+- Only implemented schema rules are enforced. Unsupported numeric formats or relation types can escape detection.
+- A confident replacement can still be semantically wrong: one such replacement occurred in the end-to-end evaluation. Pseudo-perplexity is an acceptance score, not a calibrated probability of correctness.
+- With business categories disabled, the CSV runner conservatively abstains from neural whole-cell correction; deterministic category controls remain available.
+- Results on this mostly synthetic corpus do not establish performance on every real-world table or domain.
+
+## License
+
+TabFix fine-tuning contributions are licensed under **CC BY-NC 4.0**. Commercial use requires the project owner's permission. The underlying mmBERT model retains its MIT terms and attribution; see [BASE_MODEL_CARD.md](BASE_MODEL_CARD.md). Dataset provenance and source-specific terms are documented in the [dataset card](https://huggingface.co/datasets/Antix5/tabular-errors-v1).

checkpoint.json

@@ -13,7 +13,7 @@
     "metrics.json": "05d4275b333608c81fd73556dfdeb81312e4f8dae74b33f9333e4efcdfaa8cc0",
     "categories.json": "da00946a1a8dfd24dd3bd01f47a4493abe6ade281d6a8571ae9728e0406d4aab",
     "train_tabfix.py": "b16a43dbb276461d319a05bd4719de7520e0612fa23f7d83fb5fba58bebbc24f",
-    "README.md": "b1da944908d669137ba6fc09477c8f0f8c2754e1b072c6138d0447d57a53e62c",
+    "README.md": "06bb0fa5f98814c1194fbb7dadb727d3c3719884238a2daec97c76107787f855",
     "LICENSE": "b6c1c7bb5aae03bb6b3396fc7c65cbe7306c422ed424b0b7b687d9a50cc15412",
     "BASE_MODEL_CARD.md": "35724415037ee3159aa733c16050620cc19a76219112a6ceeb3949572e0b37e9"
   },