docs/title_boundary_heuristic_experiments.md

# Title-Boundary Heuristic Experiments

This note records deterministic Rust heuristic changes and Pi RPC reviewer evidence
for DMHY template `title_spans` generation. The reviewer output is evidence only;
recipes and training JSONL are changed only by Rust code.

## Baseline

Input recipe evidence:

- `reports/dmhy_template_recipes.full_top5000.title_boundary_trial.seed.jsonl`
- total title-boundary decisions: `5631`

Decision distribution:

| decision | count |
| --- | ---: |
| `merge_dissimilar_title_continuation` | 4661 |
| `keep_similar_duplicate_or_alias` | 463 |
| `keep_structural_boundary` | 502 |
| `keep_path_boundary` | 5 |

Pi RPC stratified sample, `--threads 4`:

| stratum | rows | model result | note |
| --- | ---: | --- | --- |
| ambiguous merge, similarity `0.25..0.85` | 20 | 20 merge | Good. Reviewer cited concrete title-word pairings. |
| high-sim `keep_similar_duplicate_or_alias`, similarity `0.85..1.0` | 20 | 19 merge, 1 parse-error | Bad baseline heuristic. Examples such as `Asobi ni Iku yo!`, `Code Geass ~Akito the Exiled~`, `Yagate Kimi ni Naru`, and `Big Order` were wrongly kept separate by the heuristic. |
| structural keep | 20 | 18 keep, 1 merge, 1 skip | Mostly good. Covers category+title, multilingual aliases, and distinct fields. |
| path keep | 5 | 5 keep | Good. Covers path duplicate/category boundaries. |

Conclusion: `keep_similar_duplicate_or_alias` was too broad because it used max
literal similarity. Short connector repeats (`no`, `wa`, `ni`, `the`, `of`) and
repeated title words inflated similarity to `1.0` even for ordinary contiguous
title phrases.

## Experiment 1: strict substantive duplicate keep

Change tried:

- Keep adjacent `TITLE SEP TITLE` only when top literals include a substantive
  duplicate/near-duplicate pair.
- Ignore short connectors and literals with normalized length `< 6`.

Generated recipe output:

- `reports/dmhy_template_recipes.full_top5000.strictdup.seed.jsonl`

Decision distribution:

| decision | count |
| --- | ---: |
| `merge_dissimilar_title_continuation` | 4661 |
| `merge_adjacent_title_continuation` | 444 |
| `keep_similar_duplicate_or_alias` | 19 |
| `keep_structural_boundary` | 502 |
| `keep_path_boundary` | 5 |

Pi RPC review of all 19 remaining `keep_similar_duplicate_or_alias` decisions:

| rows | model result |
| ---: | --- |
| 19 | 19 merge |

Examples the reviewer judged as merge:

- `Air In Summer`, `Assault Lily BOUQUET`
- `Aoki Hagane no Arpeggio Ars Nova`
- `Shoujo Kageki Revue Starlight`
- `Motto! Ojamajo Doremi`
- `Fate stay night Heaven's Feel`
- `Arafou Otoko no Isekai Tsuuhan Seikatsu`
- `Watashi, Nouryoku wa Heikinchi de tte Itta yo ne!`

Conclusion: strict duplicate evidence still produced false keeps. The repeated
literal was often a real repeated word inside one title phrase or a top-literal
artifact, not a duplicate/alias boundary.

## Experiment 2: merge all adjacent separator-only title slots

Final change:

- `TITLE PATH TITLE` remains `keep_path_boundary`.
- Non-separator structural boundaries remain `keep_structural_boundary`.
- Every separator-only adjacent title boundary, `TITLE SEP TITLE`, now merges.
- Low-similarity merges keep the old decision name
  `merge_dissimilar_title_continuation`.
- High-similarity merges use `merge_adjacent_title_continuation`.
- `keep_similar_duplicate_or_alias` is no longer emitted by the deterministic
  heuristic.

Generated recipe output:

- `reports/dmhy_template_recipes.full_top5000.mergeall.seed.jsonl`

Decision distribution:

| decision | count |
| --- | ---: |
| `merge_dissimilar_title_continuation` | 4661 |
| `merge_adjacent_title_continuation` | 463 |
| `keep_structural_boundary` | 502 |
| `keep_path_boundary` | 5 |

Pi RPC sample of high-similarity `merge_adjacent_title_continuation` decisions:

| rows | model result |
| ---: | --- |
| 30 | 29 merge, 1 parse-error |

The reviewer cited concrete title continuations such as:

- `Asobi ni Iku yo!`
- `Code Geass ~Akito the Exiled~`
- `Kami nomi zo Shiru Sekai`
- `Yagate Kimi ni Naru`
- `Big Order`
- `Gate Jieitai Kanochi nite Kaku Tatakaeri`
- `JoJo's Bizarre Adventure - Stardust Crusaders`

Conclusion: merge-all adjacent separator-only title slots is the best of the
tried heuristics. It matches Pi RPC evidence and simplifies downstream synthetic
title-span replacement.

## Synthetic generation smoke

Generated schema v2 synthetic augmentation with merge-all recipes:

- `data/schema_v2_synthetic_aug.mergeall.jsonl`
- `data/schema_v2_synthetic_aug.mergeall.manifest.json`

Validator passed. Manifest summary:

| metric | value |
| --- | ---: |
| `generated_rows` | 49998 |
| `numeric_title_rows` | 30000 |
| `path_rows` | 19998 |
| `path_series_rows` | 15000 |
| `path_movie_rows` | 1666 |
| `path_special_rows` | 1666 |
| `path_confuser_rows` | 1666 |
| `dropped_media_kind_mismatch` | 19611 |

## Commands

Pi RPC high-sim adjacent merge sample:

```powershell
cargo run --release --manifest-path tools\rust_dmhy_template_apply\Cargo.toml -- `
  --review-title-boundaries-pi-rpc `
  --recipes reports\dmhy_template_recipes.full_top5000.mergeall.seed.jsonl `
  --title-boundary-decisions-output reports\dmhy_title_boundary_pi_rpc_mergeall.adjacent_highsim.jsonl `
  --title-boundary-heuristic-contains merge_adjacent_title_continuation `
  --title-boundary-min-similarity 0.85 `
  --title-boundary-max-similarity 1.0 `
  --threads 4 `
  --limit 30
```

Synthetic smoke:

```powershell
cargo run --release --manifest-path tools\schema_v2_synthetic_augment\Cargo.toml --bin schema_v2_synthetic_augment -- `
  --recipes reports\dmhy_template_recipes.full_top5000.mergeall.seed.jsonl `
  --label-schema-file label_schema.json `
  --numeric-title-seeds data\synthetic_numeric_titles.txt `
  --path-prefix-seeds data\synthetic_path_prefixes.txt `
  --limit-templates 3000 `
  --max-rows 50000 `
  --output data\schema_v2_synthetic_aug.mergeall.jsonl `
  --manifest-output data\schema_v2_synthetic_aug.mergeall.manifest.json

cargo run --release --manifest-path tools\schema_v2_synthetic_augment\Cargo.toml --bin validate_synthetic_aug_jsonl -- `
  --input data\schema_v2_synthetic_aug.mergeall.jsonl `
  --manifest data\schema_v2_synthetic_aug.mergeall.manifest.json
```