Pronunciation cache limited to the vocabulary words; README wording
Browse files- README.md +8 -6
- SHA256SUMS +1 -1
- g2p_raw_cache.json +0 -0
README.md
CHANGED
|
@@ -31,7 +31,7 @@ writes no คำสร้อย. One sample is drawn per topic at temperature 0.
|
|
| 31 |
| `model/config.json`, `TRAINED.json`, `train.log` | model configuration, training record and log |
|
| 32 |
| `lexicon.json` | the vocabulary: 62,856 words with their pronounced syllables |
|
| 33 |
| `allow.json` | allowlist of the decoder's stutter ban |
|
| 34 |
-
| `g2p_raw_cache.json` | pronunciation cache (word -> phonemes) |
|
| 35 |
| `config.json`, `configuration_klong4.py`, `modeling_klong4.py` | the `transformers` wrapper |
|
| 36 |
| `src/klong4/` | the project's code, used by the wrapper |
|
| 37 |
| `SHA256SUMS` | checksums of the weight and data files above |
|
|
@@ -74,21 +74,23 @@ Notes:
|
|
| 74 |
|
| 75 |
## Evaluation (from the paper)
|
| 76 |
|
| 77 |
-
On 75 held-out topics, one sample per
|
| 78 |
-
same pools as the human stanzas
|
| 79 |
|
| 80 |
| | Model | Human verse |
|
| 81 |
|---|---|---|
|
| 82 |
-
| Passes form
|
| 83 |
| Lexically clean | 66.7% | 92.0% |
|
| 84 |
| On topic (alignment >= 4 of 5) | 72.0% | 88.0% |
|
| 85 |
| Clean and on topic | 50.7% | 81.3% |
|
| 86 |
|
| 87 |
-
The difference in the last row is -30.7 points (95% CI [-44.0, -17.3]). No generated line
|
|
|
|
| 88 |
|
| 89 |
## Limitations
|
| 90 |
|
| 91 |
-
- Every stanza
|
|
|
|
| 92 |
narrower and more repetitive than human verse (6.8% rare words against 13.9%).
|
| 93 |
- The training set is small (2,001 stanzas).
|
| 94 |
- The numbers are judgments of one LLM judge family, not of expert readers.
|
|
|
|
| 31 |
| `model/config.json`, `TRAINED.json`, `train.log` | model configuration, training record and log |
|
| 32 |
| `lexicon.json` | the vocabulary: 62,856 words with their pronounced syllables |
|
| 33 |
| `allow.json` | allowlist of the decoder's stutter ban |
|
| 34 |
+
| `g2p_raw_cache.json` | pronunciation cache for the 62,856 vocabulary words (word -> phonemes) |
|
| 35 |
| `config.json`, `configuration_klong4.py`, `modeling_klong4.py` | the `transformers` wrapper |
|
| 36 |
| `src/klong4/` | the project's code, used by the wrapper |
|
| 37 |
| `SHA256SUMS` | checksums of the weight and data files above |
|
|
|
|
| 74 |
|
| 75 |
## Evaluation (from the paper)
|
| 76 |
|
| 77 |
+
On 75 held-out stanzas and their topics (63 distinct), one sample per item, judged by three independent runs of an LLM
|
| 78 |
+
judge (Claude Opus 5.5) in the same pools as the human stanzas:
|
| 79 |
|
| 80 |
| | Model | Human verse |
|
| 81 |
|---|---|---|
|
| 82 |
+
| Passes the form checker, three rhymes included | 100% | 100% (selected) |
|
| 83 |
| Lexically clean | 66.7% | 92.0% |
|
| 84 |
| On topic (alignment >= 4 of 5) | 72.0% | 88.0% |
|
| 85 |
| Clean and on topic | 50.7% | 81.3% |
|
| 86 |
|
| 87 |
+
The difference in the last row is -30.7 points (95% CI [-44.0, -17.3]). No generated line equals a line of the corpora searched, and
|
| 88 |
+
none shares more than five consecutive syllables with them.
|
| 89 |
|
| 90 |
## Limitations
|
| 91 |
|
| 92 |
+
- Every stanza passes the automatic form checker, which counts syllables as the pronunciation converter reads them
|
| 93 |
+
(it misreads some words), but a third contain a line the judge considers ill-formed, and the vocabulary is
|
| 94 |
narrower and more repetitive than human verse (6.8% rare words against 13.9%).
|
| 95 |
- The training set is small (2,001 stanzas).
|
| 96 |
- The numbers are judgments of one LLM judge family, not of expert readers.
|
SHA256SUMS
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
f053475b9be734873050775abc9fcd51cc01f271ce80d22352126ed9e97e8916 model/model.pt
|
| 2 |
5e47c92e00fd240544a4acf8fe8f95fd30a1b79a7899d08023039ea810e3c228 lexicon.json
|
| 3 |
fdcb862c4dcab8f370dc0bac46d9066ddee92fd3728a4ffdf7d12b99d9e0eaf5 allow.json
|
| 4 |
-
|
|
|
|
| 1 |
f053475b9be734873050775abc9fcd51cc01f271ce80d22352126ed9e97e8916 model/model.pt
|
| 2 |
5e47c92e00fd240544a4acf8fe8f95fd30a1b79a7899d08023039ea810e3c228 lexicon.json
|
| 3 |
fdcb862c4dcab8f370dc0bac46d9066ddee92fd3728a4ffdf7d12b99d9e0eaf5 allow.json
|
| 4 |
+
8d4b4ef7c9860fd4181c1eb9cd38856a900b7b5899698f7ff95c01c35c774466 g2p_raw_cache.json
|
g2p_raw_cache.json
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|