Card brought up to date for the finished study: what was found, where to read it, how to use this
Browse files
README.md
CHANGED
|
@@ -4,7 +4,7 @@ language: en
|
|
| 4 |
tags: [authorship, embeddings, book-of-mormon-authorship, 1780-1830]
|
| 5 |
---
|
| 6 |
|
| 7 |
-
# Style
|
| 8 |
|
| 9 |
Authorship style embeddings (Amendment 24): the time-locked model fine-tuned with a contrastive
|
| 10 |
objective so that passages by the same author land close together. `297m/style.pt` and
|
|
@@ -13,6 +13,42 @@ in the GitHub repository; `load_encoder(path, cfg, "base")` or `"mid"`), with th
|
|
| 13 |
logs. Benchmark: 297M 0.839 right author, 0.500 false matches (`benchmark_297m.json`); 590M 0.839,
|
| 14 |
0.628 (`benchmark_590m.json`).
|
| 15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
## About
|
| 17 |
|
| 18 |
Part of an open, pre-registered study of the authorship, borrowing and dating of the
|
|
|
|
| 4 |
tags: [authorship, embeddings, book-of-mormon-authorship, 1780-1830]
|
| 5 |
---
|
| 6 |
|
| 7 |
+
# Style models (the "fingerprint" models)
|
| 8 |
|
| 9 |
Authorship style embeddings (Amendment 24): the time-locked model fine-tuned with a contrastive
|
| 10 |
objective so that passages by the same author land close together. `297m/style.pt` and
|
|
|
|
| 13 |
logs. Benchmark: 297M 0.839 right author, 0.500 false matches (`benchmark_297m.json`); 590M 0.839,
|
| 14 |
0.628 (`benchmark_590m.json`).
|
| 15 |
|
| 16 |
+
## The second model (`v2/`): the one behind the library search
|
| 17 |
+
|
| 18 |
+
`v2/style.pt` is the second style model, on the 297M backbone. It is the model that searched all
|
| 19 |
+
23,872 authors of the period for the writing closest to the Book of Mormon. The first model
|
| 20 |
+
memorised its training authors; this one was trained on eight times as many passages per book,
|
| 21 |
+
with half its steps forcing it to match an author across different kinds of writing
|
| 22 |
+
(`style_embed/v2.py`). Load it the same way: `load_encoder("v2/style.pt", cfg)`. A book is scored
|
| 23 |
+
against an author by its closest episode (`sweep/episodes.py`). `v2/log.jsonl` and `v2/run.json`
|
| 24 |
+
are its training record. SHA-256 of `v2/style.pt`:
|
| 25 |
+
`8a4e09b0ba9cef5de5beb73c58c7412a1041d60473ae52746094bf0dbff9fd29`.
|
| 26 |
+
|
| 27 |
+
How it did, on books whose author is known (results files in the GitHub repository):
|
| 28 |
+
|
| 29 |
+
- **Finding the author in a library of 18,288:** on 727 known books, the true author comes first
|
| 30 |
+
34% of the time and in the top ten 59%. For a book unlike its author's other writing: 17% and
|
| 31 |
+
38% (`data/sweep/coverage_episodes_v2.json`).
|
| 32 |
+
- **Confirming or rejecting one named author** (200 books, 2,200 pairs,
|
| 33 |
+
`data/sweep/nobody.json`): separation 0.955, against 0.895 for LambdaG (2024) and 0.861 for
|
| 34 |
+
LUAR (2021) on the same pairs.
|
| 35 |
+
- **Its limits:** set to confirm 95% of true authors, it still accepts 33% of wrong ones. It was
|
| 36 |
+
trained and tested on English of 1780-1830 only.
|
| 37 |
+
|
| 38 |
+
## The study and what it found
|
| 39 |
+
|
| 40 |
+
This is one of the models from a finished, open study of the English text of the Book of Mormon:
|
| 41 |
+
who writes like it, what it borrows, and when its language dates from.
|
| 42 |
+
|
| 43 |
+
- **The findings in plain words, with live charts:** https://lifehug.com/book-of-mormon
|
| 44 |
+
- **The paper, the code and every results file:** https://github.com/lifehug/book-of-mormon-authorship
|
| 45 |
+
|
| 46 |
+
In short: the book copies almost nothing from earlier print. Among the eight people usually
|
| 47 |
+
proposed as its author, the closest writing is Joseph Smith's revelations, and a search of 23,872
|
| 48 |
+
authors of the period puts those revelations first or second. No tool we found, ours included, can
|
| 49 |
+
tell a person from the voice they are writing in, or reliably say "it is none of these people".
|
| 50 |
+
The study takes no position on inspiration.
|
| 51 |
+
|
| 52 |
## About
|
| 53 |
|
| 54 |
Part of an open, pre-registered study of the authorship, borrowing and dating of the
|