KD-Code commited on
Commit
1fe46f7
·
verified ·
1 Parent(s): 453a142

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -16,7 +16,7 @@ Introducing Petrichor, a new Game Master helper for TTRPGs. Petrichor is a dual-
16
 
17
  # Data Overview
18
 
19
- The curated static corpus consists of 32 Project Gutenberg novels, 8 TTRPG rulebooks or handbooks, and 5 TTRPG scenario books. The novels act as a pure and classic form of narrative creativity, while TTRPG related books depict more active and actionable narrative structures. The corpus is comprised of a wide variety of file types (PDF, txt, docx, markdown, etc.), which are either public domain, licensed under Creative Commons, or fall under other permissive licensing.
20
 
21
  ### Corpus and Campaign Notes Addition
22
 
@@ -28,7 +28,7 @@ Campaign notes may be optionally supplied to the system, which should ideally be
28
 
29
  ### Data Evaluation Process
30
 
31
- Due to the nature of the creative corpus, a standard test split was unsuitable for the task. Instead, three benchmarks were evaluated, two utilizing open-ended prompting, while one utilized a synthetic dataset created from the corpus to test competence of retrieval. All benchmarks were evaluated by LLM-as-judge and are described in more depth in the **Evaluation** section.
32
 
33
  ---
34
 
 
16
 
17
  # Data Overview
18
 
19
+ The curated static corpus consists of 32 Project Gutenberg novels, 8 TTRPG rulebooks or handbooks, and 5 TTRPG scenario books. The novels act as a pure and classic form of narrative creativity, while TTRPG related books depict more active and actionable narrative structures. The corpus is comprised of a wide variety of file types (PDF, txt, docx, markdown, etc.), which are either public domain, licensed under Creative Commons, or fall under other permissive licensing. Other than using different packages to pull text for different file types, the files themselves were not altered and all file types were treated as equally as possible.
20
 
21
  ### Corpus and Campaign Notes Addition
22
 
 
28
 
29
  ### Data Evaluation Process
30
 
31
+ Due to the nature of the creative corpus, a standard test split was unsuitable for the task. Instead, three benchmarks were evaluated, two utilizing open-ended prompting, one is domain-specific and created by hand and the other a subset of an existing database of generic creative prompts, while the final benchmark utilized a synthetic dataset created from the corpus to test competence of retrieval. These benchmarks collectively serve the purpose of a "holdout" set. All benchmarks were evaluated by LLM-as-judge and are described in more depth in the **Evaluation** section.
32
 
33
  ---
34