Update README.md

README.md

@@ -4,13 +4,13 @@ license: cc-by-nc-sa-4.0
 
 # **Petrichor — Model Card**
 
-## **Introduction**
+# **Introduction**
 
 As a creator, it is common to experience fatigue and burnout, both of which present major challenges when crafting inspirational and innovative materials. This relates directly to Game Masters (GMs) that run Tabletop Role Playing Games (TTRPGs). They run games often, which creates pressure on both time and diversity of material. LLMs have a different creative block, which is homogenous creativity, causing LLMs to be on average less creative than humans; this occurs both across LLMs (different LLM families) and within LLMs (same LLM families), where LLMs will also produce similar output to each other. Preliminary evidence shows that even a small amount of new material injected can assist in breaking an LLM out of that cycle.
 
 Introducing Petrichor, a new Game Master helper for TTRPGs. Petrichor is a dual-path hybrid retrieval RAG system that works off a diverse corpus, plus your campaign notes, to inject an LLM with diverse creative material for the next plot hook, character, thematic trap, and more. The main Claude version of Petrichor showed an increase in the actionability criterion in both benchmarks where the metric is present; exhibiting a 3.3% (0.82 to 0.86 on a 0–1 scale) absolute increase over the baseline model in domain specific queries/prompts and a 10.7% (0.44 to 0.55 on a 0–1 scale) absolute increase over generic creative prompts. All other metrics experience drops between the RAG version and the baseline version, but actionability is arguably one of the more important and difficult elements to capture in a narrative.
 
-## **Data Overview**
+# **Data Overview**
 
 The curated static corpus consists of 32 Project Gutenberg novels, 8 TTRPG rulebooks or handbooks, and 5 TTRPG scenario books. The novels act as a pure and classic form of narrative creativity, while TTRPG related books depict more active and actionable narrative structures. The corpus is comprised of a wide variety of file types (PDF, txt, docx, markdown, etc.), which are either public domain, licensed under Creative Commons, or fall under other permissive licensing.
 
@@ -26,7 +26,7 @@ Campaign notes may be optionally supplied to the system, which should ideally be
 
 Due to the nature of the creative corpus, a standard test split was unsuitable for the task. Instead, three benchmarks were evaluated, two utilizing open-ended prompting, while one utilized a synthetic dataset created from the corpus to test competence of retrieval. All benchmarks were evaluated by LLM-as-judge and are described in more depth in the **Evaluation** section.
 
-## **Methodology**
+# **Methodology**
 
 Petrichor is a hybrid-retrieval RAG system utilizing vector similarity and BM25 for both the main pipeline and the optional campaign notes pipeline; Neo4j graph retrieval is an addition to the campaign notes pipeline to visualize and assist in keeping track of complex relationships. Chunking was conducted with LangChain recursive text splitter in 500 token chunk increments for the static corpus and 250 chunk increments for the campaign notes, both with a standard 10% token overlap. These chunks were embedded using the multi-purpose embedder `BAAI/bge-m3`.
 
@@ -46,7 +46,7 @@ Neo4j and other graph-based retrieval are beneficial for queries that require mu
 
 Reciprocal rank fusion receives the combined and ranked vector similarity and BM25 list from the corpus side, the combined and ranked vector similarity and BM25 list from the campaign side when applicable, and the GraphRAG entities/relationships when applicable. These three lists are combined and reranked. The dampening constant used is k=60, as selected originally by Cormack, Clarke, and Buettcher, 2009. This result is handed to the answering LLM as context.
 
-## **Evaluation**
+# **Evaluation**
 
 ### All Evaluation Criteria
 
@@ -109,13 +109,13 @@ Benchmark 3 consists of 40 synthetic queries/prompts generated by RAGAS from the
 
 \* Likely subject to self-bias as GPT-4.1 is judging GPT-4.1.
 
-## **Model Selection/Results**
+### **Model Selection/Results**
 
 Claude was selected early in the process due to familiarity, capability, ease of use, and the fact that creativity can be a difficult task overall for LLMs. The comparison models, GPT-4.1 and Gemini 3.1 Pro, were selected because they are frontier level models that are likely of a similar size to Claude. Gemma 4 12B Q8_0 was also tested on Benchmark 1 as an open-weights option to the system due to its native creativity and strong instruction following capabilities.
 
 The most difficult non-RAG specific metric for the LLMs across the board was actionability, which is the task that Claude version Petrichor excelled at over all other models for both benchmarks. This includes a 3.3% (0.82 to 0.86 on a 0–1 scale) absolute increase over the baseline model in domain specific queries/prompts and a 10.7% (0.44 to 0.55 on a 0–1 scale) absolute increase over generic creative prompts. While Claude version Petrichor fell to the lower-middle of the pack in Benchmark 1 outside of the actionability criterion, the scores for Petrichor suffered relative to other models in Benchmark 2; despite this, Petrichor still maintained a 100% pass rate outside of narrative specificity at 0.72 points (87.5% pass rate) and actionability, which no model scored at 100%. Contextual relevancy was low across all benchmarks aside from Benchmark 3 due to a lack of a "correct answer," but faithfulness was strong across all benchmarks (lowest at 0.79 points), and contextual relevancy increased to 0.81 points in the presence of questions with "ground-truths." Petrichor also answers correctly on more statements than the baseline models. The Gemma version of Petrichor struggles compared to the Claude version Petrichor and baseline version Gemma, which was qualitatively assessed to find that the model was less capable of sophisticatedly integrating the information than Claude.
 
-## **Usage and Intended Uses**
+# **Usage and Intended Uses**
 
 The intended usage of this RAG system is to create plot hooks, characters, thematic traps, and more. None of the tested models are loadable via HuggingFace and most are instead called by API. An exception to the API calls is that Gemma 4 12B Q8_0 can be used with a `llama-cpp` wheel. The main path can be run locally or through an app, but the campaign notes portion was designed to have a UI. Campaign notes can enrich responses by adding personalization through the bespoke content of an individual's created settings, characters, and factions. This allows one to ask for hooks, characters, traps, and other elements set within their own world.
 
@@ -464,7 +464,7 @@ def clear_session(session_id):
     processed_filenames.pop(session_id, None)
 ```
 
-## Prompt Format
+# Prompt Format
 
 The system prompt is passed through `system`, which instructs the LLM to have a persona, perform, and directives to avoid hedging and unnecessary tables. This is combined with the user message, which is context chunks from RAG, and the query itself.
 
@@ -523,7 +523,7 @@ query: Write me a plot hook about a player party of jovial misfits at sea who wa
 on the sandy beaches of a mystical island that is not what it seems.
 ```
 
-## Output Format
+# Output Format
 
 The output is generated with Claude Sonnet 4.6 and a setting of 2000 max tokens, with a typical output range of ~700–1300 token responses in preliminary tests. The output includes content related to the specific element chosen (character, hook, etc.), but hooks such as this one specifically contain exposition, player engagement, and source/inspirational citation.
 
@@ -553,7 +553,7 @@ response = client.messages.create(
 >
 > *Sources: One Page Dungeon Contest 2025 Compendium [scenario] — the hexagonal membrane island, the Daughter of the Elements, the Mechanical Remnants, and the six corner-weapons; Three Men In A Boat, Jerome [novel] — the disreputable hired vessel and the tone of jovial maritime misfortune; The Arabian Nights Entertainment, Anonymous [novel] — the island that is not what it seems beneath one's feet.*
 
-## **Limitations**
+# **Limitations**
 
 ### Cost
 
@@ -567,7 +567,7 @@ Preliminary tests have been conducted, including benchmarking; however, extensiv
 
 This is a class project and version 1 of this system; therefore, there may be existing bugs or unknown issues. While the campaign notes pipeline allows for multiple users to access and independently use a hosted demo at the same time due to session IDs, the notes are strictly transient and will be wiped after each session at this time to avoid overloading the free tier cloud storage.
 
-## **Citations**
+# **Citations**
 
 ### Introduction