opensporks commited on
Commit
c3be7f0
·
verified ·
1 Parent(s): 0bcdd1d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -9
README.md CHANGED
@@ -68,15 +68,7 @@ We evaluated Schematron's real-world impact on LLM factuality using SimpleQA.
68
  3. **Structured Extraction**: Schematron extracts JSON data from retrieved pages using the schema
69
  4. **Answer Synthesis**: Primary LLM produces final answer from structured data
70
 
71
- | Base Model | Configuration | SimpleQA Accuracy |
72
- |:-----------|:--------------|------------------:|
73
- | GPT-5 Nano | Solo | 8.54% |
74
- | GPT-5 Nano | + SERP + Schematron-8B | 64.15% |
75
- | GPT-5 Nano | + Exa + **Schematron-3B** | **75.47%** |
76
- | GPT-5 Nano | + Exa + Gemini 2.5 Flash | 80.61% |
77
- | GPT-5 Nano | + Exa + **Schematron-8B** | **82.87%** |
78
- | GPT-4.1 | Solo | 41.60% |
79
- | GPT-4.1 | + Exa + **Schematron-8B** | **85.58%** |
80
 
81
  **Key findings:**
82
  - Web search paired with JSON extraction improves factuality: Adding Schematron with web retrieval improves GPT-5 Nano's accuracy from 8.54% to 82.87%—nearly a 10x improvement
 
68
  3. **Structured Extraction**: Schematron extracts JSON data from retrieved pages using the schema
69
  4. **Answer Synthesis**: Primary LLM produces final answer from structured data
70
 
71
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6626a246891c75742bd19aaf/h2CA0TpQ1hKDxexGlpdSQ.png)
 
 
 
 
 
 
 
 
72
 
73
  **Key findings:**
74
  - Web search paired with JSON extraction improves factuality: Adding Schematron with web retrieval improves GPT-5 Nano's accuracy from 8.54% to 82.87%—nearly a 10x improvement