Update README.md
Browse files
README.md
CHANGED
|
@@ -613,7 +613,7 @@ The base model Claude operates by paid API. Due to costly LLM-as-Judge benchmark
|
|
| 613 |
|
| 614 |
### Performance
|
| 615 |
|
| 616 |
-
Preliminary tests have been conducted, including benchmarking; however, extensive qualitative testing has not been performed. Benchmarked figures indicate weakened performance in many categories when the RAG is introduced,
|
| 617 |
|
| 618 |
### Complexity of Use
|
| 619 |
|
|
|
|
| 613 |
|
| 614 |
### Performance
|
| 615 |
|
| 616 |
+
Preliminary tests have been conducted, including benchmarking; however, extensive qualitative testing has not been performed. Benchmarked figures indicate weakened performance in many categories when the RAG is introduced, suggesting more testing, investigation, and development is required. The smaller tested open-weights model (Gemma 4 12B Q8_0) had difficulties performing citations and quotes without breaking tone and narrative consistency. An architectural performance limitation is that Reciprocal Rank Fusion would likely operate more effectively utilizing a more sophisticated weighting system as opposed to relying on the order in which a chunk is inserted to break ties.
|
| 617 |
|
| 618 |
### Complexity of Use
|
| 619 |
|