Update README.md
Browse files
README.md
CHANGED
|
@@ -408,7 +408,7 @@ for i, (c, t) in enumerate(zip(chunks, token_pos)):
|
|
| 408 |
```
|
| 409 |
## Evaluation
|
| 410 |
Evaluation is done by code from [brandonstarxel/chunking_evaluation](https://github.com/brandonstarxel/chunking_evaluation), most of the following results come from [Evaluating Chunking Strategies for Retrieval](https://research.trychroma.com/evaluating-chunking).
|
| 411 |
-
| Chunking | Size| Overlap | Recall | Precision | PrecisionΩ | IoU | Time complexity by token number N | Is chunk size strictly controlable|
|
| 412 |
|---------|---------|---------|---------|---------|---------|---------|---------|---------|
|
| 413 |
| Recursive | <= 800 | 400 | 85.4 ± 34.9 | 1.5 ± 1.3 | 6.7 ± 5.2 | 1.5 ± 1.3| **O(N)** | **Yes**
|
| 414 |
| TokenText | 800 | 400 | 87.9 ± 31.7| 1.4 ± 1.1 | 4.7 ± 3.1 | 1.4 ± 1.1 |**O(N)** | **Yes**
|
|
|
|
| 408 |
```
|
| 409 |
## Evaluation
|
| 410 |
Evaluation is done by code from [brandonstarxel/chunking_evaluation](https://github.com/brandonstarxel/chunking_evaluation), most of the following results come from [Evaluating Chunking Strategies for Retrieval](https://research.trychroma.com/evaluating-chunking).
|
| 411 |
+
| Chunking | Size| Overlap | Recall | Precision | PrecisionΩ | IoU | Time complexity by token number N | Is max chunk size strictly controlable|
|
| 412 |
|---------|---------|---------|---------|---------|---------|---------|---------|---------|
|
| 413 |
| Recursive | <= 800 | 400 | 85.4 ± 34.9 | 1.5 ± 1.3 | 6.7 ± 5.2 | 1.5 ± 1.3| **O(N)** | **Yes**
|
| 414 |
| TokenText | 800 | 400 | 87.9 ± 31.7| 1.4 ± 1.1 | 4.7 ± 3.1 | 1.4 ± 1.1 |**O(N)** | **Yes**
|