Update README.md
Browse files
README.md
CHANGED
|
@@ -7,11 +7,7 @@ pipeline_tag: token-classification
|
|
| 7 |
---
|
| 8 |
# bert-chunker-3
|
| 9 |
|
| 10 |
-
|
| 11 |
-
## Introduction
|
| 12 |
-
|
| 13 |
-
bert-chunker-3 is a text chunker based on BERT with a classifier head to predict the start token of chunks (for use in RAG, etc), and using a sliding window it cuts documents of any size into chunks. The training data are labeled by a LLM.
|
| 14 |
-
|
| 15 |
|
| 16 |
## Quickstart
|
| 17 |
Run the following:
|
|
|
|
| 7 |
---
|
| 8 |
# bert-chunker-3
|
| 9 |
|
| 10 |
+
bert-chunker-3 is a text chunker based on BERT with a classifier head to predict the start token of chunks (for use in RAG, etc), and using a sliding window it cuts documents of any size into chunks. Its training data are labeled by a LLM.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
## Quickstart
|
| 13 |
Run the following:
|