simple-rag-qa / chunking.py
Matvii Hotovych
Simple RAG QA
4c0ae33
raw
history blame contribute delete
245 Bytes
def chunk_text(text, chunk_size=300, overlap=50):
words = text.split()
chunks = []
for i in range(0, len(words), chunk_size - overlap):
chunk = " ".join(words[i:i + chunk_size])
chunks.append(chunk)
return chunks