eski holat with utils

eski holat

eski holat

max rows = 20, 150 + 150 bm25

chunk size = 2048 + rows=15

max rows = 20, 100 + 100 bm25

top k reranker = 20, max rows = 10, max chars= 2000 + new deduplication

top k reranker = 20, max rows = 10, max chars= 4000 + new deduplication

doc id problem fixed

doc id problem fixed

new doc id filter + 100 + 100 retrieval

Much lower reranking threshold (-0.5 instead of 0.1) + detailed score logging

Much lower reranking threshold (-0.5 instead of 0.1) + detailed score logging

removed normalization doc id

index retriever = 100 + 100

index retriever = 100 + 100

max_chars = 1500 + doc id retriever

new keyword score based index retriever + answer question

new keyword score based index retriever + answer question

chunk size = 1024 + max chars = 1200 + deduplication variant

chunk size = 1024 + max chars = 1200 + keyword based

new embeeding model + new create_quer_engine with keyword matching

simplest version

simplest version

simplest version

added sheet_name

added the load_table_data function

added the load_table_data function

added the load_table_data function

table prep changed

hybrid_retrieve_with_keywords implemented

a new way with keywords

a new chunking with 3 000 size for all types of data

bm = 40, semantic = 40. hybrid = 60, cutoff = 0.65, topk_reranker= 20

bm = 20, semantic = 30. hybrid = 40, cutoff = 0.65, topk_reranker= 20

bm = 20, semantic = 20. hybrid = 40, cutoff = 0.65, topk_reranker= 20

bm = 40, semantic = 40. hybrid = 60, cutoff = 0.5, topk_reranker= 20

bm = 30, semantic = 30. hybrid = 60, cutoff = 0.7, topk_reranker= 60

bm = 25, semantic = 25. hybrid = 50, cutoff = 0.7, topk_reranker= 40

bm = 20, semantic = 20. hybrid = 40, cutoff = 0.7, topk_reranker= 40

bm = 40, semantic = 20. hybrid = 50, cutoff = 0.7, topk_reranker= 40

bm = 45, semantic = 20. hybrid = 50, cutoff = 0.7, topk_reranker= 20

bm = 40, semantic = 20. hybrid = 50

bm = 30, semantic = 20. hybrid = 50

bm = 20, semantic = 20. hybrid = 40

top k semantic 50, cutof == 0.6, hybdrid = 0.6

top k in reranker = 30, bm = 20

bm = 60, semantic = 50. hybrid = 70

bm = 50, semantic = 50. hybrid = 50

bm = 40, semantic = 50. hybrid = 40