data/Customer-techqa.csv

Test #,Configuration purpose,Subset(s),Embedding Model,Reranker Model,Summarization,Chunking Strategy,Chunk size,Overlap,Stride,Retreival Strategy,Alpha,Retr. K,Final K,Repacking,Summ. Max,Summ. Min,8B GPT Label,RMSE=trace relevance,RMSE=trace utilization,RMSE=trace completeness,AUCROC,F1-score,# Failed/Total Samples
1,Efficiency Baseline,techqa,BAAI/bge-base-en-v1.5,cross-encoder/ms-marco-MiniLM-L-6-v2,disabled,sentence-level,256,50,N/A,Hybrid,0.6,50,5,forward,N/A,N/A,short,0.2512,0.1084,0.6559,0.6095,0.3824,63/100
2,Prove Chunking,techqa,BAAI/bge-base-en-v1.5,cross-encoder/ms-marco-MiniLM-L-6-v2,disabled,token-level,256,50,206,Hybrid,0.6,50,5,forward,N/A,N/A,short,0.203,0.0917,0.6756,0.5817,0.3077,66/100
3,Prove Hybrid/Rerank,techqa,BAAI/bge-base-en-v1.5,BAAI/bge-reranker-base,disabled,token-level,256,50,206,Hybrid,0.6,50,5,forward,N/A,N/A,long,0.3895,0.1457,0.6639,0.5862,0.3889,13/100
4,Prove Repacking,techqa,BAAI/bge-base-en-v1.5,BAAI/bge-reranker-base,disabled,sentence-level,256,50,206,Hybrid,0.6,50,5,reverse,N/A,N/A,long,0.387,0.1531,0.6581,0.62,0.5,10/100
5,Prove Summarization,techqa,BAAI/bge-base-en-v1.5,BAAI/bge-reranker-base,enabled,token-level,256,50,206,Hybrid,0.6,50,5,reverse,150,20,long_cot,0.4656,0.7848,0.6406,Not available ,0,30/100
6,Optimized Tech Support,techqa,BAAI/bge-base-en-v1.5,BAAI/bge-reranker-base,disabled,token-level,256,50,206,Hybrid,0.5,50,5,reverse,N/A,N/A,long,0.4182,0.1016,0.6794,0.6667,0.5,13/100