eval/reports/beir_fiqa_retrieval_eval.md

Retrieval Eval: beir_fiqa

Metrics

• queries: 10
• documents: 500
• top_k: 5
• mrr: 0.8000
• ndcg_at_k: 0.6582
• hit_at_1: 0.8000
• hit_at_3: 0.8000
• hit_at_5: 0.8000

Sample Cases

8

How to deposit a cheque issued to an associate in my business into my business account?

• first_hit_rank: 1
• rank 1: hit=True doc_id=65404 score=0.6844510955827177
• rank 2: hit=False doc_id=508754 score=0.6415634192002271
• rank 3: hit=False doc_id=1873 score=0.6244133153886419
• rank 4: hit=False doc_id=590102 score=0.6106401478322256
• rank 5: hit=False doc_id=1066 score=0.5854493569389293

15

Can I send a money order from USPS as a business?

• first_hit_rank: 1
• rank 1: hit=True doc_id=325273 score=0.6860931820873509
• rank 2: hit=False doc_id=3714 score=0.5383410844537323
• rank 3: hit=False doc_id=508754 score=0.5295326644960427
• rank 4: hit=False doc_id=1873 score=0.5219679418951554
• rank 5: hit=False doc_id=4457 score=0.5122406473020094

18

1 EIN doing business under multiple business names

• first_hit_rank: 1
• rank 1: hit=True doc_id=88124 score=0.5926237160250162
• rank 2: hit=False doc_id=1873 score=0.5421392202098603
• rank 3: hit=False doc_id=248624 score=0.5355707959162649
• rank 4: hit=False doc_id=590102 score=0.5349105669189491
• rank 5: hit=False doc_id=1173 score=0.5304232255229728

26

Applying for and receiving business credit

• first_hit_rank: 1
• rank 1: hit=True doc_id=350819 score=0.6130084948278423
• rank 2: hit=False doc_id=2064 score=0.5484836878784439
• rank 3: hit=False doc_id=5019 score=0.545421752024407
• rank 4: hit=False doc_id=1873 score=0.5288677740902044
• rank 5: hit=False doc_id=1766 score=0.5277730439438229

34

401k Transfer After Business Closure

• first_hit_rank: None
• rank 1: hit=False doc_id=19183 score=0.5697281829712297
• rank 2: hit=False doc_id=1506 score=0.5606544069043923
• rank 3: hit=False doc_id=1134 score=0.5594801072658324
• rank 4: hit=False doc_id=3481 score=0.5580692841866827
• rank 5: hit=False doc_id=3059 score=0.5470931591486823

42

What are the ins/outs of writing equipment purchases off as business expenses in a home based business?

• first_hit_rank: 1
• rank 1: hit=True doc_id=272709 score=0.6108084707046366
• rank 2: hit=False doc_id=2528 score=0.5915589749452431
• rank 3: hit=True doc_id=331981 score=0.5819601957870557
• rank 4: hit=False doc_id=1873 score=0.5679211375564418
• rank 5: hit=True doc_id=327263 score=0.5609058973658579

56

Can a entrepreneur hire a self-employed business owner?

• first_hit_rank: 1
• rank 1: hit=True doc_id=572690 score=0.5928112761756716
• rank 2: hit=False doc_id=1873 score=0.5329399371121925
• rank 3: hit=False doc_id=350819 score=0.49122764843847383
• rank 4: hit=False doc_id=288 score=0.48281883887294536
• rank 5: hit=False doc_id=599545 score=0.4825679577769018

68

Intentions of Deductible Amount for Small Business

• first_hit_rank: None
• rank 1: hit=False doc_id=599545 score=0.5484593654392641
• rank 2: hit=False doc_id=350819 score=0.545089604374947
• rank 3: hit=False doc_id=327263 score=0.5425303284932907
• rank 4: hit=False doc_id=272709 score=0.5367760755311749
• rank 5: hit=False doc_id=1873 score=0.5341962558469263

89

How can I deposit a check made out to my business into my personal account?

• first_hit_rank: 1
• rank 1: hit=True doc_id=508754 score=0.678210930846752
• rank 2: hit=False doc_id=3336 score=0.6219187366693569
• rank 3: hit=False doc_id=1066 score=0.6102283456272309
• rank 4: hit=False doc_id=65404 score=0.6070578770706204
• rank 5: hit=True doc_id=413229 score=0.5974145307840032

90

Filing personal with 1099s versus business s-corp?

• first_hit_rank: 1
• rank 1: hit=True doc_id=31793 score=0.6463855238248295
• rank 2: hit=False doc_id=4992 score=0.575164246858743
• rank 3: hit=False doc_id=1873 score=0.567805853646443
• rank 4: hit=False doc_id=2020 score=0.5629015874196683
• rank 5: hit=False doc_id=350819 score=0.5607360854843948