docs/appendices/paper_data/paper_task_map.csv

paper_id,paper_title,task_name,dataset_slice,metrics,source_status,canonical_score_source,notes
rx-llm,Rx-Bench: a benchmarking suite to evaluate safe large language model performance for medication-related tasks,Formulation Matching,250 cases,Precision; Recall; F1-score; Accuracy; Correctness consistency,published_table,docs/appendices/sources/rx_llm_tables_2_3.csv; Rx-Bench manuscript Tables 2-3,Uses Rx-Bench primary metric (F1) reported in Tables 2-3; base Gemma 3 27B remains N/A because MedGemma-27B is reported separately.
rx-llm,Rx-Bench: a benchmarking suite to evaluate safe large language model performance for medication-related tasks,Drug Order Gen (Sig),250 cases,Exact match; HAMeC score; Correctness consistency,published_table,docs/appendices/sources/rx_llm_tables_2_3.csv; Rx-Bench manuscript Tables 2-3,Uses Rx-Bench primary metric (Overall accuracy) reported in Tables 2-3; base Gemma 3 27B remains N/A because MedGemma-27B is reported separately.
rx-llm,Rx-Bench: a benchmarking suite to evaluate safe large language model performance for medication-related tasks,Route Matching,250 cases,Precision; Recall; F1-score; Accuracy; Correctness consistency,published_table,docs/appendices/sources/rx_llm_tables_2_3.csv; Rx-Bench manuscript Tables 2-3,Uses Rx-Bench primary metric (F1) reported in Tables 2-3; base Gemma 3 27B remains N/A because MedGemma-27B is reported separately.
rx-llm,Rx-Bench: a benchmarking suite to evaluate safe large language model performance for medication-related tasks,Rx-Bench DDI ID,pointwise two-drug format,Precision; Recall; F1-score; Accuracy; Self-consistency,published_table,docs/appendices/sources/rx_llm_tables_2_3.csv; Rx-Bench manuscript Tables 2-3,Uses Rx-Bench primary metric (Accuracy) reported in Tables 2-3; base Gemma 3 27B remains N/A because MedGemma-27B is reported separately.
rx-llm,Rx-Bench: a benchmarking suite to evaluate safe large language model performance for medication-related tasks,Renal Dose ID,250 cases,Precision; Recall; F1-score; Exact match; Correctness consistency,published_table,docs/appendices/sources/rx_llm_tables_2_3.csv; Rx-Bench manuscript Tables 2-3,Uses Rx-Bench primary metric (F1) reported in Tables 2-3; base Gemma 3 27B remains N/A because MedGemma-27B is reported separately.
rx-llm,Rx-Bench: a benchmarking suite to evaluate safe large language model performance for medication-related tasks,Drug-Indication,250 cases,Precision; Recall; F1-score; Accuracy; Correctness consistency,published_table,docs/appendices/sources/rx_llm_tables_2_3.csv; Rx-Bench manuscript Tables 2-3,Uses Rx-Bench primary metric (Accuracy) reported in Tables 2-3; base Gemma 3 27B remains N/A because MedGemma-27B is reported separately.
ddi-identification,Drug-drug interaction identification using large language models,DDI ID,pointwise two-drug format,Precision; Recall; F1-score; Accuracy; Self-consistency,published_table,docs/appendices/sources/ddi_identification_table3.csv; server/data/benchmark.json,Table 3 row: DDI ID.
ddi-identification,Drug-drug interaction identification using large language models,DDI 3-Drug Combo,pairwise three-drug format,Precision; Recall; F1-score; Accuracy; Self-consistency,published_table,docs/appendices/sources/ddi_identification_table3.csv; server/data/benchmark.json,Table 3 row: DDI 3-Drug Combo.
ddi-identification,Drug-drug interaction identification using large language models,DDI Multi-Drug,listwise 4-6 drug format,Precision; Recall; F1-score; Accuracy; Self-consistency,published_table,docs/appendices/sources/ddi_identification_table3.csv; server/data/benchmark.json,Table 3 row: DDI Multi-Drug.
medmatch,MedMatch: a first step for the automation of large language model performance benchmarking for medication-related tasks,MedMatch (Oral Solid),n=40,MedMatch score (exact field match); Micro-F1,source_derived_aggregate,docs/appendices/sources/entity_accuracy_table.csv; docs/appendices/sources/route_accuracy_table.csv; server/data/benchmark.json,Entity accuracy category: Oral Solid.
medmatch,MedMatch: a first step for the automation of large language model performance benchmarking for medication-related tasks,MedMatch (Oral Liq),n=10,MedMatch score (exact field match); Micro-F1,source_derived_aggregate,docs/appendices/sources/entity_accuracy_table.csv; docs/appendices/sources/route_accuracy_table.csv; server/data/benchmark.json,Entity accuracy category: Oral Liq.
medmatch,MedMatch: a first step for the automation of large language model performance benchmarking for medication-related tasks,MedMatch (IV Intermit),n=17,MedMatch score (exact field match); Micro-F1,source_derived_aggregate,docs/appendices/sources/entity_accuracy_table.csv; docs/appendices/sources/route_accuracy_table.csv; server/data/benchmark.json,Entity accuracy category: IV Intermit.
medmatch,MedMatch: a first step for the automation of large language model performance benchmarking for medication-related tasks,MedMatch (IV Push),n=17,MedMatch score (exact field match); Micro-F1,source_derived_aggregate,docs/appendices/sources/entity_accuracy_table.csv; docs/appendices/sources/route_accuracy_table.csv; server/data/benchmark.json,Entity accuracy category: IV Push.
medmatch,MedMatch: a first step for the automation of large language model performance benchmarking for medication-related tasks,MedMatch (Continuous Titrate),n=11,MedMatch score (exact field match); Micro-F1,source_derived_aggregate,docs/appendices/sources/entity_accuracy_table.csv; docs/appendices/sources/route_accuracy_table.csv; server/data/benchmark.json,Entity accuracy category: Continuous Titrate.
medmatch,MedMatch: a first step for the automation of large language model performance benchmarking for medication-related tasks,MedMatch (Continuous Non-Titrate),n=6,MedMatch score (exact field match); Micro-F1,source_derived_aggregate,docs/appendices/sources/entity_accuracy_table.csv; docs/appendices/sources/route_accuracy_table.csv; server/data/benchmark.json,Entity accuracy category: Continuous Non-Titrate.
medmatch,MedMatch: a first step for the automation of large language model performance benchmarking for medication-related tasks,MedMatch Route Selection,route categories,Route accuracy; MedMatch score,source_derived_aggregate,docs/appendices/sources/entity_accuracy_table.csv; docs/appendices/sources/route_accuracy_table.csv; server/data/benchmark.json,Route accuracy table mean across route categories.
drug-or-pokemon,Drug or Pokemon? Large language model performance in identification of fabricated medications,Pokémon (Generic),250 generic-name poisoned vignettes,LLM-as-a-judge; Confabulation rate,computed_from_pmc_table_2,server/scripts/build_benchmark_from_appendices.py embedded PMC Table 2 rates; server/data/benchmark.json,Suspicion detected = 100 - default dosing confabulation rate.
drug-or-pokemon,Drug or Pokemon? Large language model performance in identification of fabricated medications,Pokémon (Brand),250 brand-name poisoned vignettes,LLM-as-a-judge; Confabulation rate,computed_from_pmc_table_2,server/scripts/build_benchmark_from_appendices.py embedded PMC Table 2 rates; server/data/benchmark.json,Suspicion detected = 100 - default dosing confabulation rate.