paper_id,title,domain,doi,pmc,url,github,dataset_scope,task_count,task_names,source_files,notes
rx-llm,Rx-Bench: a benchmarking suite to evaluate safe large language model performance for medication-related tasks,Comprehensive Medication Management,10.64898/2025.12.01.25341004,,https://www.medrxiv.org/content/10.64898/2025.12.01.25341004v2,https://github.com/AIChemist-Lab,6 CMM benchmark definitions; 250 clinician-annotated cases per benchmark,6,Formulation Matching; Drug Order Gen (Sig); Route Matching; Rx-Bench DDI ID; Renal Dose ID; Drug-Indication,docs/appendices/sources/rx_llm_tables_2_3.csv; Rx-Bench manuscript Tables 2-3,Dashboard score is the macro mean of six primary task metrics from Rx-Bench Tables 2-3; Scenarios exposes each benchmark task separately.
ddi-identification,Drug-drug interaction identification using large language models,Drug-drug interaction identification,10.64898/2025.12.03.25341549,,https://www.medrxiv.org/content/10.64898/2025.12.03.25341549v2,https://github.com/AIChemist-Lab/LLM-DDI,"750 clinician-annotated DDI scenarios across pointwise, pairwise, and listwise formats",3,DDI ID; DDI 3-Drug Combo; DDI Multi-Drug,docs/appendices/sources/ddi_identification_table3.csv; server/data/benchmark.json,Scores are Table 3 experiment-level accuracies.
medmatch,MedMatch: a first step for the automation of large language model performance benchmarking for medication-related tasks,Medication order structuring and route selection,10.64898/2026.01.13.26343949,PMC12870651,https://pmc.ncbi.nlm.nih.gov/articles/PMC12870651/,https://github.com/AIChemist-Lab/MedMatch,100 clinician-annotated medication prompts; JSON slot filling and route selection,7,MedMatch (Oral Solid); MedMatch (Oral Liq); MedMatch (IV Intermit); MedMatch (IV Push); MedMatch (Continuous Titrate); MedMatch (Continuous Non-Titrate); MedMatch Route Selection,docs/appendices/sources/entity_accuracy_table.csv; docs/appendices/sources/route_accuracy_table.csv; server/data/benchmark.json,Scores are source-derived aggregates from MedMatch entity and route source tables; DrugGPT and MedGemma-27B are not reported.
drug-or-pokemon,Drug or Pokemon? Large language model performance in identification of fabricated medications,Adversarial fabricated-medication safety,10.64898/2026.01.12.26343930,PMC12870567,https://pmc.ncbi.nlm.nih.gov/articles/PMC12870567/,https://github.com/AIChemist-Lab/Pokemon-Drugs-Names,250 adversarial vignettes with fabricated generic and brand medication names,2,Pokémon (Generic); Pokémon (Brand),server/scripts/build_benchmark_from_appendices.py embedded PMC Table 2 rates; server/data/benchmark.json,"Scores are suspicion detected = 100 - default drug-dosing confabulation rate; GPT-5 Chat, MedGemma-27B, and DrugGPT are not reported."