Spaces:
Running
Running
| question_id,allow_tools,path_used,model,backend,stage,turn,label,tool,arguments,result,text,text_system,seconds,prompt_tokens,completion_tokens,error | |
| drug_tier,True,native,liquid/lfm-2.5-2.6b:free,openrouter,prompt,0,the prompt you sent,,,,"What is the formulary tier for zephadril, and does it require prior authorisation?","You are a hospital formulary assistant used in a graduate analytics class (MIS 752, Lab 6). You know NOTHING about the drugs or the arrival forecasts in this conversation except what your tools return. Rules: (1) if a tool can answer, call it instead of guessing; (2) never invent a drug fact, a formulary tier, a drug class or a number; (3) if a tool says there is no record, say plainly that there is no record; (4) answer in at most four sentences and name the tool result you are relying on.",0.0,0,0, | |
| drug_tier,True,native,liquid/lfm-2.5-2.6b:free,openrouter,model_turn,1,"model response, turn 1",,,,,,0.91,562,57, | |
| drug_tier,True,native,liquid/lfm-2.5-2.6b:free,openrouter,tool_call,1,the model asked for a tool,get_drug_info,"{""name"": ""zephadril""}",,"{""name"": ""zephadril""}",,0.0,0,0, | |
| drug_tier,True,native,liquid/lfm-2.5-2.6b:free,openrouter,tool_result,1,your function answered,get_drug_info,,"{""drug"": ""zephadril"", ""found"": true, ""synthetic"": true, ""warning"": ""SYNTHETIC TEACHING STUB -- these drug names are fictional and every value this stub returns was invented for MIS 752 Lab 6. Inside this lab the stub's answer is authoritative; outside it, nothing here is real. Never use for a clinical decision."", ""source"": ""MIS 752 Lab 6 invented formulary v1"", ""drug_class"": ""invented class 7A (teaching placeholder)"", ""route"": ""oral"", ""formulary_tier"": 3, ""prior_auth_required"": true, ""supply_days_per_fill"": 30, ""monitoring"": ""stub-monitor-1 (invented label)"", ""interaction_flags"": [""renavex"", ""cardiomyst""]}",,,6.0791999999310065e-05,0,0, | |
| drug_tier,True,native,liquid/lfm-2.5-2.6b:free,openrouter,model_turn,2,"model response, turn 2",,,,"Based on the MIS 752 Lab 6 synthetic formulary data, zephadril is assigned to **formulary tier 3** and **requires prior authorisation**. This information comes from the drug record retrieved via the `get_drug_info` tool.",,1.68,592,132, | |
| drug_tier,True,native,liquid/lfm-2.5-2.6b:free,openrouter,answer,2,the model's final answer,,,,"Based on the MIS 752 Lab 6 synthetic formulary data, zephadril is assigned to **formulary tier 3** and **requires prior authorisation**. This information comes from the drug record retrieved via the `get_drug_info` tool.",,0.0,0,0, | |
| drug_tier,False,tools-disabled,nvidia/nemotron-3.5-content-safety:free,openrouter,prompt,0,the prompt you sent,,,,"What is the formulary tier for zephadril, and does it require prior authorisation?","You are a hospital formulary assistant used in a graduate analytics class (MIS 752, Lab 6). You know NOTHING about the drugs or the arrival forecasts in this conversation except what your tools return. Rules: (1) if a tool can answer, call it instead of guessing; (2) never invent a drug fact, a formulary tier, a drug class or a number; (3) if a tool says there is no record, say plainly that there is no record; (4) answer in at most four sentences and name the tool result you are relying on.",0.0,0,0, | |
| drug_tier,False,tools-disabled,nvidia/nemotron-3.5-content-safety:free,openrouter,model_turn,1,"model response, turn 1",,,,User Safety: safe,,3.35,481,219, | |
| drug_tier,False,tools-disabled,nvidia/nemotron-3.5-content-safety:free,openrouter,answer,1,the model's final answer,,,,User Safety: safe,,0.0,0,0, | |
| drug_tier-forced,True,native,liquid/lfm-2.5-2.6b:free,openrouter,prompt,0,the prompt you sent,,,,"What is the formulary tier for zephadril, and does it require prior authorisation?","You are a hospital formulary assistant used in a graduate analytics class (MIS 752, Lab 6). You know NOTHING about the drugs or the arrival forecasts in this conversation except what your tools return. Rules: (1) if a tool can answer, call it instead of guessing; (2) never invent a drug fact, a formulary tier, a drug class or a number; (3) if a tool says there is no record, say plainly that there is no record; (4) answer in at most four sentences and name the tool result you are relying on.",0.0,0,0, | |
| drug_tier-forced,True,native,liquid/lfm-2.5-2.6b:free,openrouter,model_turn,1,"model response, turn 1",,,,,,0.91,562,57, | |
| drug_tier-forced,True,native,liquid/lfm-2.5-2.6b:free,openrouter,tool_call,1,the model asked for a tool,get_drug_info,"{""name"": ""zephadril""}",,"{""name"": ""zephadril""}",,0.0,0,0, | |
| drug_tier-forced,True,native,liquid/lfm-2.5-2.6b:free,openrouter,tool_result,1,your function answered,get_drug_info,,"{""drug"": ""zephadril"", ""found"": true, ""synthetic"": true, ""warning"": ""SYNTHETIC TEACHING STUB -- these drug names are fictional and every value this stub returns was invented for MIS 752 Lab 6. Inside this lab the stub's answer is authoritative; outside it, nothing here is real. Never use for a clinical decision."", ""source"": ""MIS 752 Lab 6 invented formulary v1"", ""drug_class"": ""invented class 7A (teaching placeholder)"", ""route"": ""oral"", ""formulary_tier"": 3, ""prior_auth_required"": true, ""supply_days_per_fill"": 30, ""monitoring"": ""stub-monitor-1 (invented label)"", ""interaction_flags"": [""renavex"", ""cardiomyst""]}",,,5.546500005948474e-05,0,0, | |
| drug_tier-forced,True,native,liquid/lfm-2.5-2.6b:free,openrouter,model_turn,2,"model response, turn 2",,,,"Based on the MIS 752 Lab 6 synthetic formulary data, zephadril is assigned to **formulary tier 3** and **requires prior authorisation**. This information comes from the drug record retrieved via the `get_drug_info` tool.",,1.68,592,132, | |
| drug_tier-forced,True,native,liquid/lfm-2.5-2.6b:free,openrouter,answer,2,the model's final answer,,,,"Based on the MIS 752 Lab 6 synthetic formulary data, zephadril is assigned to **formulary tier 3** and **requires prior authorisation**. This information comes from the drug record retrieved via the `get_drug_info` tool.",,0.0,0,0, | |
| ed_surge,True,native,cohere/north-mini-code:free,openrouter,prompt,0,the prompt you sent,,,,"Starting 2026-10-05, forecast ED arrivals for the next 7 days. Which day is the peak and how many days are flagged as a surge?","You are a hospital formulary assistant used in a graduate analytics class (MIS 752, Lab 6). You know NOTHING about the drugs or the arrival forecasts in this conversation except what your tools return. Rules: (1) if a tool can answer, call it instead of guessing; (2) never invent a drug fact, a formulary tier, a drug class or a number; (3) if a tool says there is no record, say plainly that there is no record; (4) answer in at most four sentences and name the tool result you are relying on.",0.0,0,0, | |
| ed_surge,True,native,cohere/north-mini-code:free,openrouter,model_turn,1,"model response, turn 1",,,," | |
| ",,2.98,481,164, | |
| ed_surge,True,native,cohere/north-mini-code:free,openrouter,tool_call,1,the model asked for a tool,forecast_ed_arrivals,"{""shift_date"": ""2026-10-05"", ""horizon_days"": 7}",,"{""shift_date"": ""2026-10-05"", ""horizon_days"": 7}",,0.0,0,0, | |
| ed_surge,True,native,cohere/north-mini-code:free,openrouter,tool_result,1,your function answered,forecast_ed_arrivals,,"{""synthetic"": true, ""warning"": ""SYNTHETIC TEACHING STUB -- this forecast was invented for MIS 752 Lab 6 by a fixed formula, not fitted to any hospital's data. Within this lab the numbers are the authoritative answer; outside it, nothing here is real. Never use for staffing."", ""model"": ""stub seasonal-naive (invented for teaching)"", ""start"": ""2026-10-05"", ""horizon_days"": 7, ""peak_day"": ""2026-10-10"", ""peak_arrivals"": 243, ""surge_days"": 3, ""days"": [{""date"": ""2026-10-05"", ""weekday"": ""Mon"", ""predicted_arrivals"": 201, ""ci_low"": 179, ""ci_high"": 223, ""staffing_flag"": ""surge""}, {""date"": ""2026-10-06"", ""weekday"": ""Tue"", ""predicted_arrivals"": 199, ""ci_low"": 178, ""ci_high"": 220, ""staffing_flag"": ""normal""}, {""date"": ""2026-10-07"", ""weekday"": ""Wed"", ""predicted_arrivals"": 181, ""ci_low"": 162, ""ci_high"": 200, ""staffing_flag"": ""normal""}, {""date"": ""2026-10-08"", ""weekday"": ""Thu"", ""predicted_arrivals"": 185, ""ci_low"": 165, ""ci_high"": 205, ""staffing_flag"": ""normal""}, {""date"": ""2026-10-09"", ""weekday"": ""Fri"", ""predicted_arrivals"": 229, ""ci_low"": 204, ""ci_high"": 254, ""staffing_flag"": ""surge""}, {""date"": ""2026-10-10"", ""weekday"": ""Sat"", ""predicted_arrivals"": 243, ""ci_low"": 217, ""ci_high"": 269, ""staffing_flag"": ""surge""}, {""date"": ""2026-10-11"", ""weekday"": ""Sun"", ""predicted_arrivals"": 181, ""ci_low"": 162, ""ci_high"": 200, ""staffing_flag"": ""normal""}]}",,,0.0003931439999860231,0,0, | |
| ed_surge,True,native,cohere/north-mini-code:free,openrouter,model_turn,2,"model response, turn 2",,,,"The peak day is 2026-10-10 (Saturday) with 243 predicted arrivals. There are 3 days flagged as a surge: 2026-10-05, 2026-10-09, and 2026-10-10. This information comes from the forecast_ed_arrivals tool result.",,1.71,662,196, | |
| ed_surge,True,native,cohere/north-mini-code:free,openrouter,answer,2,the model's final answer,,,,"The peak day is 2026-10-10 (Saturday) with 243 predicted arrivals. There are 3 days flagged as a surge: 2026-10-05, 2026-10-09, and 2026-10-10. This information comes from the forecast_ed_arrivals tool result.",,0.0,0,0, | |
| drug_flags,True,native,poolside/laguna-s-2.1:free,openrouter,prompt,0,the prompt you sent,,,,"Which other formulary drugs is renavex flagged against, and what monitoring label does it carry?","You are a hospital formulary assistant used in a graduate analytics class (MIS 752, Lab 6). You know NOTHING about the drugs or the arrival forecasts in this conversation except what your tools return. Rules: (1) if a tool can answer, call it instead of guessing; (2) never invent a drug fact, a formulary tier, a drug class or a number; (3) if a tool says there is no record, say plainly that there is no record; (4) answer in at most four sentences and name the tool result you are relying on.",0.0,0,0, | |
| drug_flags,True,native,poolside/laguna-s-2.1:free,openrouter,model_turn,1,"model response, turn 1",,,," | |
| ",,0.95,597,93, | |
| drug_flags,True,native,poolside/laguna-s-2.1:free,openrouter,tool_call,1,the model asked for a tool,get_drug_info,"{""name"": ""renavex""}",,"{""name"": ""renavex""}",,0.0,0,0, | |
| drug_flags,True,native,poolside/laguna-s-2.1:free,openrouter,tool_result,1,your function answered,get_drug_info,,"{""drug"": ""renavex"", ""found"": true, ""synthetic"": true, ""warning"": ""SYNTHETIC TEACHING STUB -- these drug names are fictional and every value this stub returns was invented for MIS 752 Lab 6. Inside this lab the stub's answer is authoritative; outside it, nothing here is real. Never use for a clinical decision."", ""source"": ""MIS 752 Lab 6 invented formulary v1"", ""drug_class"": ""invented class 2C (teaching placeholder)"", ""route"": ""oral"", ""formulary_tier"": 1, ""prior_auth_required"": false, ""supply_days_per_fill"": 90, ""monitoring"": ""stub-monitor-2 (invented label)"", ""interaction_flags"": [""zephadril"", ""neurvolan""]}",,,4.136600000492763e-05,0,0, | |
| drug_flags,True,native,poolside/laguna-s-2.1:free,openrouter,model_turn,2,"model response, turn 2",,,,,,4.83,838,203, | |
| drug_flags,True,native,poolside/laguna-s-2.1:free,openrouter,tool_call,2,the model asked for a tool,get_drug_info,"{""name"": ""renavex""}",,"{""name"":""renavex""}",,0.0,0,0, | |
| drug_flags,True,native,poolside/laguna-s-2.1:free,openrouter,tool_result,2,your function answered,get_drug_info,,"{""drug"": ""renavex"", ""found"": true, ""synthetic"": true, ""warning"": ""SYNTHETIC TEACHING STUB -- these drug names are fictional and every value this stub returns was invented for MIS 752 Lab 6. Inside this lab the stub's answer is authoritative; outside it, nothing here is real. Never use for a clinical decision."", ""source"": ""MIS 752 Lab 6 invented formulary v1"", ""drug_class"": ""invented class 2C (teaching placeholder)"", ""route"": ""oral"", ""formulary_tier"": 1, ""prior_auth_required"": false, ""supply_days_per_fill"": 90, ""monitoring"": ""stub-monitor-2 (invented label)"", ""interaction_flags"": [""zephadril"", ""neurvolan""]}",,,6.249999989904609e-05,0,0, | |
| drug_flags,True,native,poolside/laguna-s-2.1:free,openrouter,model_turn,3,"model response, turn 3",,,,"Renavex is flagged against **zephadril** and **neurvolan**, and it carries the monitoring label **stub-monitor-2 (invented label)** (source: MIS 752 Lab 6 invented formulary v1).",,1.92,844,52, | |
| drug_flags,True,native,poolside/laguna-s-2.1:free,openrouter,answer,3,the model's final answer,,,,"Renavex is flagged against **zephadril** and **neurvolan**, and it carries the monitoring label **stub-monitor-2 (invented label)** (source: MIS 752 Lab 6 invented formulary v1).",,0.0,0,0, | |
| drug_flags,False,tools-disabled,apodex/apodex-1.1-mini:free,openrouter,prompt,0,the prompt you sent,,,,"Which other formulary drugs is renavex flagged against, and what monitoring label does it carry?","You are a hospital formulary assistant used in a graduate analytics class (MIS 752, Lab 6). You know NOTHING about the drugs or the arrival forecasts in this conversation except what your tools return. Rules: (1) if a tool can answer, call it instead of guessing; (2) never invent a drug fact, a formulary tier, a drug class or a number; (3) if a tool says there is no record, say plainly that there is no record; (4) answer in at most four sentences and name the tool result you are relying on.",0.0,0,0, | |
| drug_flags,False,tools-disabled,apodex/apodex-1.1-mini:free,openrouter,model_turn,1,"model response, turn 1",,,,,,8.31,156,2000,"TRUNCATED: finish_reason='length', 2000 completion tokens of which 2104 went to hidden reasoning, and no answer text. Raise MAX_TOKENS (currently 2000) and re-run." | |
| drug_flags,False,tools-disabled,apodex/apodex-1.1-mini:free,openrouter,answer,1,the model's final answer,,,,,,0.0,0,0, | |
| drug_absent,True,native,inclusionai/ling-3.0-flash-sante:free,openrouter,prompt,0,the prompt you sent,,,,"What is the formulary tier for xyloprane? If you cannot find it, say so plainly.","You are a hospital formulary assistant used in a graduate analytics class (MIS 752, Lab 6). You know NOTHING about the drugs or the arrival forecasts in this conversation except what your tools return. Rules: (1) if a tool can answer, call it instead of guessing; (2) never invent a drug fact, a formulary tier, a drug class or a number; (3) if a tool says there is no record, say plainly that there is no record; (4) answer in at most four sentences and name the tool result you are relying on.",0.0,0,0, | |
| drug_absent,True,native,inclusionai/ling-3.0-flash-sante:free,openrouter,model_turn,1,"model response, turn 1",,,,,,0.57,605,48, | |
| drug_absent,True,native,inclusionai/ling-3.0-flash-sante:free,openrouter,tool_call,1,the model asked for a tool,get_drug_info,"{""name"": ""xyloprane""}",,"{""name"":""xyloprane""}",,0.0,0,0, | |
| drug_absent,True,native,inclusionai/ling-3.0-flash-sante:free,openrouter,tool_result,1,your function answered,get_drug_info,,"{""drug"": ""xyloprane"", ""found"": false, ""synthetic"": true, ""warning"": ""SYNTHETIC TEACHING STUB -- these drug names are fictional and every value this stub returns was invented for MIS 752 Lab 6. Inside this lab the stub's answer is authoritative; outside it, nothing here is real. Never use for a clinical decision."", ""message"": ""xyloprane is NOT in the MIS 752 Lab 6 invented formulary v1. There is no record to report. Say that there is no record; do not substitute a guess."", ""known_drugs"": [""cardiomyst"", ""glycofane"", ""neurvolan"", ""pulmosyn"", ""renavex"", ""zephadril""]}",,,5.375499995352584e-05,0,0, | |
| drug_absent,True,native,inclusionai/ling-3.0-flash-sante:free,openrouter,model_turn,2,"model response, turn 2",,,,"There is no record for xyloprane in the MIS 752 Lab 6 formulary — the lookup returned ""found: false."" I cannot assign a tier because the drug is not in the formulary. (I am relying on the `get_drug_info` tool result, which explicitly states xyloprane is not in the invented formulary v1.)",,1.13,774,111, | |
| drug_absent,True,native,inclusionai/ling-3.0-flash-sante:free,openrouter,answer,2,the model's final answer,,,,"There is no record for xyloprane in the MIS 752 Lab 6 formulary — the lookup returned ""found: false."" I cannot assign a tier because the drug is not in the formulary. (I am relying on the `get_drug_info` tool result, which explicitly states xyloprane is not in the invented formulary v1.)",,0.0,0,0, | |
| drug_absent,False,tools-disabled,inclusionai/ling-3.0-flash-sante:free,openrouter,prompt,0,the prompt you sent,,,,"What is the formulary tier for xyloprane? If you cannot find it, say so plainly.","You are a hospital formulary assistant used in a graduate analytics class (MIS 752, Lab 6). You know NOTHING about the drugs or the arrival forecasts in this conversation except what your tools return. Rules: (1) if a tool can answer, call it instead of guessing; (2) never invent a drug fact, a formulary tier, a drug class or a number; (3) if a tool says there is no record, say plainly that there is no record; (4) answer in at most four sentences and name the tool result you are relying on.",0.0,0,0, | |
| drug_absent,False,tools-disabled,inclusionai/ling-3.0-flash-sante:free,openrouter,model_turn,1,"model response, turn 1",,,,"<tool_call>search_formulary | |
| <arg_key>drug_name</arg_key> | |
| <arg_value>xyloprane</arg_value> | |
| </tool_call>",,1.13,165,61, | |
| drug_absent,False,tools-disabled,inclusionai/ling-3.0-flash-sante:free,openrouter,answer,1,the model's final answer,,,,"<tool_call>search_formulary | |
| <arg_key>drug_name</arg_key> | |
| <arg_value>xyloprane</arg_value> | |
| </tool_call>",,0.0,0,0, | |
| ed_surge,False,tools-disabled,liquid/lfm-2.5-2.6b:free,openrouter,prompt,0,the prompt you sent,,,,"Starting 2026-10-05, forecast ED arrivals for the next 7 days. Which day is the peak and how many days are flagged as a surge?","You are a hospital formulary assistant used in a graduate analytics class (MIS 752, Lab 6). You know NOTHING about the drugs or the arrival forecasts in this conversation except what your tools return. Rules: (1) if a tool can answer, call it instead of guessing; (2) never invent a drug fact, a formulary tier, a drug class or a number; (3) if a tool says there is no record, say plainly that there is no record; (4) answer in at most four sentences and name the tool result you are relying on.",0.0,0,0, | |
| ed_surge,False,tools-disabled,liquid/lfm-2.5-2.6b:free,openrouter,model_turn,1,"model response, turn 1",,,,"<|tool_call_start|>[tool_call(name='get_forecast', arguments={'start_date': '2026-10-05', 'days': 7})]<|tool_call_end|>",,6.18,173,609, | |
| ed_surge,False,tools-disabled,liquid/lfm-2.5-2.6b:free,openrouter,answer,1,the model's final answer,,,,"<|tool_call_start|>[tool_call(name='get_forecast', arguments={'start_date': '2026-10-05', 'days': 7})]<|tool_call_end|>",,0.0,0,0, | |