From Tool State to Scientific Action: Auditing Decision Provenance in Biomedical Research Agents
Abstract
A scientific agent can give the same verdict while reading and citing different evidence. BioAgent-DF tests this separation using paired reference and candidate values saved at the same scientific computation site. It restores or transplants those values at their original coordinates, then runs the agent on healthy hardware. Across eight SciFact batches, Qwen3.5-4B shows repeatable citation changes in three capture-claim combinations, and Gemma 3 4B IT shows repeatable read and citation changes in four. These seven combinations cover two claims. Full restoration recovers the reference evidence sets while verdicts remain unchanged. Verdict changes observed under the baseline prompt did not recur under the explicit-example prompt. Complementary tool and optimizer experiments distinguish recovery of numerical state, recovery of ranked lists, and restoration of an agent's visible history. These results show that an audit should track both the evidence an agent uses and its final answer. In hERG cardiac-safety assay allocation, silent numerical faults changed testing decisions while the fixed-budget shortlist kept the same number of compounds labeled as blockers. After the agent had used faulty values, restoring its earlier execution state and supplying certified scores recovered all 39 changed decisions, whereas replacing the prediction file recovered none; no incorrect action was committed. A streaming implementation of the arithmetic checks reduced the total time for forward execution and verification by 42.4% relative to the original FP64 implementation, with identical flags and recovery outputs.
Community
Silent GPU faults change what AI agents decide: a drug-screening agent skipped safety tests for two known blockers
👋 Author here. We study how silent GPU faults, which return wrong values with no crash and no NaN/Inf, change the scientific decisions AI agents make.
We replayed captured faulty tool outputs into a hERG cardiac safety screening task with the native Biomni A1 agent, running the agent itself on healthy hardware:
- Changed testing decisions: Two compounds labeled as hERG blockers in the benchmark, K12252 and K1461, were assigned to advance without testing instead of the reference decision to perform patch clamp testing. Each change recurred in all three repeats of its affected source.
- Unchanged aggregate count: The shortlist still contained 351 labeled blockers out of 402 compounds. The count stayed the same; the selected compounds and testing assignments changed.
- Detection and cost: Checks of data integrity, finite values and probability ranges flagged 0/16 fault sources. Checksums on linear layers flagged 16/16. On the measured workload, streaming verification added 25.8% to the time required for the native forward pass.
- State recovery: Replacing the prediction file after the agent had read faulty values recovered 0/39 changed decisions. Restoring the execution state saved before that read and supplying certified values recovered 39/39.
Selected SciFact examples show a related separation: agents can return the same verdict while reading or citing different evidence.
Takeaway: Fixing a tool output may not undo decisions already formed from it. Recovery also needs to address the execution state that depends on those values.
🔗 Project page: https://intosyn.github.io/BioAgent-DF/
We’re preparing to release Aritheia, our tool for testing GPU numerical reliability, and are looking for independent testers. If you’re interested, reply with your GPU model and driver, CUDA and PyTorch versions. We welcome both runs with no detected anomalies and reproducible mismatches.
Happy to discuss the findings and help with testing!
Get this paper in your agent:
hf papers read 2611.01527 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper