Ibis0-BED-Simulator / README.md
sid-futurehouse's picture
Simplify simulator card title
a7fd0e9 verified
|
Raw History Blame Contribute Delete
2.66 kB
metadata
license: apache-2.0
base_model: Qwen/Qwen3-30B-A3B
library_name: transformers
pipeline_tag: text-generation
datasets:
  - EdisonScientific/EDB-Bio
language:
  - en
tags:
  - biology
  - experimental-design
  - simulation

Ibis0 BED outcome simulator

paper · code · data

The outcome simulator from the Bayesian experimental design (BED) experiments in "Do Language Models Know What Compelling Scientific Evidence Looks Like?". Given a candidate experiment with its result hidden, the hypothesis θ′ that the experiment itself tests, and a requested valence, it generates a plausible outcome: either compelling support for θ′ or a clear null. During experiment selection the two outcomes are weighted by the belief model's p(θ′ | evidence) to estimate expected information gain. The belief model is Ibis0-30B-A3B-Classifier.

Clone and install ibis0

The prompt builder and BED scripts below live in the EdisonScientific/ibis0 code repository. Install Python 3.12 and uv, then clone and install the repository:

git clone --recurse-submodules https://github.com/EdisonScientific/ibis0.git
cd ibis0
uv sync --frozen --all-groups

Run the referenced scripts from this repository with uv run python <script-path>. Local GPU inference requires Linux and a CUDA GPU.

Use

build_theta_prime_sft.py builds the simulator's prompts from an experiment, its hypothesis, and the requested outcome type. bed_thetaprime.py uses the simulator and classifier together to select experiments.

The simulator returns JSON with three fields: supports_hypothesis ("yes" or "no"), weight_of_evidence_decibans, and y, which contains the simulated measurements for the domain.

Licence

Apache-2.0. Fine-tuned from Qwen/Qwen3-30B-A3B (Apache-2.0).

Citation

@article{ibis0,
  title   = {Do Language Models Know What Compelling Scientific Evidence Looks Like?},
  author  = {Igoe, Conor and Grand, Gabriel and Koch, Zane and Bou, Albert and Moss, Eli and Hinks, Michaela M. and Rodriques, Samuel G. and White, Andrew and Narayanan, Siddharth},
  journal = {arXiv preprint arXiv:ARXIV_ID},
  year    = {2026}
}