| --- |
| language: |
| - en |
| license: other |
| tags: |
| - rare-disease |
| - synthetic |
| - clinical-notes |
| - phenotype |
| - HPO |
| - gene-prioritization |
| - differential-diagnosis |
| - parquet |
| pretty_name: Rare Disease Synthetic Diagnosis v2 |
| --- |
| |
| # Rare Disease Synthetic Diagnosis (v2) |
|
|
| **Fully synthetic** rare-disease clinical notes and multi-task diagnosis / gene / variant training rows. |
| **Not for clinical use.** Contains **no real patient data** and **no RareArena case reports**. |
|
|
| > **Hub note:** Assets live on the **model** repo [`iLOVE2D/rare_sync`](https://huggingface.co/iLOVE2D/rare_sync) |
| > (`repo_type=model`). Hugging Face **native** `load_dataset("{repo}")` expects a |
| > *dataset* repo, so it will **not** resolve this URL automatically. Use download-based |
| > loading below (or mirror the parquet trees into a dataset repo if you need Hub configs). |
|
|
| ## Contents |
|
|
| | Config | Split | Rows | Description | |
| |--------|-------|------|-------------| |
| | `cases` | train / validation | **62,004** / **1,960** | Case-level records (HPO phenotypes, genes, variant tables, clinical notes) | |
| | `tasks` | train / validation | **132,619** / **600** | Multi-task verl-format rows (`synth_dx`, `synth_gene`, `synth_ddx10`, `synth_gene10`, `synth_mcq`, `synth_variant`) | |
| | `cot_sft` | train | **10,141** | Rejection-sampled chain-of-thought (Qwen3-8B teacher; see caveats) | |
|
|
| Note sources among cases: `{"llm_rewrite": 63956, "template": 8}`. |
|
|
| ## Load (download-based; model repo) |
|
|
| ```python |
| from pathlib import Path |
| from huggingface_hub import snapshot_download |
| from datasets import load_dataset |
| |
| root = Path(snapshot_download("iLOVE2D/rare_sync", repo_type="model")) |
| cases = load_dataset("parquet", data_files={ |
| "train": str(root / "cases" / "train-*.parquet"), |
| "validation": str(root / "cases" / "validation-*.parquet"), |
| }) |
| tasks = load_dataset("parquet", data_files={ |
| "train": str(root / "tasks" / "train-*.parquet"), |
| "validation": str(root / "tasks" / "validation-*.parquet"), |
| }) |
| cot = load_dataset("parquet", data_files=str(root / "cot_sft" / "train-*.parquet")) |
| ``` |
|
|
| Or read shards directly with `pandas.read_parquet` / `pyarrow.parquet`. |
|
|
| ## Provenance (summary) |
|
|
| - Generator: `hpoa_sampler_v2` (`pipeline/configs/v2.yaml`) |
| - KB: HPO + HPOA + genes_to_phenotype (see `kb_dir` in config); HGNC + MONDO refs under `data/raw/ref` |
| - Note rewrite teacher: **Qwen/Qwen3-8B** (6 styles); template fallback when rewrite missing |
| - Eval holdouts (phenopacket / mygene2 / rarebench / RDS-RDC genes & diseases) excluded at world build |
| - Val = 3.0% of MONDO groups held out entirely |
| - Git commit: `862c51d768af48b5032c14005884df5163a8038e` |
| - Packaged: `2026-10-22T00:31:33.654005+00:00` |
| - Full machine-readable provenance: `provenance.json` in this repo |
| - Pipeline snapshot: `pipeline/` (source + relative configs + `DEPENDENCIES.md`) |
|
|
| ## Calibration (difficulty) |
|
|
| Baseline phenotype→disease hit@1 (from `data/reports/calibration.jsonl`): |
|
|
| | Set | hit@1 | hit@10 | median rank | |
| |-----|-------|--------|-------------| |
| | phenopacket_test_500 | 0.228 | 0.464 | 13 | |
| | mygene2 | 0.148 | 0.345 | 29 | |
| | v2_records (synth) | 0.543 | 0.843 | 1 | |
| |
| Synthetic cases are **easier** than real eval sets; do not treat synth accuracy as clinical performance. |
| |
| ## CoT caveat |
| |
| `cot_sft` was produced with K=4 rejection sampling + answer-hinted rationalization on unsolved prompts |
| (~76% rationalized). The Qwen3-8B teacher introduces knowledge errors; regenerate with a stronger |
| teacher before relying on CoT for supervised fine-tuning. |
|
|
| ## What is intentionally excluded |
|
|
| - RareArena / real case-report text (`*_mixed` exports, `sft_ra*`) |
| - Model checkpoints, wandb secrets, API keys |
| - Raw redistributable KB dumps (rebuild from public HPO/MONDO/HGNC sources) |
| - Logs, caches, `__pycache__` |
|
|
| ## License / use |
|
|
| Ontology sources retain their upstream licenses (HPO, MONDO, HGNC, OMIM identifiers as references). |
| Synthetic text is provided for **research and model training only**. Do not use for diagnosis or patient care. |
|
|
| ## Reproduction |
|
|
| ```bash |
| # after cloning pipeline/ or the parent project |
| python scripts/generate_v2.py |
| sbatch scripts/vllm_rewrite.sbatch |
| python scripts/export_v2.py |
| python scripts/package_hf_dataset.py --upload --repo_id iLOVE2D/rare_sync |
| ``` |
|
|
