| # Contributing to CAFF |
|
|
| Thank you for your interest in contributing to CAFF. This document explains |
| how the project is structured and how to make changes that match the |
| existing style. |
|
|
| --- |
|
|
| ## Project structure |
|
|
| ``` |
| caff/ Core implementation (paper Sections 6-7) |
| configs/ YAML training configurations |
| data/ Real-data placeholder (KGs, QA splits) |
| examples/ Demonstration files (do NOT cite their numbers) |
| scripts/ Standalone utilities (KG building, BFS, annotation) |
| tests/ Unit tests + smoke fixtures |
| runs/ Training artifacts (auto-generated) |
| ``` |
|
|
| Top-level entry points: `train.py`, `evaluate.py`, `context_swap_diagnostic.py`. |
|
|
| --- |
|
|
| ## Setting up a development environment |
|
|
| ```bash |
| git clone https://github.com/marwan8086/caff.git |
| cd caff |
| python -m venv .venv |
| source .venv/bin/activate # Linux / macOS |
| # or: .venv\Scripts\Activate.ps1 # Windows PowerShell |
| |
| pip install -r requirements.txt |
| ``` |
|
|
| For the smoke test you also need to build the synthetic fixtures once: |
|
|
| ```bash |
| python tests/fixtures/build_smoke_data.py |
| ``` |
|
|
| This produces `tests/fixtures/smoke_kg.tsv` and three QA JSON splits. |
|
|
| --- |
|
|
| ## Running the test suite |
|
|
| The full suite must pass before any commit: |
|
|
| ```bash |
| python -m pytest tests/ -v |
| ``` |
|
|
| Expected: 52 tests passing in roughly 5-10 seconds (depending on machine). |
|
|
| If you add new functionality, add a corresponding unit test in `tests/`. We |
| prefer pytest's plain-function style over class-based tests; see existing |
| test files for examples. |
|
|
| --- |
|
|
| ## Smoke training (end-to-end check) |
|
|
| Before submitting changes that touch the training loop, run the CPU-friendly |
| smoke training to verify the pipeline still trains end-to-end: |
|
|
| ```bash |
| python train.py --config configs/caff_smoke.yaml --seed 42 |
| ``` |
|
|
| Expected output (abridged): |
|
|
| ``` |
| KG loaded: |V|=5,000 |E|=24,995 |R|=20 |
| Built 20,982 triple instances |
| [Epoch 1/2] loss=0.04xx dev_f1=0.0xxx ... |
| [Epoch 2/2] loss=0.04xx dev_f1=0.0xxx ... |
| Training complete. |
| ``` |
|
|
| Loss should decrease across epochs. F1 numbers will be small because the |
| fixtures are random synthetic data; this is expected. |
|
|
| --- |
|
|
| ## Coding style |
|
|
| - Python 3.10+ syntax (we use `X | Y` unions, `list[...]`, etc.) |
| - Type-annotate function signatures |
| - Docstrings reference the paper's equation numbers when applicable, e.g. |
| `Eq. 14`, `paper Section 8.4` |
| - Use ASCII characters in source files. Em-dashes, smart quotes, and other |
| Unicode punctuation should be avoided in code, configs, and the README so |
| that Windows cp1252 environments do not break logging |
| - Source files use LF line endings (enforced by `.gitattributes`) |
| - Configuration values must round-trip: every field added to `CAFFConfig` |
| must be readable from YAML and validated in `__post_init__` |
|
|
| --- |
|
|
| ## Validating configurations |
|
|
| `CAFFConfig.__post_init__` raises `ValueError` for invalid combinations. |
| Prefer `raise ValueError(...)` over `assert`, because `python -O` strips |
| asserts in production. Example: |
|
|
| ```python |
| if self.rho >= self.d: |
| raise ValueError(f"rho ({self.rho}) must be < d ({self.d})") |
| ``` |
|
|
| --- |
|
|
| ## Reporting bugs |
|
|
| If you find a discrepancy between the paper and the code, add it to |
| `PAPER_DISCREPANCIES.md` rather than silently editing the code or the |
| paper. Include: |
|
|
| 1. What the paper says |
| 2. What the code produces |
| 3. Which one is correct (with reasoning) |
| 4. Recommended resolution |
|
|
| --- |
|
|
| ## Pull requests |
|
|
| 1. Create a topic branch off `main` |
| 2. Make focused commits with clear messages (one logical change per commit) |
| 3. Run `pytest tests/` and the smoke training before pushing |
| 4. Open a pull request describing the change and its motivation |
| 5. Reference paper sections / equations when relevant |
|
|
| --- |
|
|
| ## Recent milestones |
|
|
| The following used to be open issues and have since been resolved: |
|
|
| - **Phase 2 - DC mining** (May 4, 2026): Implemented in `caff/miners.py` as |
| `DCMiner`, wired into `caff/trainer.py::__init__` and exercised by four |
| new unit tests in `tests/test_miners.py`. See PAPER_DISCREPANCIES.md |
| section 5 for the design notes. |
| - **Phase 3 - Real biomedical KG** (May 4, 2026): `scripts/build_kg.py` |
| now loads Orphanet TSV exports produced by `convert_orphanet_xml_to_tsv.py`, |
| and `merge_hpo_into_kg.py` adds HPO + OMIM annotations to yield a |
| 38,456-node / 291,335-edge KG (called `merged_kg_v2.tsv`). |
| - **3-seed validation** (May 6, 2026): The full pipeline has been run with |
| seeds 42 / 1337 / 2024 on 20K QA records, producing |
| F1 = 0.522 +/- 0.001 on a held-out test set. See PAPER_DISCREPANCIES.md |
| section 10 and the README's Implementation Reality Check. |
| - **Phase 5 - GPU + BioLinkBERT** (May 13, 2026): Training migrated to a |
| single 8 GB consumer GPU and the encoder swapped to |
| `michiyasunaga/BioLinkBERT-large`. Same 20K / 3-seed protocol lifts |
| test F1 to 0.5315 +/- 0.0003 (+1.8% over CPU baseline). See |
| PAPER_DISCREPANCIES.md section 11 and the README's `GPU + BioLinkBERT-Large |
| upgrade (Phase 5)` subsection. |
|
|
| --- |
|
|
| ## Known gaps (good first issues) |
|
|
| These items would close the remaining gap between the as-shipped |
| F1 = 0.522 and the paper's headline F1 = 0.79: |
|
|
| - **DisGeNET integration**: add a gene-disease association layer to the |
| KG. The current public DisGeNET tier requires registration and a |
| manual license agreement; an issue is open to track API access. |
| - **Open Targets parquet pipeline**: Open Targets ships association data |
| in Parquet rather than TSV. A small adapter using `pyarrow` would let |
| us reuse the merge logic in `merge_hpo_into_kg.py`. |
| - **Per-relation thresholds**: `per_hop_threshold_sweep.py` tunes one |
| theta per hop; a per-relation variant would help when the KG contains |
| many high-volume relations of different signal strength (e.g. after |
| the MONDO experiment in PAPER_DISCREPANCIES.md section 9). |
| - **MONDO follow-up**: the experimental KG v3 built by |
| `merge_mondo_into_kg.py` improves MAP and NDCG but hurts F1 because |
| the model has not learned to suppress the new candidates. Longer |
| training or candidate filtering should recover the lost precision. |
| - **Paper-spec 30-epoch training on GPU**: Phase 5 used 10 epochs to |
| match the CPU baseline budget. The paper specifies 30 epochs with |
| patience 5. Running BioLinkBERT for 30 epochs might add another |
| +0.02 to +0.05 F1 and is the cheapest remaining gain to chase |
| (~3 hours total on an 8 GB GPU). |
|
|
| If you want to tackle any of these, please open an issue first so we can |
| coordinate. |
|
|
| --- |
|
|
| ## Contact |
|
|
| For questions about the algorithm or paper, contact the corresponding author: |
|
|
| - Marwan Dhifallah (marwan@mail.dlut.edu.cn) |
| - Yu Liu (supervisor) (yuliu@dlut.edu.cn) |
|
|
| For implementation-only questions, please open a GitHub issue. |
|
|
