Contributing to CAFF

Thank you for your interest in contributing to CAFF. This document explains how the project is structured and how to make changes that match the existing style.

Project structure

caff/                 Core implementation (paper Sections 6-7)
configs/              YAML training configurations
data/                 Real-data placeholder (KGs, QA splits)
examples/             Demonstration files (do NOT cite their numbers)
scripts/              Standalone utilities (KG building, BFS, annotation)
tests/                Unit tests + smoke fixtures
runs/                 Training artifacts (auto-generated)

Top-level entry points: train.py, evaluate.py, context_swap_diagnostic.py.

Setting up a development environment

git clone https://github.com/marwan8086/caff.git
cd caff
python -m venv .venv
source .venv/bin/activate          # Linux / macOS
# or:  .venv\Scripts\Activate.ps1  # Windows PowerShell

pip install -r requirements.txt

For the smoke test you also need to build the synthetic fixtures once:

python tests/fixtures/build_smoke_data.py

This produces tests/fixtures/smoke_kg.tsv and three QA JSON splits.

Running the test suite

The full suite must pass before any commit:

python -m pytest tests/ -v

Expected: 52 tests passing in roughly 5-10 seconds (depending on machine).

If you add new functionality, add a corresponding unit test in tests/. We prefer pytest's plain-function style over class-based tests; see existing test files for examples.

Smoke training (end-to-end check)

Before submitting changes that touch the training loop, run the CPU-friendly smoke training to verify the pipeline still trains end-to-end:

python train.py --config configs/caff_smoke.yaml --seed 42

Expected output (abridged):

KG loaded: |V|=5,000  |E|=24,995  |R|=20
Built 20,982 triple instances
[Epoch 1/2] loss=0.04xx  dev_f1=0.0xxx  ...
[Epoch 2/2] loss=0.04xx  dev_f1=0.0xxx  ...
Training complete.

Loss should decrease across epochs. F1 numbers will be small because the fixtures are random synthetic data; this is expected.

Coding style

Python 3.10+ syntax (we use X | Y unions, list[...], etc.)
Type-annotate function signatures
Docstrings reference the paper's equation numbers when applicable, e.g. Eq. 14, paper Section 8.4
Use ASCII characters in source files. Em-dashes, smart quotes, and other Unicode punctuation should be avoided in code, configs, and the README so that Windows cp1252 environments do not break logging
Source files use LF line endings (enforced by .gitattributes)
Configuration values must round-trip: every field added to CAFFConfig must be readable from YAML and validated in __post_init__

Validating configurations

CAFFConfig.__post_init__ raises ValueError for invalid combinations. Prefer raise ValueError(...) over assert, because python -O strips asserts in production. Example:

if self.rho >= self.d:
    raise ValueError(f"rho ({self.rho}) must be < d ({self.d})")

Reporting bugs

If you find a discrepancy between the paper and the code, add it to PAPER_DISCREPANCIES.md rather than silently editing the code or the paper. Include:

What the paper says
What the code produces
Which one is correct (with reasoning)
Recommended resolution

Pull requests

Create a topic branch off main
Make focused commits with clear messages (one logical change per commit)
Run pytest tests/ and the smoke training before pushing
Open a pull request describing the change and its motivation
Reference paper sections / equations when relevant

Recent milestones

The following used to be open issues and have since been resolved:

Phase 2 - DC mining (May 4, 2026): Implemented in caff/miners.py as DCMiner, wired into caff/trainer.py::__init__ and exercised by four new unit tests in tests/test_miners.py. See PAPER_DISCREPANCIES.md section 5 for the design notes.
Phase 3 - Real biomedical KG (May 4, 2026): scripts/build_kg.py now loads Orphanet TSV exports produced by convert_orphanet_xml_to_tsv.py, and merge_hpo_into_kg.py adds HPO + OMIM annotations to yield a 38,456-node / 291,335-edge KG (called merged_kg_v2.tsv).
3-seed validation (May 6, 2026): The full pipeline has been run with seeds 42 / 1337 / 2024 on 20K QA records, producing F1 = 0.522 +/- 0.001 on a held-out test set. See PAPER_DISCREPANCIES.md section 10 and the README's Implementation Reality Check.
Phase 5 - GPU + BioLinkBERT (May 13, 2026): Training migrated to a single 8 GB consumer GPU and the encoder swapped to michiyasunaga/BioLinkBERT-large. Same 20K / 3-seed protocol lifts test F1 to 0.5315 +/- 0.0003 (+1.8% over CPU baseline). See PAPER_DISCREPANCIES.md section 11 and the README's GPU + BioLinkBERT-Large upgrade (Phase 5) subsection.

Known gaps (good first issues)

These items would close the remaining gap between the as-shipped F1 = 0.522 and the paper's headline F1 = 0.79:

DisGeNET integration: add a gene-disease association layer to the KG. The current public DisGeNET tier requires registration and a manual license agreement; an issue is open to track API access.
Open Targets parquet pipeline: Open Targets ships association data in Parquet rather than TSV. A small adapter using pyarrow would let us reuse the merge logic in merge_hpo_into_kg.py.
Per-relation thresholds: per_hop_threshold_sweep.py tunes one theta per hop; a per-relation variant would help when the KG contains many high-volume relations of different signal strength (e.g. after the MONDO experiment in PAPER_DISCREPANCIES.md section 9).
MONDO follow-up: the experimental KG v3 built by merge_mondo_into_kg.py improves MAP and NDCG but hurts F1 because the model has not learned to suppress the new candidates. Longer training or candidate filtering should recover the lost precision.
Paper-spec 30-epoch training on GPU: Phase 5 used 10 epochs to match the CPU baseline budget. The paper specifies 30 epochs with patience 5. Running BioLinkBERT for 30 epochs might add another +0.02 to +0.05 F1 and is the cheapest remaining gain to chase (~3 hours total on an 8 GB GPU).

If you want to tackle any of these, please open an issue first so we can coordinate.

Contact

For questions about the algorithm or paper, contact the corresponding author:

Marwan Dhifallah (marwan@mail.dlut.edu.cn)
Yu Liu (supervisor) (yuliu@dlut.edu.cn)

For implementation-only questions, please open a GitHub issue.