# Contributing to PACTBench

## Development setup

```bash
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -r requirements.txt
```

Before submitting a change, run:

```bash
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests -v
python3 pact/validate_pactbench.py --require_v5
```

Shell entry points must also pass `bash -n`.

## Code and experiment changes

- Put shared Python implementation in `pact/`.
- Keep per-game files in `games/<game>/` as thin wrappers and configuration.
- Do not hand-edit published aggregate values in `registry/registry.json`.
- A result is publishable only when its code, configuration, ordered rows,
  metrics, model identity, and source identities are available to the gate.
- Keep unreviewed runs outside the published registry and README results. Follow
  the review-isolation procedure in `docs/MAINTAINERS.md`.

## Repository hygiene

Do not commit:

- API keys, access tokens, private endpoints, or credential files;
- machine-specific absolute paths in code or documentation;
- raw caches that can be rebuilt from documented inputs;
- internal reviews, meeting notes, temporary reports, or backup copies;
- new headline numbers that have not passed maintainer review.

Use environment variables for model paths, service URLs, and credentials.
Keep one canonical implementation for every runnable tool; avoid copied scripts
that can silently diverge.

## Documentation

Update the narrowest relevant document. Dataset facts belong in `docs/DATA.md`,
experiment definitions and results in `docs/EXPERIMENTS.md`, commands in
`docs/REPRODUCE.md`, and game-specific entry points in the corresponding game
README. Historical reasoning and future experiment wish lists do not belong in
the release documentation.