Image-level person classification on EUPE-ViT-B features with no free parameters

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1,7 @@
+__pycache__/
+*.py[cod]
+.pytest_cache/
+*.egg-info/
+
+# Synthesis build products; regenerate with `make synth`.
+build/

Makefile

@@ -0,0 +1,20 @@
+# make test    run the backbone-free consistency suite
+# make rtl     regenerate rtl/ from rules.json
+# make synth   synthesize every rule with nosis
+# make clean
+
+PYTHON ?= python
+
+test:
+	$(PYTHON) -m pytest -q
+
+rtl:
+	$(PYTHON) rtl_gen.py
+
+synth: rtl
+	$(PYTHON) synth.py
+
+clean:
+	rm -rf build
+
+.PHONY: test rtl synth clean

README.md

@@ -0,0 +1,146 @@
+---
+license: other
+license_name: fair-research-license
+license_link: https://huggingface.co/facebook/EUPE-ViT-B/blob/main/LICENSE
+base_model: facebook/EUPE-ViT-B
+tags:
+  - image-classification
+  - binary-classification
+  - minimal-models
+  - interpretability
+  - vision-transformer
+  - circuit-synthesis
+library_name: pytorch
+datasets:
+  - detection-datasets/coco
+pipeline_tag: image-classification
+---
+
+# Zero-Parameter Classifier
+
+Image-level person classification on EUPE-ViT-B features. A 768 pixel image
+gives 2304 patch tokens at the final layer; layernorm across the 768 channels
+and max-pool across patches gives one 768-D vector. The decision compares two
+sums of that vector. The boundary is zero.
+
+```python
+patches = backbone(image)["x_norm_patchtokens"]      # (2304, 768)
+pooled  = layernorm(patches, 768).max(dim=0)         # (768,)
+present = pooled[pos_dims].sum() > pooled[neg_dims].sum()
+```
+
+At two dimensions:
+
+```
+person present  ⟺  feat[48] > feat[637]
+```
+
+```python
+from infer import PersonDetector
+det = PersonDetector.load('d6')
+present = det.predict('image.jpg')
+```
+
+## Rules
+
+Dimensions are selected on COCO train2017, 118,287 images, and scored on
+val2017, 5000 images. The splits are disjoint.
+
+| rule | dims | F1 | precision | recall | slices | LUT4 | CCU2C | ns |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|
+| `d2` | 2 | 0.8681 | 0.8685 | 0.8678 | 4 | 7 | 0 | 0.40 |
+| `d4` | 4 | 0.8817 | 0.9069 | 0.8578 | 10 | 8 | 10 | 0.90 |
+| `d6` | 6 | 0.8977 | 0.9461 | 0.8541 | 20 | 9 | 20 | 1.40 |
+| `d8` | 8 | 0.9039 | 0.9475 | 0.8641 | 30 | 9 | 30 | 1.90 |
+| `d12` | 12 | 0.9068 | 0.9500 | 0.8674 | 60 | 10 | 60 | 2.90 |
+| `d16` | 16 | 0.9126 | 0.9568 | 0.8723 | 84 | 10 | 84 | 3.90 |
+| `d20` | 20 | 0.9217 | 0.9587 | 0.8875 | 108 | 11 | 108 | 4.90 |
+| `d40` | 40 | 0.9307 | 0.9698 | 0.8945 | 266 | 12 | 266 | 9.90 |
+
+Train and validation F1 differ by 0.0006 at 40 dimensions and by at most 0.0078
+across the set. `d6` is the default.
+
+## Dimensions
+
+```
+d2   48 > 637
+d4   48 + 71 > 637 + 90
+d6   48 + 71 + 292 > 637 + 90 + 82
+```
+
+Selection is greedy over the 192 dimensions with the largest class-mean
+separation, alternating sides and adding whichever remaining dimension most
+improves F1 at a zero boundary. Rules nest; `tests/test_rules.py` checks the
+nesting.
+
+Dimension 48 responds to people and to person-associated objects and is
+suppressed on non-human animals and on non-anthropogenic structures.
+
+## Offset
+
+A decision of the form `sum(pos) - sum(neg) > t` requires `t` because the two
+sums carry a relative offset. Sets selected under a zero boundary carry none. A
+40-dimension set selected at `t = 25.28` scores F1 0.7410 on these images when
+`t` is set to zero.
+
+Dimension indices and signs are fixed structure. Each rule has no free
+parameters and 2 to 40 fixed ones.
+
+## Circuit
+
+`rtl_gen.py` emits one Verilog module per rule. `synth.py` synthesizes them with
+[nosis](https://github.com/CharlesCNorton/nosis) for a Lattice ECP5 LFE5U-25F.
+Counts are LUT4s, carry cells and slices on that device. Inputs are the selected
+channels as signed INT8, post-layernorm and post-max-pool. Output is one bit,
+combinational, with no multipliers, no memory and no constants.
+
+`d2` contains no adder and is LUT-bound at 0.40 ns. Wider rules are carry-bound,
+with area and delay linear in dimension count.
+
+`tests/test_rtl.py` simulates each module against a Python reference under
+Icarus Verilog, on uniform inputs and on inputs at the decision boundary.
+
+## Layout
+
+```
+common/      pooled features, the comparison rule, metrics, named pools
+cache.py     pooled feature cache for a COCO split
+choose.py    dimension selection on train2017, writes rules.json
+verify.py    scoring on val2017, writes eval.json
+rtl_gen.py   Verilog generation from rules.json
+synth.py     nosis synthesis, writes synth.json
+infer.py     loader for every rule
+rtl/         one module per rule, all generated
+tests/       consistency suite, no backbone or dataset required
+```
+
+Each measured JSON opens with a provenance block naming its generating script
+and the pool it read. `tests/test_artifacts.py` enforces the pairing and that
+selection and scoring name different splits.
+
+## Running
+
+```
+pip install -e .
+python cache.py --split train2017
+python cache.py --split val2017
+python choose.py
+python verify.py
+make synth
+make test
+```
+
+`COCO_ROOT` is the dataset root. `BACKBONE` is the backbone repo id or a local
+path. `BACKBONE_SRC` supplies `argus.py` from a local directory; otherwise it is
+fetched from the backbone repo. Caching the two splits is a backbone forward
+over 123,287 images; every later step reads the cache.
+
+bfloat16 kernels select reduction orders by batch size, so cached values depend
+on `--batch`. A cache must be built at one batch size throughout.
+
+## Source backbone
+
+EUPE-ViT-B from Meta FAIR ([arXiv:2603.22387](https://arxiv.org/abs/2603.22387),
+Zhu et al., March 2026), distilled from PEcore-G + PElang-G + DINOv3-H+ via a
+1.9B proxy teacher. License: FAIR Research License, non-commercial. This
+classifier is an artifact derived from that backbone's feature geometry.

cache.py

@@ -0,0 +1,132 @@
+"""Cache the pooled 768-D vector for every image in a COCO split.
+
+    python cache.py --split train2017
+    python cache.py --split val2017
+
+Selection and evaluation both read these, so the backbone is forwarded once per
+split rather than once per experiment. Values are written straight through to a
+memmap alongside a per-row status byte, and meta.json is written last, so an
+interrupted run resumes with --resume instead of restarting.
+
+bfloat16 kernels select reduction orders by batch size, so cached values depend
+on --batch. A cache must be built at one batch size throughout.
+"""
+import argparse
+import json
+import time
+from pathlib import Path
+
+import numpy as np
+import torch
+from PIL import Image
+
+from common import COCO_ROOT, D, RES, coco_split, device, image_paths, normalize, pool
+from common.artifacts import write_artifact
+from common.models import load_backbone
+from common.paths import BACKBONE
+
+TODO, DONE, UNREADABLE = 0, 1, 2
+
+
+class Images(torch.utils.data.Dataset):
+    """Normalized images for a list of (row, path); unreadable files yield None."""
+
+    def __init__(self, work):
+        self.work = work
+
+    def __len__(self):
+        return len(self.work)
+
+    def __getitem__(self, i):
+        row, path = self.work[i]
+        try:
+            return row, normalize(Image.open(path), RES, 'cpu')[0]
+        except Exception:                       # noqa: BLE001
+            # One undecodable image is not worth losing an hour of forwards.
+            return row, None
+
+
+def collate(batch):
+    good = [(r, x) for r, x in batch if x is not None]
+    bad = [r for r, x in batch if x is None]
+    if not good:
+        return None, bad
+    rows, xs = zip(*good)
+    return (torch.tensor(rows), torch.stack(xs)), bad
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument('--split', default='val2017', choices=('train2017', 'val2017'))
+    ap.add_argument('--backbone', default=BACKBONE)
+    ap.add_argument('--batch', type=int, default=8)
+    ap.add_argument('--workers', type=int, default=6)
+    ap.add_argument('--resume', action='store_true')
+    ap.add_argument('--out', type=Path, default=None)
+    args = ap.parse_args()
+
+    out = args.out or COCO_ROOT / f'pooled_{args.split}'
+    out.mkdir(parents=True, exist_ok=True)
+    dev = device()
+
+    coco, id_to_file = coco_split(args.split)
+    img_ids = sorted(coco.getImgIds())
+    paths = image_paths(id_to_file, img_ids, args.split)
+    n = len(img_ids)
+
+    present = all((out / f).exists() for f in ('pooled.dat', 'status.dat'))
+    if args.resume and not present:
+        raise SystemExit(f'--resume asked for but {out} has no partial run')
+    mode = 'r+' if (args.resume and present) else 'w+'
+    pooled = np.memmap(out / 'pooled.dat', np.float16, mode, shape=(n, D))
+    status = np.memmap(out / 'status.dat', np.uint8, mode, shape=(n,))
+
+    work = [(r, p) for r, p in enumerate(paths) if status[r] == TODO]
+    print(f'[init] {args.split}: {n} images, {len(work)} to compute', flush=True)
+    backbone = load_backbone(args.backbone).to(dev).eval()
+
+    loader = torch.utils.data.DataLoader(
+        Images(work), batch_size=args.batch, shuffle=False,
+        num_workers=args.workers, collate_fn=collate)
+
+    t0, seen = time.time(), 0
+    with torch.inference_mode():
+        for good, bad in loader:
+            for r in bad:
+                status[r] = UNREADABLE
+            if good is None:
+                continue
+            idx, x = good
+            with torch.autocast(dev, dtype=torch.bfloat16):
+                tok = backbone.forward_features(x.to(dev))['x_norm_patchtokens']
+            rows = idx.numpy()
+            pooled[rows] = pool(tok.float()).half().cpu().numpy()
+            status[rows] = DONE
+            seen += len(rows)
+            if seen % (args.batch * 50) < args.batch:
+                pooled.flush()
+                status.flush()
+                rate = seen / max(time.time() - t0, 1e-6)
+                print(f'  {seen}/{len(work)}  {rate:.1f} img/s  '
+                      f'ETA {(len(work) - seen) / max(rate, 1e-6) / 60:.1f} min',
+                      flush=True)
+
+    pooled.flush()
+    status.flush()
+    done = int((np.asarray(status) == DONE).sum())
+    np.save(out / 'pooled.npy', np.asarray(pooled))
+    (out / 'img_ids.json').write_text(json.dumps([int(i) for i in img_ids]))
+    write_artifact(out / 'meta.json', {
+        'split': args.split,
+        'n_images': done,
+        'n_unreadable': int((np.asarray(status) == UNREADABLE).sum()),
+        'resolution': RES,
+        'files': {'pooled.npy': {'shape': [n, D], 'dtype': 'float16'}},
+    }, generator='cache.py', backbone=args.backbone, batch=args.batch,
+        protocol='live backbone forward at 768 px, layernorm over 768 channels '
+                 'per patch, max over patches')
+    print(f'[done] {done}/{n} -> {out}  ({time.time() - t0:.0f}s)', flush=True)
+
+
+if __name__ == '__main__':
+    main()

choose.py

@@ -0,0 +1,85 @@
+"""Choose the two dim sets on train2017 and write rules.json.
+
+    python choose.py
+
+The decision is sum(feat[pos]) > sum(feat[neg]), evaluated at zero. A threshold
+would absorb the offset between the two sums; choosing the sets so the offset is
+already zero removes it instead, which is why the rule carries no free parameter.
+
+Dims are added greedily, alternating sides, each step taking whichever remaining
+dim most improves F1 with the boundary pinned at zero. Selection reads only
+train2017. Nothing here touches val2017.
+"""
+import argparse
+from pathlib import Path
+
+import torch
+
+from common import COCO_ROOT, prf1, write_artifact
+from common.cached import load_pooled
+from common.pools import TRAIN2017
+
+HERE = Path(__file__).resolve().parent
+
+
+def greedy(X, y, k, candidates):
+    """Grow pos and neg sets to k each, scoring only at threshold zero."""
+    pos, neg = [], []
+    cur = torch.zeros(X.shape[0])
+    for step in range(2 * k):
+        side, sign = (pos, 1.0) if step % 2 == 0 else (neg, -1.0)
+        used = set(pos) | set(neg)
+        best_f1, best_d = -1.0, None
+        for d in candidates:
+            if d in used:
+                continue
+            f1 = prf1(cur + sign * X[:, d] > 0, y).f1
+            if f1 > best_f1:
+                best_f1, best_d = f1, d
+        side.append(int(best_d))
+        cur = cur + sign * X[:, best_d]
+    return pos, neg, prf1(cur > 0, y).f1
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument('--cache', type=Path, default=None)
+    ap.add_argument('--sizes', type=int, nargs='+',
+                    default=[1, 2, 3, 4, 6, 8, 10, 20])
+    ap.add_argument('--candidates', type=int, default=192)
+    ap.add_argument('--out', type=Path, default=HERE / 'rules.json')
+    args = ap.parse_args()
+
+    cache = args.cache or COCO_ROOT / 'pooled_train2017'
+    X, y = load_pooled(cache, 'train2017')
+    print(f'[train] {X.shape[0]} images, person rate {y.float().mean():.3f}',
+          flush=True)
+
+    # Restrict the search to dims with real class separation, for tractability.
+    sep = (X[y].mean(0) - X[~y].mean(0)).abs()
+    candidates = torch.topk(sep, args.candidates).indices.tolist()
+
+    rules = {}
+    for k in args.sizes:
+        pos, neg, f1 = greedy(X, y, k, candidates)
+        rules[f'd{2 * k}'] = {'pos_dims': pos, 'neg_dims': neg,
+                              'n_dims': 2 * k, 'free_parameters': 0,
+                              'F1_train': round(f1, 4)}
+        print(f'  d{2 * k:<3} F1 {f1:.4f}  {pos} > {neg}', flush=True)
+
+    write_artifact(args.out, {
+        'candidates': args.candidates,
+        'sizes': [2 * k for k in args.sizes],
+        'rules': rules,
+    }, generator='choose.py',
+        pool_info={'pool': TRAIN2017.name, 'split': TRAIN2017.split,
+                   'n_images': int(X.shape[0]),
+                   'positive_rate': round(y.float().mean().item(), 4),
+                   'selection': TRAIN2017.selection},
+        decision='sum(feat[pos_dims]) > sum(feat[neg_dims])',
+        search='greedy, alternating sides, boundary pinned at zero')
+    print(f'[done] wrote {args.out}', flush=True)
+
+
+if __name__ == '__main__':
+    main()

common/__init__.py

@@ -0,0 +1,22 @@
+"""Pooled features, the comparison rule, metrics, named pools, artifact contract."""
+from .paths import BACKBONE, BACKBONE_SRC, COCO_ROOT, REPO, UPSTREAM_BACKBONE, device
+from .data import (MEAN, STD, coco_split, image_paths, load_image, normalize,
+                   person_labels)
+from .features import D, RES, backbone_pooled, decide, pool, score
+from .metrics import Metrics, f1_at, prf1
+from .pools import POOLS, TRAIN2017, VAL5000, Pool, by_name
+from .pool import LoadedPool, load_pool
+from .artifacts import (REGISTRY, ArtifactSpec, provenance, read_artifact,
+                        sha256_of, write_artifact)
+
+__all__ = [
+    'BACKBONE', 'BACKBONE_SRC', 'COCO_ROOT', 'REPO', 'UPSTREAM_BACKBONE', 'device',
+    'MEAN', 'STD', 'coco_split', 'image_paths', 'load_image', 'normalize',
+    'person_labels',
+    'D', 'RES', 'backbone_pooled', 'decide', 'pool', 'score',
+    'Metrics', 'f1_at', 'prf1',
+    'POOLS', 'TRAIN2017', 'VAL5000', 'Pool', 'by_name',
+    'LoadedPool', 'load_pool',
+    'REGISTRY', 'ArtifactSpec', 'provenance', 'read_artifact', 'sha256_of',
+    'write_artifact',
+]

common/artifacts.py

@@ -0,0 +1,58 @@
+"""Artifact provenance stamping and the generator/artifact registry."""
+import hashlib
+import json
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Optional, Tuple
+
+from .paths import REPO
+
+
+@dataclass(frozen=True)
+class ArtifactSpec:
+    """Owning script, source pool, and required top-level payload keys."""
+
+    generator: Optional[str]
+    pool: Optional[str]
+    payload_keys: Tuple[str, ...]
+
+
+REGISTRY = {
+    'rules.json': ArtifactSpec(
+        'choose.py', 'TRAIN2017', ('candidates', 'sizes', 'rules')),
+    'eval.json': ArtifactSpec('verify.py', 'VAL5000', ('rules',)),
+    'synth.json': ArtifactSpec(
+        'synth.py', None, ('tool', 'target', 'variants')),
+}
+
+
+def sha256_of(path) -> str:
+    """Content hash of a file."""
+    return hashlib.sha256(Path(path).read_bytes()).hexdigest()
+
+
+def provenance(generator: str, classifier=None, pool_info: Optional[dict] = None,
+               **extra) -> dict:
+    """Assemble a provenance block from repository-recoverable fields only."""
+    block = {'generator': generator}
+    if classifier is not None:
+        path = Path(classifier)
+        block['classifier'] = str(path.resolve().relative_to(REPO)).replace('\\', '/')
+        block['classifier_sha256'] = sha256_of(path)
+    if pool_info:
+        block.update(pool_info)
+    block.update(extra)
+    return block
+
+
+def write_artifact(path, payload: dict, *, generator: str, classifier=None,
+                   pool_info: Optional[dict] = None, **extra):
+    """Write `payload` beneath a provenance block and return the document."""
+    doc = {'provenance': provenance(generator, classifier, pool_info, **extra)}
+    doc.update(payload)
+    Path(path).write_text(json.dumps(doc, indent=2) + '\n', encoding='utf-8')
+    return doc
+
+
+def read_artifact(path) -> dict:
+    return json.loads(Path(path).read_text(encoding='utf-8'))

common/cached.py

@@ -0,0 +1,20 @@
+"""Reading the pooled feature caches that cache.py writes."""
+import json
+from pathlib import Path
+from typing import Tuple
+
+import numpy as np
+import torch
+
+from .data import coco_split, person_labels
+
+
+def load_pooled(cache, split: str) -> Tuple[torch.Tensor, torch.Tensor]:
+    """(N, 768) pooled vectors and their person labels, in cache row order."""
+    cache = Path(cache)
+    pooled = np.load(cache / 'pooled.npy').astype(np.float32)
+    img_ids = json.loads((cache / 'img_ids.json').read_text())
+    if pooled.shape[0] != len(img_ids):
+        raise ValueError(f'{cache} has {pooled.shape[0]} rows and {len(img_ids)} ids')
+    coco, _ = coco_split(split)
+    return torch.from_numpy(pooled), person_labels(coco, img_ids, 'cpu')

common/data.py

@@ -0,0 +1,77 @@
+"""COCO loading and input normalization.
+
+Images are resized to a square `resolution` with bilinear interpolation and
+normalized with ImageNet statistics, matching the protocol every stage was
+measured under.
+"""
+from pathlib import Path
+from typing import Iterable, List, Sequence, Tuple, Union
+
+import numpy as np
+import torch
+from PIL import Image
+
+from .paths import COCO_ROOT
+
+MEAN = (0.485, 0.456, 0.406)
+STD = (0.229, 0.224, 0.225)
+PERSON_CATEGORY_ID = 1
+
+
+def _stats(device: str) -> Tuple[torch.Tensor, torch.Tensor]:
+    mean = torch.tensor(MEAN).view(1, 3, 1, 1).to(device)
+    std = torch.tensor(STD).view(1, 3, 1, 1).to(device)
+    return mean, std
+
+
+def normalize(img: Image.Image, resolution: int, device: str) -> torch.Tensor:
+    """PIL image -> (1, 3, R, R) normalized float tensor."""
+    img = img.convert('RGB').resize((resolution, resolution), Image.BILINEAR)
+    arr = np.asarray(img, dtype=np.uint8).copy()
+    x = torch.from_numpy(arr).permute(2, 0, 1).unsqueeze(0).to(device).float() / 255.0
+    mean, std = _stats(device)
+    return (x - mean) / std
+
+
+def load_image(image: Union[str, Path, Image.Image, np.ndarray, torch.Tensor],
+               resolution: int, device: str) -> torch.Tensor:
+    """Accept a path, PIL image, HWC array, or CHW tensor; return a batch of 1."""
+    if isinstance(image, (str, Path)):
+        img = Image.open(image)
+    elif isinstance(image, Image.Image):
+        img = image
+    elif isinstance(image, np.ndarray):
+        img = Image.fromarray(image)
+    elif isinstance(image, torch.Tensor):
+        arr = image.cpu().numpy() if image.ndim == 3 else image[0].cpu().numpy()
+        if arr.shape[0] == 3:
+            arr = arr.transpose(1, 2, 0)
+        img = Image.fromarray((arr * 255).astype('uint8'))
+    else:
+        raise TypeError(f'unsupported image type: {type(image)}')
+    return normalize(img, resolution, device)
+
+
+def coco_split(split: str = 'val2017'):
+    """Return (COCO handle, image-file lookup) for a COCO split."""
+    from pycocotools.coco import COCO
+    coco = COCO(str(COCO_ROOT / 'annotations' / f'instances_{split}.json'))
+    id_to_file = {i['id']: i['file_name'] for i in coco.loadImgs(coco.getImgIds())}
+    return coco, id_to_file
+
+
+def person_labels(coco, img_ids: Sequence[int], device: str = 'cpu') -> torch.Tensor:
+    """Image-level person presence for each id, as a bool tensor."""
+    labels = [
+        any(a['category_id'] == PERSON_CATEGORY_ID
+            for a in coco.loadAnns(coco.getAnnIds(imgIds=i, iscrowd=False)))
+        for i in img_ids
+    ]
+    return torch.tensor(labels, dtype=torch.bool, device=device)
+
+
+def image_paths(id_to_file: dict, img_ids: Iterable[int],
+                split: str = 'val2017') -> List[Path]:
+    """Absolute paths for a sequence of image ids within a split."""
+    root = COCO_ROOT / split
+    return [root / id_to_file[i] for i in img_ids]

common/features.py

@@ -0,0 +1,48 @@
+"""Pooled feature extraction and the comparison rule.
+
+One 768-D vector per image: layernorm across the 768 channels of every patch
+token, then max-pool across patches. The decision compares two sums of that
+vector against each other, so the boundary sits at zero and carries no offset.
+"""
+from typing import Sequence
+
+import torch
+import torch.nn.functional as F
+
+D = 768
+RES = 768
+
+
+def pool(patch_tokens: torch.Tensor) -> torch.Tensor:
+    """(N, D) or (B, N, D) patch tokens -> (D,) or (B, D) pooled vector."""
+    ln = F.layer_norm(patch_tokens.float(), [D])
+    return ln.max(dim=-2).values
+
+
+@torch.inference_mode()
+def backbone_pooled(backbone, x: torch.Tensor, autocast: bool = True) -> torch.Tensor:
+    """Forward a normalized batch through the backbone and pool it."""
+    if autocast:
+        dev = 'cuda' if x.is_cuda else 'cpu'
+        with torch.autocast(dev, dtype=torch.bfloat16):
+            out = backbone.forward_features(x)
+    else:
+        out = backbone.forward_features(x)
+    return pool(out['x_norm_patchtokens'].float())
+
+
+def _as_index(idx, like: torch.Tensor) -> torch.Tensor:
+    if torch.is_tensor(idx):
+        return idx
+    return torch.tensor(list(idx), dtype=torch.long, device=like.device)
+
+
+def score(pooled: torch.Tensor, pos: Sequence[int], neg: Sequence[int]) -> torch.Tensor:
+    """sum(pooled[pos]) - sum(pooled[neg]), over the last axis."""
+    p, n = _as_index(pos, pooled), _as_index(neg, pooled)
+    return pooled.index_select(-1, p).sum(-1) - pooled.index_select(-1, n).sum(-1)
+
+
+def decide(pooled: torch.Tensor, pos: Sequence[int], neg: Sequence[int]) -> torch.Tensor:
+    """sum(pooled[pos]) > sum(pooled[neg]). No threshold, no free parameter."""
+    return score(pooled, pos, neg) > 0

common/metrics.py

@@ -0,0 +1,47 @@
+"""Binary classification metrics, single-sourced so every stage scores identically."""
+from typing import NamedTuple
+
+import torch
+
+
+class Metrics(NamedTuple):
+    """F1, precision, recall, and the threshold they were measured at."""
+
+    f1: float
+    precision: float
+    recall: float
+    threshold: float = float('nan')
+
+    def asdict(self) -> dict:
+        d = {'F1': self.f1, 'precision': self.precision, 'recall': self.recall}
+        if self.threshold == self.threshold:  # excludes NaN
+            d['threshold'] = self.threshold
+        return d
+
+
+def prf1(pred: torch.Tensor, labels: torch.Tensor) -> Metrics:
+    """Metrics for boolean prediction and label tensors."""
+    tp = (pred & labels).sum().float()
+    fp = (pred & ~labels).sum().float()
+    fn = (~pred & labels).sum().float()
+    precision = tp / (tp + fp).clamp(min=1)
+    recall = tp / (tp + fn).clamp(min=1)
+    f1 = 2 * precision * recall / (precision + recall).clamp(min=1e-9)
+    return Metrics(float(f1), float(precision), float(recall))
+
+
+def f1_at(scores: torch.Tensor, labels: torch.Tensor, threshold: float) -> Metrics:
+    """Metrics at a fixed threshold."""
+    return prf1(scores > threshold, labels)._replace(threshold=float(threshold))
+
+
+def f1_sweep(scores: torch.Tensor, labels: torch.Tensor, n_candidates: int = 500) -> Metrics:
+    """Best metrics over candidate thresholds drawn evenly from the sorted unique scores."""
+    uniq = torch.unique(scores).sort().values
+    stride = max(1, len(uniq) // n_candidates)
+    best = Metrics(0.0, 0.0, 0.0, 0.0)
+    for t in uniq.tolist()[::stride]:
+        m = prf1(scores > t, labels)
+        if m.f1 > best.f1:
+            best = m._replace(threshold=float(t))
+    return best

common/models.py

@@ -0,0 +1,32 @@
+"""Backbone loading."""
+import sys
+from pathlib import Path
+from typing import Optional
+
+from .paths import BACKBONE, BACKBONE_SRC
+
+_argus = None
+
+
+def argus_module():
+    """Import argus.py from the environment, BACKBONE_SRC, or the backbone repo."""
+    global _argus
+    if _argus is not None:
+        return _argus
+    try:
+        import argus
+    except ImportError:
+        if BACKBONE_SRC:
+            sys.path.insert(0, str(BACKBONE_SRC))
+        else:
+            from huggingface_hub import hf_hub_download
+            sys.path.insert(0, str(Path(hf_hub_download(BACKBONE, 'argus.py')).parent))
+        import argus
+    _argus = argus
+    return argus
+
+
+def load_backbone(repo: Optional[str] = None):
+    """Load the stock backbone in eval mode."""
+    from transformers import AutoModel
+    return AutoModel.from_pretrained(repo or BACKBONE, trust_remote_code=True).eval().backbone

common/paths.py

@@ -0,0 +1,25 @@
+"""Filesystem and repository locations, overridable by environment.
+
+BACKBONE      HF repo id or local path for the backbone wrapper (alias: ARGUS_PATH)
+BACKBONE_SRC  local directory supplying argus.py (alias: ARGUS_SRC)
+COCO_ROOT     dataset root holding annotations/, train2017/, val2017/
+DEVICE        torch device string
+"""
+import os
+from pathlib import Path
+
+REPO = Path(__file__).resolve().parent.parent
+
+UPSTREAM_BACKBONE = 'facebook/EUPE-ViT-B'
+BACKBONE = os.environ.get('BACKBONE') or os.environ.get('ARGUS_PATH') or 'phanerozoic/argus'
+BACKBONE_SRC = os.environ.get('BACKBONE_SRC') or os.environ.get('ARGUS_SRC') or None
+COCO_ROOT = Path(os.environ.get('COCO_ROOT', '/home/zootest/datasets/coco'))
+
+ARGUS = BACKBONE
+
+
+def device() -> str:
+    if 'DEVICE' in os.environ:
+        return os.environ['DEVICE']
+    import torch
+    return 'cuda' if torch.cuda.is_available() else 'cpu'

common/pool.py

@@ -0,0 +1,72 @@
+"""Evaluation-pool loading."""
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Iterator, List, Optional
+
+import torch
+from PIL import Image
+
+from .data import coco_split, image_paths, normalize, person_labels
+from .features import RES
+from .pools import Pool
+
+
+@dataclass
+class LoadedPool:
+    """Image ids, paths and labels for one named pool, images optionally resident."""
+
+    pool: Pool
+    img_ids: List[int]
+    paths: List[Path]
+    labels: torch.Tensor
+    device: str
+    images: Optional[List[torch.Tensor]] = None
+
+    def __len__(self) -> int:
+        return len(self.img_ids)
+
+    def __iter__(self) -> Iterator[torch.Tensor]:
+        """Yield each image as a normalized (1, 3, RES, RES) tensor."""
+        if self.images is not None:
+            yield from self.images
+            return
+        for path in self.paths:
+            yield normalize(Image.open(path), RES, self.device)
+
+    @property
+    def positive_rate(self) -> float:
+        return round(self.labels.float().mean().item(), 4)
+
+    def provenance(self) -> dict:
+        """Pool fields recorded in an artifact's provenance block."""
+        return {'pool': self.pool.name, 'split': self.pool.split,
+                'n_images': len(self), 'positive_rate': self.positive_rate,
+                'selection': self.pool.selection}
+
+
+def balanced_indices(labels: torch.Tensor, seed: int = 0) -> torch.Tensor:
+    """Indices subsampling `labels` to equal positive and negative counts, seeded."""
+    generator = torch.Generator(device='cpu').manual_seed(seed)
+    cpu = labels.cpu()
+    pos = cpu.nonzero(as_tuple=True)[0]
+    neg = (~cpu).nonzero(as_tuple=True)[0]
+    n = min(len(pos), len(neg))
+    sel = torch.cat([pos[torch.randperm(len(pos), generator=generator)[:n]],
+                     neg[torch.randperm(len(neg), generator=generator)[:n]]])
+    return sel[torch.randperm(len(sel), generator=generator)]
+
+
+def load_pool(pool: Pool, device: str, preload: bool = False, seed: int = 0) -> LoadedPool:
+    """Resolve a named pool to ids, paths and labels; `preload` holds images in memory."""
+    coco, id_to_file = coco_split(pool.split)
+    img_ids = sorted(coco.getImgIds())
+    if pool.n is not None:
+        img_ids = img_ids[:pool.n]
+    labels = person_labels(coco, img_ids, device)
+    if pool.balanced:
+        sel = balanced_indices(labels, seed)
+        img_ids = [img_ids[i] for i in sel.tolist()]
+        labels = labels[sel.to(labels.device)]
+    paths = image_paths(id_to_file, img_ids, pool.split)
+    images = [normalize(Image.open(p), RES, device) for p in paths] if preload else None
+    return LoadedPool(pool, img_ids, paths, labels, device, images)

common/pools.py

@@ -0,0 +1,30 @@
+"""Named pools, cited by name in every artifact's provenance block.
+
+Dims are selected on TRAIN2017 and reported on VAL5000. The two never overlap,
+so no figure is measured on data that chose it.
+"""
+from typing import NamedTuple, Optional
+
+
+class Pool(NamedTuple):
+    name: str
+    split: str
+    n: Optional[int]
+    selection: str
+
+
+TRAIN2017 = Pool(
+    'TRAIN2017', 'train2017', None,
+    'all 118287 train2017 images, used only to select dims')
+
+VAL5000 = Pool(
+    'VAL5000', 'val2017', None,
+    'all 5000 val2017 images, used only to report')
+
+POOLS = {p.name: p for p in (TRAIN2017, VAL5000)}
+
+
+def by_name(name: str) -> Pool:
+    if name not in POOLS:
+        raise ValueError(f'unknown pool {name!r}; expected one of {sorted(POOLS)}')
+    return POOLS[name]

eval.json

@@ -0,0 +1,218 @@
+{
+  "provenance": {
+    "generator": "verify.py",
+    "pool": "VAL5000",
+    "split": "val2017",
+    "n_images": 5000,
+    "positive_rate": 0.5386,
+    "selection": "all 5000 val2017 images, used only to report",
+    "task": "image-level person presence (binary)",
+    "protocol": "dims selected on train2017, not refit here"
+  },
+  "rules": {
+    "d2": {
+      "n_dims": 2,
+      "free_parameters": 0,
+      "pos_dims": [
+        48
+      ],
+      "neg_dims": [
+        637
+      ],
+      "F1_train": 0.8734,
+      "F1": 0.8681,
+      "precision": 0.8685,
+      "recall": 0.8678
+    },
+    "d4": {
+      "n_dims": 4,
+      "free_parameters": 0,
+      "pos_dims": [
+        48,
+        71
+      ],
+      "neg_dims": [
+        637,
+        90
+      ],
+      "F1_train": 0.8874,
+      "F1": 0.8817,
+      "precision": 0.9069,
+      "recall": 0.8578
+    },
+    "d6": {
+      "n_dims": 6,
+      "free_parameters": 0,
+      "pos_dims": [
+        48,
+        71,
+        292
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82
+      ],
+      "F1_train": 0.9055,
+      "F1": 0.8977,
+      "precision": 0.9461,
+      "recall": 0.8541
+    },
+    "d8": {
+      "n_dims": 8,
+      "free_parameters": 0,
+      "pos_dims": [
+        48,
+        71,
+        292,
+        111
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82,
+        269
+      ],
+      "F1_train": 0.9104,
+      "F1": 0.9039,
+      "precision": 0.9475,
+      "recall": 0.8641
+    },
+    "d12": {
+      "n_dims": 12,
+      "free_parameters": 0,
+      "pos_dims": [
+        48,
+        71,
+        292,
+        111,
+        724,
+        406
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82,
+        269,
+        565,
+        140
+      ],
+      "F1_train": 0.9133,
+      "F1": 0.9068,
+      "precision": 0.95,
+      "recall": 0.8674
+    },
+    "d16": {
+      "n_dims": 16,
+      "free_parameters": 0,
+      "pos_dims": [
+        48,
+        71,
+        292,
+        111,
+        724,
+        406,
+        510,
+        665
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82,
+        269,
+        565,
+        140,
+        537,
+        49
+      ],
+      "F1_train": 0.917,
+      "F1": 0.9126,
+      "precision": 0.9568,
+      "recall": 0.8723
+    },
+    "d20": {
+      "n_dims": 20,
+      "free_parameters": 0,
+      "pos_dims": [
+        48,
+        71,
+        292,
+        111,
+        724,
+        406,
+        510,
+        665,
+        318,
+        267
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82,
+        269,
+        565,
+        140,
+        537,
+        49,
+        283,
+        114
+      ],
+      "F1_train": 0.9213,
+      "F1": 0.9217,
+      "precision": 0.9587,
+      "recall": 0.8875
+    },
+    "d40": {
+      "n_dims": 40,
+      "free_parameters": 0,
+      "pos_dims": [
+        48,
+        71,
+        292,
+        111,
+        724,
+        406,
+        510,
+        665,
+        318,
+        267,
+        552,
+        323,
+        382,
+        305,
+        127,
+        293,
+        85,
+        721,
+        207,
+        326
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82,
+        269,
+        565,
+        140,
+        537,
+        49,
+        283,
+        114,
+        310,
+        113,
+        494,
+        571,
+        694,
+        92,
+        518,
+        91,
+        307,
+        437
+      ],
+      "F1_train": 0.9301,
+      "F1": 0.9307,
+      "precision": 0.9698,
+      "recall": 0.8945
+    }
+  }
+}

infer.py

@@ -0,0 +1,67 @@
+"""Score images for person presence.
+
+    from infer import PersonDetector
+    det = PersonDetector.load('d6')
+    present = det.predict('image.jpg')
+
+`predict` returns a bool. `margin` returns the signed difference between the two
+sums, which is positive exactly when the answer is yes; it carries no threshold.
+"""
+import argparse
+import sys
+from pathlib import Path
+
+import torch
+
+from common import BACKBONE, RES, backbone_pooled, load_image, read_artifact, score
+from common.models import load_backbone
+
+HERE = Path(__file__).resolve().parent
+
+
+class PersonDetector:
+    def __init__(self, forward_fn, pos_dims, neg_dims, dev):
+        self._forward = forward_fn
+        self._dev = dev
+        self._pos = torch.tensor(pos_dims, dtype=torch.long, device=dev)
+        self._neg = torch.tensor(neg_dims, dtype=torch.long, device=dev)
+
+    @property
+    def dims(self):
+        return self._pos.tolist(), self._neg.tolist()
+
+    @torch.inference_mode()
+    def margin(self, image) -> float:
+        pooled = self._forward(load_image(image, RES, self._dev))
+        return float(score(pooled, self._pos, self._neg))
+
+    def predict(self, image) -> bool:
+        return self.margin(image) > 0.0
+
+    @classmethod
+    def load(cls, rule: str = None, backbone_repo: str = BACKBONE, root=None):
+        from common import device
+        root = Path(root) if root else HERE
+        doc = read_artifact(root / 'rules.json')
+        rules = doc['rules']
+        rule = rule or 'd6'
+        if rule not in rules:
+            raise ValueError(f'unknown rule {rule!r}; expected one of {sorted(rules)}')
+        r = rules[rule]
+        dev = device()
+        backbone = load_backbone(backbone_repo).to(dev).eval()
+        return cls(lambda x: backbone_pooled(backbone, x)[0],
+                   r['pos_dims'], r['neg_dims'], dev)
+
+
+if __name__ == '__main__':
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument('rule')
+    ap.add_argument('images', nargs='+')
+    args = ap.parse_args()
+    det = PersonDetector.load(args.rule)
+    pos, neg = det.dims
+    print(f'rule {args.rule}: sum{pos} > sum{neg}')
+    for path in args.images:
+        m = det.margin(path)
+        print(f'{path}  margin={m:+.3f}  person={m > 0}')

pyproject.toml

@@ -0,0 +1,28 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "zero-parameter-classifier"
+version = "0.0.0"
+description = "Image-level person classification on EUPE-ViT-B features with no free parameters"
+requires-python = ">=3.9"
+dependencies = [
+    "torch>=2.0",
+    "numpy",
+    "pillow",
+    "transformers>=4.40",
+    "huggingface-hub",
+    "pycocotools",
+]
+
+[project.optional-dependencies]
+dev = ["pytest>=7"]
+synth = ["nosis"]
+
+[tool.setuptools]
+packages = ["common"]
+py-modules = ["infer", "cache", "choose", "verify", "rtl_gen", "synth"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

rtl/person_d12.v

@@ -0,0 +1,21 @@
+// Zero-parameter person classifier, 12 dims.
+// Generated by rtl_gen.py; do not edit by hand.
+//
+// sum(pos) > sum(neg), on signed INT8 channels taken from the pooled feature
+// vector. No threshold, so no constant appears anywhere in this module.
+
+module person_d12 (
+    input  signed [7:0] f48, f71, f292, f111, f724, f406, f637, f90,
+    input  signed [7:0] f82, f269, f565, f140,
+    output person_present
+);
+    wire signed [10:0] pos_sum =
+        {{3{f48[7]}}, f48} + {{3{f71[7]}}, f71} + {{3{f292[7]}}, f292} + {{3{f111[7]}}, f111} +
+        {{3{f724[7]}}, f724} + {{3{f406[7]}}, f406};
+
+    wire signed [10:0] neg_sum =
+        {{3{f637[7]}}, f637} + {{3{f90[7]}}, f90} + {{3{f82[7]}}, f82} + {{3{f269[7]}}, f269} +
+        {{3{f565[7]}}, f565} + {{3{f140[7]}}, f140};
+
+    assign person_present = pos_sum > neg_sum;
+endmodule

rtl/person_d16.v

@@ -0,0 +1,21 @@
+// Zero-parameter person classifier, 16 dims.
+// Generated by rtl_gen.py; do not edit by hand.
+//
+// sum(pos) > sum(neg), on signed INT8 channels taken from the pooled feature
+// vector. No threshold, so no constant appears anywhere in this module.
+
+module person_d16 (
+    input  signed [7:0] f48, f71, f292, f111, f724, f406, f510, f665,
+    input  signed [7:0] f637, f90, f82, f269, f565, f140, f537, f49,
+    output person_present
+);
+    wire signed [10:0] pos_sum =
+        {{3{f48[7]}}, f48} + {{3{f71[7]}}, f71} + {{3{f292[7]}}, f292} + {{3{f111[7]}}, f111} +
+        {{3{f724[7]}}, f724} + {{3{f406[7]}}, f406} + {{3{f510[7]}}, f510} + {{3{f665[7]}}, f665};
+
+    wire signed [10:0] neg_sum =
+        {{3{f637[7]}}, f637} + {{3{f90[7]}}, f90} + {{3{f82[7]}}, f82} + {{3{f269[7]}}, f269} +
+        {{3{f565[7]}}, f565} + {{3{f140[7]}}, f140} + {{3{f537[7]}}, f537} + {{3{f49[7]}}, f49};
+
+    assign person_present = pos_sum > neg_sum;
+endmodule

rtl/person_d2.v

@@ -0,0 +1,12 @@
+// Zero-parameter person classifier, 2 dims.
+// Generated by rtl_gen.py; do not edit by hand.
+//
+// sum(pos) > sum(neg), on signed INT8 channels taken from the pooled feature
+// vector. No threshold, so no constant appears anywhere in this module.
+
+module person_d2 (
+    input  signed [7:0] f48, f637,
+    output person_present
+);
+    assign person_present = f48 > f637;
+endmodule

rtl/person_d20.v

@@ -0,0 +1,24 @@
+// Zero-parameter person classifier, 20 dims.
+// Generated by rtl_gen.py; do not edit by hand.
+//
+// sum(pos) > sum(neg), on signed INT8 channels taken from the pooled feature
+// vector. No threshold, so no constant appears anywhere in this module.
+
+module person_d20 (
+    input  signed [7:0] f48, f71, f292, f111, f724, f406, f510, f665,
+    input  signed [7:0] f318, f267, f637, f90, f82, f269, f565, f140,
+    input  signed [7:0] f537, f49, f283, f114,
+    output person_present
+);
+    wire signed [11:0] pos_sum =
+        {{4{f48[7]}}, f48} + {{4{f71[7]}}, f71} + {{4{f292[7]}}, f292} + {{4{f111[7]}}, f111} +
+        {{4{f724[7]}}, f724} + {{4{f406[7]}}, f406} + {{4{f510[7]}}, f510} + {{4{f665[7]}}, f665} +
+        {{4{f318[7]}}, f318} + {{4{f267[7]}}, f267};
+
+    wire signed [11:0] neg_sum =
+        {{4{f637[7]}}, f637} + {{4{f90[7]}}, f90} + {{4{f82[7]}}, f82} + {{4{f269[7]}}, f269} +
+        {{4{f565[7]}}, f565} + {{4{f140[7]}}, f140} + {{4{f537[7]}}, f537} + {{4{f49[7]}}, f49} +
+        {{4{f283[7]}}, f283} + {{4{f114[7]}}, f114};
+
+    assign person_present = pos_sum > neg_sum;
+endmodule

rtl/person_d4.v

@@ -0,0 +1,18 @@
+// Zero-parameter person classifier, 4 dims.
+// Generated by rtl_gen.py; do not edit by hand.
+//
+// sum(pos) > sum(neg), on signed INT8 channels taken from the pooled feature
+// vector. No threshold, so no constant appears anywhere in this module.
+
+module person_d4 (
+    input  signed [7:0] f48, f71, f637, f90,
+    output person_present
+);
+    wire signed [8:0] pos_sum =
+        {{1{f48[7]}}, f48} + {{1{f71[7]}}, f71};
+
+    wire signed [8:0] neg_sum =
+        {{1{f637[7]}}, f637} + {{1{f90[7]}}, f90};
+
+    assign person_present = pos_sum > neg_sum;
+endmodule

rtl/person_d40.v

@@ -0,0 +1,30 @@
+// Zero-parameter person classifier, 40 dims.
+// Generated by rtl_gen.py; do not edit by hand.
+//
+// sum(pos) > sum(neg), on signed INT8 channels taken from the pooled feature
+// vector. No threshold, so no constant appears anywhere in this module.
+
+module person_d40 (
+    input  signed [7:0] f48, f71, f292, f111, f724, f406, f510, f665,
+    input  signed [7:0] f318, f267, f552, f323, f382, f305, f127, f293,
+    input  signed [7:0] f85, f721, f207, f326, f637, f90, f82, f269,
+    input  signed [7:0] f565, f140, f537, f49, f283, f114, f310, f113,
+    input  signed [7:0] f494, f571, f694, f92, f518, f91, f307, f437,
+    output person_present
+);
+    wire signed [12:0] pos_sum =
+        {{5{f48[7]}}, f48} + {{5{f71[7]}}, f71} + {{5{f292[7]}}, f292} + {{5{f111[7]}}, f111} +
+        {{5{f724[7]}}, f724} + {{5{f406[7]}}, f406} + {{5{f510[7]}}, f510} + {{5{f665[7]}}, f665} +
+        {{5{f318[7]}}, f318} + {{5{f267[7]}}, f267} + {{5{f552[7]}}, f552} + {{5{f323[7]}}, f323} +
+        {{5{f382[7]}}, f382} + {{5{f305[7]}}, f305} + {{5{f127[7]}}, f127} + {{5{f293[7]}}, f293} +
+        {{5{f85[7]}}, f85} + {{5{f721[7]}}, f721} + {{5{f207[7]}}, f207} + {{5{f326[7]}}, f326};
+
+    wire signed [12:0] neg_sum =
+        {{5{f637[7]}}, f637} + {{5{f90[7]}}, f90} + {{5{f82[7]}}, f82} + {{5{f269[7]}}, f269} +
+        {{5{f565[7]}}, f565} + {{5{f140[7]}}, f140} + {{5{f537[7]}}, f537} + {{5{f49[7]}}, f49} +
+        {{5{f283[7]}}, f283} + {{5{f114[7]}}, f114} + {{5{f310[7]}}, f310} + {{5{f113[7]}}, f113} +
+        {{5{f494[7]}}, f494} + {{5{f571[7]}}, f571} + {{5{f694[7]}}, f694} + {{5{f92[7]}}, f92} +
+        {{5{f518[7]}}, f518} + {{5{f91[7]}}, f91} + {{5{f307[7]}}, f307} + {{5{f437[7]}}, f437};
+
+    assign person_present = pos_sum > neg_sum;
+endmodule

rtl/person_d6.v

@@ -0,0 +1,18 @@
+// Zero-parameter person classifier, 6 dims.
+// Generated by rtl_gen.py; do not edit by hand.
+//
+// sum(pos) > sum(neg), on signed INT8 channels taken from the pooled feature
+// vector. No threshold, so no constant appears anywhere in this module.
+
+module person_d6 (
+    input  signed [7:0] f48, f71, f292, f637, f90, f82,
+    output person_present
+);
+    wire signed [9:0] pos_sum =
+        {{2{f48[7]}}, f48} + {{2{f71[7]}}, f71} + {{2{f292[7]}}, f292};
+
+    wire signed [9:0] neg_sum =
+        {{2{f637[7]}}, f637} + {{2{f90[7]}}, f90} + {{2{f82[7]}}, f82};
+
+    assign person_present = pos_sum > neg_sum;
+endmodule

rtl/person_d8.v

@@ -0,0 +1,18 @@
+// Zero-parameter person classifier, 8 dims.
+// Generated by rtl_gen.py; do not edit by hand.
+//
+// sum(pos) > sum(neg), on signed INT8 channels taken from the pooled feature
+// vector. No threshold, so no constant appears anywhere in this module.
+
+module person_d8 (
+    input  signed [7:0] f48, f71, f292, f111, f637, f90, f82, f269,
+    output person_present
+);
+    wire signed [9:0] pos_sum =
+        {{2{f48[7]}}, f48} + {{2{f71[7]}}, f71} + {{2{f292[7]}}, f292} + {{2{f111[7]}}, f111};
+
+    wire signed [9:0] neg_sum =
+        {{2{f637[7]}}, f637} + {{2{f90[7]}}, f90} + {{2{f82[7]}}, f82} + {{2{f269[7]}}, f269};
+
+    assign person_present = pos_sum > neg_sum;
+endmodule

rtl_gen.py

@@ -0,0 +1,81 @@
+"""Emit one Verilog module per rule in rules.json.
+
+    python rtl_gen.py
+
+Inputs are the selected channels as signed INT8, post-LayerNorm and post-max-pool.
+Output is one bit. Combinational, no multipliers, no memory, no constants: the
+comparison is between the two sums, so there is nothing to bake in.
+"""
+import argparse
+import math
+from pathlib import Path
+
+from common import read_artifact
+
+HERE = Path(__file__).resolve().parent
+HEADER = '''// Zero-parameter person classifier, {n} dims.
+// Generated by rtl_gen.py; do not edit by hand.
+//
+// sum(pos) > sum(neg), on signed INT8 channels taken from the pooled feature
+// vector. No threshold, so no constant appears anywhere in this module.
+'''
+
+
+def _decl(dims, per_line=8):
+    rows = []
+    for i in range(0, len(dims), per_line):
+        names = ', '.join(f'f{d}' for d in dims[i:i + per_line])
+        rows.append(f'    input  signed [7:0] {names},')
+    return '\n'.join(rows)
+
+
+def _sum(dims, width, per_line=4):
+    pad = width - 8
+    terms = [(f'{{{{{pad}{{f{d}[7]}}}}, f{d}}}' if pad else f'f{d}') for d in dims]
+    rows = [' + '.join(terms[i:i + per_line]) for i in range(0, len(terms), per_line)]
+    return ' +\n        '.join(rows)
+
+
+def emit(name, pos, neg):
+    k = max(len(pos), len(neg))
+    width = 8 + max(1, math.ceil(math.log2(k))) if k > 1 else 8
+    body = f'''
+module {name} (
+{_decl(list(pos) + list(neg))}
+    output person_present
+);
+'''
+    if k == 1:
+        body += f'    assign person_present = f{pos[0]} > f{neg[0]};\nendmodule\n'
+        return HEADER.format(n=len(pos) + len(neg)) + body
+    body += f'''    wire signed [{width - 1}:0] pos_sum =
+        {_sum(pos, width)};
+
+    wire signed [{width - 1}:0] neg_sum =
+        {_sum(neg, width)};
+
+    assign person_present = pos_sum > neg_sum;
+endmodule
+'''
+    return HEADER.format(n=len(pos) + len(neg)) + body
+
+
+def generate(out_dir=None, rules_json=None):
+    out_dir = Path(out_dir or HERE / 'rtl')
+    out_dir.mkdir(parents=True, exist_ok=True)
+    doc = read_artifact(rules_json or HERE / 'rules.json')
+    written = []
+    for name, r in doc['rules'].items():
+        path = out_dir / f'person_{name}.v'
+        path.write_text(emit(f'person_{name}', r['pos_dims'], r['neg_dims']),
+                        encoding='utf-8')
+        written.append(path)
+    return written
+
+
+if __name__ == '__main__':
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument('--out', type=Path, default=None)
+    args = ap.parse_args()
+    for p in generate(args.out):
+        print(f'wrote {p}')

rules.json

@@ -0,0 +1,205 @@
+{
+  "provenance": {
+    "generator": "choose.py",
+    "pool": "TRAIN2017",
+    "split": "train2017",
+    "n_images": 118287,
+    "positive_rate": 0.542,
+    "selection": "all 118287 train2017 images, used only to select dims",
+    "decision": "sum(feat[pos_dims]) > sum(feat[neg_dims])",
+    "search": "greedy, alternating sides, boundary pinned at zero"
+  },
+  "candidates": 192,
+  "sizes": [
+    2,
+    4,
+    6,
+    8,
+    12,
+    16,
+    20,
+    40
+  ],
+  "rules": {
+    "d2": {
+      "pos_dims": [
+        48
+      ],
+      "neg_dims": [
+        637
+      ],
+      "n_dims": 2,
+      "free_parameters": 0,
+      "F1_train": 0.8734
+    },
+    "d4": {
+      "pos_dims": [
+        48,
+        71
+      ],
+      "neg_dims": [
+        637,
+        90
+      ],
+      "n_dims": 4,
+      "free_parameters": 0,
+      "F1_train": 0.8874
+    },
+    "d6": {
+      "pos_dims": [
+        48,
+        71,
+        292
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82
+      ],
+      "n_dims": 6,
+      "free_parameters": 0,
+      "F1_train": 0.9055
+    },
+    "d8": {
+      "pos_dims": [
+        48,
+        71,
+        292,
+        111
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82,
+        269
+      ],
+      "n_dims": 8,
+      "free_parameters": 0,
+      "F1_train": 0.9104
+    },
+    "d12": {
+      "pos_dims": [
+        48,
+        71,
+        292,
+        111,
+        724,
+        406
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82,
+        269,
+        565,
+        140
+      ],
+      "n_dims": 12,
+      "free_parameters": 0,
+      "F1_train": 0.9133
+    },
+    "d16": {
+      "pos_dims": [
+        48,
+        71,
+        292,
+        111,
+        724,
+        406,
+        510,
+        665
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82,
+        269,
+        565,
+        140,
+        537,
+        49
+      ],
+      "n_dims": 16,
+      "free_parameters": 0,
+      "F1_train": 0.917
+    },
+    "d20": {
+      "pos_dims": [
+        48,
+        71,
+        292,
+        111,
+        724,
+        406,
+        510,
+        665,
+        318,
+        267
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82,
+        269,
+        565,
+        140,
+        537,
+        49,
+        283,
+        114
+      ],
+      "n_dims": 20,
+      "free_parameters": 0,
+      "F1_train": 0.9213
+    },
+    "d40": {
+      "pos_dims": [
+        48,
+        71,
+        292,
+        111,
+        724,
+        406,
+        510,
+        665,
+        318,
+        267,
+        552,
+        323,
+        382,
+        305,
+        127,
+        293,
+        85,
+        721,
+        207,
+        326
+      ],
+      "neg_dims": [
+        637,
+        90,
+        82,
+        269,
+        565,
+        140,
+        537,
+        49,
+        283,
+        114,
+        310,
+        113,
+        494,
+        571,
+        694,
+        92,
+        518,
+        91,
+        307,
+        437
+      ],
+      "n_dims": 40,
+      "free_parameters": 0,
+      "F1_train": 0.9301
+    }
+  }
+}

synth.json

@@ -0,0 +1,110 @@
+{
+  "provenance": {
+    "generator": "synth.py",
+    "inputs": "signed INT8 channels of the pooled feature vector",
+    "note": "LUT4 and slice counts on an ECP5, not abstract gate counts"
+  },
+  "tool": "nosis",
+  "target": {
+    "family": "ecp5",
+    "device": "LFE5U-25F"
+  },
+  "variants": {
+    "d2": {
+      "slices": 4,
+      "lut4": 7,
+      "ccu2c": 0,
+      "ffs": 0,
+      "bound": "lut",
+      "critical_path_ns": 0.4,
+      "max_freq_mhz": 2040.8,
+      "device": "LFE5U-25F",
+      "rtl": "rtl/person_d2.v",
+      "n_dims": 2
+    },
+    "d4": {
+      "slices": 10,
+      "lut4": 8,
+      "ccu2c": 10,
+      "ffs": 0,
+      "bound": "carry",
+      "critical_path_ns": 0.9,
+      "max_freq_mhz": 973.4,
+      "device": "LFE5U-25F",
+      "rtl": "rtl/person_d4.v",
+      "n_dims": 4
+    },
+    "d6": {
+      "slices": 20,
+      "lut4": 9,
+      "ccu2c": 20,
+      "ffs": 0,
+      "bound": "carry",
+      "critical_path_ns": 1.4,
+      "max_freq_mhz": 654.8,
+      "device": "LFE5U-25F",
+      "rtl": "rtl/person_d6.v",
+      "n_dims": 6
+    },
+    "d8": {
+      "slices": 30,
+      "lut4": 9,
+      "ccu2c": 30,
+      "ffs": 0,
+      "bound": "carry",
+      "critical_path_ns": 1.9,
+      "max_freq_mhz": 493.3,
+      "device": "LFE5U-25F",
+      "rtl": "rtl/person_d8.v",
+      "n_dims": 8
+    },
+    "d12": {
+      "slices": 60,
+      "lut4": 10,
+      "ccu2c": 60,
+      "ffs": 0,
+      "bound": "carry",
+      "critical_path_ns": 2.9,
+      "max_freq_mhz": 330.3,
+      "device": "LFE5U-25F",
+      "rtl": "rtl/person_d12.v",
+      "n_dims": 12
+    },
+    "d16": {
+      "slices": 84,
+      "lut4": 10,
+      "ccu2c": 84,
+      "ffs": 0,
+      "bound": "carry",
+      "critical_path_ns": 3.9,
+      "max_freq_mhz": 248.3,
+      "device": "LFE5U-25F",
+      "rtl": "rtl/person_d16.v",
+      "n_dims": 16
+    },
+    "d20": {
+      "slices": 108,
+      "lut4": 11,
+      "ccu2c": 108,
+      "ffs": 0,
+      "bound": "carry",
+      "critical_path_ns": 4.9,
+      "max_freq_mhz": 198.4,
+      "device": "LFE5U-25F",
+      "rtl": "rtl/person_d20.v",
+      "n_dims": 20
+    },
+    "d40": {
+      "slices": 266,
+      "lut4": 12,
+      "ccu2c": 266,
+      "ffs": 0,
+      "bound": "carry",
+      "critical_path_ns": 9.9,
+      "max_freq_mhz": 99.0,
+      "device": "LFE5U-25F",
+      "rtl": "rtl/person_d40.v",
+      "n_dims": 40
+    }
+  }
+}

synth.py

@@ -0,0 +1,91 @@
+"""Synthesize every generated module with nosis and record the cell counts.
+
+    python synth.py
+
+nosis is a pure-Python SystemVerilog to Lattice ECP5 synthesizer. It is the
+backend used here rather than Yosys, whose gate counts are not comparable.
+Counts are LUT4s and slices on an ECP5, not abstract gates.
+"""
+import argparse
+import json
+import re
+import subprocess
+import sys
+from pathlib import Path
+
+from common import read_artifact, write_artifact
+
+HERE = Path(__file__).resolve().parent
+NOSIS_ROOT = Path(r'D:\nosis')
+
+
+def synth_one(src: Path, top: str, build: Path, nosis_root: Path):
+    build.mkdir(parents=True, exist_ok=True)
+    r = subprocess.run(
+        [sys.executable, '-m', 'nosis', str(src), '--top', top, '--stats',
+         '-o', str(build / f'{top}.json')],
+        cwd=str(nosis_root), capture_output=True, text=True)
+    if r.returncode != 0:
+        raise SystemExit(f'nosis failed on {src.name}:\n{r.stdout[-2000:]}{r.stderr[-2000:]}')
+    (build / f'{top}.log').write_text(r.stdout, encoding='utf-8')
+
+    def grab(pattern, cast=int):
+        m = re.search(pattern, r.stdout)
+        return cast(m.group(1)) if m else None
+
+    def text(pattern):
+        m = re.search(pattern, r.stdout)
+        return m.group(1) if m else None
+
+    # Adder trees land on the ECP5 carry chain, so CCU2C rather than LUT4 is the
+    # size that moves; `bound` records which resource the design is limited by.
+    return {
+        'slices': grab(r'Slices:\s+(\d+)'),
+        'lut4': grab(r'LUTs:\s+(\d+)'),
+        'ccu2c': grab(r'CCU2C:\s+(\d+)'),
+        'ffs': grab(r'FFs:\s+(\d+)'),
+        'bound': text(r'Bound:\s+(\S+)'),
+        'critical_path_ns': grab(r'Critical path delay:\s+([\d.]+)', float),
+        'max_freq_mhz': grab(r'Max frequency \(logic\):\s+([\d.]+)', float),
+        'device': text(r'Device:\s+(\S+)'),
+    }
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument('--rtl', type=Path, default=HERE / 'rtl')
+    ap.add_argument('--build', type=Path, default=HERE / 'build')
+    ap.add_argument('--nosis', type=Path, default=NOSIS_ROOT)
+    ap.add_argument('--out', type=Path, default=HERE / 'synth.json')
+    args = ap.parse_args()
+
+    rules = read_artifact(HERE / 'rules.json')['rules']
+    variants = {}
+    print(f"{'rule':>6}{'dims':>6}{'slices':>8}{'LUT4':>7}{'CCU2C':>7}"
+          f"{'bound':>7}{'ns':>7}")
+    for name in rules:
+        top = f'person_{name}'
+        src = args.rtl / f'{top}.v'
+        if not src.exists():
+            raise SystemExit(f'{src} missing; run rtl_gen.py first')
+        s = synth_one(src, top, args.build, args.nosis)
+        s['rtl'] = f'rtl/{top}.v'
+        s['n_dims'] = rules[name]['n_dims']
+        variants[name] = s
+        print(f'{name:>6}{s["n_dims"]:>6}{s["slices"]:>8}{s["lut4"]:>7}'
+              f'{s["ccu2c"]:>7}{s["bound"]:>7}{s["critical_path_ns"]:>7.2f}',
+              flush=True)
+
+    device = next((v['device'] for v in variants.values() if v['device']), None)
+    write_artifact(args.out, {
+        'tool': 'nosis',
+        'target': {'family': 'ecp5', 'device': device},
+        'variants': variants,
+    }, generator='synth.py',
+        inputs='signed INT8 channels of the pooled feature vector',
+        note='LUT4 and slice counts on an ECP5, not abstract gate counts')
+    print(f'\n[done] wrote {args.out}')
+
+
+if __name__ == '__main__':
+    main()

tests/conftest.py

@@ -0,0 +1,29 @@
+"""Shared fixtures for the consistency suite."""
+import json
+import sys
+from pathlib import Path
+
+import pytest
+
+REPO = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(REPO))
+
+
+def load(rel: str) -> dict:
+    """Parse a repo-relative JSON file."""
+    return json.loads((REPO / rel).read_text(encoding='utf-8'))
+
+
+@pytest.fixture(scope='session')
+def repo() -> Path:
+    return REPO
+
+
+@pytest.fixture(scope='session')
+def rules() -> dict:
+    return load('rules.json')
+
+
+@pytest.fixture(scope='session')
+def evaluation() -> dict:
+    return load('eval.json')

tests/test_artifacts.py

@@ -0,0 +1,45 @@
+"""Every committed artifact matches the schema its generator declares."""
+import pytest
+
+from common.artifacts import REGISTRY
+from conftest import REPO, load
+
+
+@pytest.mark.parametrize('rel', sorted(REGISTRY))
+def test_artifact_exists(rel):
+    assert (REPO / rel).exists(), f'{rel} is registered but missing'
+
+
+@pytest.mark.parametrize('rel', sorted(REGISTRY))
+def test_artifact_opens_with_provenance(rel):
+    doc = load(rel)
+    assert next(iter(doc)) == 'provenance', f'{rel} does not open with provenance'
+
+
+@pytest.mark.parametrize('rel', sorted(REGISTRY))
+def test_artifact_payload_keys(rel):
+    spec = REGISTRY[rel]
+    got = tuple(k for k in load(rel) if k != 'provenance')
+    assert got == spec.payload_keys, f'{rel} payload {got} != declared {spec.payload_keys}'
+
+
+@pytest.mark.parametrize('rel', sorted(REGISTRY))
+def test_artifact_generator(rel):
+    assert load(rel)['provenance']['generator'] == REGISTRY[rel].generator
+
+
+@pytest.mark.parametrize('rel', sorted(REGISTRY))
+def test_artifact_pool(rel):
+    spec = REGISTRY[rel]
+    pool = load(rel)['provenance'].get('pool')
+    if spec.pool is not None:
+        assert pool == spec.pool, f'{rel} names pool {pool!r}, registry says {spec.pool!r}'
+
+
+def test_selection_and_evaluation_read_different_splits():
+    """No figure may be measured on the images that chose it."""
+    sel = load('rules.json')['provenance']
+    ev = load('eval.json')['provenance']
+    assert sel['split'] == 'train2017'
+    assert ev['split'] == 'val2017'
+    assert sel['split'] != ev['split']

tests/test_rtl.py

@@ -0,0 +1,106 @@
+"""The generated RTL carries the selected dims and computes the comparison.
+
+The structural checks run everywhere. The simulation checks run when Icarus
+Verilog is available, on PATH or via the IVERILOG and VVP environment variables.
+"""
+import os
+import re
+import shutil
+import subprocess
+
+import pytest
+
+from conftest import REPO, load
+
+RTL = REPO / 'rtl'
+RULES = load('rules.json')['rules']
+NAMES = sorted(RULES)
+N_VECTORS = 256
+
+
+def tool(name, env_var):
+    return os.environ.get(env_var) or shutil.which(name)
+
+
+IVERILOG, VVP = tool('iverilog', 'IVERILOG'), tool('vvp', 'VVP')
+needs_sim = pytest.mark.skipif(not (IVERILOG and VVP),
+                               reason='Icarus Verilog not found')
+
+
+def source(name):
+    return (RTL / f'person_{name}.v').read_text(encoding='utf-8')
+
+
+@pytest.mark.parametrize('name', NAMES)
+def test_module_exists(name):
+    assert (RTL / f'person_{name}.v').exists()
+
+
+@pytest.mark.parametrize('name', NAMES)
+def test_ports_are_exactly_the_selected_dims(name):
+    r = RULES[name]
+    declared = re.findall(r'\bf(\d+)\b', source(name).split(');')[0])
+    assert sorted(int(d) for d in set(declared)) == sorted(r['pos_dims'] + r['neg_dims'])
+
+
+@pytest.mark.parametrize('name', NAMES)
+def test_no_constant_is_baked_in(name):
+    """A zero-parameter rule must not contain a fitted threshold."""
+    body = source(name).split('\n')
+    body = '\n'.join(l for l in body if not l.strip().startswith('//'))
+    assert 'localparam' not in body, f'{name} declares a constant'
+    for lit in re.findall(r"\d+'s?d(\d+)", body):
+        assert int(lit) == 0, f'{name} compares against a non-zero constant'
+
+
+def vectors(n, dims, seed=0):
+    import random
+    rng = random.Random(seed)
+    return [{d: rng.randint(-128, 127) for d in dims} for _ in range(n)]
+
+
+def reference(vec, pos, neg):
+    return sum(vec[d] for d in pos) > sum(vec[d] for d in neg)
+
+
+@needs_sim
+@pytest.mark.parametrize('name', NAMES)
+def test_rtl_matches_the_reference(tmp_path, name):
+    r = RULES[name]
+    dims = r['pos_dims'] + r['neg_dims']
+    vecs = vectors(N_VECTORS, dims)
+    # Half the vectors are pushed onto the boundary, where > must reject ties.
+    for i in range(0, len(vecs), 2):
+        v = vecs[i]
+        v[r['pos_dims'][0]] = (sum(v[d] for d in r['neg_dims'])
+                               - sum(v[d] for d in r['pos_dims'][1:]))
+    vecs = [v for v in vecs if all(-128 <= x <= 127 for x in v.values())]
+
+    top = f'person_{name}'
+    conns = ',\n        '.join(f'.f{d}(f{d})' for d in dims)
+    decls = ', '.join(f'f{d}' for d in dims)
+    lines = []
+    for v in vecs:
+        lines.append('    ' + ' '.join(f'f{d} = {v[d]};' for d in dims) + ' #1;'
+                     ' $display("%b", out);')
+    tb = f'''`timescale 1ns/1ps
+module tb;
+  reg signed [7:0] {decls};
+  wire out;
+  {top} dut (
+        {conns},
+        .person_present(out));
+  initial begin
+{chr(10).join(lines)}
+    $finish;
+  end
+endmodule
+'''
+    (tmp_path / 'tb.v').write_text(tb)
+    subprocess.run([IVERILOG, '-g2005', '-o', 'tb.vvp',
+                    str(RTL / f'{top}.v'), 'tb.v'],
+                   cwd=tmp_path, check=True, capture_output=True)
+    out = subprocess.run([VVP, 'tb.vvp'], cwd=tmp_path, check=True,
+                         capture_output=True, text=True).stdout
+    got = [l.strip() == '1' for l in out.splitlines() if l.strip() in ('0', '1')]
+    assert got == [reference(v, r['pos_dims'], r['neg_dims']) for v in vecs]

tests/test_rules.py

@@ -0,0 +1,61 @@
+"""The rules are well formed and say the same thing everywhere they appear."""
+import pytest
+
+from conftest import load
+
+RULE_NAMES = sorted(load('rules.json')['rules'])
+
+
+@pytest.mark.parametrize('name', RULE_NAMES)
+def test_sets_are_disjoint_and_balanced(name, rules):
+    r = rules['rules'][name]
+    pos, neg = r['pos_dims'], r['neg_dims']
+    assert len(pos) == len(neg), f'{name}: the two sums must have equal term counts'
+    assert not set(pos) & set(neg), f'{name}: a dim appears on both sides'
+    assert len(set(pos)) == len(pos) and len(set(neg)) == len(neg)
+    assert r['n_dims'] == len(pos) + len(neg)
+
+
+@pytest.mark.parametrize('name', RULE_NAMES)
+def test_dims_are_in_range(name, rules):
+    for d in rules['rules'][name]['pos_dims'] + rules['rules'][name]['neg_dims']:
+        assert 0 <= d < 768
+
+
+@pytest.mark.parametrize('name', RULE_NAMES)
+def test_no_free_parameters(name, rules):
+    assert rules['rules'][name]['free_parameters'] == 0
+
+
+@pytest.mark.parametrize('name', RULE_NAMES)
+def test_name_matches_size(name, rules):
+    assert name == f'd{rules["rules"][name]["n_dims"]}'
+
+
+def test_rules_nest(rules):
+    """Greedy selection grows the sets, so each rule extends the one below it."""
+    by_size = sorted(rules['rules'].values(), key=lambda r: r['n_dims'])
+    for small, large in zip(by_size, by_size[1:]):
+        assert small['pos_dims'] == large['pos_dims'][:len(small['pos_dims'])]
+        assert small['neg_dims'] == large['neg_dims'][:len(small['neg_dims'])]
+
+
+@pytest.mark.parametrize('name', RULE_NAMES)
+def test_eval_carries_the_same_dims(name, rules, evaluation):
+    r, e = rules['rules'][name], evaluation['rules'][name]
+    assert e['pos_dims'] == r['pos_dims']
+    assert e['neg_dims'] == r['neg_dims']
+    assert e['F1_train'] == r['F1_train']
+
+
+def test_more_dims_do_not_score_worse(evaluation):
+    by_size = sorted(evaluation['rules'].values(), key=lambda r: r['n_dims'])
+    f1 = [r['F1'] for r in by_size]
+    assert f1 == sorted(f1), f'F1 is not monotone in dim count: {f1}'
+
+
+def test_val_tracks_train(evaluation):
+    """Selection on 118k images should not overfit; val must stay close to train."""
+    for name, r in evaluation['rules'].items():
+        assert abs(r['F1'] - r['F1_train']) < 0.02, \
+            f'{name}: train {r["F1_train"]} against val {r["F1"]}'

verify.py

@@ -0,0 +1,55 @@
+"""Score every rule in rules.json on val2017 and write eval.json.
+
+    python verify.py
+
+Dimensions were chosen on train2017 and are not refit here.
+"""
+import argparse
+from pathlib import Path
+
+from common import COCO_ROOT, prf1, read_artifact, write_artifact
+from common.cached import load_pooled
+from common.pools import VAL5000
+
+HERE = Path(__file__).resolve().parent
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument('--cache', type=Path, default=None)
+    ap.add_argument('--rules', type=Path, default=HERE / 'rules.json')
+    ap.add_argument('--out', type=Path, default=HERE / 'eval.json')
+    args = ap.parse_args()
+
+    cache = args.cache or COCO_ROOT / 'pooled_val2017'
+    X, y = load_pooled(cache, 'val2017')
+    print(f'[val] {X.shape[0]} images, person rate {y.float().mean():.3f}\n',
+          flush=True)
+
+    doc = read_artifact(args.rules)
+    out = {}
+    print(f"{'rule':>6}{'dims':>6}{'F1 train':>10}{'F1 val':>9}{'P':>9}{'R':>9}")
+    for name, r in doc['rules'].items():
+        s = X[:, r['pos_dims']].sum(1) - X[:, r['neg_dims']].sum(1)
+        m = prf1(s > 0, y)
+        out[name] = {'n_dims': r['n_dims'], 'free_parameters': 0,
+                     'pos_dims': r['pos_dims'], 'neg_dims': r['neg_dims'],
+                     'F1_train': r['F1_train'],
+                     'F1': round(m.f1, 4), 'precision': round(m.precision, 4),
+                     'recall': round(m.recall, 4)}
+        print(f'{name:>6}{r["n_dims"]:>6}{r["F1_train"]:>10.4f}{m.f1:>9.4f}'
+              f'{m.precision:>9.4f}{m.recall:>9.4f}', flush=True)
+
+    write_artifact(args.out, {'rules': out},
+                   generator='verify.py',
+                   pool_info={'pool': VAL5000.name, 'split': VAL5000.split,
+                              'n_images': int(X.shape[0]),
+                              'positive_rate': round(y.float().mean().item(), 4),
+                              'selection': VAL5000.selection},
+                   task='image-level person presence (binary)',
+                   protocol='dims selected on train2017, not refit here')
+    print(f'\n[done] wrote {args.out}', flush=True)
+
+
+if __name__ == '__main__':
+    main()