AIP-AD — Liars Are Information for Multi-Agent Industrial Anomaly Detection

Swarm decision fusion for multi-view / multi-modal industrial anomaly detection, with a GRPO-reinforced arbiter, agent calibration, and continual learning. This repository is the industrial-anomaly-detection continuation of the "Liars Are Information" research program (AIP — Adversarial Informativeness Pooling, Byzantine-robust decentralized LLM swarms, Dhruv1000/Liars_Are_Information).

The idea

In Liars Are Information we showed that coherent adversarial agents inside a swarm should not be discarded — their reports can be inverted and pooled as information. AIP-AD transfers that principle from LLM swarms to industrial inspection:

• Real-IAD (Wang et al., CVPR 2024) photographs every physical sample with 5 camera agents; Real-IAD D³ (Zhu et al., CVPR 2025) captures every sample with 4 modality agents (RGB, photometric stereo, XYZ surface map, point cloud). The official Real-IAD protocol integrates the camera scores with a plain max. A defect usually shows in only 1–2 views, so the majority of agents "lie" (report normal).
• Agent Calibration Layer (ACL) — each camera/modality agent has a systematic reporting bias (e.g. in bottle_cap, camera C1's normal-score median is ~38% above cameras C2–C5). ACL is a label-free quantile standardization of every agent's reports against the normal-only training swarm. After ACL, Byzantine-robust poolers (median, trimmed mean) that were destroyed by the bias become the strongest fixed rules — the AIP thesis, empirically.
• AIP mechanisms make dissent first-class: aip_d (dissent amplification), aip_c (consensus shrinkage), aip_mt/aip_wt (consensus-anchored dissent blends).
• GRPO arbiter: a policy over the pooling strategies (state = swarm statistics: sorted scores, gaps, dispersion) trained in the group-relative style of GRPO — every episode evaluates all K poolings on a mini-batch, rewards (batch AUROC) are standardized within the group of strategies, and REINFORCE moves the softmax policy toward high-advantage actions.
• Continual learning: the arbiter streams over categories with experience replay (no catastrophic forgetting), plus an online multiplicative-weights (Hedge) variant that learns from delayed operator feedback on the test stream.

Datasets

dataset	agents	scale	access
Real-IAD (Wang et al., CVPR 2024)	5 cameras / sample	151,050 images, 30 categories	gated (auto-approve) on HF
Real-IAD D³ (Zhu et al., CVPR 2025)	4 modalities / sample (RGB, PS, XYZ, PCD)	20 categories, 8,450 samples	gated on HF

Both are CC-BY-NC-SA-4.0, research use only. Accept the conditions on the dataset pages with your HF account; run_gpu_full.py then downloads them automatically via HF_TOKEN.

Previous-best context (full 30-category benchmark, published in the Real-IAD paper, Table 2): multi-view sample-level S-AUROC — PatchCore 93.7 (max-pooled views), DeSTSeg 94.0, SimpleNet 94.9. Our contribution is a drop-in decision-fusion layer: with the same per-view scorer, the sample-level AUROC improves substantially once the swarm is calibrated and fused with the RL arbiter instead of plain max.

Results (this repo's run, 6-category CPU-scale configuration)

Real-IAD subset (usb, pcb, mint, eraser, bottle_cap, zipper), official train/test JSONs, 256×256 protocol, ResNet-18 PatchCore-style scorer, 300 test samples/category (25% calibration / 75% eval). Subset numbers measure the fusion effect under a fixed scorer — they are not directly comparable to the paper's full 30-category WRN-50 benchmark; run run_gpu_full.py for that.

RAW agents (no calibration) — the paper protocol is the worst good rule here:

strategy	usb	pcb	mint	eraser	bottle_cap	zipper	MACRO
max (paper protocol)	71.7	73.1	56.3	87.3	72.1	97.7	76.4
mean	79.5	85.1	57.1	91.3	74.9	98.0	81.0
median	73.7	83.9	57.1	88.9	77.9	96.5	79.7
top2	73.3	82.2	56.7	90.0	67.4	98.2	78.0
trimmed	77.0	86.4	56.9	89.5	72.6	97.0	79.9
aip_d05	69.5	65.4	54.5	85.2	68.9	96.2	73.3
aip_c03	69.9	66.2	54.7	85.5	69.2	96.4	73.7
aip_soft2	73.4	72.2	56.6	87.5	73.4	97.6	76.8
aip_mt01	78.8	84.6	57.2	91.1	74.9	98.1	80.8
aip_mt025	77.9	83.2	57.5	90.5	74.6	98.2	80.3
aip_mt05	75.9	79.9	57.6	89.5	74.2	98.2	79.2
aip_mt075	73.7	76.2	56.9	88.4	73.1	98.0	77.7
aip_wt05	76.6	84.3	56.8	90.7	70.9	98.3	79.6
aip_wt075	74.9	83.3	56.9	90.3	68.9	98.3	78.8
GRPO arbiter (ours)	79.6	84.1	57.8	89.5	86.5	97.7	82.5

ACL-calibrated agents — robust poolers recover, the arbiter tops everything:

strategy	usb	pcb	mint	eraser	bottle_cap	zipper	MACRO
max (paper protocol)	77.3	84.7	55.9	87.4	62.4	98.0	77.6
mean	79.8	85.1	57.9	91.1	75.0	98.0	81.1
median	77.7	82.0	58.6	89.1	88.1	96.7	82.0
top2	78.3	86.1	57.2	90.0	67.1	98.2	79.5
trimmed	79.6	83.9	58.1	88.6	84.3	97.5	82.0
aip_d05	76.4	82.6	55.1	85.3	59.7	97.6	76.1
aip_c03	76.5	82.9	55.1	85.6	60.0	97.7	76.3
aip_soft2	77.5	84.8	56.0	87.4	62.6	98.0	77.7
aip_mt01	79.5	85.7	57.6	90.9	72.5	98.1	80.7
aip_mt025	79.1	85.9	57.0	90.4	69.5	98.1	80.0
aip_mt05	78.4	85.8	56.4	89.5	66.2	98.1	79.1
aip_mt075	77.9	85.4	56.2	88.5	64.1	98.1	78.4
aip_wt05	79.0	86.0	57.3	90.6	70.0	98.2	80.2
aip_wt075	78.7	86.1	57.3	90.4	68.4	98.2	79.9
GRPO arbiter (ours)	79.6	84.1	57.8	89.5	86.5	97.7	82.5

• GRPO arbiter vs paper-protocol max: +6.2 S-AUROC points macro.
• GRPO arbiter vs best fixed rule on raw agents (mean): +1.5 points.
• GRPO arbiter vs best fixed rule after ACL (median): +0.5 points.
• The learned policy concentrates on trimmed / median (see results/figures/fig4_policy.png) — reinforcement learning rediscovers Byzantine-robust pooling once agents are calibrated.
• Known limitation: mint is unsolved by this configuration (all poolers ≈ 0.56–0.59) — the ResNet-18 features do not separate its subtle surface defects; the GPU/WRN-50 configuration targets exactly this.

Figures: results/figures/fig1…fig5*.png · video: results/aipad_overview.mp4 · raw metrics: results/results.json (includes calibration-split tables used for all hyperparameter selection).

Repository layout

src/data.py        Real-IAD loader: official JSONs -> 5-view sample grouping
src/data_d3.py     Real-IAD D3 loader: RGB/PS/XYZ/PCD modality agents
src/scorer.py      PatchCore-style per-view scorer (resnet18 / wide_resnet50_2, CPU or CUDA)
src/fusion.py      poolers: max/mean/median/top2/trimmed + AIP-D/C/MT/WT mechanisms
src/arbiter.py     CalibNormalizer + GRPO arbiter + online Hedge
src/evalmetrics.py AUROC utilities
run_experiments.py stage score | stage analyze (6-category CPU run)
run_gpu_full.py    full-scale GPU run: official HF download, all 30 categories, WRN-50 (+ D3)
scripts/           agent calibration, figures, video, notebook generation
AIPAD_Kaggle.ipynb one-click Kaggle GPU notebook (pulls code from this repo, pushes results back)
results/           scores npz, results.json, agent_calibration.json, figures, video

Run it

# local / CPU subset (this configuration)
python3 run_experiments.py --stage score   --categories usb pcb mint eraser bottle_cap zipper
python3 scripts/score_train_normals.py     # agent calibration statistics (ACL)
python3 run_experiments.py --stage analyze --categories usb pcb mint eraser bottle_cap zipper

# arc-3 / any CUDA box: full official benchmark (auto-downloads gated data)
HF_TOKEN=hf_xxx python3 run_gpu_full.py --dataset realiad --backbone wide_resnet50_2
HF_TOKEN=hf_xxx python3 run_gpu_full.py --dataset d3 --categories power_jack fuse_holder

Kaggle: open AIPAD_Kaggle.ipynb, add the HF_TOKEN secret, select GPU, run all.

License & citation

Code: MIT. Data: follow the official Real-IAD / Real-IAD D³ licenses (CC-BY-NC-SA-4.0). If this repository helps your research, please cite the Real-IAD papers (Wang et al., CVPR 2024; Zhu et al., CVPR 2025) and the predecessor AIP work.