# BARG-CC — document-only TNNLS manuscript and experimental package **Allowed datasets: RVL-CDIP and FUNSD only.** The two tasks are **not interchangeable**: * **RVL-CDIP:** page-level document category clustering, K=16, evaluated with ARI/NMI using official page-category labels. * **FUNSD:** *word-level* semantic role clustering, K=4 (header/question/answer/other), evaluated on OCR-detected words matched to ground-truth annotated word boxes. Parent form IDs define data splits and uncertainty units. FUNSD **does not** supply 16-class document-category labels. The source here contains a complete 12-page IEEEtran two-column draft, document-specific data parsers, and a strict train/validation/test benchmark runner. It **does not** include either dataset or genuine benchmark accuracy. The raw images/annotations were not available locally. The software tests use generated fixtures only. The draft is **not submission-ready** until the actual data are processed and the empirical tables are populated from verified runs. ## Files - `main.tex`: editable LaTeX manuscript (IEEEtran, 10pt journal). - `main.pdf`: compiled 12-page manuscript. - `prepare_documents.py`: two-corpus image/OCR/geometry extraction and split integrity. - `run_document_benchmark.py`: frozen-protocol neural-denoising, masking, PCA, shuffled-score and prior controls, and ARI/NMI readout. - `paired_group_bootstrap.py`: paired ARI/NMI uncertainty by independent page/form group. - `test_document_protocol.py`: synthetic software, group-bootstrap, and theorem checks. - `DATASET_STATUS.json`: honest dataset/execution status. - `ENVIRONMENT.json`: local software versions. - `SHA256SUMS.txt`: integrity hashes. - `requirements.txt`: Python dependencies. ## Source-identity preflight (required before a journal benchmark) Run `preflight_corpora.py` before feature extraction. With default strict counts it refuses a partial dataset while recording available SHA-256 metadata; an explicit `--allow-subset` flag labels any pilot as a subset. **Counts and local hashes alone do not independently authenticate provenance or grant redistribution rights.** ``` python preflight_corpora.py rvlcdip --root /path/to/RVL-CDIP \\ --output attestations/rvlcdip_preflight.json python preflight_corpora.py funsd --root /path/to/FUNSD \\ --output attestations/funsd_preflight.json ``` RVL-CDIP checks the 320,000/40,000/40,000 official list counts and verifies the referenced paths exist, sample image hashes, category ranges and split identity. FUNSD checks its 149/50 form counts, image and annotation integrity, word-box schemas and all local media hashes. Neither preflight uses the labels for training. ## Dataset 1: RVL-CDIP Supply an authorized RVL-CDIP unpacked root that contains: ``` rvl_root/ labels/ train.txt val.txt test.txt images/ ... images referenced by the three label lists ... ``` Each label-list line must be ` `. The script defaults to a *clearly labeled resource-limited pilot*: 6000 training, 1000 validation and 1000 test pages, selected by a stable file-ID hash **without inspecting labels**. Set all three `--max-*` counts to `0` for complete official splits; OCR on the full corpus can require considerable storage and computation. ``` python prepare_documents.py rvlcdip \ --root /path/to/RVL-CDIP --out data/rvlcdip.npz \ --max-train 6000 --max-validation 1000 --max-test 1000 ``` Do not describe a hash-subset run as a full RVL-CDIP benchmark. Exact-byte duplicates across partitions cause a fail-closed error. ## Dataset 2: FUNSD Supply the canonical image and annotation hierarchy: ``` funsd_root/ training_data/ images/*.png annotations/*.json testing_data/ images/*.png annotations/*.json ``` Word-level roles come from the `form[].label` fields **only for final scoring**; OCR text, detected boxes and word crops supply features. Every OCR detection is retained as an unlabeled clustering record. A one-to-one spatial match against annotated boxes attaches a role solely for evaluation; unmatched detections receive label -1 (unscored, **not** the official "other" category). Both matched and unmatched predictions are exported. The benchmark's `n_test_scored`/`n_test` and `test_detection_match_fraction` describe OCR-detection match coverage; the preparation audit separately reports annotation recall and per-form coverage (including forms with zero matches). Different words from the same page never enter different splits. FUNSD's official 50 test forms remain untouched. ``` python prepare_documents.py funsd \ --root /path/to/FUNSD --out data/funsd.npz --min-iou 0.30 ``` The script fails if OCR yields no scoreable words in an essential split. Please inspect the coverage audit in the adjacent `funsd.json` before interpreting word-role clustering. ## Frozen evaluation (repeat for EACH dataset) ``` python run_document_benchmark.py freeze \ --npz data/funsd.npz --lock protocols/funsd_lock.json ``` Review the protocol **before** any test-score release. It locks the source-file SHA-256 and train/validation/test sample identity. Scoring is explicitly gated: ``` python run_document_benchmark.py execute \ --npz data/funsd.npz --lock protocols/funsd_lock.json \ --output results/funsd --release-test ``` Run equivalent commands with the RVL-CDIP NPZ and a separate lock. The seven local methods are masked concatenation, train-only PCA16, denoised uniform gating, denoised predicted gating, denoised shuffled scoring, denoised fixed-view prior and identity-anchored fusion. All methods use train-fitted K-means and held-out assignments. The runner also saves `predictions.csv.gz`, allowing independent recomputation of per-group ARI/NMI uncertainty. The model is a **train-only MLP denoising reference**; it is **not an executed reproduction of the full jointly contrastive BARG-CC objective**. ### Limitations you must not misstate 1. The implemented stress tests add **feature-space** dropout and noise, not real image blur, crop, bleed-through or OCR reruns. 2. FUNSD word-level conditional ARI is calculated only for role-matched OCR words, although *all* OCR detections are clustered. It does not establish document-category performance or measure missed annotation words; always publish both detection-match fraction and annotation recall, including zero-match forms. 3. The supplied package has **no executed official-code recent neural comparators**. Never insert external benchmark numbers without source/checkpoint/prediction provenance. 4. Freezing a protocol file does not, by itself, demonstrate prospective preregistration. 5. Repeated neural seeds on the same page/form test set are not independent dataset replicates. 6. No dataset labels enter feature preparation or training; labels are used for K according to the published ontology and final scoring. ## Reproducibility checks ``` python -m unittest -v test_document_protocol.py latexmk -pdf -interaction=nonstopmode -halt-on-error main.tex pdfinfo main.pdf | grep Pages ``` The LaTeX build is self-contained (TikZ included); it does not depend on cached figures from earlier versions. **Raw data are not redistributed.** Dataset download, rights, authenticity and journal requirements must be verified by the submitting authors. ## Published-method comparison identity contract The script below exports the exact same held-out sample/seed/condition keys from the local masked baseline, **without labels or model predictions**: ``` python export_official_reference.py \\ --predictions results/funsd/predictions.csv.gz \\ --reference-out results/funsd/official_reference_ids.csv \\ --contract-out results/funsd/official_reference_contract.json ``` This is an **unexecuted comparator interface**, not an official-code result. Once a licensed official implementation has actually been run with the exact features, masks, train/test identities and training-label restrictions, verify its provenance and keyed predictions with: ``` python ../archived_evidence/reproducibility/verify_official_comparator.py \\ --manifest /path/to/verified_official_run.json \\ --results /path/to/official_predictions.csv \\ --reference results/funsd/official_reference_ids.csv ``` A provenance contract or GitHub URL alone does NOT constitute an executed published baseline. No comparator result has been added to this package. ## Paired, form-grouped uncertainty (after a genuine FUNSD run) ``` python paired_group_bootstrap.py \ --predictions results/funsd/predictions.csv.gz \ --dataset funsd --condition M20_D50 --seed 11 \ --method-a denoised_gated --method-b masked \ --replicates 500 --output results/funsd/group_uncertainty_seed11.json ``` This resamples complete *forms*, not individual OCR words. The same command accepts `--dataset rvlcdip` for page-level groups or deduplicated source-document IDs. It is a conditional inference check for that fixed test split and does not make repeated training seeds independent datasets. ## New, distinct joint-contrastive reference implementation `run_joint_contrastive_reference.py` adds jointly trained view encoders, reliability-conditioned residual maps, learned alignment projections and all three loss terms. Its source-locked predictions are compatible with local baseline scoring, but it is **not** an original checkpoint replay or a result on canonical RVL-CDIP/FUNSD. Use `compare_joint_reference.py` for source-matched, group-level descriptive differences. Full instructions: `JOINT_REFERENCE_AND_DATA_GUIDE.md`.