Download research_code/document_protocol/README.md from Debmalya/BARG_CC_HuggingFace_Space_Deployment_v2: direct link, hf CLI and curl.
- Browser
- Download file 9.6 kB
-
https://huggingface.co/Debmalya/BARG_CC_HuggingFace_Space_Deployment_v2/resolve/main/research_code/document_protocol/README.md
- Command line
-
hf download hf://Debmalya/BARG_CC_HuggingFace_Space_Deployment_v2/research_code/document_protocol/README.md
-
curl -L -o README.md https://huggingface.co/Debmalya/BARG_CC_HuggingFace_Space_Deployment_v2/resolve/main/research_code/document_protocol/README.md
BARG-CC — document-only TNNLS manuscript and experimental package
Allowed datasets: RVL-CDIP and FUNSD only.
The two tasks are not interchangeable:
- RVL-CDIP: page-level document category clustering, K=16, evaluated with ARI/NMI using official page-category labels.
- FUNSD: word-level semantic role clustering, K=4 (header/question/answer/other), evaluated on OCR-detected words matched to ground-truth annotated word boxes. Parent form IDs define data splits and uncertainty units. FUNSD does not supply 16-class document-category labels.
The source here contains a complete 12-page IEEEtran two-column draft, document-specific data parsers, and a strict train/validation/test benchmark runner. It does not include either dataset or genuine benchmark accuracy. The raw images/annotations were not available locally. The software tests use generated fixtures only. The draft is not submission-ready until the actual data are processed and the empirical tables are populated from verified runs.
Files
main.tex: editable LaTeX manuscript (IEEEtran, 10pt journal).main.pdf: compiled 12-page manuscript.prepare_documents.py: two-corpus image/OCR/geometry extraction and split integrity.run_document_benchmark.py: frozen-protocol neural-denoising, masking, PCA, shuffled-score and prior controls, and ARI/NMI readout.paired_group_bootstrap.py: paired ARI/NMI uncertainty by independent page/form group.test_document_protocol.py: synthetic software, group-bootstrap, and theorem checks.DATASET_STATUS.json: honest dataset/execution status.ENVIRONMENT.json: local software versions.SHA256SUMS.txt: integrity hashes.requirements.txt: Python dependencies.
Source-identity preflight (required before a journal benchmark)
Run preflight_corpora.py before feature extraction. With default strict counts it
refuses a partial dataset while recording available SHA-256 metadata; an explicit
--allow-subset flag labels any pilot as a subset. Counts and local hashes alone
do not independently authenticate provenance or grant redistribution rights.
python preflight_corpora.py rvlcdip --root /path/to/RVL-CDIP \\
--output attestations/rvlcdip_preflight.json
python preflight_corpora.py funsd --root /path/to/FUNSD \\
--output attestations/funsd_preflight.json
RVL-CDIP checks the 320,000/40,000/40,000 official list counts and verifies the referenced paths exist, sample image hashes, category ranges and split identity. FUNSD checks its 149/50 form counts, image and annotation integrity, word-box schemas and all local media hashes. Neither preflight uses the labels for training.
Dataset 1: RVL-CDIP
Supply an authorized RVL-CDIP unpacked root that contains:
rvl_root/
labels/
train.txt
val.txt
test.txt
images/
... images referenced by the three label lists ...
Each label-list line must be <relative_image_path> <category_integer_0_to_15>.
The script defaults to a clearly labeled resource-limited pilot: 6000 training, 1000 validation and 1000 test pages, selected by a stable file-ID hash without inspecting labels. Set all three --max-* counts to 0 for complete official splits; OCR on the full corpus can require considerable storage and computation.
python prepare_documents.py rvlcdip \
--root /path/to/RVL-CDIP --out data/rvlcdip.npz \
--max-train 6000 --max-validation 1000 --max-test 1000
Do not describe a hash-subset run as a full RVL-CDIP benchmark. Exact-byte duplicates across partitions cause a fail-closed error.
Dataset 2: FUNSD
Supply the canonical image and annotation hierarchy:
funsd_root/
training_data/
images/*.png
annotations/*.json
testing_data/
images/*.png
annotations/*.json
Word-level roles come from the form[].label fields only for final scoring; OCR text, detected boxes and word crops supply features. Every OCR detection is retained as an unlabeled clustering record. A one-to-one spatial match against annotated boxes attaches a role solely for evaluation; unmatched detections receive label -1 (unscored, not the official "other" category). Both matched and unmatched predictions are exported. The benchmark's n_test_scored/n_test and test_detection_match_fraction describe OCR-detection match coverage; the preparation audit separately reports annotation recall and per-form coverage (including forms with zero matches). Different words from the same page never enter different splits. FUNSD's official 50 test forms remain untouched.
python prepare_documents.py funsd \
--root /path/to/FUNSD --out data/funsd.npz --min-iou 0.30
The script fails if OCR yields no scoreable words in an essential split. Please inspect the coverage audit in the adjacent funsd.json before interpreting word-role clustering.
Frozen evaluation (repeat for EACH dataset)
python run_document_benchmark.py freeze \
--npz data/funsd.npz --lock protocols/funsd_lock.json
Review the protocol before any test-score release. It locks the source-file SHA-256 and train/validation/test sample identity. Scoring is explicitly gated:
python run_document_benchmark.py execute \
--npz data/funsd.npz --lock protocols/funsd_lock.json \
--output results/funsd --release-test
Run equivalent commands with the RVL-CDIP NPZ and a separate lock.
The seven local methods are masked concatenation, train-only PCA16, denoised uniform gating, denoised predicted gating, denoised shuffled scoring, denoised fixed-view prior and identity-anchored fusion. All methods use train-fitted K-means and held-out assignments. The runner also saves predictions.csv.gz, allowing independent recomputation of per-group ARI/NMI uncertainty. The model is a train-only MLP denoising reference; it is not an executed reproduction of the full jointly contrastive BARG-CC objective.
Limitations you must not misstate
- The implemented stress tests add feature-space dropout and noise, not real image blur, crop, bleed-through or OCR reruns.
- FUNSD word-level conditional ARI is calculated only for role-matched OCR words, although all OCR detections are clustered. It does not establish document-category performance or measure missed annotation words; always publish both detection-match fraction and annotation recall, including zero-match forms.
- The supplied package has no executed official-code recent neural comparators. Never insert external benchmark numbers without source/checkpoint/prediction provenance.
- Freezing a protocol file does not, by itself, demonstrate prospective preregistration.
- Repeated neural seeds on the same page/form test set are not independent dataset replicates.
- No dataset labels enter feature preparation or training; labels are used for K according to the published ontology and final scoring.
Reproducibility checks
python -m unittest -v test_document_protocol.py
latexmk -pdf -interaction=nonstopmode -halt-on-error main.tex
pdfinfo main.pdf | grep Pages
The LaTeX build is self-contained (TikZ included); it does not depend on cached figures from earlier versions.
Raw data are not redistributed. Dataset download, rights, authenticity and journal requirements must be verified by the submitting authors.
Published-method comparison identity contract
The script below exports the exact same held-out sample/seed/condition keys from the local masked baseline, without labels or model predictions:
python export_official_reference.py \\
--predictions results/funsd/predictions.csv.gz \\
--reference-out results/funsd/official_reference_ids.csv \\
--contract-out results/funsd/official_reference_contract.json
This is an unexecuted comparator interface, not an official-code result. Once a licensed official implementation has actually been run with the exact features, masks, train/test identities and training-label restrictions, verify its provenance and keyed predictions with:
python ../archived_evidence/reproducibility/verify_official_comparator.py \\
--manifest /path/to/verified_official_run.json \\
--results /path/to/official_predictions.csv \\
--reference results/funsd/official_reference_ids.csv
A provenance contract or GitHub URL alone does NOT constitute an executed published baseline. No comparator result has been added to this package.
Paired, form-grouped uncertainty (after a genuine FUNSD run)
python paired_group_bootstrap.py \
--predictions results/funsd/predictions.csv.gz \
--dataset funsd --condition M20_D50 --seed 11 \
--method-a denoised_gated --method-b masked \
--replicates 500 --output results/funsd/group_uncertainty_seed11.json
This resamples complete forms, not individual OCR words. The same command accepts
--dataset rvlcdip for page-level groups or deduplicated source-document IDs.
It is a conditional inference check for that fixed test split and does not
make repeated training seeds independent datasets.
New, distinct joint-contrastive reference implementation
run_joint_contrastive_reference.py adds jointly trained view encoders,
reliability-conditioned residual maps, learned alignment projections and all
three loss terms. Its source-locked predictions are compatible with local
baseline scoring, but it is not an original checkpoint replay or a result
on canonical RVL-CDIP/FUNSD. Use compare_joint_reference.py for source-matched,
group-level descriptive differences. Full instructions:
JOINT_REFERENCE_AND_DATA_GUIDE.md.