Orienter / docs /RELEASE_CHECKLIST.md
stereoid's picture
Add files using upload-large-folder tool
6d35aff verified
|
Raw
History Blame Contribute Delete
4.77 kB

Public-release checklist

Completed in this sanitized copy

  • Preserved the original experiment directory unchanged.
  • Created a separate sibling copy for release preparation.
  • Removed logs, caches, .nfs* files, editor state, generated API responses, predictions, plots, baseline checkpoints, training outputs, previous experiment results, and raw/full dataset payloads.
  • Removed broken absolute symlinks and private machine-operation scripts.
  • Retained evaluation GT subsets and RQ3 simulation source without historical result outputs.
  • Added an honest repository overview, pipeline map, asset inventory, third-party notice, secret-safe example environment file, and release-focused .gitignore.
  • Centralized default/paper model profiles and switched the default path to OpenRouter.
  • Replaced source-edited APE JSONL slicing with CLI range/shard arguments.
  • Added offline unit tests for reflection crops, APE output shaping, provider config, and evaluation conversion.
  • Added configurable run_vlm.py and run_ape.py entrypoints with explicit data, output, model, APE, range, shard, resume, and reflection settings.
  • Added isolated shard filenames, zero-detection progress tracking, and a fail-closed shard merger with overlap/coverage validation.
  • Rebased legacy method*.py path construction on the current checkout while preserving historical stage branches and comments.
  • Added machine-readable manuscript author and ORCID metadata in CITATION.cff.
  • Bundled the APE-L(_D) checkpoint needed by the main detector and recorded its checksum in docs/MODEL_MANIFEST.md.
  • Configured Hugging Face/Git LFS tracking for the bundled checkpoint.
  • Added public asset verification helpers without committing private data, historical caches, generated predictions, or compiled APE artifacts.
  • Added environment/build notes without pretending the CUDA/APE stack is a universal lockfile.
  • Ran one-image live smoke tests for the default OpenRouter VLM, APE-L(_D), and one PII.5–PII.7 advisor round without retaining generated artifacts.
  • Added fail-closed offline/read-only semantic evaluation controls.
  • Removed the evaluator's shared temporary-file race with per-process paths and cleanup-on-failure tests.
  • Kept prior RQ3 raw/final experiment runs outside the release copy.
  • Added deterministic question-manifest generation from mounted screenshots.
  • Added optional pinned app metadata cache support for reproducible VLM prompts without live Steam drift.
  • Added project-local agent guidance forbidding direct OpenRouter key retrieval or exposure.
  • Added the published PACMSE/FSE DOI, article metadata, BibTeX, and preferred paper citation.
  • Added explicit remote-provider screenshot privacy guidance and fail-closed path/cache validation.
  • Added automatic SHA-256 verification before the released PyTorch checkpoint is loaded.
  • Added a deterministic one-image control-path smoke that uses no API or model weights and retains no outputs.

Required before publication

  • Decide and add the repository license.
  • Resolve every UNKNOWN and license mismatch in docs/THIRD_PARTY.md; either pin and clear each retained vendor/asset or remove it from the chosen public scope.
  • Add the final public repository/contact/demo/model links; add the dataset link when the dataset is published.
  • Publish the dataset assets with versions, checksums, schemas, and licenses; preserve the bundled model manifest and attribution in the final Hugging Face revision.
  • Pin a validated paper-scale CUDA/APE environment from a fresh public clone; resolve conflicting or duplicate dependency pins after the final target GPU host is selected.
  • Add deterministic regression fixtures for the published paper tables without committing generated predictions or the 548 MB embedding cache.
  • Decide how to repair or disclose the dynamic-coverage sample protocol; new comparisons must use one fixed GT image set and report coverage.
  • Recover model/round provenance for the ablation artifacts and replace the non-model-controlled w/o Context comparison.
  • Recover the three LMM rounds and verify the averaging procedure claimed in the paper.
  • Reconcile the abstract's 103.1% RQ3 coverage increase with the stored K=60 gains, or state the exact split and time point used for that percentage.
  • Run a fresh-clone installation, inference smoke test, evaluation smoke test, and full selected-paper experiment.
  • Run final secret, large-file, license, and personal-path scans on the exact commit to be published.
  • Create a versioned tag/release with known limitations and asset revisions.