social-data-attribution-w2/src/data_attribution/evaluation/README.md

# Evaluation Package Scope

## OlmoBaseEval

I keep OlmoBaseEval support as a separate expansion track from the current
SocialTDA paper benchmark set.

The code in this package may live with the shared evaluation infrastructure
because it reuses OLMES export, query-manifest conversion, and TrackStar
analysis paths. That does not make `base_olmobaseeval` result artifacts part of
the SocialTDA paper outputs.

Current merge scope:

- Registry and conversion support for cloze-compatible OlmoBaseEval probes.
- Tests that prevent accidental export or aggregation of unintended probes.
- No tracked cluster manifests, rendered result pages, or paper-facing
  OlmoBaseEval result tables.

Result policy:

- Treat prior `base_olmobaseeval` PACE and WIP outputs as separate-analysis
  artifacts.
- Do not mix OlmoBaseEval results into SocialTDA paper result folders, report
  tables, figures, or claims unless a later release explicitly promotes that
  scope.
- Keep cluster-specific paths, run manifests, and large generated outputs out of
  the implementation PR unless they are reviewed as a separate results PR.