social-data-attribution-w2/src/data_attribution/evaluation/README.md

Evaluation Package Scope

OlmoBaseEval

I keep OlmoBaseEval support as a separate expansion track from the current SocialTDA paper benchmark set.

The code in this package may live with the shared evaluation infrastructure because it reuses OLMES export, query-manifest conversion, and TrackStar analysis paths. That does not make base_olmobaseeval result artifacts part of the SocialTDA paper outputs.

Current merge scope:

• Registry and conversion support for cloze-compatible OlmoBaseEval probes.
• Tests that prevent accidental export or aggregation of unintended probes.
• No tracked cluster manifests, rendered result pages, or paper-facing OlmoBaseEval result tables.

Result policy:

• Treat prior base_olmobaseeval PACE and WIP outputs as separate-analysis artifacts.
• Do not mix OlmoBaseEval results into SocialTDA paper result folders, report tables, figures, or claims unless a later release explicitly promotes that scope.
• Keep cluster-specific paths, run manifests, and large generated outputs out of the implementation PR unless they are reviewed as a separate results PR.