ounce100m-code / kernels /p6_cli_probe.py

Commit History

p6_cli_probe v3: build metric keys from the YAML's own metric/aggregation fields and rank candidate config files -- the first pass guessed metric_uri and picked a mmlu subject file
5f5b188
verified

Cion-lab commited on

p6_cli_probe: locate lm_eval via a child process -- a user-site dir created mid-run is not on the parent's sys.path
8ec129e
verified

Cion-lab commited on

p6_cli_probe: check run_benchmarks' flag list and PRIMARY_METRIC against the installed 0.4.13 CLI and its shipped YAMLs
48949db
verified

Cion-lab commited on