docs/RESULTS.md

# Results Index

This page is the quick index to generated evaluation outputs.

## Community challenge eval

- Report (markdown): `docs/hf_hub_community_challenge_report.md`
- Report (json): `docs/hf_hub_community_challenge_report.json`
- Inputs: `scripts/hf_hub_community_challenges.txt`
- Generator: `scripts/score_hf_hub_community_challenges.py`

## Community coverage eval

- Report (markdown): `docs/hf_hub_community_coverage_report.md`
- Report (json): `docs/hf_hub_community_coverage_report.json`
- Inputs: `scripts/hf_hub_community_coverage_prompts.json`
- Generator: `scripts/score_hf_hub_community_coverage.py`

## Prompt/card A/B eval (community)

- Summary:
  - `docs/hf_hub_prompt_ab/prompt_ab_summary.md`
  - `docs/hf_hub_prompt_ab/prompt_ab_summary.json`
  - `docs/hf_hub_prompt_ab/prompt_ab_summary.csv`
- Visuals (if matplotlib available):
  - `docs/hf_hub_prompt_ab/prompt_ab_composite_<model>.png`
  - `docs/hf_hub_prompt_ab/prompt_ab_scatter_tokens_vs_challenge.png`
- Generator:
  - `scripts/eval_hf_hub_prompt_ab.py`

## Tool routing eval

- Batch summary:
  - `docs/tool_routing_eval/tool_routing_batch_summary.md`
  - `docs/tool_routing_eval/tool_routing_batch_summary.json`
  - `docs/tool_routing_eval/tool_routing_batch_summary.csv`
- Per-model reports: `docs/tool_routing_eval/tool_routing_*.md` (+ `.json`)
- Inputs:
  - `scripts/tool_routing_challenges.txt`
  - `scripts/tool_routing_expected.json`
- Generators:
  - `scripts/score_tool_routing_confusion.py`
  - `scripts/run_tool_routing_batch.py`

## Tool description A/B eval

- Summary:
  - `docs/tool_description_eval/tool_description_ab_summary.md`
  - `docs/tool_description_eval/tool_description_ab_summary.json`
  - `docs/tool_description_eval/tool_description_ab_summary.csv`
- Detailed/pairwise:
  - `docs/tool_description_eval/tool_description_ab_detailed.json`
  - `docs/tool_description_eval/tool_description_ab_pairwise.json`
  - `docs/tool_description_eval/tool_description_ab_pairwise.csv`
  - `docs/tool_description_eval/tool_description_ab_ranking.json`
- Visuals:
  - `docs/tool_description_eval/heat_first_call_ok.png`
  - `docs/tool_description_eval/heat_avg_score.png`
  - `docs/tool_description_eval/heat_avg_calls.png`
  - `docs/tool_description_eval/scatter_calls_vs_first_ok.png`
  - `docs/tool_description_eval/tool_description_interpretation.md`
- Inputs:
  - `scripts/hf_hub_community_challenges.txt`
  - `scripts/tool_description_variants.json`
- Generators:
  - `scripts/eval_tool_description_ab.py`
  - `scripts/plot_tool_description_eval.py`

---

## One-command regeneration

```bash
scripts/run_all_evals.sh
```

Optional environment overrides:

```bash
MODELS=gpt-oss,gpt-5-mini ROUTER_AGENT=hf_hub_community scripts/run_all_evals.sh
```