docs/SPACE.md

Workspace Guide ("What lives where")

This is the single orientation page for new contributors.

1) Production surface

Use these when you want real user-facing behavior:

• Community agent/tooling

  • Card: .fast-agent/tool-cards/hf_hub_community.md
  • Backend function tool: .fast-agent/tool-cards/hf_api_tool.py
  • Focus: Hub users/orgs/discussions/collections/activity API workflows

• Papers search agent/tooling

  • Card: .fast-agent/tool-cards/hf_paper_search.md
  • Backend function tool: .fast-agent/tool-cards/hf_papers_tool.py
  • Focus: /api/daily_papers filtering and retrieval

---

2) Eval inputs (challenge sets)

• scripts/hf_hub_community_challenges.txt
• scripts/hf_hub_community_coverage_prompts.json
• scripts/tool_routing_challenges.txt
• scripts/tool_routing_expected.json
• scripts/tool_description_variants.json

These are the canonical prompt sets/configs used for reproducible scoring.

---

3) Eval execution scripts

• scripts/score_hf_hub_community_challenges.py

  • Runs + scores the community challenge pack.

• scripts/score_hf_hub_community_coverage.py

  • Runs + scores endpoint-coverage prompts that avoid overlap with the core challenge pack.

• scripts/score_tool_routing_confusion.py

  • Scores tool-routing quality for a single model.

• scripts/run_tool_routing_batch.py

  • Runs routing eval across many models + creates aggregate summary.

• scripts/eval_tool_description_ab.py

  • A/B tests tool-description variants across models.

• scripts/eval_hf_hub_prompt_ab.py

  • A/B compares prompt/card variants using both challenge and coverage packs, with summary plots.

• scripts/plot_tool_description_eval.py

  • Generates plots from A/B summary CSV.

---

4) Eval outputs (results)

• Community challenge reports:

  • docs/hf_hub_community_challenge_report.md
  • docs/hf_hub_community_challenge_report.json

• Tool routing results:

  • docs/tool_routing_eval/

• Tool description A/B outputs:

  • docs/tool_description_eval/

---

5) Instructions / context docs

• docs/hf_hub_community_challenge_pack.md
• docs/tool_description_eval_setup.md
• docs/tool_description_eval/tool_description_interpretation.md
• bench.md

---

6) Suggested newcomer workflow

1. Read this file + top-level README.md.
2. Run one production query for each agent.
3. Run one scoring script (community or routing).
4. Inspect generated markdown report in docs/.
5. Only then edit tool cards or script logic.

---

7) Results at a glance

• docs/RESULTS.md is the index page for all generated reports and plots.