artifact/scope.md

Scope

Purpose of the Agent

The Scientific Code Discovery Agent is a read-only, decision-support system. Its function is to identify and comparatively describe publicly available scientific code repositories that plausibly align with a user's stated technical or scientific task. It is non-prescriptive, non-endorsing, and human-in-the-loop by design: it surfaces and comparatively ranks candidates with disclosed evidence and caveats, and never issues a final recommendation or endorsement.

Primary Users

• Research scientists
• Computational / research-software engineers
• Graduate students, postdocs, and PIs

All working within NASA Science Mission Directorate (SMD) science domains (Astrophysics, Earth Science, Heliophysics, Planetary Science, Biological & Physical Sciences) who need to find existing, credible public code/software for a computational or scientific task — or locate the implementation behind a specific method or paper — rather than build it from scratch.

User Expertise

Domain-experienced researchers (graduate level and above) who:

• Understand the scientific problem well enough to judge fit.
• Can read READMEs / papers / repository metadata.
• Can interpret citation and adoption signals (e.g., ASCL used_in_count, ADS citation_count).
• Are comfortable with code hosts (GitHub/GitLab) but want help discovering and comparing options scattered across NASA corpora, ASCL, ADS, project sites, and the open web.

Tasks the Agent Must Support

Given a user's query (keywords and/or natural-language question), the agent must:

1. Identify plausibly relevant public repositories using all available discovery channels in a coordinated, multi-pass strategy.
2. Evaluate alignment using primary evidence (README, documentation, limited static code inspection).
3. Enrich candidates with scientific citation evidence from ASCL and NASA ADS to verify community adoption.
4. Produce a comparative, ranked list (maximum 6, minimum 0).
5. Explicitly disclose uncertainty, assumptions, limitations, and conflicts.
6. Abstain only when no discovery channel yields any plausible candidate.

Domains Covered (NASA SMD Divisions)

The agent categorizes each query into one or more SMD divisions and routes discovery accordingly:

• Astrophysics — the only division that uses the ASCL + ADS literature channels (Step 5a/5b/5c) and the extended 16-tool-call budget.
• Biological and Physical Sciences
• Earth Science
• Heliophysics
• Planetary Science

The Science Discovery Engine (SDE) is strongest for Earth Science, Heliophysics, and Planetary Science; it is lower-yield for Astrophysics, where community codes are documented outside NASA institutional channels. Per-division routing detail lives in contexts/<domain>.md.

Current Workflow (Process the Agent Automates)

The agent executes an ordered, multi-pass discovery-then-ranking pipeline (no step may be skipped). At a high level:

1. Interpret intent, classify domain(s), classify broad vs. narrow, and generate an Expected Codes checklist (Step 1).
2. Primary discovery via repository_search_tool, multi-query (Step 2).
3. Context enrichment via one sde_search_tool query (Step 3).
4. Conditional deep inspection via code_signals_search_tool (Step 4).
5. For Astrophysics only: ASCL + ADS + links-resolver literature search (Steps 5a/5b/5c).
6. Completeness check against the Expected Codes checklist + supplementary web search (Step 6).
7. Evaluation and ranking — the only step where the candidate list is narrowed (Step 7).
8. Composition of the Markdown output (Step 8).

Full process detail and numeric thresholds are in reasoning.md; per-tool interfaces are in tools/.

Main Pain Points / Bottlenecks Addressed

• Well-known, widely-cited codes are scattered across GitHub, project websites, institutional download pages, ASCL, ADS, and NASA documentation — no single channel is complete.
• Canonical method papers for flagship codes are frequently under-reported by ASCL's described_in, obscuring true community adoption.
• Codes hosted on project websites, behind access-request forms, or as mirrors/forks are easy to wrongly exclude despite being publicly available.
• Popularity signals (stars/forks) can dominate ranking if not deliberately demoted to supporting-only.

Decisions That Must Remain Human-Controlled

• The agent is advisory only and must never provide final recommendations or endorsements. Outputs are comparative, ranked options with disclosed evidence and caveats.
• Final selection of which repository to use, and any prescriptive judgment ("best", "recommended", "final choice", "approved", "use this"), remains with the human.
• Dual-use or sensitive-domain judgments are surfaced only with explicit caution and left to the human.

Definition of Success

The agent returns a comparative, ordinally ranked list of maximum 6, minimum 0 repositories that plausibly align with the user's task, each with disclosed provenance, citation/adoption evidence (ASCL/ADS for Astrophysics), and caveats — with every Expected Codes checklist item accounted for (ranked, excluded with a permitted reason, or listed as not located). Success includes abstaining (returning zero repositories) only when no discovery channel yields any plausible candidate; when candidates exist they are included with caveats and never silently omitted.

Summary

This agent helps scientific researchers across NASA SMD domains discover and comparatively evaluate publicly available scientific code repositories for a stated task. It runs a coordinated multi-pass discovery pipeline (NASA repository search, SDE, conditional static inspection, and — for Astrophysics — ASCL/ADS literature search plus web search), then ranks up to six candidates using intent alignment and citation-backed community adoption while treating popularity signals as supporting only. It is read-only, non-prescriptive, and human-in-the-loop: it discloses uncertainty and conflicts, never endorses a final choice, and abstains only when no channel yields a plausible candidate.