IMPLEMENTATION_PLAN.md

# GCMD Science Keyword Classifier MVP Implementation Plan

## 1. MVP Objective

Build the smallest tested MVP that demonstrates constrained, progressive GCMD Science Keyword classification from article title and abstract:

```text
Topic -> Term -> Variable_Level_1 -> Variable_Level_2 -> Variable_Level_3
```

The MVP must prove that selecting only from application-supplied direct-child candidates produces structurally valid, scientifically defensible recommendations without forcing unsupported specificity. The fixed deterministic target is 100% vocabulary validity for accepted classifications.

## 2. Scope

The MVP includes:

- Recursive loading of `data/gcmd_hierarchy.json`.
- Canonical vocabulary records and hierarchy lookup structures.
- Article loading and validation from `data/articles.json`.
- DOI as the unique article identifier.
- Exact preservation of source article fields: `DOI`, `Title`, `Year`, and `Abstract`.
- Multi-label Topic routing.
- Dynamic Term routing within each selected Topic.
- Variable-level controlled descent using direct children only.
- Stopping at any supported UUID-bearing level.
- Multiple independent classifications per article.
- Structured model responses using application-supplied candidate IDs.
- Application-side construction of UUIDs, labels, hierarchy levels, and paths.
- Deterministic validation for every final classification.
- Redundancy removal.
- Valid no-classification results.
- Batch processing, incremental JSON persistence, basic caching, retries, and logging.
- Lightweight Gradio demonstration separated from core classification logic.
- Hugging Face compatibility through a future thin root-level `app.py`.

## 3. Non-Goals and Deferred Features

Deferred unless explicitly approved:

- Independent semantic validation as a separate model task.
- Production-grade human-review UI.
- Vector databases and embedding-based retrieval.
- External bibliographic lookup, full-text retrieval, CMR integration, and automated vocabulary synchronization.
- Advanced model-comparison infrastructure, confidence calibration, risk-based policies, distributed processing, and production authentication.
- Evaluation dashboards and production deployment infrastructure.

The MVP should include extension points for these features but should not implement them prematurely.

## 4. Current Repository Assessment

Current structure:

```text
.
├── AGENTS.md
├── IMPLEMENTATION_PLAN.md
├── PROJECT_SPEC.md
├── README.md
├── data/
│   ├── articles.json
│   └── gcmd_hierarchy.json
├── prototype/
│   └── app_hf_poc.py
├── schemas/
├── src/
└── tests/
```

Observations:

- `IMPLEMENTATION_PLAN.md` was empty before this plan.
- `pyproject.toml` is not present.
- `src/`, `schemas/`, and `tests/` are empty skeleton directories.
- `README.md` is brief and describes the broad project goal, not implementation details.
- `prototype/app_hf_poc.py` is a frozen proof-of-concept baseline and must not be modified.
- `data/gcmd_hierarchy.json` root is a non-UUID `Category` named `EARTH SCIENCE`.
- Current full-data smoke inspection observed 14 UUID-bearing Topics, 144 Terms, 1,368 `Variable_Level_1`, 1,456 `Variable_Level_2`, and 553 `Variable_Level_3` records, with no duplicate UUIDs observed. These are assertions about the current source file, not generic application invariants.
- Topic direct-child counts range from 4 to 25 Terms; Term direct-child counts range from 0 to 52; Variable child counts range up to 28 and 16. This supports direct-child prompting without retrieval for the initial MVP.
- Current full-data smoke inspection observed 468 articles in `data/articles.json`, with required keys present in sampled records and no duplicate DOIs observed. This count is a current-data assertion, not a generic application invariant.

### GCMD Vocabulary Scope and Expected Growth

This project classifies concepts only within the `EARTH SCIENCE` Category represented by `data/gcmd_hierarchy.json`.

All current Topic and Term nodes are UUID-bearing and assignable.

The vocabulary file is updated periodically as new concepts are added, typically by only a small number of terms per week. The loader and index must therefore discover all concepts dynamically at runtime and must not hard-code counts, names, UUIDs, or branches.

The current vocabulary size is representative of the expected operational scale. The MVP does not need to design for an order-of-magnitude increase in vocabulary size. Current record counts remain useful full-data smoke-test assertions, while the application logic must continue to work when ordinary incremental vocabulary updates change those counts.


## 5. Proof-of-Concept Reuse and Replacement Analysis

Reusable ideas from `prototype/app_hf_poc.py`:

- Hugging Face-compatible Gradio launch pattern.
- Simple text-entry interface for title and abstract.
- Multi-label Topic routing.
- Structured Pydantic model responses.
- Model-backed classification using an OpenAI-compatible chat model.
- Basic validation of model selections against application-provided options.
- Branch fan-out concept for multiple selected Topics.

Conflicts with the MVP:

- The prototype loads `gcmd_hierarchy.json` from the repository root, while the MVP should use `data/gcmd_hierarchy.json`.
- It builds Topic-wide complete-path lists and asks the model to extract complete paths. The MVP must use progressive direct-child classification.
- It treats paths as strings selected by the model, while the MVP must construct authoritative records from a vocabulary index.
- It builds a static LangGraph node for every Topic, while the MVP should use reusable services and dynamic branches.
- It hard-codes `gpt-4o` in classification functions.
- It combines data loading, index creation, model interaction, graph orchestration, validation, and UI in one file.
- It uses a fallback pseudo-topic instead of a formal no-classification result.
- It does not preserve article records, batch process `articles.json`, cache results, save incrementally, or emit the formal structured JSON result.

Components to replace:

- Replace Topic-wide path extraction with direct-child routers.
- Replace string-path validation with UUID/candidate-ID validation against a canonical vocabulary index.
- Replace global mutable prototype state with injectable services.
- Replace in-file Gradio classification logic with `gcmd_classifier.ui` calling application services.
- Replace hard-coded model construction with a provider-neutral model interface and settings.

## 6. Key Architectural Decisions

- Use simple Python orchestration for MVP rather than LangGraph. The workflow is recursive and branch-based, but the required branching can be represented clearly with service calls, queues, and model fakes. LangGraph may be reconsidered after the MVP if observability or complex branch recovery needs justify it.
- Use candidate IDs generated by the application for prompts. Candidate IDs can be stable within a prompt, such as `c1`, `c2`, while authoritative UUIDs remain application-owned.
- The model must never generate final UUIDs, canonical paths, hierarchy levels, or labels.
- Include all direct children in prompts by default. Candidate limits should be configurable, but retrieval/pre-ranking is deferred unless a measured prompt limit is exceeded.
- Treat every UUID-bearing node as assignable, including internal nodes with children.
- Treat the known non-UUID root `Category` as routing/context only. Topic routing selects UUID-bearing Topic records supplied by the vocabulary index; the root `Category` is never selectable. Any other UUID-less hierarchy node is an error unless an approved vocabulary rule explicitly permits it.
- Use Pydantic for internal schemas and structured model responses.
- Keep independent semantic validation out of the MVP implementation unless separately approved; retain schema fields or metadata hooks where needed for later compatibility.
- Introduce root-level `app.py` only when the Gradio interface exists and is tested. It should import from `src/gcmd_classifier/ui/` and launch the app.
- Keep source JSON files read-only. Generated outputs, caches, schemas, and logs must be stored separately.

## 7. Proposed Repository and Package Structure

```text
.
├── .env.example
├── AGENTS.md
├── IMPLEMENTATION_PLAN.md
├── MVP_PROGRESS.md
├── PROJECT_SPEC.md
├── README.md
├── app.py
├── data/
│   ├── articles.json
│   └── gcmd_hierarchy.json
├── prototype/
│   └── app_hf_poc.py
├── pyproject.toml
├── schemas/
│   ├── classification_result.schema.json
│   └── run_summary.schema.json
├── src/
│   └── gcmd_classifier/
│       ├── __init__.py
│       ├── config.py
│       ├── errors.py
│       ├── logging_config.py
│       ├── models.py
│       ├── articles/
│       │   ├── __init__.py
│       │   └── loader.py
│       ├── classification/
│       │   ├── __init__.py
│       │   ├── candidates.py
│       │   ├── redundancy.py
│       │   ├── traversal.py
│       │   ├── routers.py
│       │   └── validator.py
│       ├── llm/
│       │   ├── __init__.py
│       │   ├── base.py
│       │   ├── fake.py
│       │   ├── openai_provider.py
│       │   ├── prompts.py
│       │   └── schemas.py
│       ├── persistence/
│       │   ├── __init__.py
│       │   ├── cache.py
│       │   └── json_store.py
│       ├── pipeline/
│       │   ├── __init__.py
│       │   ├── batch.py
│       │   └── service.py
│       ├── ui/
│       │   ├── __init__.py
│       │   └── gradio_app.py
│       └── vocabulary/
│           ├── __init__.py
│           ├── index.py
│           └── loader.py
└── tests/
    ├── fixtures/
    │   ├── articles_valid.json
    │   ├── articles_invalid.json
    │   └── gcmd_hierarchy_small.json
    ├── test_articles.py
    ├── test_cache.py
    ├── test_output_schema.py
    ├── test_pipeline.py
    ├── test_redundancy.py
    ├── test_structured_responses.py
    ├── test_validation.py
    └── test_vocabulary.py
```

Justification:

- `vocabulary/` owns deterministic GCMD loading, canonical records, and lookup structures.
- `articles/` owns read-only article parsing and validation.
- `llm/` isolates providers, prompts, structured schemas, retries, and test fakes.
- `classification/` owns candidate construction, routing logic, traversal, redundancy removal, and deterministic validation.
- `pipeline/` coordinates article-level and batch workflows without UI concerns.
- `persistence/` owns incremental result storage and cache identity.
- `ui/` owns Gradio only and calls pipeline services.
- `schemas/` stores formal output schemas, separate from runtime Python models.

## 8. Component Responsibilities

- `config.py`: typed settings for paths, model provider/name, parameters, retries, timeouts, prompt versions, output paths, cache options, and candidate limits.
- `errors.py`: explicit exception classes for malformed vocabulary, malformed articles, duplicate UUIDs, duplicate DOIs, invalid candidate IDs, model failures, output validation, and persistence errors.
- `models.py`: shared Pydantic models for canonical records, articles, classification records, results, metadata, warnings, and errors.
- `vocabulary.loader`: read JSON, recursively traverse variable-depth hierarchy, calculate file hash, detect malformed nodes and duplicate UUIDs.
- `vocabulary.index`: UUID lookup, path lookup, parent lookup, direct-child lookup, Topic-to-Term lookup, Term-to-Variable lookup, descendant/ancestor checks.
- `articles.loader`: load articles, validate required fields, preserve source values exactly, detect duplicate DOIs.
- `llm.base`: provider-neutral interface returning typed responses plus usage/cost metadata.
- `llm.openai_provider`: optional OpenAI implementation behind the interface, installed through an optional dependency group rather than required by the deterministic core.
- `llm.fake`: deterministic fake responses for tests.
- `llm.schemas`: Pydantic structured response models for Topic, Term, and Variable decisions.
- `llm.prompts`: prompt builders that delimit untrusted article text and list candidates.
- `classification.candidates`: convert vocabulary records into prompt candidates with application-supplied candidate IDs.
- `classification.routers`: Topic and Term routers using the model interface.
- `classification.traversal`: controlled variable-level descent and branch stopping.
- `classification.validator`: deterministic vocabulary validation.
- `classification.redundancy`: duplicate and ancestor/descendant final-result removal.
- `pipeline.service`: classify a single article.
- `pipeline.batch`: process all valid articles, continue after failures, and call persistence incrementally.
- `persistence.cache`: compute cache keys from article fingerprint, vocabulary hash, model config, prompt versions, thresholds, and app config.
- `persistence.json_store`: atomic incremental JSON writes and result loading.
- `ui.gradio_app`: lightweight demonstration UI, installed through an optional dependency group rather than required by the deterministic core.

## 9. End-to-End Data Flow

```text
data/gcmd_hierarchy.json
        ↓
VocabularyLoader
        ↓
VocabularyIndex with file hash
        ↓
data/articles.json
        ↓
ArticleLoader and schema validation
        ↓
BatchRunner
        ↓
Per article:
  TopicRouter selects zero or more UUID-bearing Topic candidate IDs from the vocabulary index
        ↓
  For each selected Topic:
    TermRouter selects Terms or stops at Topic
        ↓
  For each selected Term/Variable parent:
    Variable traversal evaluates direct children only
        ↓
  Branch stops at deepest supported UUID-bearing node
        ↓
  Application constructs authoritative classification records
        ↓
  Redundancy removal
        ↓
  Deterministic validation
        ↓
  Article result with classifications or no-classification outcome
        ↓
  Incremental JSON persistence and cache update
```

## 10. Canonical Vocabulary Data Model

Canonical concept fields:

- `UUID`: authoritative UUID from the source hierarchy.
- `name`: authoritative label.
- `level`: one of `Topic`, `Term`, `Variable_Level_1`, `Variable_Level_2`, `Variable_Level_3`.
- `topic`: Topic name for this branch.
- `term`: Term name when applicable, otherwise `None`.
- `path_components`: ordered labels excluding the root `Category`.
- `canonical_path`: labels joined with ` > `.
- `parent_uuid`: immediate parent UUID when the parent is UUID-bearing, otherwise `None`.
- `parent_name`: immediate parent label when present.
- `child_uuids`: direct UUID-bearing child UUIDs.
- `has_children`: whether the source node has children.
- `assignable`: `True` for every UUID-bearing node.
- `definition`: optional source definition.
- `vocabulary_version`: file hash unless a future source version exists.

Index structures:

- `by_uuid: dict[str, ConceptRecord]`
- `by_path: dict[str, ConceptRecord]`
- `parent_by_uuid: dict[str, str | None]`
- `children_by_uuid: dict[str | None, list[str]]`
- `topic_uuids: list[str]`
- `terms_by_topic_uuid: dict[str, list[str]]`
- `variables_by_parent_uuid: dict[str, list[str]]`

Validation concerns:

- Duplicate UUIDs are errors.
- Missing `level` or `name` on any node is an error.
- Missing `UUID` is allowed only for the known non-assignable root `Category`, unless an approved vocabulary rule explicitly permits another case.
- Child levels must follow the GCMD order.
- Same concept names in different branches are allowed when their canonical paths differ.
- The same canonical path associated with different UUIDs is an error unless explicitly permitted by an approved vocabulary rule.

## 11. Article Data Model

Source article fields are preserved exactly:

- `DOI: str`
- `Title: str`
- `Year: int`
- `Abstract: str`

Generated article result fields:

- `processing_status`
- `classification_outcome`
- `classifications`
- `no_classification_reason`
- `review_status`
- `warnings`
- `errors`
- `processing_metadata`

Validation rules:

- Each record must be an object.
- Required fields must be present.
- `DOI`, `Title`, and `Abstract` must be non-empty strings.
- `Year` must be an integer.
- DOI values must be unique.
- Invalid records are reported explicitly and do not block remaining records.

## 12. Structured Model Response Schemas

Common candidate decision item:

- `candidate_id: str`
- `confidence: float | None`
- `evidence: str`
- `support_type: Literal["explicit", "inferred", "mixed"] | None`
- `reason: str | None`

Topic response:

- `selected: list[CandidateDecision]`
- `ambiguous_alternatives: list[str]`
- `no_selection_reason: str | None`

Term response:

- `selected: list[CandidateDecision]`
- `stop_at_parent: bool`
- `stop_reason: str | None`
- `ambiguous_alternatives: list[str]`

Variable response:

- `selected: list[CandidateDecision]`
- `stop_at_parent: bool`
- `stop_reason: str | None`
- `ambiguous_alternatives: list[str]`

Rules:

- The model selects only candidate IDs supplied in the prompt.
- Unknown candidate IDs cause explicit validation errors or retry.
- `stop_at_parent: true` requires an empty `selected` list.
- A non-empty `selected` list requires `stop_at_parent: false`.
- Contradictory structured responses fail schema validation or trigger retry according to the retry policy.
- Model confidence is optional, uncalibrated metadata. It cannot independently establish scientific support, justify descent, or cause acceptance without candidate validation, evidence, and final deterministic vocabulary validation.
- Free-form prose is not parsed for critical decisions.
- Prompts must delimit title and abstract and instruct the model to ignore commands inside article text.

## 13. Configuration Design

Configuration sources:

- Environment variables.
- Optional `.env` for local development.
- Typed defaults in `config.py`.

Settings:

- `GCMD_HIERARCHY_PATH`
- `ARTICLES_PATH`
- `OUTPUT_PATH`
- `CACHE_PATH`
- `MODEL_PROVIDER`
- `MODEL_NAME`
- `MODEL_TEMPERATURE`
- `MODEL_TIMEOUT_SECONDS`
- `MODEL_MAX_RETRIES`
- `PROMPT_VERSION_TOPIC`
- `PROMPT_VERSION_TERM`
- `PROMPT_VERSION_VARIABLE`
- `MAX_CANDIDATES_PER_PROMPT`
- `ENABLE_CACHE`
- `FORCE_REPROCESS`
- `APP_VERSION`
- `LOG_LEVEL`

Credentials:

- API keys are read from environment variables only.
- `.env.example` should document names but contain no secrets.

## 14. Error-Handling Strategy

Error principles:

- Do not silently convert failures into successful classifications.
- Failure of one article must not stop the batch.
- Retries must not duplicate accepted results.
- Errors must be structured with stage, code, message, and retry count when applicable.
- Stage-level candidate validation and final deterministic vocabulary validation are separate. Stage-level validation checks model response shape, known candidate IDs, and parent-scoped candidate membership before traversal continues. Final deterministic vocabulary validation checks the constructed final classification record against the canonical vocabulary index before it can be accepted.

Explicit error categories:

- Malformed hierarchy node.
- Duplicate UUID.
- Invalid hierarchy level or parent-child level transition.
- Malformed article record.
- Duplicate DOI.
- Structured response validation failure.
- Invalid model-selected candidate ID.
- Model timeout, rate limit, or provider error.
- Deterministic validation failure.
- Output schema failure.
- Persistence failure.

Processing status policy:

- `completed`: workflow completed, including valid no-classification.
- `partial`: some required branch failed after other branch work completed.
- `failed`: article could not be processed.
- `skipped`: explicit operator choice only, not cache reuse.
- Review-related enum values may remain schema-compatible for future use, but the initial MVP must not emit `pending_review` or `review_required` without an explicitly approved MVP rule.

## 15. Persistence and Caching Strategy

Outputs:

- `outputs/classification_results.json`
- `outputs/classification_diagnostics.json`
- `outputs/run_summary.json`
- `outputs/cache/`

Incremental persistence:

- Save an article result immediately after processing completes or fails.
- Use temporary file plus atomic replace for JSON writes; the temporary file and destination must be on the same filesystem for atomic replace guarantees.
- Consider an append-safe internal checkpoint representation for crash recovery before producing the final consolidated JSON output.
- Preserve source article order when practical.
- Retain errors and warnings per article.

Cache identity includes:

- DOI.
- Fingerprint of exact `DOI`, `Title`, `Year`, and `Abstract`.
- Vocabulary file hash.
- Model provider, model name, and parameters.
- Prompt versions.
- Relevant thresholds and candidate limits.
- Application version and configuration hash.

Cache behavior:

- A cache hit returns `processing_status: completed` with `cache_used: true`.
- Cache reuse is not `skipped`.
- Force reprocessing should be configurable.

## 16. Hugging Face and Gradio Integration Strategy

The Gradio interface should be implemented in `src/gcmd_classifier/ui/gradio_app.py` after core services work.

UI responsibilities:

- Accept title and abstract.
- Call the single-article pipeline service.
- Display final classifications, UUIDs, canonical paths, confidence signals, evidence, stopping reasons, validation status, and no-classification reasons.
- Avoid embedding classification logic.

Root `app.py` timing:

- Introduce root-level `app.py` only in the Gradio/Hugging Face milestone.
- It should be a thin launcher that imports `create_demo()` or equivalent from `gcmd_classifier.ui.gradio_app`.

Hugging Face compatibility:

- Use environment variables for credentials and configuration.
- Avoid local absolute paths.
- Keep generated outputs under configurable writable paths.
- Keep business logic under `src/gcmd_classifier/`.
- Preserve `prototype/app_hf_poc.py` as the baseline.

## 17. Milestone Sequence

1. Repository setup and tooling.
2. Hierarchy loader and vocabulary index.
3. Article loading and validation.
4. Shared models, output schema draft, and structured response schemas.
5. Model abstraction, prompts, fake model, and retries.
6. Topic routing.
7. Term routing.
8. Variable-level controlled descent.
9. Deterministic validation and redundancy removal.
10. Batch runner, incremental persistence, caching, and logging.
11. Gradio integration and Hugging Face launcher.
12. MVP smoke/evaluation harness.

## 18. Tasks Within Each Milestone

### Milestone 1: Repository Setup and Tooling

Objective: establish installable, testable project scaffolding without classifier behavior.

Included requirements: `AGENTS.md / Required Application Components`, `AGENTS.md / Code Quality`, `PROJECT_SPEC.md / Non-Functional Requirements / Maintainability and Modularity`.

Tasks:

- Create `pyproject.toml` with package metadata, Python version, deterministic-core dependencies, pytest configuration, and Ruff configuration.
- Include Pydantic, pytest, and Ruff in the initial tooling set.
- Define OpenAI and Gradio as optional dependency groups rather than deterministic-core requirements.
- Defer choosing mypy versus Pyright.
- Create package skeleton under `src/gcmd_classifier/`.
- Create `.env.example`.
- Create `.gitignore`.
- Create `MVP_PROGRESS.md`.
- Add minimal import tests.

Likely files: `pyproject.toml`, `.env.example`, `.gitignore`, `MVP_PROGRESS.md`, `src/gcmd_classifier/__init__.py`, `tests/test_package_import.py`.

Tests: `python -m pytest`; import smoke test for `gcmd_classifier`.

Acceptance criteria:

- Package imports from `src`.
- Test command succeeds.
- Ruff configuration is present.
- Type-checker choice remains deferred.
- No source JSON files modified.
- No prototype changes.

Excluded work: vocabulary parsing, model calls, UI.

### Milestone 2: Hierarchy Loader and Vocabulary Index

Objective: create deterministic canonical vocabulary records and lookups.

Included requirements: `PROJECT_SPEC.md / Dataset Specification / GCMD Keyword Dataset`, `PROJECT_SPEC.md / Functional Requirements / Load the GCMD hierarchy`, `Create canonical GCMD records`, `Create hierarchy lookup structures`, `AGENTS.md / Vocabulary Authority`, `Assignable Concepts`.

Tasks:

- Implement recursive traversal of variable-depth hierarchy.
- Calculate vocabulary file hash.
- Build canonical records for all UUID-bearing nodes.
- Preserve definitions.
- Build UUID, path, parent, child, Topic, and Term lookups.
- Validate duplicate UUIDs, required fields, and level transitions.
- Add small fixture hierarchy covering Topic through `Variable_Level_3`, internal assignable nodes, leaf nodes, missing definitions, and malformed records.

Likely files: `src/gcmd_classifier/models.py`, `src/gcmd_classifier/errors.py`, `src/gcmd_classifier/vocabulary/loader.py`, `src/gcmd_classifier/vocabulary/index.py`, `tests/fixtures/gcmd_hierarchy_small.json`, `tests/test_vocabulary.py`.

Tests:

- Recursive traversal.
- UUID preservation.
- Duplicate UUID detection.
- Canonical path construction.
- Parent and direct-child lookup.
- Topic-to-Term and Term-to-Variable lookup.
- Assignable internal nodes.
- Optional definitions.
- Malformed nodes and invalid level transitions.
- Source-file immutability check.

Acceptance criteria:

- Current full-data smoke test confirms `data/gcmd_hierarchy.json` loads into 3,535 UUID-bearing records.
- Root `Category` is not assignable.
- All UUID-bearing internal and leaf nodes are assignable.
- Tests pass without model access.
- For the current EARTH SCIENCE vocabulary, every Topic and Term must contain a UUID.

Excluded work: article loading and classification.

### Milestone 3: Article Loading and Validation

Objective: load and validate `articles.json` while preserving source fields exactly.

Included requirements: `PROJECT_SPEC.md / Dataset Specification / Article Dataset`, `PROJECT_SPEC.md / Functional Requirements / Load and validate article records`, `Preserve article source data`, `AGENTS.md / Article Data Rules`.

Tasks:

- Implement article model preserving exact source field names.
- Validate required keys, strict field types, non-empty DOI and Title strings, and DOI uniqueness. `Abstract` is required and must be a string, but may be empty.
- Report invalid records without modifying source records.
- Add valid and invalid article fixtures.

Likely files: `src/gcmd_classifier/articles/loader.py`, `tests/fixtures/articles_valid.json`, `tests/fixtures/articles_invalid.json`, `tests/test_articles.py`.

Tests:

- Valid article loading.
- Missing required fields.
- Empty DOI and Title. Empty `Abstract` is allowed and must be preserved exactly.
- Non-integer Year, including Boolean values.
- Duplicate DOI.
- Source values preserved exactly.

Acceptance criteria:

- Current full-data smoke test confirms `data/articles.json` contains 468 source records, 467 records with valid non-empty unique DOI values, exactly one invalid record at index 467 due to empty DOI, and 14 otherwise valid records with empty `Abstract` values.
- Invalid fixtures produce explicit errors.
- No normalization or rewriting occurs.

Excluded work: model prompts and batch classification.

### Milestone 4: Shared Models, Output Schema Draft, and Structured Response Schemas

Objective: define typed contracts for model responses and result output.

Included requirements: `PROJECT_SPEC.md / Functional Requirements / Generate structured model responses`, `Output Specification`, `Status Scope`, `Naming Conventions`, `AGENTS.md / Structured Model Responses`, `Output Rules`.

Tasks:

- Define Pydantic response schemas for Topic, Term, and Variable decisions.
- Define Pydantic result models for article results, classification records, validation result, warnings, errors, and processing metadata.
- Draft JSON Schemas for classification result and run summary.
- Validate status enumerations and confidence ranges.

Likely files: `src/gcmd_classifier/llm/schemas.py`, `schemas/classification_result.schema.json`, `schemas/run_summary.schema.json`, `tests/test_structured_responses.py`, `tests/test_output_schema.py`.

Tests:

- Valid structured responses.
- Unknown/malformed fields rejected according to policy.
- Confidence range validation.
- Status scope validation.
- Minimal no-classification output validation.

Acceptance criteria:

- Schemas can represent classified, pending-review, failed, partial, and not-classified results.
- Critical model decisions are structured, not prose-parsed.

Excluded work: live model provider and semantic validator implementation.

### Milestone 5: Model Abstraction, Prompts, Fake Model, and Retries

Objective: isolate model calls behind a testable interface.

Included requirements: `PROJECT_SPEC.md / System Architecture / Model Abstraction Layer`, `Structured Model Interface`, `AGENTS.md / Model Abstraction`, `Article Data Rules`.

Tasks:

- Define model client protocol/interface.
- Implement fake model for deterministic tests.
- Implement optional OpenAI provider behind interface.
- Implement prompt builders for Topic, Term, and Variable decisions.
- Delimit article text and include prompt-injection instruction.
- Add retry wrapper for temporary model failures.
- Return usage/cost metadata when available.

Likely files: `src/gcmd_classifier/llm/base.py`, `src/gcmd_classifier/llm/fake.py`, `src/gcmd_classifier/llm/openai_provider.py`, `src/gcmd_classifier/llm/prompts.py`, `src/gcmd_classifier/config.py`, `tests/test_structured_responses.py`.

Tests:

- Fake model returns typed responses.
- Invalid candidate IDs and contradictory stop/select responses are rejected.
- Prompt includes candidates and delimited article content.
- Retry behavior for temporary failures.
- No API calls in unit tests.

Acceptance criteria:

- Classification components can depend on an interface, not a provider.
- Model name and parameters are configurable.

Excluded work: end-to-end classification and unmarked live tests.

### Milestone 6: Topic Routing

Objective: select zero, one, or multiple valid Topics from the loaded vocabulary.

Included requirements: `PROJECT_SPEC.md / Functional Requirements / Identify relevant GCMD Topics`, `MVP Classification Workflow / Select Relevant Topics`, `AGENTS.md / Candidate Selection Rules`.

Tasks:

- Build Topic candidates from the vocabulary index.
- Route article title/abstract through model interface.
- Validate selected candidate IDs.
- Produce branch seed records with evidence/confidence.
- Support valid no-topic/no-classification outcome.

Likely files: `src/gcmd_classifier/classification/candidates.py`, `src/gcmd_classifier/classification/routers.py`, `tests/test_pipeline.py`.

Tests:

- Single Topic selection.
- Multi-Topic selection.
- No Topic selection.
- Invalid candidate ID retry/error.
- Topic UUID/name/path populated from index.

Acceptance criteria:

- Topic router never accepts non-vocabulary Topics.
- Topic routing can return no classification without failure.

Excluded work: Term routing and variable descent.

### Milestone 7: Term Routing

Objective: evaluate only direct Terms beneath each selected Topic.

Included requirements: `PROJECT_SPEC.md / Functional Requirements / Identify relevant Terms`, `MVP Classification Workflow / Select Relevant Terms Within Each Topic`, `AGENTS.md / Assignable Concepts`.

Tasks:

- Build direct Term candidates for selected Topic.
- Allow one or more Term selections.
- Allow stop at Topic when supported and no Term is adequately supported.
- Carry evidence and confidence per branch.
- Validate Topic-to-Term relationship.

Likely files: `src/gcmd_classifier/classification/routers.py`, `src/gcmd_classifier/classification/traversal.py`, `tests/test_pipeline.py`.

Tests:

- Term selected only from selected Topic.
- Stop at Topic.
- Multiple Terms within one Topic.
- Topic with no supported Term.
- Invalid Term candidate ID rejected.

Acceptance criteria:

- Term router never evaluates all Terms globally.
- Topic-level final classification is possible when Topic has UUID.

Excluded work: Variable descent.

### Milestone 8: Variable-Level Controlled Descent

Objective: classify through Variable levels using only direct children and stop at supported assignable parent.

Included requirements: `PROJECT_SPEC.md / Functional Requirements / Evaluate Variable-level concepts`, `Apply controlled hierarchical descent`, `Prioritize accuracy over specificity`, `Support multiple classifications`, `AGENTS.md / Core Classification Rule`.

Tasks:

- Implement reusable recursive/iterative variable descent.
- Evaluate direct children for Term or Variable parent.
- Support multiple independent child branches.
- Give every selected child its own independent branch with `branch_id` or equivalent provenance.
- Ensure stopping or failure in one branch does not stop sibling branches.
- Stop at Term or Variable parent when descent is unsupported.
- Stop naturally at leaf nodes.
- Track confidence/evidence by stage.

Likely files: `src/gcmd_classifier/classification/traversal.py`, `tests/test_pipeline.py`.

Tests:

- Stop at Term.
- Stop at `Variable_Level_1`.
- Stop at `Variable_Level_2`.
- Select `Variable_Level_3`.
- Multiple independent Variable branches.
- Sibling branch continuation after one branch stops or fails.
- Leaf-node handling.
- No forced leaf descent.

Acceptance criteria:

- No descendant is evaluated unless it is a direct child of the current parent.
- Branches return deepest supported selected UUID-bearing concept with branch provenance.

Excluded work: Redundancy cleanup and final deterministic validation.

### Milestone 9: Deterministic Validation and Redundancy Removal

Objective: ensure final results are structurally valid and non-redundant.

Included requirements: `PROJECT_SPEC.md / Functional Requirements / Perform deterministic vocabulary validation`, `Support multiple classifications`, `AGENTS.md / Deterministic Validation`, `Redundancy Rules`.

Tasks:

- Validate UUID existence, name, level, canonical path, parent-child chain, Topic membership, Term membership, and assignability.
- Remove duplicate UUIDs and duplicate paths.
- Remove redundant ancestors using UUID ancestry and branch provenance when a descendant in the same branch is retained.
- Preserve ancestor classifications only when branch provenance shows they represent an independent classification decision.
- Retain diagnostic information for rejected invalid candidates.

Likely files: `src/gcmd_classifier/classification/validator.py`, `src/gcmd_classifier/classification/redundancy.py`, `tests/test_validation.py`, `tests/test_redundancy.py`.

Tests:

- Valid candidate accepted.
- Nonexistent UUID rejected.
- UUID/name mismatch rejected.
- Wrong hierarchy level rejected.
- Invalid canonical path rejected.
- Invalid parent-child relationship rejected.
- Wrong Topic or Term membership rejected.
- Duplicate and ancestor-descendant redundancy removal using UUID ancestry and branch provenance.

Acceptance criteria:

- Accepted classifications all pass final deterministic vocabulary validation.
- Rejected invalid candidates do not appear in final `classifications`.

Excluded work: independent semantic validation.

### Milestone 10: Batch Runner, Incremental Persistence, Caching, and Logging

Objective: process article collections safely and preserve completed results.

Included requirements: `PROJECT_SPEC.md / Functional Requirements / Support batch processing`, `Save results incrementally`, `Support caching and controlled reprocessing`, `Record errors and diagnostics`, `Produce machine-readable output`, `AGENTS.md / Persistence and Caching`, `Logging`, `Error Handling`.

Tasks:

- Implement single-article pipeline service.
- Implement batch runner over valid article records.
- Save article-level results incrementally.
- Evaluate append-safe checkpoint storage for internal recovery before final JSON emission.
- Continue after article and branch failures.
- Implement cache key and cache reuse.
- Emit run summary and diagnostics.
- Add structured logging without secrets.

Likely files: `src/gcmd_classifier/pipeline/service.py`, `src/gcmd_classifier/pipeline/batch.py`, `src/gcmd_classifier/persistence/cache.py`, `src/gcmd_classifier/persistence/json_store.py`, `src/gcmd_classifier/logging_config.py`, `tests/test_cache.py`, `tests/test_pipeline.py`.

Tests:

- Batch continues after one article failure.
- Incremental or checkpointed result survives simulated later failure.
- Cache hit sets `cache_used: true` and remains `completed`.
- Cache key changes when article, vocabulary hash, model config, prompt version, or config changes.
- Output schema validation.
- No-classification result is completed with empty classifications.

Acceptance criteria:

- Representative batch can run with fake model and persist valid JSON.
- Source JSON files remain unchanged.

Excluded work: Gradio UI and live model smoke unless explicitly configured.

### Milestone 11: Gradio Integration and Hugging Face Launcher

Objective: provide lightweight demonstration UI without moving classification logic into UI.

Included requirements: `PROJECT_SPEC.md / MVP User Interface`, `AGENTS.md / Hugging Face Compatibility`, `Proof-of-Concept Application`.

Tasks:

- Implement `create_demo()` in UI package.
- Display selected classifications, paths, evidence, validation status, errors, and no-classification reason.
- Add root-level thin `app.py`.
- Confirm prototype remains unchanged.

Likely files: `src/gcmd_classifier/ui/gradio_app.py`, `app.py`, UI import test.

Tests:

- UI module imports.
- `create_demo()` constructs Gradio app.
- Root `app.py` imports launcher only.

Acceptance criteria:

- UI calls pipeline service.
- Business logic remains under `src/gcmd_classifier/`.
- Hugging Face launch path is clear.

Excluded work: production review UI.

### Milestone 12: MVP Smoke/Evaluation Harness

Objective: provide a small repeatable harness for fake-model and optional live-model checks.

Included requirements: `PROJECT_SPEC.md / MVP Evaluation`, `Evaluation Plan`, `MVP Evaluation Baseline`, `AGENTS.md / MVP Evaluation Baseline`.

Tasks:

- Add smoke command or script for a small subset.
- Add optional integration marker for live model checks.
- Record basic metrics: article count, classifications, validation failures, model calls, processing time.
- Preserve prototype baseline for future comparison.

Likely files: smoke tests and optional script only if justified during implementation.

Tests:

- Fake-model smoke run.
- Optional live integration test marked and skipped by default.

Acceptance criteria:

- Smoke run proves the end-to-end path without live API calls.
- No accepted result fails deterministic validation.

Excluded work: full expert-reviewed evaluation dashboard.

## 19. Dependencies Between Tasks

- Milestone 1 precedes all implementation.
- Milestone 2 is required before candidates, validation, routing, and cache vocabulary identity.
- Milestone 3 is required before article-level pipeline and batch processing.
- Milestone 4 is required before model abstraction and output validation.
- Milestone 5 is required before model-backed routers.
- Milestone 6 is required before Term routing.
- Milestone 7 is required before Variable descent.
- Milestone 8 is required before final candidate validation and redundancy cleanup.
- Milestone 9 is required before accepted output can be trusted.
- Milestone 10 is required before UI and smoke testing are meaningful.
- Milestone 11 depends on the single-article pipeline service.
- Milestone 12 depends on the batch runner and deterministic validation.

## 20. Acceptance Criteria for Every Milestone

Global acceptance criteria:

- Only the approved milestone scope is implemented.
- Automated tests for new deterministic behavior are added.
- `python -m pytest` passes, or failures are reported honestly.
- Source JSON files remain unchanged.
- `prototype/app_hf_poc.py` remains unchanged.
- `MVP_PROGRESS.md` is updated during implementation milestones.
- No live model API call is required for unit tests.
- Completion report covers files changed, requirements implemented, design decisions, tests, formatting/static checks, source JSON status, deviations, limitations, deferred work, and acceptance status.

Milestone-specific acceptance criteria are listed in section 18.

## 21. Unit-Test Strategy

Unit tests should cover deterministic behavior without live APIs:

- Recursive hierarchy traversal.
- Variable-depth branches.
- Topic, Term, and Variable-level records.
- UUID-bearing internal nodes as assignable.
- Leaf nodes.
- UUID preservation and duplicate UUID detection.
- Canonical path construction.
- Parent, direct-child, Topic-to-Term, and Term-to-Variable lookup.
- Optional definitions.
- Malformed hierarchy records.
- Article schema validation and DOI uniqueness.
- Structured response validation.
- Invalid candidate IDs.
- Deterministic vocabulary validation.
- Redundancy removal.
- No-classification behavior.
- Status handling.
- Caching behavior.
- Incremental persistence.
- Output schema validation.
- Source-file immutability.

## 22. Integration-Test Strategy

Integration tests should use model fakes by default:

- Load small fixture vocabulary and articles.
- Run Topic -> Term -> Variable descent with scripted fake responses.
- Validate output JSON schema.
- Simulate model failures and retries.
- Simulate invalid model candidate IDs.
- Simulate partial branch failure.
- Verify cache reuse.

Optional live tests:

- Mark with `pytest.mark.integration`.
- Skip unless credentials and explicit environment flag are present.
- Process one or two articles only.
- Assert structural validity, not scientific correctness.

## 23. Smoke-Test Strategy

Smoke tests:

- `python -m pytest`
- Import package and create settings.
- Load full `data/gcmd_hierarchy.json`.
- Load full `data/articles.json`.
- Run fake-model batch on two fixture articles.
- Optionally launch/import Gradio app after the UI milestone.

The first full-data smoke should occur after Milestone 3 for loaders and after Milestone 10 for the end-to-end fake pipeline.

## 24. Test Fixtures and Model Fakes

Fixtures:

- `gcmd_hierarchy_small.json`: root Category, two Topics, Terms, variable-depth branches through `Variable_Level_3`, an internal assignable node, a leaf node, definitions, and sibling branches.
- Malformed hierarchy fixtures created inline in tests: duplicate UUID, missing name, bad level transition, duplicate canonical path if applicable.
- `articles_valid.json`: at least three valid articles, including single-topic, multi-topic, and no-classification cases.
- `articles_invalid.json`: missing DOI, empty Title, non-integer Year, duplicate DOI.

Model fakes:

- Scripted fake selecting known candidate IDs by stage.
- Fake returning no selections.
- Fake returning invalid candidate IDs.
- Fake raising timeout once then succeeding.
- Fake exhausting retries.
- Fake returning malformed structured data.

## 25. Risks and Mitigations

- Risk: Topic omission blocks correct descendants. Mitigation: Topic prompts favor recall while requiring evidence; evaluate Topic recall separately.
- Risk: Model over-descends. Mitigation: explicit stopping option at every UUID-bearing level; confidence/evidence required; deterministic direct-child constraints.
- Risk: Candidate sets exceed prompt limits. Mitigation: configure candidate limit and log exceedance; defer retrieval until measured need.
- Risk: GCMD labels are ambiguous without definitions. Mitigation: preserve definitions and make optional prompt inclusion configurable.
- Risk: Model returns invalid candidate IDs. Mitigation: structured validation, retries, and explicit errors.
- Risk: Cache becomes stale. Mitigation: include article fingerprint, vocabulary hash, model parameters, prompt versions, thresholds, app version, and config hash.
- Risk: UI logic drifts from batch logic. Mitigation: UI calls the same pipeline service.
- Risk: Full project semantic validation conflicts with MVP deferral. Mitigation: include future-compatible fields/hooks but do not implement independent validator until approved.
- Risk: No Git repository means change tracking may rely on file inspection. Mitigation: report files modified carefully at each milestone.

## 26. Requirement-to-Task Traceability Matrix

| Specification requirement | MVP task | Milestone | Test or verification method | Status |
| ------------------------- | -------- | --------- | --------------------------- | ------ |
| `PROJECT_SPEC.md / Minimum Viable Product / MVP Goals / load and recursively parse gcmd_hierarchy.json` | Implement recursive hierarchy loader | 2 | `tests/test_vocabulary.py` full and fixture loads | planned |
| `PROJECT_SPEC.md / Functional Requirements / Create canonical GCMD records` | Build `ConceptRecord` and canonical paths | 2 | Canonical record and path tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Create hierarchy lookup structures` | Build UUID/path/parent/child/topic/term lookups | 2 | Lookup tests | planned |
| `PROJECT_SPEC.md / Dataset Specification / Assignable Concepts` | Mark UUID-bearing internal and leaf nodes assignable | 2 | Internal-node and leaf-node tests | planned |
| `PROJECT_SPEC.md / Dataset Specification / Vocabulary Authority and Integrity` | Detect malformed nodes and duplicate UUIDs | 2 | Error fixture tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Load and validate article records` | Implement article loader | 3 | Article validation tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Preserve article source data` | Preserve exact source fields | 3 | Round-trip source-value tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Identify relevant GCMD Topics` | Implement Topic router | 6 | Fake-model Topic routing tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Identify relevant Terms` | Implement Term router using direct children | 7 | Term routing and wrong-parent tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Evaluate Variable-level concepts` | Implement reusable variable descent | 8 | Variable-level traversal tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Apply controlled hierarchical descent` | Stop at parent when child support is absent | 8 | Stop-at-each-level tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Prioritize accuracy over specificity` | Require stop decisions and evidence at each level | 8 | Fake over-descent and stop tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Support multiple classifications` | Branch for multiple Topics/Terms/Variables | 6-8 | Multi-branch fake tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Generate structured model responses` | Define Pydantic response schemas | 4-5 | Structured response validation tests | planned |
| `PROJECT_SPEC.md / System Architecture / Model Abstraction Layer` | Implement provider-neutral model interface | 5 | Fake provider tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Perform deterministic vocabulary validation` | Implement deterministic validator | 9 | Positive/negative validation tests | planned |
| `PROJECT_SPEC.md / Classification and Validation Workflow / Remove Redundant Candidates` | Implement redundancy removal | 9 | Duplicate and ancestor-descendant tests | planned |
| `PROJECT_SPEC.md / Classification and Validation Workflow / Handle No-Classification Outcomes` | Return completed not-classified result | 6, 10 | No-classification output schema test | planned |
| `PROJECT_SPEC.md / Functional Requirements / Support batch processing` | Implement batch runner | 10 | Batch fake-model integration tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Save results incrementally` | Implement JSON store with per-article saves | 10 | Simulated interruption test | planned |
| `PROJECT_SPEC.md / Functional Requirements / Support caching and controlled reprocessing` | Implement cache identity and reuse | 10 | Cache hit/miss tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Record errors and diagnostics` | Structured warnings/errors and diagnostics | 10 | Failure-path tests | planned |
| `PROJECT_SPEC.md / Functional Requirements / Produce machine-readable output` | Emit JSON result and run summary schemas | 4, 10 | JSON Schema validation tests | planned |
| `PROJECT_SPEC.md / Non-Functional Requirements / Security and Configuration Management` | Use env config and redact secrets | 1, 5, 10 | Config tests and log review | planned |
| `PROJECT_SPEC.md / MVP User Interface` | Implement Gradio app calling services | 11 | UI import/create test | planned |
| `AGENTS.md / Hugging Face Compatibility` | Add thin root `app.py` launcher | 11 | Root launcher inspection/import test | planned |
| `AGENTS.md / Proof-of-Concept Application` | Preserve frozen prototype | all | File unchanged verification | planned |
| `PROJECT_SPEC.md / MVP Evaluation` | Add fake smoke/evaluation harness | 12 | Smoke command | planned |

## 27. Decisions Requiring Human Approval

- Approve this implementation plan before any implementation begins.
- Confirm whether Milestone 1 should add or update README setup instructions.
- Confirm preferred dependency set in `pyproject.toml`: deterministic core should include Pydantic, pytest, and Ruff; OpenAI and Gradio should be optional groups; mypy versus Pyright remains deferred.
- Confirm the initial model provider and environment variable names for live use.
- Confirm whether independent semantic validation remains deferred for MVP, as directed by AGENTS, despite full-project references in `PROJECT_SPEC.md`.
- Confirm whether root `app.py` should wait until Milestone 11.
- Confirm output directory names and whether generated outputs should be gitignored once a repository exists.

## 28. Recommended First Implementation Milestone

Recommended first milestone: Milestone 1, repository setup and tooling.

Exact files/modules to create:

- `pyproject.toml`
- `.env.example`
- `.gitignore`
- `MVP_PROGRESS.md`
- `src/gcmd_classifier/__init__.py`
- `tests/test_package_import.py`

Exact behavior:

- Make the project installable/importable from `src`.
- Define pytest and Ruff configuration.
- Include Pydantic as a deterministic-core dependency.
- Define OpenAI and Gradio as optional dependency groups.
- Document expected environment variables without secrets.
- Establish a progress log for milestone completion reports.

Exact tests to add:

- A minimal import test, `tests/test_package_import.py`.

Exact commands that should pass:

```bash
python -m pytest
```

Work that must not begin yet:

- Do not parse `gcmd_hierarchy.json`.
- Do not load `articles.json`.
- Do not create model prompts.
- Do not choose mypy versus Pyright.
- Do not implement routers or traversal.
- Do not create root `app.py`.
- Do not modify `prototype/app_hf_poc.py`.

## 29. MVP Definition of Done

The MVP is done when:

- Full hierarchy loading creates canonical records and lookups from `data/gcmd_hierarchy.json`.
- Articles load from `data/articles.json` with DOI uniqueness and source preservation.
- Topic, Term, and Variable classification use only supplied candidates and valid direct children.
- Classification can stop at Topic, Term, `Variable_Level_1`, `Variable_Level_2`, or `Variable_Level_3`.
- Multiple independent classifications per article are supported.
- No-classification is represented as completed processing with empty `classifications`; the initial MVP does not emit `pending_review` or `review_required` without an approved rule.
- Every accepted classification passes deterministic validation.
- Duplicate paths and redundant ancestor classifications are removed.
- Batch processing saves results incrementally and continues after failures.
- Cache reuse is correct and marked as `cache_used: true`.
- JSON output validates against project schemas.
- Unit and fake integration tests pass without live model access.
- Lightweight Gradio demo calls application services.
- Root `app.py` is a thin launcher.
- `data/gcmd_hierarchy.json`, `data/articles.json`, and `prototype/app_hf_poc.py` remain unchanged.

## Final Review Section

### Unresolved Assumptions

- The initial MVP will not implement a separate independent semantic validator; this follows `AGENTS.md / Deferred Features` over the broader full-project architecture.
- The root `Category` is context only and excluded from canonical assignable output because it has no UUID.
- Direct-child candidate sets are small enough for default prompts in the initial MVP based on current data inspection.
- A simple orchestrator is preferable to LangGraph for the MVP.
- `pyproject.toml` can be introduced in Milestone 1 because it is absent.
- The repository may not be initialized as Git in the current environment.

### Decisions Requiring Approval

- Approve or revise the milestone sequence.
- Approve deferring independent semantic validation.
- Approve avoiding LangGraph for the MVP implementation.
- Approve adding `pyproject.toml`, `.env.example`, `.gitignore`, `MVP_PROGRESS.md`, and import tests in the first milestone.
- Approve waiting to add root `app.py` until Gradio integration.

### Deviations From `PROJECT_SPEC.md` or `AGENTS.md`

- No deviations from `AGENTS.md`.
- The plan intentionally defers independent semantic validation from the full `PROJECT_SPEC.md` architecture because `AGENTS.md` explicitly lists it as deferred for the initial MVP. The implementation should keep future-compatible interfaces where practical.

### Recommended First Codex Implementation Prompt

```text
Implement Milestone 1 from IMPLEMENTATION_PLAN.md only.

Create the repository setup and tooling files needed for an installable, testable Python package:
- pyproject.toml
- .env.example
- .gitignore
- MVP_PROGRESS.md
- src/gcmd_classifier/__init__.py
- tests/test_package_import.py

Do not implement vocabulary loading, article loading, model integration, classification, persistence, UI, or root app.py.
Do not modify data/gcmd_hierarchy.json, data/articles.json, or prototype/app_hf_poc.py.
Run python -m pytest and update MVP_PROGRESS.md with the milestone result.
```

### Plan Approval Checklist

- [ ] The plan follows `AGENTS.md` source-of-truth precedence.
- [ ] The MVP uses progressive direct-child classification, not Topic-wide complete-path selection.
- [ ] The prototype remains frozen.
- [ ] The first milestone is small and deterministic, with OpenAI and Gradio kept as optional dependency groups.
- [ ] The proposed package structure keeps UI, persistence, model interaction, vocabulary logic, and classification logic separate.
- [ ] LangGraph is not required for the MVP unless later evidence justifies it.
- [ ] Independent semantic validation is deferred unless explicitly approved.
- [ ] The root `app.py` is introduced only as a thin launcher during the UI milestone.
- [ ] Tests do not require live model API calls by default.
- [ ] Source JSON files remain read-only.