PROJECT_SPEC.md

# GCMD Science Keyword Classifier: Project Specification

* Specification Conventions
* Problem Statement
* Project Objectives
* Dataset Specification
* Functional Requirements
* Non-Functional Requirements
* Minimum Viable Product
* System Architecture
* Classification and Validation Workflow
* Evaluation Plan
* Output Specification

## Specification Conventions

The following conventions apply throughout this document:

* The source article fields `DOI`, `Title`, `Year`, and `Abstract` retain their original names and values.
* The authoritative GCMD concept identifier is represented as `UUID` when a complete GCMD concept record is serialized. Generated reference fields use snake_case, such as `parent_uuid` and `deepest_supported_uuid`.
* `processing_status` describes execution of the article-processing workflow and uses `completed`, `partial`, `failed`, or `skipped`.
* `classification_outcome` describes the article-level semantic outcome and uses `classified`, `not_classified`, or `pending_review` when processing completes.
* `final_status` describes an individual candidate decision and uses `accepted`, `reduced_to_ancestor`, `review_required`, or `rejected`.
* `review_status` describes human-review progress and uses `not_required`, `pending`, or `completed`.
* “Reduced to ancestor” means that the original proposal was replaced by the deepest UUID-bearing ancestor supported by the article; the retained concept is not necessarily the immediate parent.
* Formal JSON Schemas are normative for field names, types, required properties, and enumerations. JSON examples in this document are illustrative.

## Problem Statement

NASA Earth science publications must be described with standardized Global Change Master Directory (GCMD) Science Keywords to support consistent metadata, search, discovery, interoperability, and analysis. Assigning these keywords manually requires subject-matter expertise, detailed knowledge of the GCMD hierarchy, and substantial review effort. This process becomes increasingly difficult to scale as the volume and disciplinary diversity of scientific publications grow.

Scientific titles and abstracts rarely use the exact terminology found in the GCMD controlled vocabulary. Authors may describe the same concept through synonyms, abbreviations, specialized terminology, indirect language, or relationships spanning multiple Earth science domains. A single article may therefore require several GCMD classifications from different branches of the hierarchy.

GCMD Science Keywords are organized hierarchically:

`Topic → Term → Variable_Level_1 → Variable_Level_2 → Variable_Level_3`

Each level may contain an authoritative UUID and may represent a valid final classification. A concept does not need to be a terminal leaf node to be assigned. The appropriate classification is the deepest hierarchy level that is adequately supported by the article title and abstract. When the evidence supports only a broad Topic or Term, the classifier should stop at that level rather than force selection of a more specific descendant.

An automated classifier must therefore do more than match words in an article to vocabulary labels. It must interpret the scientific meaning of the article, identify its primary subjects, distinguish substantive concepts from incidental mentions, recognize multidisciplinary content, and determine how far the available evidence justifies descending through each relevant GCMD branch.

Large language models can assist with semantic interpretation, but unrestricted generation introduces significant risks. A model may invent terms, modify official labels, construct invalid paths, omit relevant concepts, select an incorrect sibling, or assign a scientifically plausible concept that is not supported by the source text. It may also descend too far into the hierarchy, creating the appearance of precision while reducing classification accuracy. Supplying the entire vocabulary in every prompt may further increase cost, latency, and classification error.

The project must therefore design and implement a scalable, controlled, and auditable GCMD Science Keyword Classifier that:

1. accepts scientific article metadata consisting primarily of a title and abstract;
2. identifies all relevant GCMD Topics represented in the article;
3. evaluates valid Terms and Variable-level concepts within each selected branch;
4. makes constrained decisions using candidates drawn directly from the current GCMD hierarchy;
5. permits classification to stop at any UUID-bearing hierarchy level when further descent is not adequately supported;
6. assigns one or more valid GCMD concepts when an article addresses multiple scientific subjects;
7. prevents the acceptance of invented terms, altered labels, invalid UUIDs, or invalid hierarchical paths;
8. prioritizes well-supported broader classifications over uncertain, overly specific classifications;
9. provides evidence and confidence information for each recommendation;
10. distinguishes accepted classifications from uncertain cases requiring human review;
11. remains maintainable as the GCMD vocabulary and language-model capabilities change; and
12. can be evaluated systematically for precision, recall, hierarchical correctness, unsupported specificity, consistency, processing time, and cost.

The resulting system is intended to assist metadata specialists rather than replace expert judgment. Its purpose is to reduce the effort required to identify appropriate GCMD Science Keywords while preserving scientific accuracy, vocabulary integrity, transparency, and human oversight.

## Project Objectives

The primary objective of the GCMD Keyword Classifier project is to develop a reliable, scalable, and auditable prototype for recommending valid GCMD Science Keywords for scientific journal articles based on their titles and abstracts.

The system will use the GCMD hierarchy to make a sequence of constrained classification decisions rather than generate complete keyword paths from unrestricted model knowledge. It will support multi-topic classification, controlled hierarchical descent, deterministic vocabulary validation, independent semantic validation, structured output, and human review.

The project has the following objectives:

1. **Automate GCMD keyword recommendation**

   Develop a system that analyzes article titles and abstracts and recommends one or more relevant GCMD Science Keyword concepts.

2. **Preserve GCMD vocabulary integrity**

   Ensure that every accepted classification corresponds to an existing GCMD concept, authoritative UUID, hierarchy level, and valid canonical path in `gcmd_hierarchy.json`.

3. **Support hierarchical classification**

   Classify articles progressively through the GCMD hierarchy:

   `Topic → Term → Variable_Level_1 → Variable_Level_2 → Variable_Level_3`

   At each stage, the model should evaluate only valid child concepts beneath the selected parent.

4. **Permit classification at any supported level**

   Allow Topic, Term, and Variable-level nodes to serve as final classifications when they contain authoritative UUIDs and are supported by the article.

   The classifier should stop when the evidence does not justify further descent.

5. **Support multidisciplinary articles**

   Allow an article to receive multiple classifications across different Topics, Terms, or Variable branches when the title and abstract provide sufficient evidence for more than one scientific subject.

6. **Prioritize accuracy over maximum specificity**

   Select the deepest assignable GCMD concept that is adequately supported by the article.

   A broader Topic, Term, or Variable classification with strong support is preferable to a more specific descendant with weak or uncertain support.

7. **Reduce unsupported classifications**

   Minimize false-positive assignments, including classifications that are scientifically related to the article but not sufficiently supported by its title or abstract.

8. **Provide explainable recommendations**

   Return concise evidence for each recommendation so that a metadata specialist can understand why the concept was selected and why classification stopped at the chosen level.

9. **Apply deterministic vocabulary validation**

   Validate every proposed UUID, concept name, hierarchy level, canonical path, and parent–child relationship against `gcmd_hierarchy.json`.

10. **Apply independent semantic validation**

    Evaluate whether each structurally valid classification is supported by the article title and abstract.

    The semantic validator should be able to accept the proposed classification, reduce it to the deepest supported ancestor, reject it, or require human review.

11. **Support human oversight**

    Present classifications in a form that allows metadata specialists to inspect, accept, reject, or revise system recommendations.

    Original automated results and subsequent human decisions must remain distinguishable.

12. **Process article collections efficiently**

    Support batch classification of records in `articles.json`, save results incrementally, continue processing when individual articles fail, and avoid unnecessary reprocessing when valid cached results exist.

13. **Use a maintainable and configurable architecture**

    Separate data loading, hierarchy normalization, model interaction, routing, validation, confidence handling, persistence, logging, and result generation into modular components.

    Models, prompts, thresholds, and output settings should be configurable without requiring major changes to the complete workflow.

14. **Support model comparison and future upgrades**

    Provide a model abstraction layer that allows different current or future language models to be evaluated within the same classification workflow.

    Model selection should be based on measured project-specific performance rather than model age, size, or domain branding alone.

15. **Measure classification quality systematically**

    Evaluate the system using an expert-reviewed dataset and metrics that include:

    * Topic-, Term-, and Variable-level precision, recall, and F1;
    * exact-path accuracy;
    * hierarchical and ancestor-aware correctness;
    * unsupported descent rate;
    * premature stopping rate;
    * false-positive rate;
    * classification coverage;
    * semantic-validator confirmation, reduction, and rejection rates;
    * human-review rate;
    * consistency across repeated runs;
    * processing time; and
    * model cost.

16. **Produce structured and reusable output**

    Save the original article metadata together with selected GCMD UUIDs, hierarchy levels, canonical paths, confidence information, supporting evidence, validation decisions, review status, and processing metadata in a machine-readable JSON format.

The successful project outcome will be a prototype that produces structurally valid and scientifically defensible GCMD Science Keyword recommendations, with measurable emphasis on classification precision, appropriate hierarchy depth, explainability, vocabulary integrity, and expert review.

### Scope and Non-Goals

The initial project will classify journal articles using only the `Title` and `Abstract` fields provided in `articles.json`. The DOI and publication year will be preserved as metadata but will not serve as primary semantic classification inputs.

The initial implementation will:

* load and normalize the GCMD hierarchy from `gcmd_hierarchy.json`;
* classify articles through constrained hierarchical decisions;
* validate every accepted classification against the source hierarchy;
* perform article-level semantic validation;
* support batch processing and incremental result storage;
* generate structured JSON classifications, diagnostic output, and a human-review queue; and
* record the model, prompt, configuration, and vocabulary version used for each run.

The initial project will not:

* retrieve or analyze full-text articles;
* query external bibliographic or scientific information services;
* modify, extend, or automatically correct the GCMD controlled vocabulary;
* automatically retrieve newer GCMD vocabulary releases;
* publish classifications to CMR or another external metadata system;
* treat model-generated terms as additions to the vocabulary; or
* require a production-grade graphical review interface.

A basic review interface may be developed as a convenience, but the required initial review capability may be satisfied through structured review-queue output. Production deployment, external-system integration, automated vocabulary synchronization, and full operational user-interface development are outside the initial scope unless added as separate requirements.

## Dataset Specification

The project uses two JSON input files:

* `gcmd_hierarchy.json`, which contains the authoritative GCMD Science Keyword hierarchy; and
* `articles.json`, which contains the scientific journal articles to be classified.

The two datasets serve different purposes. `gcmd_hierarchy.json` defines the complete controlled set of valid classification targets, while `articles.json` provides the scientific text that the classifier must interpret.

### 1. GCMD Keyword Dataset: `gcmd_hierarchy.json`

The `gcmd_hierarchy.json` file contains the Global Change Master Directory Science Keywords represented as a nested hierarchical structure. The vocabulary contains approximately 3,500 UUID-bearing concepts distributed across multiple Earth science domains and hierarchy levels.

The hierarchy follows this general structure:

`Topic → Term → Variable_Level_1 → Variable_Level_2 → Variable_Level_3`

The hierarchy has variable depth. Some branches terminate at `Topic`, `Term`, `Variable_Level_1`, or `Variable_Level_2`, while others continue through `Variable_Level_3`. The application must therefore traverse the hierarchy recursively and must not assume that every branch contains the same number of levels.

#### 1.1 Source Node Structure

Each hierarchy node is represented as a JSON object and may contain the following fields:

| Field        | Type            | Description                                                                                                                                                    |
| ------------ | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `level`      | String          | Identifies the node’s position in the GCMD hierarchy. Expected values include `Topic`, `Term`, `Variable_Level_1`, `Variable_Level_2`, and `Variable_Level_3`. |
| `name`       | String          | The authoritative controlled-vocabulary label for the concept.                                                                                                 |
| `UUID`       | String          | The authoritative GCMD identifier for the concept.                                                                                                             |
| `definition` | String          | An optional textual definition of the concept.                                                                                                                 |
| `children`   | List of objects | The concepts located directly beneath the current node. This field may be absent or empty when the node has no descendants.                                    |

A representative portion of the hierarchy is shown below:

```json
{
  "level": "Topic",
  "name": "ATMOSPHERE",
  "UUID": "c47f6052-634e-40ef-a5ac-13f69f6f4c2a",
  "children": [
    {
      "level": "Term",
      "name": "ATMOSPHERIC CHEMISTRY",
      "UUID": "b9c56939-c624-467d-b196-e56a5b660334",
      "children": [
        {
          "level": "Variable_Level_1",
          "name": "CARBON AND HYDROCARBON COMPOUNDS",
          "UUID": "example-variable-level-1-uuid",
          "children": [
            {
              "level": "Variable_Level_2",
              "name": "ACETYLENE",
              "UUID": "04833f72-ac6d-40b0-b1ae-1f55eb25b5dd",
              "definition": "Acetylene appears as a colorless gas..."
            },
            {
              "level": "Variable_Level_2",
              "name": "ATMOSPHERIC CARBON DIOXIDE",
              "UUID": "example-variable-level-2-uuid",
              "children": [
                {
                  "level": "Variable_Level_3",
                  "name": "CARBON DIOXIDE PROFILES",
                  "UUID": "03ddc432-906d-4469-bb00-179c828dbea4",
                  "definition": "A graph showing the variation..."
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}
```

The UUID values labeled as examples are placeholders used only to illustrate the structure. The application must use the authoritative UUID values contained in the actual source file.

#### 1.2 Assignable Concepts

Every UUID-bearing node in `gcmd_hierarchy.json` is considered an assignable GCMD concept, regardless of whether the node has children.

Assignable concepts may occur at any of the following levels:

* `Topic`;
* `Term`;
* `Variable_Level_1`;
* `Variable_Level_2`; or
* `Variable_Level_3`.

A concept does not need to be a terminal leaf node to serve as a final classification.

The classifier may stop at any adequately supported hierarchy level. For example:

* it may return a Topic when the broad scientific domain is clear but no Term is sufficiently supported;
* it may return a Term when the article supports that subject area but not a particular Variable;
* it may return an intermediate Variable when further descent would introduce unsupported specificity; or
* it may return a deepest-level Variable when the article provides sufficiently specific evidence.

The classification objective is to identify the deepest assignable concept adequately supported by the article, not automatically the deepest concept available in the hierarchy.

#### 1.3 Canonical GCMD Records

The nested hierarchy will be transformed into canonical records for classification, lookup, deterministic validation, and output generation.

Each canonical record should contain at least:

* `UUID`;
* `name`;
* `level`;
* `topic`;
* `term`, when applicable;
* `path_components`;
* `canonical_path`;
* immediate parent UUID and name, when applicable;
* child UUIDs, when applicable;
* `has_children`;
* `assignable`;
* `definition`, when available; and
* vocabulary version or file-hash identifier.

A representative canonical record is:

```json
{
  "UUID": "03ddc432-906d-4469-bb00-179c828dbea4",
  "name": "CARBON DIOXIDE PROFILES",
  "level": "Variable_Level_3",
  "topic": "ATMOSPHERE",
  "term": "ATMOSPHERIC CHEMISTRY",
  "path_components": [
    "ATMOSPHERE",
    "ATMOSPHERIC CHEMISTRY",
    "CARBON AND HYDROCARBON COMPOUNDS",
    "ATMOSPHERIC CARBON DIOXIDE",
    "CARBON DIOXIDE PROFILES"
  ],
  "canonical_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY > CARBON AND HYDROCARBON COMPOUNDS > ATMOSPHERIC CARBON DIOXIDE > CARBON DIOXIDE PROFILES",
  "parent_uuid": "example-variable-level-2-uuid",
  "parent_name": "ATMOSPHERIC CARBON DIOXIDE",
  "child_uuids": [],
  "has_children": false,
  "assignable": true,
  "definition": "A graph showing the variation of a meteorological event with height."
}
```

The canonical representation will support:

* recursive hierarchy traversal;
* Topic-to-Term lookup;
* parent-to-child lookup;
* UUID lookup;
* canonical path construction;
* constrained candidate generation;
* deterministic validation;
* redundancy detection;
* human review; and
* structured result generation.

#### 1.4 Vocabulary Authority and Integrity

The source file `gcmd_hierarchy.json` is authoritative for:

* valid GCMD concepts;
* UUIDs;
* controlled-vocabulary labels;
* hierarchy levels;
* parent–child relationships; and
* canonical paths.

UUIDs will serve as the primary concept identifiers. Canonical paths will be retained for display, validation, comparison, and human review.

The system must not:

* alter authoritative GCMD labels;
* abbreviate or inaccurately reconstruct paths;
* invent concepts or UUIDs;
* create parent–child relationships that do not exist in the source hierarchy; or
* silently modify the controlled vocabulary.

Definitions will be preserved in the canonical records but will not be included automatically in every model prompt. They may be used selectively when:

* a label is ambiguous;
* sibling concepts are difficult to distinguish;
* a term has a specialized meaning; or
* controlled experiments show that definitions improve classification performance.

If the source file does not include an explicit vocabulary version, the system should calculate a reproducible file hash and use it as the vocabulary-version identifier.

### 2. Article Dataset: `articles.json`

The `articles.json` file contains the scientific journal articles to be classified. It is represented as a JSON list in which each item is an object corresponding to one article.

Each article contains the following required fields:

| Field      | Type    | Description                                                                                          |
| ---------- | ------- | ---------------------------------------------------------------------------------------------------- |
| `DOI`      | String  | The article’s Digital Object Identifier. It is required and serves as the unique article identifier. |
| `Title`    | String  | The full article title and one of the primary semantic inputs to classification.                     |
| `Year`     | Integer | The publication year.                                                                                |
| `Abstract` | String  | The article abstract and the primary source of detailed scientific context for classification.       |

A representative record is:

```json
[
  {
    "DOI": "10.xxxx/example.doi",
    "Title": "Example Earth Science Article",
    "Year": 2025,
    "Abstract": "This study investigates atmospheric carbon dioxide profiles..."
  }
]
```

The project assumes that:

* every article contains all four required fields;
* every article contains a DOI;
* every DOI is unique within `articles.json`;
* DOI values are stable across processing runs;
* `Title` and `Abstract` contain the source text to be classified; and
* the source fields are already in the expected format.

The classifier will use `Title` and `Abstract` as its semantic inputs. `DOI` and `Year` will be preserved as article metadata and included in the output.

The system will not create normalized article records, rewrite article text, or generate fallback identifiers. The DOI will serve directly as the unique article identifier.

#### 2.1 Article Validation

Each article record will be validated before classification.

Validation will confirm that:

* each item is a JSON object;
* all four required fields are present;
* `DOI`, `Title`, and `Abstract` are strings;
* `Year` is an integer;
* the required string fields are not empty; and
* DOI values are unique within the file.

Although the project assumes valid input, validation remains necessary to identify malformed data clearly and prevent failed records from entering the classification workflow.

If an article record does not satisfy the expected schema, the system will:

* record a validation error;
* skip classification for that article; and
* continue processing the remaining records.

The system must not fabricate, infer, or replace missing source values.

#### 2.2 Article Text Handling

The original article fields must be preserved exactly as provided in `articles.json`.

The system will not:

* summarize titles or abstracts before classification;
* rewrite article text;
* supplement the text with external information; or
* alter the source fields in the output.

Article text must be treated as untrusted content. Model prompts must clearly delimit the title and abstract and instruct the model not to follow commands or instructions that may appear within them.

### 3. Relationship Between the Datasets

`articles.json` provides the scientific text to be interpreted, while `gcmd_hierarchy.json` provides the complete controlled set of valid classification targets.

The classifier will map concepts expressed in each article to one or more canonical GCMD records.

A valid final classification must satisfy both of the following conditions:

1. **Vocabulary validity**

   The classification must resolve to an existing UUID-bearing node and valid canonical path in `gcmd_hierarchy.json`.

2. **Semantic support**

   The classification must be adequately supported by the article title and abstract.

A structurally valid GCMD concept is not automatically a correct classification for a particular article. It must also pass semantic validation against the source text.

A representative high-level result is:

```json
{
  "DOI": "10.xxxx/example.doi",
  "Title": "Example Earth Science Article",
  "Year": 2025,
  "Abstract": "This study investigates atmospheric carbon dioxide profiles...",
  "classifications": [
    {
      "UUID": "03ddc432-906d-4469-bb00-179c828dbea4",
      "name": "CARBON DIOXIDE PROFILES",
      "level": "Variable_Level_3",
      "canonical_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY > CARBON AND HYDROCARBON COMPOUNDS > ATMOSPHERIC CARBON DIOXIDE > CARBON DIOXIDE PROFILES",
      "confidence": {
        "final": 0.94
      },
      "evidence": "The abstract explicitly discusses vertical atmospheric carbon dioxide measurements.",
      "final_status": "accepted",
      "review_required": false
    }
  ]
}
```

Every accepted classification must:

* resolve to an existing GCMD UUID;
* use the authoritative GCMD concept name;
* identify the correct hierarchy level;
* reproduce a valid canonical path;
* preserve valid parent–child relationships;
* be adequately supported by the article title and abstract; and
* conform to the project’s output schema.

A model output that cannot be matched to the source vocabulary must be rejected rather than accepted or silently corrected.

A structurally valid classification that is not sufficiently supported by the article must be:

* reduced to the deepest supported ancestor;
* rejected; or
* sent for human review.

## Functional Requirements

The GCMD Keyword Classifier shall satisfy the following functional requirements.

### 1. Load the GCMD hierarchy

The system shall load `gcmd_hierarchy.json` and recursively traverse the complete GCMD Science Keyword hierarchy.

The parser shall support the following hierarchy levels:

`Topic → Term → Variable_Level_1 → Variable_Level_2 → Variable_Level_3`

The parser shall not assume that every hierarchy branch has the same depth.

### 2. Create canonical GCMD records

The system shall transform each UUID-bearing hierarchy node into a canonical GCMD record.

Each canonical record shall preserve:

* UUID;
* authoritative concept name;
* hierarchy level;
* complete path components;
* canonical path;
* Topic;
* Term, when applicable;
* immediate parent;
* direct children;
* definition, when available;
* whether the concept has children; and
* whether the concept is assignable.

The system shall treat UUID-bearing internal nodes and leaf nodes as assignable concepts.

### 3. Create hierarchy lookup structures

The system shall create lookup structures that support:

* UUID-to-concept lookup;
* canonical-path lookup;
* parent-to-child lookup;
* child-to-parent lookup;
* Topic-to-Term lookup;
* Term-to-Variable lookup; and
* deterministic validation of hierarchy relationships.

### 4. Load and validate article records

The system shall load article records from `articles.json`.

Each article shall contain:

* `DOI`;
* `Title`;
* `Year`; and
* `Abstract`.

The system shall validate that:

* each article is represented as a JSON object;
* all required fields are present;
* `DOI`, `Title`, and `Abstract` are non-empty strings;
* `Year` is an integer; and
* each DOI is unique within the input file.

The DOI shall serve as the unique article identifier.

### 5. Preserve article source data

The system shall preserve the original `DOI`, `Title`, `Year`, and `Abstract` values exactly as provided.

The system shall not normalize, rewrite, summarize, or supplement article text before classification.

### 6. Identify relevant GCMD Topics

The system shall analyze each article’s title and abstract and identify one or more relevant GCMD Topics.

Topic selection shall be multi-label so that an article may be routed to more than one Topic when it addresses multiple Earth science domains.

The Topic classifier shall select only from valid Topics contained in `gcmd_hierarchy.json`.

### 7. Identify relevant Terms

For each selected Topic, the system shall evaluate only the valid Terms located beneath that Topic.

The system may select one or more Terms or stop at the Topic level when no Term is adequately supported.

### 8. Evaluate Variable-level concepts

For each selected Term or Variable-level parent, the system shall evaluate only the valid direct children beneath that parent.

The system shall support classification through:

* `Variable_Level_1`;
* `Variable_Level_2`; and
* `Variable_Level_3`.

The system may select multiple independent child concepts when they are adequately supported by the article.

### 9. Apply controlled hierarchical descent

At each hierarchy level, the system shall determine whether the available article evidence supports descent to a more specific child.

The classifier shall return the deepest assignable concept adequately supported by the article.

The system shall not require classification to reach a leaf node.

### 10. Prioritize accuracy over specificity

The system shall prefer a broader, well-supported concept over a more specific concept supported by weak or uncertain evidence.

The system shall stop at a Topic, Term, or intermediate Variable level when further descent would introduce unsupported specificity.

### 11. Support multiple classifications

The system shall support multiple final GCMD classifications for one article when the article substantively addresses multiple scientific concepts.

Multiple classifications may occur:

* across different Topics;
* across different Terms within a Topic; or
* across independent Variable branches.

The system shall avoid returning a redundant ancestor as a separate classification when a supported descendant from the same branch is already returned.

### 12. Generate structured model responses

All language-model classification stages shall return structured responses conforming to predefined schemas.

Structured responses shall include, as applicable:

* selected candidate identifiers drawn from the supplied candidate set;
* confidence signals;
* supporting evidence;
* support type;
* reason for stopping;
* alternative candidate identifiers; and
* review recommendation.

The application, not the model, shall populate authoritative concept names, UUIDs, hierarchy levels, and canonical paths from the Canonical Vocabulary Index.

### 13. Perform deterministic vocabulary validation

The system shall validate every proposed classification against `gcmd_hierarchy.json`.

Deterministic validation shall confirm that:

* the UUID exists;
* the concept name matches the UUID;
* the hierarchy level is correct;
* the canonical path is valid;
* every parent–child relationship is valid;
* the concept occurs beneath the selected Topic and Term; and
* no path component has been invented or altered.

A classification that fails deterministic validation shall not be accepted.

### 14. Perform independent semantic validation

After deterministic validation, the system shall evaluate whether each proposed classification is supported by the article title and abstract.

The semantic validator shall assess an existing proposed path rather than generate an unrestricted replacement list.

The semantic validator shall be able to:

* accept the proposed concept;
* reduce it to the deepest supported ancestor;
* reject it;
* require human review; or
* determine that no defensible classification is available.

Independent semantic validation shall use a separate validation task, prompt, and structured-output schema. It does not require a different underlying model.

### 15. Apply a final decision policy

The system shall combine classification results, deterministic validation, semantic validation, hierarchy depth, evidence strength, and configured thresholds to determine a final status.

Supported candidate `final_status` values shall include:

* `accepted`;
* `reduced_to_ancestor`;
* `review_required`; and
* `rejected`.

When processing completes without a defensible classification, the article-level `classification_outcome` shall be `not_classified`. When only unresolved review-required candidates remain, it shall be `pending_review`. Neither value shall be represented as an individual classification status.

The system shall not treat raw model confidence as a calibrated probability unless calibration has been performed.

### 16. Provide evidence and confidence information

Each accepted, reduced, or review-required classification shall include:

* the selected UUID;
* the authoritative concept name;
* the hierarchy level;
* the canonical path;
* a confidence signal;
* concise supporting evidence;
* the semantic validation decision; and
* the reason classification stopped at the selected level.

### 17. Support human review

The system shall generate sufficient information for a metadata specialist to:

* accept a recommendation;
* reject a recommendation;
* select a broader supported ancestor;
* replace a recommendation with another valid GCMD concept;
* add another valid classification; or
* mark the article as not classifiable from the available text.

The system shall preserve the original automated recommendation separately from any human-reviewed result.

A production-grade graphical review interface is not required for the initial project. A structured human-review queue shall satisfy this requirement.

### 18. Support batch processing

The system shall process all valid article records in `articles.json` without requiring manual submission of individual articles.

The failure of one article shall not prevent processing of the remaining valid records.

### 19. Save results incrementally

The system shall save completed results incrementally so that previously processed articles are not lost if a later record or system component fails.

### 20. Support caching and controlled reprocessing

The system shall avoid reprocessing an article when a valid result already exists for the same:

* DOI;
* a deterministic fingerprint of `DOI`, `Title`, `Year`, and `Abstract`;
* GCMD vocabulary version;
* model;
* prompt version;
* threshold configuration; and
* application configuration.

The system shall allow explicit reprocessing when requested.

### 21. Record errors and diagnostics

The system shall record:

* article-schema validation failures;
* malformed hierarchy nodes;
* model or API failures;
* timeouts;
* retry attempts;
* structured-output parsing failures;
* invalid UUIDs;
* invalid paths;
* deterministic validation failures;
* semantic validation reductions or rejections;
* articles with no defensible classification; and
* processing time and token usage.

Errors shall not be silently converted into valid classifications.

### 22. Produce machine-readable output

The system shall save classification results in JSON format.

The output shall preserve the original article metadata and include:

* final classifications;
* rejected or reduced candidates when retained;
* validation results;
* review status;
* processing status;
* warnings and errors;
* model information;
* prompt versions;
* application version;
* vocabulary version; and
* processing metadata.

## Non-Functional Requirements

### 1. Classification Quality

The system shall prioritize scientifically correct and well-supported classifications over maximum hierarchy depth.

A broader Topic, Term, or Variable-level concept selected with strong evidence shall be preferred over a more specific descendant selected with weak or uncertain evidence.

The system shall minimize:

* unsupported classifications;
* false-positive classifications;
* incorrect sibling selections;
* unrelated branch selections; and
* unsupported hierarchical descent.

Classification quality shall be evaluated using an expert-reviewed dataset.

Relevant quality measures shall include:

* precision;
* recall;
* F1 score;
* exact-path accuracy;
* hierarchical correctness;
* ancestor-aware correctness;
* false-positive rate;
* unsupported descent rate;
* premature stopping rate;
* classification coverage; and
* agreement with expert-reviewed classifications.

Precision and hierarchical correctness shall receive greater importance than average classification depth.

### 2. Explainability

Every accepted, reduced, or review-required recommendation shall contain sufficient information for a metadata specialist to understand the decision.

The system shall provide:

* the selected UUID;
* the authoritative concept name;
* the hierarchy level;
* the canonical path;
* concise supporting evidence;
* a confidence signal;
* the semantic validation result;
* the reason for stopping at the selected level; and
* an indication of whether human review is required.

Explanations shall be grounded in the article title and abstract and shall not introduce unsupported external information.

### 3. Traceability and Auditability

Every classification shall be traceable to:

* the article DOI;
* the original title and abstract;
* the GCMD vocabulary version;
* the model and model parameters;
* the prompt versions;
* the threshold configuration;
* the deterministic validation result;
* the semantic validation result;
* the final automated decision; and
* any subsequent human-review decision.

The system shall distinguish clearly among:

* model-generated recommendations;
* deterministic validation results;
* semantic validation decisions;
* automated final decisions; and
* human-reviewed outcomes.

### 4. Vocabulary Integrity

The system shall treat `gcmd_hierarchy.json` as the authoritative source of valid GCMD concepts and hierarchy relationships.

The system shall not:

* alter authoritative labels;
* invent UUIDs;
* construct nonexistent paths;
* create invalid parent–child relationships; or
* accept a classification that cannot be resolved to the source vocabulary.

Every accepted classification shall pass deterministic vocabulary validation.

### 5. Reproducibility and Consistency

The system should produce stable and reasonably consistent results when processing the same article with the same:

* article data;
* GCMD vocabulary;
* model;
* model parameters;
* prompts;
* thresholds; and
* application configuration.

The system shall record sufficient configuration information to reproduce a classification run.

When the GCMD source file does not include a formal version identifier, the system shall use a reproducible file hash as the vocabulary-version identifier.

Where complete determinism is not possible, the system shall use model settings that reduce unnecessary variation.

### 6. Performance and Scalability

The system shall support batch processing without requiring the entire GCMD hierarchy to be included in every model request.

At each classification stage, the model should receive only the valid candidate concepts relevant to the current hierarchy branch.

The system should present all direct children of the selected parent when the candidate set fits within configured prompt limits.

Retrieval or pre-ranking should be introduced only when:

* a candidate set exceeds configured limits; or
* evaluation demonstrates that retrieval measurably improves accuracy, latency, or cost.

The system should use hierarchical filtering, caching, parallel branch execution, and controlled retries where they provide measurable benefit.

Performance evaluation shall include:

* processing time per article;
* number of model calls;
* token usage;
* model cost;
* cache-hit rate;
* retry rate; and
* failure rate.

### 7. Reliability and Fault Tolerance

The system shall continue processing valid articles when an individual article or classification branch fails.

Temporary failures should be retried according to a configurable retry policy.

Retries shall not create duplicate accepted results.

The system shall preserve completed work through incremental result storage.

A failure shall be represented explicitly and shall not be silently converted into:

* an accepted classification;
* an empty successful result; or
* fabricated source data.

### 8. Maintainability and Modularity

The system shall be organized into modular components with clear responsibilities, including:

* GCMD loading and recursive parsing;
* canonical record construction;
* hierarchy lookup creation;
* article loading and validation;
* Topic routing;
* Term routing;
* Variable-level classification;
* deterministic validation;
* semantic validation;
* final decision management;
* caching and persistence;
* logging;
* output generation; and
* human-review queue generation.

The system shall allow models, prompts, thresholds, retry policies, and output settings to be changed without requiring major modifications to unrelated components.

Updates to `gcmd_hierarchy.json` shall not require classification logic to be rewritten.

### 9. Testability

Core deterministic behavior shall be covered by automated tests.

Testing shall include:

* recursive traversal of variable-depth branches;
* Topic, Term, and Variable node handling;
* UUID preservation;
* canonical path construction;
* parent–child lookup;
* treatment of internal nodes as assignable concepts;
* article schema validation;
* DOI uniqueness validation;
* structured model-output parsing;
* deterministic vocabulary validation;
* redundancy removal;
* decision-status handling;
* caching behavior;
* output-schema validation; and
* error handling.

Model-based components shall be evaluated using a separate expert-reviewed dataset.

Articles used to design prompts or tune thresholds shall not be the only articles used for final evaluation.

### 10. Data Preservation

The system shall treat `gcmd_hierarchy.json` and `articles.json` as read-only source files.

The original article fields shall be preserved exactly as provided.

The system shall not create normalized article text, modify titles or abstracts, or supplement source text with external content.

Canonical GCMD records, lookup structures, caches, logs, classifications, diagnostics, and review results shall be stored separately from the source files.

### 11. Security and Configuration Management

API keys, tokens, and other credentials shall not be stored directly in source code or committed to version control.

Sensitive values shall be loaded from environment variables or another secure configuration mechanism.

Article titles and abstracts shall be treated as untrusted content. Model prompts shall delimit article text clearly and instruct the model not to follow instructions contained within the article content.

Logs shall not expose:

* credentials;
* authentication tokens;
* protected configuration values; or
* complete sensitive API request data.

### 12. Output Integrity

All output shall conform to a defined JSON schema.

Output validation shall confirm that:

* required fields are present;
* field types are correct;
* status values use approved enumerations;
* accepted classifications contain valid UUIDs and paths;
* confidence values use the defined representation;
* review fields are present when required; and
* original article metadata is preserved.

Output that fails schema validation shall not be treated as a completed production result.

### 13. Usability and Human Review

Classification and review output shall allow a metadata specialist to determine quickly:

* which concept was recommended;
* where it appears in the GCMD hierarchy;
* why it was recommended;
* what article evidence supports it;
* why the system stopped at that hierarchy level;
* whether another candidate was considered;
* whether semantic validation confirmed or reduced it; and
* whether human review is required.

The review output shall distinguish clearly among:

* accepted classifications;
* classifications reduced to a supported ancestor;
* uncertain recommendations;
* rejected or invalid recommendations;
* articles with no defensible classification; and
* completed human-review decisions.

The system shall not imply certainty when the article title and abstract do not support a reliable decision.

### 14. Model Independence and Upgradeability

Model-specific implementation details shall be isolated behind a common interface.

The system shall allow a model to be replaced or upgraded without redesigning:

* hierarchy traversal;
* candidate construction;
* deterministic validation;
* decision policies;
* persistence; or
* output generation.

A model shall be selected based on measured task-specific performance rather than model age, size, or domain branding.

### 15. Cost Awareness

The system shall record token usage, model calls, processing time, and estimated cost when that information is available.

Cost-reduction techniques shall not be adopted when they materially reduce classification quality.

Changes intended to reduce cost or latency shall be evaluated against:

* precision;
* hierarchical correctness;
* unsupported descent rate;
* review rate; and
* overall classification coverage.

## Minimum Viable Product

The Minimum Viable Product will provide an end-to-end implementation of the core GCMD Science Keyword classification workflow for a limited article collection.

Its purpose is to verify that hierarchical, constrained language-model classification can produce structurally valid and scientifically plausible GCMD recommendations before the complete production-oriented workflow is implemented.

The MVP will build on the existing proof of concept while replacing full-Topic path selection with progressive classification through the GCMD hierarchy.

### 1. MVP Goals

The MVP will demonstrate that the system can:

1. load and recursively parse `gcmd_hierarchy.json`;
2. treat UUID-bearing Topic, Term, and Variable-level nodes as assignable concepts;
3. load article records from `articles.json`;
4. classify an article into one or more GCMD Topics;
5. select Terms only from the children of each selected Topic;
6. descend through Variable levels using only valid direct children of the selected parent;
7. stop at any supported hierarchy level;
8. construct authoritative UUID-based canonical paths from the vocabulary index;
9. reject structurally invalid model selections;
10. provide concise supporting evidence and confidence signals;
11. process multiple articles; and
12. save structured JSON results.

The MVP is intended to validate the architectural direction rather than provide the complete operational system.

### 2. MVP Classification Workflow

The MVP workflow will be:

```text
Load GCMD hierarchy
        ↓
Create canonical vocabulary index
        ↓
Load and validate articles
        ↓
Select one or more Topics
        ↓
For each Topic, select one or more Terms or stop at Topic
        ↓
For each Term, evaluate direct Variable_Level_1 children
        ↓
Continue through Variable_Level_2 and Variable_Level_3 when supported
        ↓
Construct the selected concept from the vocabulary index
        ↓
Perform deterministic validation
        ↓
Save structured results
```

The language model will select only from candidate identifiers supplied by the application. It will not generate complete GCMD paths or UUIDs from memory.

### 3. MVP Components

The MVP will contain the following components:

#### 3.1 GCMD Hierarchy Loader

The hierarchy loader will:

* recursively traverse `gcmd_hierarchy.json`;
* preserve authoritative names, UUIDs, hierarchy levels, and definitions;
* construct canonical paths;
* record parent–child relationships; and
* identify UUID-bearing assignable concepts.

#### 3.2 Vocabulary Index

The vocabulary index will support:

* UUID lookup;
* direct-child lookup;
* parent lookup;
* Topic lookup;
* Term lookup;
* canonical-path construction; and
* deterministic classification validation.

A vector database or semantic retrieval index is not required for the MVP.

#### 3.3 Article Loader

The article loader will read `articles.json` and validate the required:

* `DOI`;
* `Title`;
* `Year`; and
* `Abstract`.

The DOI will serve as the unique article identifier.

#### 3.4 Multi-Topic Router

The Topic Router will evaluate all valid GCMD Topics in one structured model request.

It will return zero, one, or multiple selected Topic identifiers with concise evidence and confidence signals.

#### 3.5 Dynamic Term Router

For each selected Topic, the reusable Term Router will receive only the direct Term children of that Topic.

It may:

* select one or more Terms; or
* stop at the Topic when no Term is adequately supported.

#### 3.6 Hierarchical Variable Classifier

The reusable Variable Classifier will receive only the direct children of the current Term or Variable parent.

It may:

* select one or more supported children;
* stop at the current parent; or
* determine that no classification should continue in the branch.

The same component will be reused for all Variable levels.

#### 3.7 Deterministic Validator

The deterministic validator will confirm that:

* the selected UUID exists;
* the concept name and level match the UUID;
* the canonical path is valid; and
* all parent–child relationships are valid.

Only deterministically valid classifications will appear as accepted MVP results.

#### 3.8 Result Writer

The MVP will save article-level results in JSON.

Each classification will include:

* UUID;
* authoritative name;
* hierarchy level;
* canonical path;
* path components;
* confidence signals;
* supporting evidence;
* reason for stopping; and
* deterministic validation result.

### 4. MVP User Interface

A lightweight Gradio interface may be retained for interactive testing.

The interface should allow a user to enter:

* article title; and
* article abstract.

It should display:

* selected Topics;
* final GCMD classifications;
* UUIDs and canonical paths;
* confidence signals;
* supporting evidence;
* stopping reasons; and
* deterministic validation status.

The Gradio interface is a testing aid and is not the authoritative output mechanism. Structured JSON output remains the required MVP result.

### 5. MVP Scope

The MVP will include:

* one configurable language model;
* structured model responses;
* multi-label Topic routing;
* dynamic Term routing;
* controlled Variable-level descent;
* support for multiple branches;
* deterministic validation;
* batch processing;
* incremental JSON output;
* basic retry handling;
* basic caching; and
* a lightweight interactive demonstration.

### 6. Deferred Features

The following features may be deferred until after the MVP demonstrates acceptable classification behavior:

* independent semantic validation;
* confidence calibration;
* risk-based validation policies;
* formal human-review workflow;
* production-grade review interface;
* advanced semantic retrieval;
* vector indexing;
* alternative-model comparison;
* extensive diagnostics;
* full run-summary reporting;
* detailed cost optimization;
* parallel execution tuning;
* formal evaluation dashboards; and
* integration with external metadata systems.

Independent semantic validation may be included as an MVP extension if the initial classifier produces too many scientifically plausible but unsupported results.

### 7. MVP Success Criteria

The MVP will be considered successful when it demonstrates that:

1. all returned classifications resolve to valid UUID-bearing concepts in `gcmd_hierarchy.json`;
2. all returned canonical paths preserve valid parent–child relationships;
3. the system can stop at Topic, Term, or Variable levels;
4. the system can produce multiple classifications for multidisciplinary articles;
5. the system avoids unrestricted generation of GCMD paths;
6. the system processes a representative batch of articles without losing completed results;
7. results are saved in valid structured JSON; and
8. expert inspection indicates that the hierarchical workflow is more reliable than the original full-path proof of concept.

The required deterministic-validity target for MVP output is 100%.

Model-dependent precision, recall, and hierarchy-depth targets will be established after an initial expert-reviewed MVP evaluation.

### 8. MVP Evaluation

The MVP should be tested on a small expert-reviewed sample containing:

* single-topic articles;
* multidisciplinary articles;
* articles requiring only Topic-level classification;
* articles requiring Term-level classification;
* articles with deep Variable-level classifications;
* ambiguous articles; and
* articles for which no defensible classification exists.

The MVP should be compared with the original proof of concept using:

* valid-path rate;
* accepted-classification precision;
* Topic recall;
* exact-path accuracy;
* ancestor-aware correctness;
* unsupported descent rate;
* average classification depth;
* processing time; and
* model cost.

The main MVP decision will be whether constrained hierarchical descent produces more defensible classifications than selecting complete paths from all paths available beneath a Topic.

### 9. Transition Beyond the MVP

After MVP evaluation, the project will decide whether to:

* add independent semantic validation;
* adjust prompts and confidence thresholds;
* introduce retrieval for large sibling candidate sets;
* evaluate alternative models;
* expand the human-review workflow;
* implement the formal output schemas; and
* proceed toward the complete architecture described in this specification.

The MVP should reuse the same canonical vocabulary index, model abstraction, and structured response interfaces planned for the complete system so that successful components do not need to be rewritten.

## System Architecture

The GCMD Keyword Classifier will use a hierarchical, multi-stage architecture aligned with the structure of the GCMD Science Keyword vocabulary.

The architecture will combine:

* recursive GCMD hierarchy processing;
* multi-label Topic routing;
* dynamic Term routing;
* constrained Variable-level classification;
* deterministic vocabulary validation;
* independent semantic validation;
* configurable decision logic; and
* structured persistence.

The system will not create a separate language-model agent for every GCMD concept. Instead, it will use a small set of reusable components that receive valid candidate concepts dynamically from `gcmd_hierarchy.json`.

This design preserves the benefits of hierarchical specialization without creating hundreds or thousands of statically defined agents.

### 1. Architectural Principles

The architecture will follow these principles:

1. **Hierarchy-driven classification**

   Classification decisions will follow the GCMD structure:

   `Topic → Term → Variable_Level_1 → Variable_Level_2 → Variable_Level_3`

2. **Dynamic candidates**

   Topics, Terms, and Variable-level concepts will be loaded from the current GCMD hierarchy rather than hard-coded into the application.

3. **Constrained model decisions**

   Each model call will evaluate only valid concepts from the relevant hierarchy level or branch.

4. **Reusable components**

   The same routing and classification components will be reused across multiple Topics, Terms, and Variable branches.

5. **Controlled descent**

   The classifier may stop at any UUID-bearing Topic, Term, or Variable-level concept when the article does not support a more specific child.

6. **Separation of concerns**

   Model reasoning, hierarchy traversal, vocabulary validation, semantic validation, decision policy, persistence, and output generation will remain separate components.

7. **Vocabulary authority**

   `gcmd_hierarchy.json` will remain the authoritative source for UUIDs, concept names, hierarchy levels, and parent–child relationships.

### 2. High-Level Component Model

The principal architectural components will be:

```text
GCMD Hierarchy Loader
        ↓
Canonical Vocabulary Index
        ↓
Article Loader and Validator
        ↓
Multi-Topic Router
        ↓
Dynamic Term Router
        ↓
Hierarchical Variable Classifier
        ↓
Deterministic Vocabulary Validator
        ↓
Independent Semantic Validator
        ↓
Decision Manager
        ↓
Persistence and Output Services
```

Topic, Term, and Variable branches may fan out dynamically and execute concurrently when an article supports multiple scientific concepts.

The graph structure will be generated from classification results and hierarchy relationships rather than from a static node for every GCMD concept.

### 3. GCMD Hierarchy Loader

The GCMD Hierarchy Loader will read `gcmd_hierarchy.json` and recursively traverse the complete vocabulary.

Its responsibilities will include:

* reading all hierarchy levels;
* preserving UUIDs and authoritative labels;
* identifying parent–child relationships;
* recognizing variable-depth branches;
* identifying UUID-bearing assignable nodes;
* preserving definitions when available; and
* detecting malformed hierarchy records.

The loader will not modify the source vocabulary.

### 4. Canonical Vocabulary Index

The Canonical Vocabulary Index will transform the nested GCMD hierarchy into structures optimized for traversal, prompting, and validation.

It will provide access to:

* concepts by UUID;
* concepts by canonical path;
* direct children of a parent;
* the parent of a child;
* Terms beneath each Topic;
* Variable concepts beneath each Term or Variable parent;
* complete path components;
* hierarchy level;
* definitions when available; and
* whether a concept has descendants.

The index will support both internal nodes and leaf nodes as assignable concepts when they contain authoritative UUIDs.

The vocabulary index may be implemented using dictionaries, adjacency mappings, or equivalent in-memory structures. It should not require a semantic vector index unless later evaluation demonstrates a need for one.

### 5. Article Loader and Validator

The Article Loader will read `articles.json` and validate that each record contains:

* `DOI`;
* `Title`;
* `Year`; and
* `Abstract`.

The DOI will serve as the unique article identifier.

The component will preserve article data exactly as provided. It will not normalize, summarize, rewrite, or supplement article text.

The Article Loader will pass valid records to the classification workflow and record schema failures without preventing the remaining articles from being processed.

### 6. Multi-Topic Router

The Multi-Topic Router will identify all top-level GCMD Topics substantively represented in an article.

The Topic candidate set is small enough for the router to evaluate all valid Topics in a single constrained request.

The router will be multi-label because an article may address multiple Earth science domains.

Its architectural responsibilities are to:

* receive the article title and abstract;
* receive the complete valid Topic candidate set;
* return zero, one, or multiple Topic selections;
* return structured evidence and confidence signals; and
* initiate one downstream branch for each selected Topic.

Topic branches may be processed concurrently.

Because an omitted Topic prevents all descendants beneath it from being considered, Topic routing should be configured to favor recall while still excluding incidental or unsupported domains.

### 7. Dynamic Term Router

Each selected Topic branch will invoke the same reusable Term Router.

The router will receive only the Terms that are direct children of the selected Topic. It will not evaluate all Terms in the vocabulary.

Its responsibilities are to:

* compare valid sibling Terms;
* select one or more supported Terms;
* identify ambiguous Term candidates;
* provide structured evidence and confidence signals; and
* permit the branch to stop at the Topic when no Term is adequately supported.

The architecture will not create a permanent agent for each of the approximately 144 Terms.

Terms are dynamic candidates supplied to a reusable classification component.

### 8. Hierarchical Variable Classifier

The Hierarchical Variable Classifier will evaluate the direct children of a selected Term or Variable-level parent.

The same component will be reused at:

* `Variable_Level_1`;
* `Variable_Level_2`; and
* `Variable_Level_3`.

At each invocation, the component will receive:

* the article title and abstract;
* the selected parent concept;
* the valid direct children of that parent; and
* the current hierarchy context.

The component may:

* select one or more supported children;
* stop at the current parent;
* identify ambiguous sibling concepts; or
* indicate that no defensible classification exists within that branch.

The system will normally provide all direct children of the selected parent when the candidate set fits within configured prompt limits.

Retrieval or pre-ranking should be introduced only when:

* the direct child set exceeds configured limits; or
* evaluation demonstrates a measurable improvement in classification quality, latency, or cost.

The architecture will not create separate agents for the approximately 1,368 `Variable_Level_1`, 1,456 `Variable_Level_2`, or 553 `Variable_Level_3` concepts.

### 9. Controlled Hierarchical Descent

The classification architecture will support controlled descent through a selected GCMD branch.

A classification branch may terminate at:

* `Topic`;
* `Term`;
* `Variable_Level_1`;
* `Variable_Level_2`; or
* `Variable_Level_3`.

The classifier will descend only when the article provides sufficient evidence for a valid child concept.

Internal nodes with descendants remain valid final classifications when they are UUID-bearing and adequately supported.

This architecture supports the project’s central rule:

> The system should return the deepest assignable concept adequately supported by the article, not automatically the deepest concept available in the hierarchy.

The detailed stopping criteria and classification decisions are defined in the Classification and Validation Workflow section.

### 10. Treatment of GCMD Definitions

Definitions will be preserved in the Canonical Vocabulary Index but excluded from default model prompts.

The primary candidate representation will normally include:

* concept name;
* hierarchy level;
* UUID or internal candidate identifier; and
* canonical path or relevant path context.

Definitions may be added selectively when:

* a label is ambiguous;
* sibling concepts are difficult to distinguish;
* a concept has a specialized meaning; or
* controlled evaluation shows that definitions improve performance.

Definition use will therefore be configurable rather than embedded in the core architecture.

### 11. Model Abstraction Layer

All language-model components will access the selected model through a common interface.

The Model Abstraction Layer will separate model-specific implementation details from:

* hierarchy traversal;
* candidate construction;
* deterministic validation;
* semantic validation;
* decision management;
* persistence; and
* output generation.

The interface should support:

* configurable model names;
* configurable model parameters;
* structured input;
* structured output;
* schema validation;
* timeouts;
* retries;
* token usage reporting;
* cost reporting when available; and
* prompt-version tracking.

The initial implementation will use a configurable GPT model selected for strong structured-output reliability, instruction following, and constrained candidate comparison.

The architecture will not hard-code a permanent model name.

### 12. Alternative Model Evaluation

Alternative models may be evaluated through the same Model Abstraction Layer.

INDUS will not be a required production component. Previous testing showed that it was not sufficiently reliable for complete GCMD hierarchical sequence classification.

It may be evaluated as an experimental baseline for narrow Topic-, Term-, or Variable-level decisions, but it will be adopted only if project-specific evaluation demonstrates a measurable advantage.

Alternative model selection belongs to model evaluation rather than to the core routing architecture.

### 13. Structured Model Interface

Each model-based component will return structured output conforming to a predefined schema.

A routing response should identify:

* selected candidate UUIDs;
* selected candidate names;
* confidence signals;
* supporting evidence;
* ambiguous alternatives when applicable; and
* whether the branch should stop or continue.

A classification component should not generate uncontrolled GCMD path strings from memory.

Instead, it should select from candidate identifiers supplied by the application. The application will use those identifiers to retrieve the authoritative UUID, name, level, and canonical path from the vocabulary index.

This design reduces:

* invented labels;
* modified path strings;
* UUID mismatches;
* parsing ambiguity; and
* unnecessary dependence on exact model-generated formatting.

### 14. Deterministic Vocabulary Validator

The Deterministic Vocabulary Validator will operate independently of the language model.

Its purpose is to confirm that each proposed classification is structurally valid within `gcmd_hierarchy.json`.

The validator will use the Canonical Vocabulary Index to verify:

* UUID existence;
* name-to-UUID consistency;
* hierarchy level;
* canonical path;
* parent–child relationships; and
* membership beneath the selected Topic and Term.

The deterministic validator will reject structurally invalid candidates before semantic validation.

Its role is vocabulary and hierarchy integrity, not evaluation of whether the article supports the concept.

Detailed validation rules are defined in the Classification and Validation Workflow section.

### 15. Independent Semantic Validator

The Independent Semantic Validator will evaluate whether a structurally valid proposed classification is supported by the article title and abstract.

It will be implemented as a separate model task with:

* a separate prompt;
* a restricted validation objective; and
* a separate structured-output schema.

Independent validation refers to procedural independence. It does not require a different underlying model.

To reduce anchoring, the semantic validator should receive:

* the article title;
* the article abstract;
* the proposed UUID and canonical path;
* relevant hierarchy context; and
* sibling or parent concepts when needed.

It should not rely on the original classifier’s confidence as evidence that the recommendation is correct.

The semantic validator will not generate a new unrestricted list of keywords. It will evaluate the supplied candidate and identify the deepest supported concept within the proposed path.

### 16. Decision Manager

The Decision Manager will combine outputs from:

* hierarchical classification;
* deterministic vocabulary validation;
* semantic validation;
* configured confidence thresholds; and
* review policy.

Its purpose is to assign a final automated outcome to each candidate.

The Decision Manager will remain separate from model components so that decision thresholds and policies can be changed without rewriting prompts or hierarchy logic.

The detailed outcome rules and status values are defined in the Classification and Validation Workflow and Output Specification sections.

### 17. Persistence and Recovery Services

Persistence services will save article-level progress incrementally.

Their responsibilities will include:

* storing completed classifications;
* storing validation results;
* storing failures and diagnostics;
* supporting restart after interruption;
* preventing duplicate accepted results; and
* supporting cache reuse when relevant inputs and configuration have not changed.

Cache identity should account for:

* DOI;
* a deterministic fingerprint of `DOI`, `Title`, `Year`, and `Abstract`;
* vocabulary version;
* model and model parameters;
* prompt versions;
* decision thresholds; and
* application configuration.

Persistence details and final output fields are defined in the Output Specification.

### 18. Logging and Observability

The architecture will include centralized logging for:

* component execution;
* branch creation;
* selected candidates;
* model calls;
* structured-output failures;
* validation failures;
* retries;
* processing time;
* token usage;
* estimated cost when available; and
* final automated outcomes.

Logs must distinguish among:

* model recommendations;
* deterministic validation results;
* semantic validation decisions;
* automated final decisions; and
* human-reviewed outcomes.

Logging must not expose credentials or protected configuration values.

### 19. Human Review Integration

Human review will be integrated through structured review data rather than embedded directly into the classification components.

The architecture will generate a review queue containing the information required for a metadata specialist to inspect and revise uncertain results.

A production-grade graphical review interface is not required for the initial architecture. A lightweight interface may be added later without changing the classification, validation, or persistence components.

Human-review actions and fields are defined in the Classification and Validation Workflow and Output Specification sections.

### 20. Architectural Data Flow

The high-level architectural data flow is:

```text
gcmd_hierarchy.json
        ↓
GCMD Hierarchy Loader
        ↓
Canonical Vocabulary Index
        ├─────────────────────────────┐
        ↓                             │
articles.json                         │
        ↓                             │
Article Loader and Validator          │
        ↓                             │
Multi-Topic Router                    │
        ↓                             │
Dynamic Term Router                   │
        ↓                             │
Hierarchical Variable Classifier      │
        ↓                             │
Deterministic Vocabulary Validator ◀──┘
        ↓
Independent Semantic Validator
        ↓
Decision Manager
        ↓
Persistence, Diagnostics, and Review Queue
```

### 21. Component Boundaries

The architecture will maintain the following boundaries:

* The vocabulary components determine which concepts and paths are valid.
* The routing components identify relevant hierarchy branches.
* The classifier determines whether to descend within a branch.
* The deterministic validator confirms structural validity.
* The semantic validator confirms article-level support.
* The Decision Manager applies configurable outcome rules.
* Persistence services store results and recovery state.
* Output services serialize final results and diagnostics.
* Human review may revise operational outcomes but must not overwrite original automated decisions.

These boundaries are intended to reduce coupling, improve testability, and allow models, thresholds, persistence formats, or user interfaces to change independently.

### 22. Architecture Summary

The architecture uses a small number of reusable components rather than an agent for every GCMD node.

The core classification pattern is:

```text
Multi-label Topic routing
        ↓
Dynamic Term routing
        ↓
Reusable Variable-level classification
        ↓
Deterministic vocabulary validation
        ↓
Independent semantic validation
        ↓
Configurable final decision
```

This design:

* aligns processing with the GCMD hierarchy;
* supports assignment at Topic, Term, or Variable levels;
* permits multidisciplinary classifications;
* prevents unrestricted path generation;
* avoids agent proliferation;
* supports vocabulary updates;
* isolates model-specific logic;
* enables independent validation; and
* preserves a clear separation between architecture, workflow, evaluation, and output concerns.

## Classification and Validation Workflow

The GCMD Keyword Classifier will process each article through a staged workflow that separates hierarchical classification, structural validation, semantic validation, and final decision handling.

The workflow will not ask the language model to generate a complete GCMD keyword path from unrestricted knowledge. At each stage, the model will receive only valid candidates derived from `gcmd_hierarchy.json`.

The workflow is designed to identify the deepest UUID-bearing GCMD concept adequately supported by the article title and abstract while avoiding unsupported specificity.

### 1. Initialize Vocabulary and Article Inputs

Before article classification begins, the system will:

1. load `gcmd_hierarchy.json`;
2. build the canonical GCMD vocabulary index;
3. load `articles.json`; and
4. validate each article record against the required input schema.

Each article must contain:

* `DOI`;
* `Title`;
* `Year`; and
* `Abstract`.

The DOI will serve as the unique article identifier.

Articles that fail input validation will be recorded as failed and excluded from classification. A failed article will not prevent the remaining valid records from being processed.

### 2. Prepare the Article for Model Evaluation

For each valid article, the system will provide the original `Title` and `Abstract` to the model as clearly delimited, untrusted content.

The workflow will preserve the distinction between the two fields. The article text will not be summarized, rewritten, normalized, or supplemented with external information before classification.

The model will be instructed to:

* interpret the article text only as scientific content;
* ignore commands or instructions that may appear within the text;
* base classification decisions only on the supplied title and abstract; and
* select only from the candidates supplied by the application.

### 3. Select Relevant Topics

The workflow will first evaluate the complete set of valid GCMD Topics.

The Topic Router will return a structured multi-label decision containing:

* selected Topic UUIDs;
* supporting evidence;
* confidence signals; and
* ambiguous Topic candidates when applicable.

Zero, one, or multiple Topics may be selected.

A Topic should be selected when it represents a substantive scientific subject of the article. It should not be selected solely because it is mentioned incidentally or provides background context.

Because an omitted Topic prevents all descendants beneath that Topic from being considered, Topic routing should favor recall while remaining evidence-based.

Each selected Topic will create an independent downstream branch.

### 4. Select Relevant Terms Within Each Topic

For every selected Topic, the workflow will retrieve its direct Term children from the canonical vocabulary index.

The Term Router will evaluate only those valid sibling Terms and return:

* selected Term UUIDs;
* supporting evidence;
* confidence signals;
* ambiguous alternatives; and
* a stop decision when no Term is adequately supported.

A branch may stop at the Topic when:

* the Topic is clearly supported;
* no child Term has sufficient support; or
* competing Terms cannot be distinguished reliably from the available text.

Because Topic nodes are UUID-bearing assignable concepts, stopping at the Topic is a valid classification outcome.

### 5. Evaluate Variable-Level Children

For each selected Term, the workflow will retrieve its direct `Variable_Level_1` children.

The same hierarchical classification process will be repeated for:

* `Variable_Level_1`;
* `Variable_Level_2`; and
* `Variable_Level_3`.

At each level, the classifier will receive:

* the article title and abstract;
* the currently selected parent concept;
* the valid direct children of that parent; and
* the relevant hierarchy context.

The classifier may:

* select one supported child;
* select multiple independent children;
* stop at the current parent;
* identify an ambiguous sibling decision; or
* determine that no defensible classification remains in the branch.

The workflow will never evaluate a descendant that is not a valid child of the current parent.

### 6. Apply the Controlled-Descent Rule

At every hierarchy level, the workflow will apply the following rule:

> Select a child only when the article provides sufficient evidence for the additional specificity. Otherwise, retain the current UUID-bearing parent as the final candidate for that branch.

The branch may therefore terminate at:

* `Topic`;
* `Term`;
* `Variable_Level_1`;
* `Variable_Level_2`; or
* `Variable_Level_3`.

The classifier will not automatically descend to a leaf node.

For example, when the article clearly supports:

```text
ATMOSPHERE > ATMOSPHERIC CHEMISTRY
```

but does not adequately support:

```text
ATMOSPHERE > ATMOSPHERIC CHEMISTRY > CARBON AND HYDROCARBON COMPOUNDS
```

the workflow will retain `ATMOSPHERIC CHEMISTRY`.

The central decision criterion is evidential support, not hierarchy depth.

### 7. Construct Candidate Classifications

When a branch stops, the application will construct a candidate classification using authoritative data from the canonical vocabulary index.

The model should return the selected candidate identifier rather than reconstruct the complete GCMD path from memory.

The application will populate the candidate with:

* DOI;
* UUID;
* authoritative concept name;
* hierarchy level;
* path components;
* canonical path;
* classifier evidence;
* confidence signal;
* support type;
* reason for stopping; and
* ambiguous alternatives when applicable.

A candidate is provisional until it passes deterministic and semantic validation.

### 8. Remove Redundant Candidates

Before validation, the workflow will identify redundant candidates within the same hierarchy branch.

When both an ancestor and its supported descendant are proposed as separate final classifications, the workflow will normally retain only the deepest candidate.

For example, when the following concept is selected:

```text
ATMOSPHERE > ATMOSPHERIC CHEMISTRY > CARBON AND HYDROCARBON COMPOUNDS
```

the workflow will not also retain:

```text
ATMOSPHERE > ATMOSPHERIC CHEMISTRY
```

as a separate final classification from the same branch.

An ancestor may remain in diagnostic data, but it will not be counted as a separate final assignment unless it represents an independent classification decision.

### 9. Perform Deterministic Vocabulary Validation

Every candidate will first undergo deterministic validation against the canonical GCMD vocabulary index.

The validator will confirm that:

* the UUID exists;
* the concept name matches the UUID;
* the hierarchy level is correct;
* the canonical path is valid;
* every adjacent parent–child relationship is valid;
* the concept belongs beneath the selected Topic and Term; and
* no label, UUID, or path component was invented or altered.

A candidate that fails deterministic validation will be rejected immediately.

Invalid candidates may be retained in diagnostic output, but they will not proceed to semantic validation and will not be accepted as final classifications.

Deterministic validation establishes vocabulary and hierarchy integrity only. It does not establish that the article supports the concept.

### 10. Perform Independent Semantic Validation

Each structurally valid candidate will next undergo semantic validation against the original article title and abstract.

The semantic validator will assess one proposed candidate path at a time.

Its task is to determine whether:

* the proposed concept is supported;
* every level of the proposed path is supported;
* the classification depth is justified;
* a broader ancestor is better supported;
* the concept should be rejected; or
* human review is required.

The validator will not generate a new unrestricted keyword list.

To preserve procedural independence, it will use:

* a separate validation prompt;
* a separate structured-output schema; and
* a restricted validation objective.

The same underlying model may be used for classification and validation, but the validation task will remain logically separate.

The validator should not treat the classifier’s confidence as evidence that the candidate is correct.

### 11. Identify the Deepest Supported Concept

When the proposed path is only partially supported, the semantic validator will identify the deepest supported UUID-bearing ancestor.

The appropriate action may be:

* retain the proposed concept;
* reduce to the immediate parent;
* reduce to a higher ancestor;
* reject the entire branch; or
* require human review.

The phrase “reduce to the deepest supported ancestor” therefore refers to reduction to the deepest supported ancestor, which may not be the immediate parent.

A representative semantic-validation response is:

```json
{
  "decision": "reduce_to_ancestor",
  "deepest_supported_uuid": "b9c56939-c624-467d-b196-e56a5b660334",
  "deepest_supported_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY",
  "support_type": "explicit",
  "support_level": "strong",
  "unsupported_components": [
    "CARBON AND HYDROCARBON COMPOUNDS"
  ],
  "evidence": "The abstract supports atmospheric chemistry but does not identify carbon or hydrocarbon compounds.",
  "review_required": false
}
```

Any reduced concept must itself pass deterministic vocabulary validation before it becomes a final result.

### 12. Apply the Final Decision Policy

After semantic validation, the Decision Manager will assign a final automated status.

The supported candidate `final_status` values are:

* `accepted`;
* `reduced_to_ancestor`;
* `review_required`; and
* `rejected`.

If no candidate survives the workflow, the article-level `classification_outcome` will be `not_classified`. If the only remaining candidates require human review, it will be `pending_review`.

The Decision Manager may consider:

* the classifier’s confidence signal;
* the semantic validator’s decision;
* the strength and type of evidence;
* ambiguity among sibling concepts;
* hierarchy depth;
* agreement between classification and validation stages;
* deterministic validation status; and
* configured decision thresholds.

Raw model confidence will be treated as a decision signal rather than a calibrated probability unless formal calibration has been performed.

### 13. Determine Human-Review Requirements

A classification may be routed to human review when:

* sibling concepts remain difficult to distinguish;
* semantic support is plausible but incomplete;
* the classifier and semantic validator disagree;
* multiple branches produce conflicting results;
* evidence is primarily inferred rather than explicit;
* the proposed depth is near a decision threshold; or
* the system cannot determine a defensible final status automatically.

Human review is not required merely because a classification stops at Topic or Term level. A broad classification may be accepted automatically when it is strongly supported and further descent is not justified.

The workflow will generate structured review data rather than require a production-grade graphical interface.

### 14. Handle Multiple Valid Classifications

An article may receive multiple final classifications when it substantively addresses multiple independent scientific concepts.

The workflow may return:

* classifications from different Topics;
* multiple Terms within one Topic;
* multiple independent Variable branches; or
* classifications at different hierarchy depths.

Each candidate will be validated independently.

The workflow will avoid:

* duplicate UUIDs;
* redundant ancestor–descendant pairs;
* unsupported extra labels; and
* multiple records representing the same canonical concept.

### 15. Handle No-Classification Outcomes

The workflow will set `classification_outcome` to `not_classified` when:

* no Topic is adequately supported;
* all branches are rejected;
* the article text does not provide sufficient scientific information; or
* no UUID-bearing concept can be defended from the available evidence.

A no-classification outcome is a valid result, not a processing failure.

The system will record the reason no classification was produced.

### 16. Persist Results and Diagnostics

After final decisions are made, the workflow will save:

* accepted classifications;
* reduced classifications;
* review-required recommendations;
* rejected candidates when retained;
* no-classification outcomes;
* deterministic validation results;
* semantic validation results;
* processing errors; and
* run metadata.

Results will be saved incrementally so that completed work is preserved if a later article or model request fails.

The exact output schema is defined in the Output Specification section.

### 17. Continue After Recoverable Failures

When a recoverable failure occurs, such as a temporary model timeout or rate-limit response, the workflow may retry according to the configured retry policy.

When retries are exhausted, the workflow will:

* record the failed stage;
* preserve any completed branch results;
* assign the appropriate article processing status; and
* continue with the remaining articles.

Failures will not be converted silently into accepted classifications or empty successful results.

### 18. Workflow Summary

The runtime workflow for each article is:

```text
Validate article record
        ↓
Provide original title and abstract as delimited input
        ↓
Select zero, one, or multiple Topics
        ↓
For each Topic, select Terms or stop at Topic
        ↓
For each Term, evaluate Variable_Level_1 children
        ↓
Descend through Variable_Level_2 and Variable_Level_3 only when supported
        ↓
Construct candidates from authoritative vocabulary records
        ↓
Remove redundant ancestor–descendant candidates
        ↓
Perform deterministic vocabulary validation
        ↓
Perform independent semantic validation
        ↓
Accept, reduce to the deepest supported ancestor, review, reject, or return no classification
        ↓
Save results, diagnostics, and processing metadata
```

This workflow ensures that every accepted result is:

* an existing UUID-bearing GCMD concept;
* structurally valid within the source hierarchy;
* supported by the article title and abstract;
* no more specific than the available evidence permits; and
* traceable through classification and validation decisions.

## Evaluation Plan

The GCMD Keyword Classifier will be evaluated using an expert-reviewed set of scientific articles with validated GCMD Science Keyword assignments.

The evaluation will measure whether the system:

* identifies the correct GCMD branches;
* returns structurally valid UUID-bearing concepts;
* selects the deepest concept supported by the article;
* avoids unsupported hierarchical descent;
* handles multiple classifications correctly;
* improves classification quality through semantic validation;
* produces stable and traceable results; and
* provides sufficient benefit to justify its cost and complexity.

Evaluation will prioritize precision and hierarchical correctness over maximum classification depth.

### 1. Evaluation Dataset

A dedicated evaluation dataset will be created from a representative subset of `articles.json`.

The dataset should include:

* articles from all represented GCMD Topics;
* multiple Terms within each Topic;
* classifications at Topic, Term, and Variable levels;
* single-topic and multidisciplinary articles;
* common and infrequently used GCMD concepts;
* articles that use exact GCMD terminology;
* articles that describe concepts through synonyms or indirect language;
* technically detailed articles;
* ambiguous articles;
* articles for which only a broad classification is defensible; and
* articles that should receive no classification.

The evaluation data will be divided into:

* **Development set** — used for prompt design, workflow development, and qualitative error analysis;
* **Validation set** — used for model selection, threshold setting, and decision-policy tuning; and
* **Test set** — reserved for final performance reporting.

Articles used to design prompts or tune thresholds must not be included in the final test set.

If the expert-reviewed dataset is too small for a fixed split, repeated stratified sampling or cross-validation may be used. Article-level separation must be maintained so that the same DOI does not appear in both tuning and final evaluation data.

### 2. Gold-Standard Annotations

Each evaluation article will have one or more expert-reviewed reference classifications.

Each reference record should include:

* DOI;
* accepted GCMD UUID;
* authoritative concept name;
* hierarchy level;
* canonical path;
* deepest supported concept;
* supporting article evidence;
* acceptable broader ancestors, when applicable;
* alternative valid classifications, when multiple answers are defensible;
* concepts that should not be assigned; and
* reviewer notes for ambiguous cases.

The gold standard must distinguish among:

* the deepest supported classification;
* a correct but broader classification;
* an unsupported descendant;
* an incorrect sibling;
* an incorrect hierarchy branch;
* an unnecessary extra classification; and
* an appropriate no-classification outcome.

This distinction is necessary because exact-match scoring alone would treat a defensible broader classification and an unrelated classification as equivalent errors.

### 3. Expert Annotation Process

Whenever practical, each evaluation article should be reviewed independently by at least two subject-matter experts or experienced metadata specialists.

Reviewers should determine:

* which Topics are substantively represented;
* which Terms are supported;
* the deepest defensible UUID-bearing concept in each branch;
* whether multiple classifications are required;
* whether a broader classification is acceptable;
* whether the title and abstract contain sufficient evidence; and
* whether the article is too ambiguous for strict exact-match evaluation.

Disagreements should be resolved through adjudication or recorded as multiple acceptable reference outcomes.

Inter-reviewer agreement should be measured to estimate the inherent ambiguity of the task. Articles with unresolved disagreement may remain useful for human-review evaluation but should not be treated as strict exact-match test cases.

### 4. Evaluation Scope

Evaluation will be performed at two levels:

1. **Stage-level evaluation**

   * Topic routing;
   * Term routing;
   * Variable-level classification;
   * deterministic validation; and
   * semantic validation.

2. **End-to-end evaluation**

   * final article-level classification sets;
   * hierarchy depth;
   * final decision status;
   * review routing;
   * consistency;
   * latency; and
   * cost.

Stage-level evaluation will identify where errors enter the workflow. End-to-end evaluation will measure the quality and usefulness of the complete system.

### 5. Stage-Level Classification Metrics

#### 5.1 Topic Routing

The Topic Router will be evaluated as a multi-label classifier.

Metrics should include:

* micro-averaged precision, recall, and F1;
* macro-averaged precision, recall, and F1;
* exact Topic-set match;
* Topic omission rate;
* unnecessary Topic rate; and
* average number of selected Topics per article.

Topic recall is important because an omitted Topic prevents every correct descendant beneath it from being considered.

However, excessive Topic selection should also be measured because over-routing increases model calls and false-positive risk.

#### 5.2 Term Routing

Term selection will be evaluated within the correct Topic branches.

Metrics should include:

* Term precision, recall, and F1;
* exact Term-set match;
* incorrect sibling-Term rate;
* missed supported Term rate;
* unnecessary Term rate; and
* correct Topic-level stopping rate.

The evaluation must distinguish between correctly stopping at a Topic and failing to select a supported Term.

#### 5.3 Variable-Level Classification

Performance will be measured separately at:

* `Variable_Level_1`;
* `Variable_Level_2`; and
* `Variable_Level_3`.

Metrics should include:

* precision, recall, and F1 at each level;
* exact concept accuracy;
* sibling-confusion rate;
* unsupported descent rate;
* premature stopping rate; and
* correct stopping rate.

Unsupported descent measures how often the system selects a child that is more specific than the article evidence supports.

Premature stopping measures how often the system returns an ancestor despite sufficient evidence for a deeper child.

Because the project prioritizes reliability over depth, unsupported descent will be treated as a more serious error than defensible early stopping.

### 6. End-to-End Hierarchical Metrics

The final system will be evaluated using both exact and hierarchy-aware metrics.

#### 6.1 Exact-Path Accuracy

Exact-path accuracy will measure how often a predicted canonical path exactly matches an accepted reference path.

This metric is useful but insufficient because it does not distinguish among:

* a correct broader ancestor;
* an incorrect sibling;
* an unsupported descendant; and
* an unrelated branch.

#### 6.2 Hierarchy-Aware Evaluation

Hierarchy-aware evaluation should include:

* hierarchical precision;
* hierarchical recall;
* hierarchical F1;
* path-prefix accuracy;
* ancestor-aware accuracy;
* lowest common ancestor depth; and
* hierarchical distance between predicted and reference concepts.

A correct supported ancestor should receive more credit than an incorrect sibling or unrelated concept.

An unsupported descendant should receive an explicit over-specificity penalty.

#### 6.3 Classification Outcome Categories

Each predicted classification should be assigned to one of the following categories:

* **Exact supported classification**
* **Correct broader classification**
* **Unsupported over-specific classification**
* **Incorrect sibling classification**
* **Incorrect branch classification**
* **Missed classification**
* **Unnecessary classification**
* **Correct no-classification outcome**
* **Incorrect no-classification outcome**

These categories will make the accuracy–specificity tradeoff visible and prevent average hierarchy depth from being mistaken for quality.

### 7. Multi-Label Evaluation

Because an article may require multiple GCMD classifications, article-level results will be evaluated as multi-label sets.

Metrics should include:

* per-article precision, recall, and F1;
* micro-averaged precision, recall, and F1;
* macro-averaged precision, recall, and F1;
* exact classification-set match;
* Jaccard similarity;
* missed-label rate;
* extra-label rate; and
* average number of final classifications per article.

Redundant ancestor–descendant pairs will be removed before scoring when the descendant already represents the final accepted classification in that branch.

### 8. Deterministic Validator Evaluation

The deterministic validator will be evaluated independently using automated test cases.

It must correctly accept:

* valid UUIDs;
* valid concept names;
* valid hierarchy levels;
* valid canonical paths;
* valid parent–child relationships; and
* valid Topic and Term membership.

It must correctly reject:

* nonexistent UUIDs;
* altered concept names;
* UUID/name mismatches;
* invalid hierarchy levels;
* invalid path order;
* incorrect parent–child relationships;
* valid concepts placed under the wrong Topic or Term; and
* model-invented path components.

The required target for deterministic validation is:

| Measure                                                              | Requirement |
| -------------------------------------------------------------------- | ----------: |
| Valid test cases accepted                                            |        100% |
| Invalid test cases rejected                                          |        100% |
| Accepted production classifications passing deterministic validation |        100% |

Any deterministic-validation failure in an accepted final result will be treated as a system defect.

### 9. Semantic Validator Evaluation

The semantic validator will be compared with expert decisions for individual proposed classifications.

Metrics should include:

* acceptance precision;
* acceptance recall;
* false acceptance rate;
* false rejection rate;
* correct reduction-to-supported-ancestor rate;
* unnecessary reduction rate;
* correct rejection rate;
* correct human-review recommendation rate; and
* agreement with expert decisions.

The validator will also be evaluated by its net effect on the complete system:

* change in final precision;
* change in final recall;
* change in unsupported descent rate;
* change in review rate;
* change in processing cost; and
* change in latency.

The semantic validator will be considered beneficial only if it selectively removes, reduces, or flags weak predictions while preserving most correct classifications.

### 10. Confidence and Decision Threshold Evaluation

Model-reported confidence values will be treated as decision signals rather than calibrated probabilities unless calibration is demonstrated empirically.

Evaluation may include:

* accuracy by confidence range;
* precision–coverage curves;
* risk–coverage analysis;
* reliability diagrams;
* expected calibration error; and
* Brier score when probabilistic interpretation is appropriate.

The validation set will be used to establish thresholds for:

* automatic acceptance;
* semantic validation;
* reduction to a supported ancestor;
* human review; and
* rejection.

After selection, thresholds must be frozen before final test-set evaluation.

Threshold evaluation must consider hierarchy depth. A threshold appropriate for Topic acceptance may not be appropriate for `Variable_Level_3`.

### 11. Human-Review Evaluation

The structured review workflow will be evaluated for operational usefulness.

Measures should include:

* percentage of articles routed to review;
* percentage of reviewed recommendations accepted unchanged;
* percentage reduced to a broader concept;
* percentage replaced with another valid concept;
* percentage rejected;
* average review time per article;
* average review time per classification;
* agreement between automated and expert decisions; and
* estimated review-time reduction compared with fully manual classification.

A useful review process should reduce expert effort without encouraging uncritical acceptance of weak recommendations.

The initial project may evaluate structured review output without requiring a production graphical interface.

### 12. Baseline and Ablation Comparisons

The proposed system will be compared with simpler approaches to determine whether each architectural component adds measurable value.

At minimum, the following baselines should be evaluated:

1. exact or normalized lexical keyword matching;
2. one unrestricted model call that predicts complete GCMD paths;
3. hierarchical classification without semantic validation; and
4. hierarchical classification with semantic validation.

Additional comparisons may include:

* title-only versus title-and-abstract classification;
* Topic routing without Term routing;
* full sibling candidate lists versus retrieval or pre-ranking;
* prompts with and without definitions;
* single-topic versus multi-topic routing;
* classifier–validator prompt separation;
* alternative confidence thresholds; and
* different language models through the common model interface.

Each ablation should use the same evaluation records and scoring rules.

The purpose is to identify whether a component improves classification quality or merely increases complexity, latency, or cost.

### 13. Model Comparison

When multiple models are evaluated, they should use equivalent:

* article datasets;
* GCMD vocabulary versions;
* candidate sets;
* structured-output schemas;
* validation rules;
* test splits; and
* evaluation metrics.

Prompts should be kept as comparable as model interfaces permit.

Model comparison should include:

* precision;
* recall;
* F1;
* exact-path accuracy;
* hierarchical correctness;
* unsupported descent rate;
* consistency;
* structured-output failure rate;
* semantic-validator effectiveness;
* processing latency;
* token usage; and
* estimated cost.

Models will be selected based on measured task-specific performance. Greater classification depth or a larger number of predictions will not be treated as evidence of better performance.

### 14. Consistency and Reproducibility

A representative evaluation subset will be processed repeatedly using the same configuration.

Consistency measures should include:

* identical Topic-set rate;
* identical Term-set rate;
* identical final classification-set rate;
* variation in confidence signals;
* variation in semantic-validator decisions; and
* variation in review routing.

Every evaluation run must record:

* model name and version;
* model parameters;
* prompt versions;
* GCMD vocabulary version or file hash;
* decision thresholds;
* application version; and
* evaluation date.

Any remaining nondeterminism should be documented.

### 15. Performance and Cost Evaluation

Operational evaluation should include:

* average and median processing time per article;
* processing time by hierarchy depth;
* model calls per article;
* input and output tokens per article;
* estimated cost per article;
* total batch cost;
* semantic-validation overhead;
* cache-hit rate;
* retry rate;
* structured-output failure rate; and
* article failure rate.

Results should be examined separately for:

* single-topic articles;
* multi-topic articles;
* shallow classifications;
* deep classifications;
* articles producing multiple branches; and
* articles requiring review.

Architecture changes intended to reduce cost or latency must also be evaluated for their effect on precision, hierarchical correctness, coverage, and unsupported descent.

### 16. Error Analysis

Quantitative evaluation will be supplemented with structured error analysis.

Errors should be categorized as:

* missed Topic;
* unnecessary Topic;
* incorrect Term;
* sibling confusion;
* unsupported descent;
* premature stopping;
* substantive concept treated as incidental;
* incidental or background concept treated as primary;
* method classified as article subject;
* acronym or synonym misunderstanding;
* multidisciplinary branch omission;
* semantic-validator false acceptance;
* semantic-validator false rejection;
* incorrect reduction to ancestor;
* unnecessary human-review routing;
* structured-output failure; or
* gold-standard disagreement.

Each error should also be associated, when possible, with its likely source:

* prompt design;
* candidate construction;
* hierarchy traversal;
* model capability;
* threshold policy;
* vocabulary ambiguity;
* article ambiguity; or
* annotation disagreement.

Error analysis will guide subsequent prompt, threshold, workflow, and model changes.

### 17. Acceptance Criteria

Project acceptance will use a combination of fixed requirements and performance targets.

#### 17.1 Fixed Requirements

The system must:

* return only existing GCMD UUIDs and valid canonical paths;
* achieve 100% deterministic-validation correctness on the automated test suite;
* preserve original article metadata;
* produce output conforming to the defined JSON schema;
* retain completed results when later articles fail;
* record model, prompt, vocabulary, and configuration metadata;
* distinguish automated results from human-reviewed outcomes; and
* avoid accepting structurally invalid classifications.

#### 17.2 Comparative Requirements

The final hierarchical system must:

* outperform unrestricted complete-path generation in hierarchical correctness;
* reduce unsupported descent relative to the unrestricted baseline;
* improve accepted-classification precision when semantic validation is enabled, unless the added validator is rejected based on evaluation;
* demonstrate that added workflow components provide measurable benefit; and
* avoid gaining apparent specificity by increasing false-positive or unsupported classifications.

#### 17.3 Initial Performance Targets

Exact numeric targets for model-dependent metrics will be established after baseline evaluation.

The following table will be completed before final test execution:

| Metric                               |                    Initial target | Requirement type   |
| ------------------------------------ | --------------------------------: | ------------------ |
| Deterministic validation correctness |                              100% | Fixed              |
| Output schema validity               |                              100% | Fixed              |
| Accepted-classification precision    |                TBD after baseline | Performance target |
| Topic recall                         |                TBD after baseline | Performance target |
| Term recall                          |                TBD after baseline | Performance target |
| Unsupported descent rate             |  Lower than unrestricted baseline | Comparative        |
| Hierarchical correctness             | Higher than unrestricted baseline | Comparative        |
| Structured-output success rate       |                TBD after baseline | Performance target |
| Human-review rate                    |        TBD after workflow testing | Operational target |
| Average cost per article             |                TBD after baseline | Operational target |

Targets will be selected using the validation set and will not be changed after the final test run begins.

### 18. Final Evaluation Procedure

The final evaluation will follow this sequence:

```text id="h9bmwc"
Create and adjudicate expert-reviewed reference data
        ↓
Separate development, validation, and test records
        ↓
Establish lexical and unrestricted-model baselines
        ↓
Evaluate Topic, Term, and Variable stages
        ↓
Evaluate deterministic and semantic validators
        ↓
Tune prompts, models, thresholds, and decision policies on development and validation data
        ↓
Freeze configuration
        ↓
Run the final test set once
        ↓
Measure classification quality, consistency, latency, and cost
        ↓
Conduct structured error analysis
        ↓
Compare results with acceptance criteria
        ↓
Document conclusions and remaining limitations
```

### 19. Evaluation Reporting

The final evaluation report should include:

* evaluation dataset composition;
* Topic and hierarchy-level distribution;
* expert annotation and adjudication procedures;
* inter-reviewer agreement;
* model and prompt configurations;
* decision thresholds;
* baseline results;
* stage-level results;
* end-to-end results;
* hierarchy-aware metrics;
* semantic-validator impact;
* human-review findings;
* consistency results;
* latency and cost;
* ablation results;
* error analysis;
* performance by GCMD Topic;
* representative correct and incorrect examples;
* known limitations; and
* recommendations for future work.

Results should be reported both in aggregate and by GCMD Topic. Aggregate metrics alone may conceal weak performance in smaller or specialized vocabulary branches.

The principal measure of success will be the system’s ability to return valid and scientifically defensible GCMD concepts at the deepest level supported by the article evidence. Classification depth by itself will not be treated as a measure of quality.

## Output Specification

The GCMD Keyword Classifier will produce structured JSON output for each article processed from `articles.json`.

The output must:

* preserve the original article metadata;
* record final GCMD classifications;
* distinguish automated decisions from human-reviewed outcomes;
* include validation and review status;
* retain sufficient metadata for traceability and reproducibility; and
* conform to a formal JSON Schema.

Detailed diagnostic and run-summary data may be written to separate supporting files.

### 1. Output Files

The primary output file will be:

```text
classification_results.json
```

It will contain one article-level result for each record in `articles.json`.

Optional supporting files may include:

```text
classification_diagnostics.json
human_review_queue.json
run_summary.json
```

The files serve the following purposes:

| File                              | Purpose                                                                           |
| --------------------------------- | --------------------------------------------------------------------------------- |
| `classification_results.json`     | Final article-level classification results                                        |
| `classification_diagnostics.json` | Rejected candidates, validation failures, model errors, and debugging information |
| `human_review_queue.json`         | Classifications requiring metadata-specialist review                              |
| `run_summary.json`                | Aggregate processing, cost, latency, and outcome statistics                       |

Results will be written incrementally so that completed work is preserved if later processing fails or is interrupted.

### 2. Article-Level Result

Each article-level result will preserve the four source fields exactly as provided:

* `DOI`;
* `Title`;
* `Year`; and
* `Abstract`.

The DOI will serve as the article identifier. A separate `article_id` field is not required.

Each result will also include:

* `processing_status`;
* `classification_outcome`;
* `classifications`;
* `reviewed_classifications`, when human review has produced revised final results;
* `review_status`;
* `warnings`;
* `errors`; and
* `processing_metadata`.

A representative structure is:

```json
{
  "DOI": "10.xxxx/example.doi",
  "Title": "Example Earth Science Article",
  "Year": 2025,
  "Abstract": "This study investigates atmospheric carbon dioxide profiles...",
  "processing_status": "completed",
  "classification_outcome": "not_classified",
  "classifications": [],
  "no_classification_reason": "No defensible GCMD concept was supported by the title and abstract.",
  "review_status": "not_required",
  "warnings": [],
  "errors": [],
  "processing_metadata": {}
}
```

### 3. Article Processing Status

`processing_status` describes whether the article-processing workflow completed successfully.

Allowed values are:

| Value       | Meaning                                                         |
| ----------- | --------------------------------------------------------------- |
| `completed` | All required processing stages completed                        |
| `partial`   | Some branches completed, but at least one required stage failed |
| `failed`    | The article could not be processed                              |
| `skipped`   | The article was intentionally excluded from processing          |

A completed article may have zero, one, or multiple classifications.

A no-classification result will therefore use:

```json
{
  "processing_status": "completed",
  "classification_outcome": "not_classified",
  "classifications": [],
  "no_classification_reason": "No defensible GCMD concept was supported by the title and abstract."
}
```

`review_required` will not be used as a processing status because review state is represented separately.

A reused cached result will normally remain:

```json
{
  "processing_status": "completed",
  "processing_metadata": {
    "cache_used": true
  }
}
```

`classification_outcome` is populated when `processing_status` is `completed`:

| Value | Meaning |
|---|---|
| `classified` | One or more accepted or reduced final classifications were produced |
| `pending_review` | No classification is final yet, but one or more candidates require human review |
| `not_classified` | No defensible GCMD concept was supported by the available text |

For `partial`, `failed`, or `skipped` processing, `classification_outcome` should be `null` or omitted according to the formal schema.

### 4. Final Classification Record

The `classifications` array will contain accepted, reduced, or review-required GCMD assignments. Rejected candidates belong in diagnostic output and must not appear in this array.

Each classification record will contain:

| Field                      | Type             | Description                                     |
| -------------------------- | ---------------- | ----------------------------------------------- |
| `UUID`                     | String           | Authoritative GCMD UUID                         |
| `name`                     | String           | Authoritative GCMD concept name                 |
| `level`                    | String           | `Topic`, `Term`, or Variable hierarchy level    |
| `canonical_path`           | String           | Complete valid GCMD path                        |
| `path_components`          | Array of strings | Ordered hierarchy components                    |
| `topic`                    | String           | Top-level GCMD Topic                            |
| `term`                     | String or null   | GCMD Term, when applicable                      |
| `confidence`               | Object           | Stage-level and final confidence signals        |
| `classifier_evidence`      | String           | Evidence supporting the original recommendation |
| `support_type`             | String           | Whether support is explicit, inferred, or mixed |
| `reason_for_stopping`      | String           | Why the workflow stopped at this level          |
| `deterministic_validation` | Object           | Vocabulary and hierarchy validation result      |
| `semantic_validation`      | Object           | Independent semantic validation result          |
| `final_status`             | String           | Final automated classification outcome          |
| `review_required`          | Boolean          | Whether expert review is required               |

A representative classification is:

```json
{
  "UUID": "03ddc432-906d-4469-bb00-179c828dbea4",
  "name": "CARBON DIOXIDE PROFILES",
  "level": "Variable_Level_3",
  "canonical_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY > CARBON AND HYDROCARBON COMPOUNDS > ATMOSPHERIC CARBON DIOXIDE > CARBON DIOXIDE PROFILES",
  "path_components": [
    "ATMOSPHERE",
    "ATMOSPHERIC CHEMISTRY",
    "CARBON AND HYDROCARBON COMPOUNDS",
    "ATMOSPHERIC CARBON DIOXIDE",
    "CARBON DIOXIDE PROFILES"
  ],
  "topic": "ATMOSPHERE",
  "term": "ATMOSPHERIC CHEMISTRY",
  "confidence": {
    "topic": 0.98,
    "term": 0.96,
    "variable_level_1": 0.94,
    "variable_level_2": 0.93,
    "variable_level_3": 0.91,
    "final": 0.94
  },
  "classifier_evidence": "The title and abstract explicitly describe vertical atmospheric carbon dioxide profiles.",
  "support_type": "explicit",
  "reason_for_stopping": "The selected concept is the deepest supported concept in this branch.",
  "deterministic_validation": {
    "valid": true,
    "errors": []
  },
  "semantic_validation": {
    "decision": "accept",
    "support_level": "strong",
    "deepest_supported_uuid": "03ddc432-906d-4469-bb00-179c828dbea4",
    "deepest_supported_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY > CARBON AND HYDROCARBON COMPOUNDS > ATMOSPHERIC CARBON DIOXIDE > CARBON DIOXIDE PROFILES",
    "unsupported_components": [],
    "evidence": "The article explicitly studies vertical atmospheric carbon dioxide profiles."
  },
  "final_status": "accepted",
  "review_required": false
}
```

### 5. Classification Status

`final_status` describes the final automated outcome for one proposed GCMD classification.

Allowed values are:

| Value               | Meaning                                                                                  |
| ------------------- | ---------------------------------------------------------------------------------------- |
| `accepted`          | The proposed concept is valid and adequately supported                                   |
| `reduced_to_ancestor` | The original proposal was too specific and was reduced to the deepest supported ancestor |
| `review_required`   | The classification is plausible but cannot be finalized automatically                    |
| `rejected`          | The proposed concept is invalid or unsupported                                           |

`not_classified` is an article-level outcome and will not be used as a classification record status.

Rejected candidates will normally be stored in diagnostic output rather than in the final `classifications` array.

### 6. Reduced Classifications

When semantic validation determines that the proposed concept is too specific, the final record will represent the deepest supported UUID-bearing ancestor.

The record will preserve the original proposal:

```json
{
  "UUID": "b9c56939-c624-467d-b196-e56a5b660334",
  "name": "ATMOSPHERIC CHEMISTRY",
  "level": "Term",
  "canonical_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY",
  "final_status": "reduced_to_ancestor",
  "original_candidate": {
    "UUID": "example-child-uuid",
    "canonical_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY > CARBON AND HYDROCARBON COMPOUNDS"
  },
  "semantic_validation": {
    "decision": "reduce_to_ancestor",
    "deepest_supported_uuid": "b9c56939-c624-467d-b196-e56a5b660334",
    "deepest_supported_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY",
    "unsupported_components": [
      "CARBON AND HYDROCARBON COMPOUNDS"
    ],
    "evidence": "The article supports atmospheric chemistry but not the more specific compound family."
  },
  "review_required": false
}
```

Although the status name is `reduced_to_ancestor`, the retained concept may be any supported ancestor, not necessarily the immediate parent.

### 7. Deterministic Validation Result

The deterministic validation object will record whether the final candidate is structurally valid within `gcmd_hierarchy.json`.

The minimum structure is:

```json
{
  "valid": true,
  "errors": []
}
```

Diagnostic output may retain more detailed checks, such as:

* UUID validity;
* UUID/name consistency;
* hierarchy-level validity;
* canonical-path validity;
* parent–child validity;
* Topic membership; and
* Term membership.

A classification that fails deterministic validation cannot receive `accepted` or `reduced_to_ancestor` status.

### 8. Semantic Validation Result

The semantic validation object will contain:

| Field                    | Type             | Description                                                 |
| ------------------------ | ---------------- | ----------------------------------------------------------- |
| `decision`               | String           | `accept`, `reduce_to_ancestor`, `reject`, or `require_review` |
| `support_level`          | String           | `strong`, `moderate`, `weak`, or `unsupported`              |
| `deepest_supported_uuid` | String or null   | UUID of the deepest supported concept                       |
| `deepest_supported_path` | String or null   | Deepest path supported by the article                       |
| `unsupported_components` | Array of strings | Unsupported parts of the proposed path                      |
| `evidence`               | String           | Concise explanation grounded in the article                 |

`support_type` and `support_level` represent separate dimensions:

* `support_type`: `explicit`, `inferred`, or `mixed`;
* `support_level`: `strong`, `moderate`, `weak`, or `unsupported`.

### 9. Confidence Representation

Confidence values will be represented as numbers from `0.0` through `1.0`.

The standard structure is:

```json
{
  "confidence": {
    "topic": 0.97,
    "term": 0.92,
    "variable_level_1": 0.81,
    "variable_level_2": 0.58,
    "variable_level_3": null,
    "final": 0.86
  }
}
```

A field may be `null` when the corresponding hierarchy level was not evaluated.

Confidence values are decision signals and must not be described as calibrated probabilities unless calibration has been demonstrated through evaluation.

### 10. Review Status and Human Decisions

The article-level `review_status` field will use:

| Value | Meaning |
|---|---|
| `not_required` | No classification requires review |
| `pending` | At least one classification requires review |
| `completed` | All required human review has been completed |

Because one article may contain multiple classifications, human decisions will be stored in a `human_reviews` array. Each review record must identify the automated classification or candidate to which it applies.

Before review:

```json
{
  "review_status": "pending",
  "review_reasons": [
    "The classifier and semantic validator disagree about the supported hierarchy depth."
  ],
  "human_reviews": []
}
```

After review:

```json
{
  "review_status": "completed",
  "human_reviews": [
    {
      "automated_candidate_uuid": "example-child-uuid",
      "decision": "accepted_with_revision",
      "final_uuid": "b9c56939-c624-467d-b196-e56a5b660334",
      "final_canonical_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY",
      "review_notes": "The Term is supported, but the Variable-level concept is not.",
      "reviewed_at": "2026-07-06T18:30:00Z"
    }
  ]
}
```

Human review must not overwrite or erase the original automated recommendation. When review changes the operational result, the article record should include a `reviewed_classifications` array containing the expert-approved classifications. Downstream consumers should use `reviewed_classifications` when `review_status` is `completed`; otherwise they should use the automated `classifications` array subject to its statuses.

### 11. Warnings and Errors

Warnings describe conditions that do not prevent article processing.

Errors describe failures that prevent all or part of the workflow from completing.

Each warning should contain:

```json
{
  "code": "WARNING_CODE",
  "message": "Human-readable description."
}
```

Each error should contain:

```json
{
  "stage": "term_routing",
  "code": "MODEL_TIMEOUT",
  "message": "The model request exceeded the configured timeout.",
  "retry_count": 2
}
```

Errors must provide enough information for diagnosis without exposing credentials or protected configuration values.

### 12. Processing Metadata

Each article result will include enough metadata to identify the configuration that produced it.

The minimum recommended structure is:

```json
{
  "processing_metadata": {
    "run_id": "example-run-id",
    "processed_at": "2026-07-06T18:25:18Z",
    "model_name": "configured-model-name",
    "model_parameters": {
      "temperature": 0
    },
    "prompt_versions": {
      "topic_router": "1.0",
      "term_router": "1.0",
      "variable_classifier": "1.0",
      "semantic_validator": "1.0"
    },
    "vocabulary_version": "configured-version-or-file-hash",
    "application_version": "0.1.0",
    "configuration_hash": "example-hash",
    "article_fingerprint": "example-content-hash",
    "cache_used": false,
    "processing_time_seconds": 8.42,
    "model_calls": 6,
    "input_tokens": 7421,
    "output_tokens": 913,
    "estimated_cost": null
  }
}
```

Unavailable optional values may be omitted or represented as `null`, as defined by the formal schema.

### 13. Multiple and Redundant Classifications

The output may contain multiple classifications when an article supports multiple independent scientific concepts.

The final `classifications` array must not contain:

* duplicate UUIDs;
* duplicate canonical paths; or
* redundant ancestors when a supported descendant from the same branch is already present.

Ancestors will remain visible within `path_components` but will not be counted as separate final classifications.

No fixed maximum number of classifications will be imposed in the initial implementation.

### 14. Diagnostic Output

Rejected or superseded candidates may be retained in `classification_diagnostics.json`.

A diagnostic record may include:

```json
{
  "DOI": "10.xxxx/example.doi",
  "candidate": {
    "UUID": "example-uuid",
    "canonical_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY > CARBON AND HYDROCARBON COMPOUNDS"
  },
  "stage": "semantic_validation",
  "outcome": "rejected",
  "reason": "The article does not identify carbon or hydrocarbon compounds."
}
```

Diagnostic candidates must not be presented as final classifications.

### 15. Run Summary

The optional `run_summary.json` file will contain aggregate counts and processing statistics.

It should distinguish between article counts and classification counts.

Recommended fields include:

* run identifier;
* start and completion timestamps;
* articles received;
* articles completed;
* articles partially completed;
* articles failed;
* articles with no classification;
* articles requiring review;
* accepted classifications;
* reduced classifications;
* rejected candidates;
* average classifications per article;
* average processing time;
* total model calls;
* token usage; and
* estimated total cost.

### 16. Ordering and Formatting

Output files will use:

* UTF-8 encoding;
* valid JSON;
* ISO 8601 timestamps;
* authoritative GCMD spelling and capitalization;
* unchanged UUID values; and
* ordered `path_components`.

Article records should retain the order of `articles.json` when practical. If parallel processing changes completion order, each result may include a `source_index`.

Classifications should be ordered consistently, such as by canonical path or descending final confidence.

### 17. JSON Schema and Integrity Rules

The project will define formal JSON Schemas for:

```text
schemas/classification_result.schema.json
schemas/run_summary.schema.json
```

Schema validation will confirm:

* required fields;
* field types;
* status enumerations and status scope;
* generated-field naming conventions;
* confidence ranges;
* validation-object structure;
* review-field requirements and reviewed-result precedence;
* preservation of article metadata; and
* article-level and run-level consistency.

The following integrity rules are mandatory:

1. Every accepted or reduced classification must resolve to an existing UUID in `gcmd_hierarchy.json`.
2. Every accepted canonical path must contain valid parent–child relationships.
3. The UUID, name, level, and canonical path must describe the same GCMD concept.
4. A classification cannot be accepted when deterministic validation fails.
5. A reduced result must preserve its original proposed candidate.
6. Human review must preserve the original automated result.
7. When `review_status` is `completed`, any `reviewed_classifications` must be treated as the operational result while the original `classifications` remain immutable.
8. Model-generated concepts must not be added silently to the vocabulary.
9. Missing values and failures must not be replaced with fabricated data.
10. Original article metadata must remain unchanged.
11. Every result must be traceable to the model, prompts, vocabulary version, and application configuration that produced it.

### 18. Representative Complete Result

```json
{
  "DOI": "10.xxxx/example.doi",
  "Title": "Vertical Distribution of Atmospheric Carbon Dioxide",
  "Year": 2025,
  "Abstract": "This study examines vertical carbon dioxide concentration profiles in the lower atmosphere.",
  "processing_status": "completed",
  "classification_outcome": "classified",
  "classifications": [
    {
      "UUID": "03ddc432-906d-4469-bb00-179c828dbea4",
      "name": "CARBON DIOXIDE PROFILES",
      "level": "Variable_Level_3",
      "canonical_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY > CARBON AND HYDROCARBON COMPOUNDS > ATMOSPHERIC CARBON DIOXIDE > CARBON DIOXIDE PROFILES",
      "path_components": [
        "ATMOSPHERE",
        "ATMOSPHERIC CHEMISTRY",
        "CARBON AND HYDROCARBON COMPOUNDS",
        "ATMOSPHERIC CARBON DIOXIDE",
        "CARBON DIOXIDE PROFILES"
      ],
      "topic": "ATMOSPHERE",
      "term": "ATMOSPHERIC CHEMISTRY",
      "confidence": {
        "topic": 0.98,
        "term": 0.96,
        "variable_level_1": 0.94,
        "variable_level_2": 0.93,
        "variable_level_3": 0.91,
        "final": 0.94
      },
      "classifier_evidence": "The title and abstract explicitly describe vertical atmospheric carbon dioxide profiles.",
      "support_type": "explicit",
      "reason_for_stopping": "The selected concept is the deepest supported concept in this branch.",
      "deterministic_validation": {
        "valid": true,
        "errors": []
      },
      "semantic_validation": {
        "decision": "accept",
        "support_level": "strong",
        "deepest_supported_uuid": "03ddc432-906d-4469-bb00-179c828dbea4",
        "deepest_supported_path": "ATMOSPHERE > ATMOSPHERIC CHEMISTRY > CARBON AND HYDROCARBON COMPOUNDS > ATMOSPHERIC CARBON DIOXIDE > CARBON DIOXIDE PROFILES",
        "unsupported_components": [],
        "evidence": "The article explicitly studies vertical atmospheric carbon dioxide profiles."
      },
      "final_status": "accepted",
      "review_required": false
    }
  ],
  "review_status": "not_required",
  "warnings": [],
  "errors": [],
  "processing_metadata": {
    "run_id": "example-run-id",
    "processed_at": "2026-07-06T18:25:18Z",
    "model_name": "configured-model-name",
    "model_parameters": {
      "temperature": 0
    },
    "prompt_versions": {
      "topic_router": "1.0",
      "term_router": "1.0",
      "variable_classifier": "1.0",
      "semantic_validator": "1.0"
    },
    "vocabulary_version": "configured-version-or-file-hash",
    "application_version": "0.1.0",
    "configuration_hash": "example-hash",
    "article_fingerprint": "example-content-hash",
    "cache_used": false,
    "processing_time_seconds": 8.42,
    "model_calls": 6,
    "input_tokens": 7421,
    "output_tokens": 913,
    "estimated_cost": null
  }
}
```

The output design principle is:

> Every final classification must be valid, evidence-based, traceable, and clearly distinguishable from rejected, uncertain, reduced, or human-revised recommendations.