docs/HARNESS_MATURITY.md

# Harness Maturity Ladder

This ladder defines how `repository-harness` should progress from static
agent instructions to measurable harness improvement.

The levels are intentionally verifiable. A level is achieved only when its
criteria can be inspected in repository files, durable Harness records, or
benchmark output.

## Levels

### H0 - Bare Environment

The model operates with no repository harness. It receives a prompt and may
produce a patch, but the repo does not tell it how to classify, validate, or
record work.

Criteria:

- No `AGENTS.md` Harness block exists.
- No feature intake policy exists.
- No story, decision, validation, or trace artifact exists.

Required files:

- None.

Benchmark indicators:

- Functional score is the only meaningful metric.
- Harness compliance: 0%.
- Trace quality: 0/3.

Current status:

- Passed. This repository is beyond H0.

Activated responsibilities:

- None.

### H1 - Scaffolding And Policy

The repository contains static operating instructions, templates, risk lanes,
and source-of-truth rules. Agents can follow a documented workflow, but durable
state may still be manual or incomplete.

Criteria:

- `AGENTS.md` points agents to the Harness operating docs.
- `docs/HARNESS.md`, `docs/FEATURE_INTAKE.md`, and `docs/ARCHITECTURE.md`
  exist.
- Story, decision, and validation templates exist under `docs/templates/`.
- `docs/TEST_MATRIX.md` defines proof columns and status meanings.

Required files:

- `AGENTS.md`
- `docs/HARNESS.md`
- `docs/FEATURE_INTAKE.md`
- `docs/ARCHITECTURE.md`
- `docs/TEST_MATRIX.md`
- `docs/templates/story.md`
- `docs/templates/decision.md`
- `docs/templates/validation-report.md`

Benchmark indicators:

- Harness compliance: 20-40%.
- Lane accuracy improves when agents read the intake policy.
- Trace quality remains low unless traces are separately requested.

Current status:

- Achieved. H1 files exist and are used by current Harness instructions.

Activated responsibilities:

- Task specification.
- Permissions.
- Project memory.
- Verification.

### H2 - Durable State And Observability

The repository has structured operational records and explicit observation
rules. Agents can record what happened, connect work to stories, and write
traces with predictable depth.

Criteria:

- `scripts/bin/harness-cli` can record intake, story, decision, backlog, and trace
  data in `harness.db`.
- `scripts/schema/001-init.sql` defines durable tables for intake, story,
  decision, backlog, and trace records.
- `docs/HARNESS_COMPONENTS.md` maps files and responsibilities.
- `docs/HARNESS_MATURITY.md` defines H0-H5 with measurable criteria.
- `docs/TRACE_SPEC.md` defines trace fields, quality tiers, and friction
  capture.
- `docs/CONTEXT_RULES.md` defines phase-by-lane context rules.
- `AGENTS.md` and `docs/HARNESS.md` reference the Phase 2 operating docs.

Required files:

- `scripts/bin/harness-cli`
- `scripts/schema/001-init.sql`
- `docs/HARNESS_COMPONENTS.md`
- `docs/HARNESS_MATURITY.md`
- `docs/TRACE_SPEC.md`
- `docs/CONTEXT_RULES.md`

Benchmark indicators:

- Harness compliance: 75-90%.
- Trace quality: at least 2.0/3 on normal-lane tasks.
- Lane accuracy: 6/6 on the current benchmark suite.
- Friction captured: at least 4/6 benchmark tasks when friction exists.

Current status:

- Achieved. Durable state exists, and the Phase 2 docs define the
  observability and context specification. Phase 3 active scoring builds on
  this layer.

Activated responsibilities:

- Task state.
- Observability.
- Failure attribution.
- Context selection.
- Entropy auditing.

### H3 - Active Observability And Evolution

The harness can evaluate its own operational data and turn repeated failures
into prioritized improvements.

Criteria:

- Trace quality can be scored by a repeatable command or benchmark step.
- Harness friction can be grouped by component from `docs/HARNESS_COMPONENTS.md`.
- Backlog items include predicted impact and actual outcome after completion.
- Benchmark comparison output identifies which harness responsibility moved or
  regressed.

Required files:

- H2 files.
- A benchmark protocol or report that references maturity levels.
- A documented trace quality scoring method.
- A documented friction-to-backlog review loop.

Benchmark indicators:

- Harness compliance: 85-95%.
- Trace quality: 2.3-2.7/3.
- Friction captured and classified by component for most failed or awkward
  tasks.
- Regressions include an attributed harness component.

Current status:

- Partially achieved by Phase 3. `scripts/bin/harness-cli score-trace` scores trace
  quality against tier rules, `query friction` includes linked intake context,
  the `trace` command now prints that score at write time, and the backlog
  outcome loop documents predicted impact versus actual outcome. Full H3 still
  requires benchmark comparison output that attributes moved or regressed
  responsibilities.

Activated responsibilities:

- Observability.
- Failure attribution.
- Entropy auditing.
- Intervention recording.

### H4 - Automated Verification

The harness can run or orchestrate proof checks consistently and can reject or
flag incomplete work before the final response.

Criteria:

- A documented verification command or protocol runs the expected checks for a
  selected story and lane.
- Stories can store and execute a `verify_command`.
- Trace recording warns when a linked story has a verification command that has
  not passed.
- Missing validation evidence is surfaced before a task is marked implemented.

Required files:

- H3 files.
- A verification protocol or command reference.
- Validation report examples tied to story proof columns.
- Story verification command documentation.

Benchmark indicators:

- Functional score remains stable.
- Harness compliance: at least 90%.
- Fewer false "done" claims in benchmark review.
- Missing proof is detected before merge or final response.

Current status:

- Achieved by Phase 5. `scripts/bin/harness-cli story verify <id>` runs
  story-level proof commands, records pass/fail state, `trace --story` warns
  before close when verification has not passed, and
  `scripts/bin/harness-cli story verify-all` runs all configured story proof
  commands in one pass. Proof-column automation remains a future enhancement,
  but H4's required automated verification gate is now present.

Activated responsibilities:

- Verification.
- Task state.
- Permissions.
- Intervention recording.

### H5 - Self-Improving Harness

The harness can use traces, benchmark results, and backlog outcomes to propose
or apply safe improvements to itself.

Criteria:

- Repeated friction patterns are summarized into proposed harness changes.
- Proposed changes include predicted impact, risk, validation plan, and rollback
  criteria.
- Completed changes compare predicted impact with actual benchmark or trace
  outcomes.
- High-risk harness changes pause for human confirmation before changing source
  hierarchy, architecture direction, or validation requirements.

Required files:

- H4 files.
- Self-improvement protocol.
- Historical improvement reports.
- Backlog outcome reviews.

Benchmark indicators:

- Harness compliance remains at least 90% across repeated benchmark runs.
- Trace quality remains at least 2.5/3.
- Improvements show measurable positive deltas or are explicitly reverted.
- Scope creep and validation weakening are caught by policy.

Current status:

- Partially achieved by Phase 5. `scripts/bin/harness-cli audit` detects
  durable-state drift, `scripts/bin/harness-cli propose` generates structured
  improvement proposals from friction, interventions, and audit results, and
  `docs/IMPROVEMENT_PROTOCOL.md` defines the review loop. H5 is not fully
  achieved until repeated benchmark outcomes prove proposed improvements create
  measurable positive deltas or are explicitly reverted.

Activated responsibilities:

- Entropy auditing.
- Failure attribution.
- Intervention recording.
- Permissions.

## Current Assessment

| Level | Status | Evidence |
| --- | --- | --- |
| H0 | Passed | Harness docs, templates, and durable records exist. |
| H1 | Achieved | `AGENTS.md`, `docs/HARNESS.md`, `docs/FEATURE_INTAKE.md`, `docs/ARCHITECTURE.md`, `docs/templates/*`, and `docs/TEST_MATRIX.md` exist. |
| H2 | Achieved | `scripts/bin/harness-cli`, `scripts/schema/001-init.sql`, durable story records, `docs/HARNESS_COMPONENTS.md`, `docs/HARNESS_MATURITY.md`, `docs/TRACE_SPEC.md`, and `docs/CONTEXT_RULES.md` define the Phase 2 surface. |
| H3 | Partial | Phase 3 adds `scripts/bin/harness-cli score-trace`, enriched friction context, and the backlog outcome loop; Phase 4 auto-scores traces on write. Component-level benchmark attribution remains open. |
| H4 | Achieved | Phase 4 adds story-level `verify_command`, `story verify`, and trace-time verification warnings. Phase 5 adds `story verify-all` for batch story proof. |
| H5 | Partial | Phase 5 adds `audit`, `score-context`, `intervention add/query`, `propose`, `docs/HARNESS_AUDIT.md`, and `docs/IMPROVEMENT_PROTOCOL.md`; repeated benchmark outcome proof remains open. |

## Responsibility Activation

| Responsibility | H0 | H1 | H2 | H3 | H4 | H5 |
| --- | --- | --- | --- | --- | --- | --- |
| Task specification | Missing | Covered | Covered | Covered | Covered | Covered |
| Context selection | Missing | Partial | Covered | Covered | Covered | Covered |
| Tool access | Missing | Partial | Partial | Partial | Covered | Covered |
| Project memory | Missing | Covered | Covered | Covered | Covered | Covered |
| Task state | Missing | Partial | Covered | Covered | Covered | Covered |
| Observability | Missing | Missing | Partial | Covered | Covered | Covered |
| Failure attribution | Missing | Missing | Partial | Covered | Covered | Covered |
| Verification | Missing | Partial | Partial | Partial | Covered | Covered |
| Permissions | Missing | Partial | Partial | Partial | Covered | Covered |
| Entropy auditing | Missing | Missing | Partial | Covered | Covered | Covered |
| Intervention recording | Missing | Partial | Partial | Covered | Covered | Covered |

## Phase 3 Interpretation

Phase 3 starts the H2 to H3 transition. It claims active trace scoring and a
documented improvement feedback loop, but it does not claim full H3 because
benchmark comparison and component-level regression attribution are explicitly
outside this repository's Phase 3 scope.

## Phase 4 Interpretation

Phase 4 starts the H3 to H4 transition. It gives stories the same mechanical
verification pattern that decisions already had, records story verification
results in the durable layer, auto-scores traces when they are written, and
warns before close when a linked story's verification has not passed. It does
not claim full H4 because benchmark execution, batch verification, and automatic
proof-column updates remain separate work.

## Phase 5 Interpretation

Phase 5 completes H4 by adding batch story verification and starts H5 by adding
tool discovery, intervention records, context scoring, drift audit, and
deterministic proposal generation. The repository may claim H5 partial only
when those commands and docs are present and validated; it must not claim full
H5 until benchmark runs or trace outcomes prove the proposal loop improves the
harness over time.