# Customer-Facing Evaluation Bundle (v7.21)

This directory contains a self-contained evaluation bundle suitable for sharing with stakeholders.

## Evaluation Philosophy

This bundle reports two tiers of signals:

- Tier 1 evaluates planning feasibility and externally verified outcome correctness.
- Tier 2 tracks final reporting discipline (exact final-state emission) and is not required for Tier 1 pass.

Tier 1 is treated as the primary indicator of agent readiness because most deployed systems compute and validate state outside the model.

## Contents

- `../comparison_results_v7_21_Mistral_Nemo_Instruct_2407_AWQ_20260529_152601_20260629_184555.csv`: full per-case evaluation results (CSV).
- `../run_manifest.json`: run provenance (paths, hashes, artifact sizes).
- `../fixtures/golden_oracle_fixtures_v7_21__sha256_6d71a0b9147c.json`: the exact versioned test cases used for this run.
- `rollup.json`: machine-readable rollup derived from the CSV.
- `summary.md`: human-readable summary derived from the CSV and manifest.
- `model_card.md`: publishable model-card style summary.

## Provenance

- Run directory: `/mnt/e/Quantization_Agent_Templates_SaaS/QUANT_EVAL/runs/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601_20260629_184555`
- Artifacts directory: `/mnt/e/Quantization_Agent_Templates_SaaS/QUANT_EVAL/runs/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601_20260629_184555/artifacts`
- Fixtures directory: `/mnt/e/Quantization_Agent_Templates_SaaS/QUANT_EVAL/runs/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601_20260629_184555/fixtures`

## How to Use This Bundle

- `summary.md` is the decision-ready overview: coverage, pass-rate signals, pass rates, readiness snapshot, and top failure reasons.
- `model_card.md` is a publishable summary suitable for a model catalog entry.
- The CSV contains authoritative per-case diagnostics (see the `detail` column and related fields).

## How to Verify

The `run_manifest.json` records artifact SHA256 hashes and file sizes. These can be used to verify model artifacts and provenance.

## How to Reproduce

From the project root, rerun the harness and select the same model directory and options recorded in `../run_manifest.json`.

```bash
python3 -m quant_eval
```

This run used fixtures: `../fixtures/golden_oracle_fixtures_v7_21__sha256_6d71a0b9147c.json` and wrote results to: `../comparison_results_v7_21_Mistral_Nemo_Instruct_2407_AWQ_20260529_152601_20260629_184555.csv`.