Hugging Face's logo

pbhappliedsystems
/
Mistral_Nemo_Instruct_2407_4bit_AWQ_Async

Safetensors
vllm
mistral
compressed-tensors
awq
w4a16
quantization
4bit
mistral-nemo
instruct
agentic
tool-calling
structured-output
pbh-applied-systems
quant-eval
Model card Files
xet
 Community
Mistral_Nemo_Instruct_2407_4bit_AWQ_Async / reports /README.md
pbhappliedsystems's picture
pbhappliedsystems
Upload 22 files
fe6076f verified
preview code
|
Raw
History Blame Contribute Delete
2.58 kB

Customer-Facing Evaluation Bundle (v7.21)

This directory contains a self-contained evaluation bundle suitable for sharing with stakeholders.

Evaluation Philosophy

This bundle reports two tiers of signals:

  • Tier 1 evaluates planning feasibility and externally verified outcome correctness.
  • Tier 2 tracks final reporting discipline (exact final-state emission) and is not required for Tier 1 pass.

Tier 1 is treated as the primary indicator of agent readiness because most deployed systems compute and validate state outside the model.

Contents

  • ../comparison_results_v7_21_Mistral_Nemo_Instruct_2407_AWQ_20260529_152601_20260629_184555.csv: full per-case evaluation results (CSV).
  • ../run_manifest.json: run provenance (paths, hashes, artifact sizes).
  • ../fixtures/golden_oracle_fixtures_v7_21__sha256_6d71a0b9147c.json: the exact versioned test cases used for this run.
  • rollup.json: machine-readable rollup derived from the CSV.
  • summary.md: human-readable summary derived from the CSV and manifest.
  • model_card.md: publishable model-card style summary.

Provenance

  • Run directory: /mnt/e/Quantization_Agent_Templates_SaaS/QUANT_EVAL/runs/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601_20260629_184555
  • Artifacts directory: /mnt/e/Quantization_Agent_Templates_SaaS/QUANT_EVAL/runs/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601_20260629_184555/artifacts
  • Fixtures directory: /mnt/e/Quantization_Agent_Templates_SaaS/QUANT_EVAL/runs/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601/Mistral_Nemo_Instruct_2407_AWQ_20260529_152601_20260629_184555/fixtures

How to Use This Bundle

  • summary.md is the decision-ready overview: coverage, pass-rate signals, pass rates, readiness snapshot, and top failure reasons.
  • model_card.md is a publishable summary suitable for a model catalog entry.
  • The CSV contains authoritative per-case diagnostics (see the detail column and related fields).

How to Verify

The run_manifest.json records artifact SHA256 hashes and file sizes. These can be used to verify model artifacts and provenance.

How to Reproduce

From the project root, rerun the harness and select the same model directory and options recorded in ../run_manifest.json.

python3 -m quant_eval

This run used fixtures: ../fixtures/golden_oracle_fixtures_v7_21__sha256_6d71a0b9147c.json and wrote results to: ../comparison_results_v7_21_Mistral_Nemo_Instruct_2407_AWQ_20260529_152601_20260629_184555.csv.