tools/validation/README.md

# Validation reporting — staging branch

## Install the Claude Code plugin

The `simready-report` plugin ships as a Claude Code marketplace under
`validation/.claude-plugin/`. From a checkout of this repo, run inside Claude Code:

```
/plugin marketplace add <path-to-repo>/validation
/plugin install simready-report@simready-playbook
```

That registers two slash commands: `/simready-report` and `/simready-package`.
You still need a Python with the SimReady runtime deps and the foundations + sdk
checkouts on disk — see [To run the demo as-is](#to-run-the-demo-as-is) below.

> [!IMPORTANT]
> ## Read [`PROBLEMS.md`](PROBLEMS.md) before trusting any report
>
> Reports produced by this prototype lean on a **local patch** for
> [PROBLEMS.md P1](PROBLEMS.md): without it, every Robot-Body profile
> silently drops 4 of its 5 declared features at load time, because
> the foundation `repo usd_profiles_codegen` step doesn't read the
> JSON variant definitions next to each feature's markdown. A
> "passing" report against `Robot-Body-Runnable` would only have
> checked 1 feature out of 5.
>
> The patch lives in
> `plugins/simready-report/skills/simready-report/validate.py` and is
> loud about itself — every run prints a `[PATCH P1]` line. With the
> patch active, `Robot-Body-Runnable` resolves to **5/5 features, 41
> requirements** instead of 1/8.
>
> The dashboard's **Caveats** panel still surfaces the gap if anyone
> runs without the patch.
>
> [`PROBLEMS.md`](PROBLEMS.md) **P1** has the full root cause, the
> patch walkthrough, and the upstream fix path. The patch is meant to
> be temporary — once foundations codegen is fixed, delete
> `_patch_register_json_variant_features()` and its call site.

---

> **Status: demo / review staging.** Not part of `simready-explorer` proper yet.
>
> This directory is a verbatim copy of the prototype work currently living at
> [`loginowskid/simready-playbook`](https://github.com/loginowskid/simready-playbook),
> dropped here on the `dev/dloginowski/validation-reporting` branch so the
> SimReady team can review the artifact before we refactor it into the repo's
> existing library + skills patterns.

## What this is

A Claude-Code-driven validation reporting pipeline for SimReady customer assets.
On top of the existing `omni.asset_validator` engine and SimReady profile registry,
it adds:

- **HTML dashboard** with summary cards, per-asset feature pass/fail, an
  asset filter, and a "Other failed requirements" panel for issues outside the
  loaded profile.
- **External-dependencies provenance report** — walks USD composition arcs per
  asset, classifies each layer as internal/external relative to the target,
  records actions taken to obtain each.
- **Foundation doc rendering** — renders the markdown source for every
  feature/requirement code referenced by the report into HTML inside the output.
- **Auto-packaging convenience** — for a raw asset bundle under
  `assets_to_validate/<name>/`, runs `simready ingest usd` per interface USD
  before validating against the packaged tree.
- **Spec selector** — `--list-profiles` enumerates registered profiles so the
  user can pick a SimReady profile interactively.
- **Parallel asset validation** — `ProcessPoolExecutor` workers, default auto.

Two slash commands ship together as a single Claude Code plugin:

- `/simready-report` — validate + dashboard + provenance + docs.
- `/simready-package` — standalone packaging (no validation).

## Layout (mirrors the source repo)

```
validation/
├── bootstrap.ps1                    # Windows convenience installer
├── marketplace.json                 # Claude Code marketplace manifest
├── plugins/simready-report/
│   ├── plugin.json
│   └── skills/
│       ├── simready-report/         # HTML dashboard + validation
│       └── simready-package/        # standalone packaging
└── playbooks/
    └── foundations-deviations.md    # observed gaps between specs and asset_validator
```

## Why it doesn't fit the repo's existing patterns yet

`simready-explorer` already ships `source/libraries/validate/` with a clean
public API (`validate_asset`, `validate_asset_list`, etc.) and three skills in
the `agentskills.io/specification` format (`validate-asset`, `validate-batch`,
`validate-metadata`). The work in this `validation/` directory:

- Reinvents a thinner version of `simready.validate.validate_asset_list` —
  this is duplication that should be removed during integration.
- Uses a different skill format (Claude Code marketplace plugin) — should be
  rewritten as `agentskills.io` skills that call into the library.
- Is a standalone-script architecture (`python validate.py …`) — should be
  exposed via the existing `repo.bat` / library entry points.

## Proposed integration after review

1. New skill `validate-report` under
   `source/libraries/validate/src/validate/skills/validate-report/` —
   instructional doc that drives a new `simready.validate.report` module.
2. New module `source/libraries/validate/src/validate/report/` containing the
   dashboard renderer, doc renderer, external-deps tracker, thumbnail mirrorer
   — extracted from `report.py` + `external_deps.py` and adapted to consume
   `AssetValidationResult` instead of the raw `omni.asset_validator` issue
   stream.
3. Optional new skill `validate-package` — thin wrapper around
   `simready ingest usd` (independent of validation).
4. Drop `bootstrap.ps1` / `marketplace.json` / Claude-Code-plugin layout — not
   relevant here; the team uses `repo.bat` + `_build/` flow.

## To run the demo as-is

The scripts in this directory are self-contained and need a Python with the
SimReady runtime deps installed plus the foundations + sdk checkouts on disk.
Easiest path on Windows: `& bootstrap.ps1` (creates a venv at
`%LOCALAPPDATA%\simready\` and persists `SIMREADY_PYTHON` /
`SIMREADY_FOUNDATIONS_PATH` / `SIMREADY_SDK_PATH` env vars). On other systems
or custom layouts, install the deps into your Python and set those env vars
manually.

After that:

```
python plugins/simready-report/skills/simready-report/validate.py \
    <path-to-asset-bundle> --profile Robot-Body-Isaac --version 1.0.0
```

Output lands at `<bundle>/.reports/<bundle-name>.<profile>/index.html`.