tools/validation/PROBLEMS.md

# Known problems

Issues encountered with the validation pipeline, the foundation specs, or the
SimReady SDK that aren't bugs in this repo's code but affect the report
output. Each entry: **symptom → root cause → workaround / how to address →
status**.

When you hit a new issue, add it here. Keep the list short — fold resolved
items into git history once they're fixed.

---

## P1 — Profiles silently drop features that aren't registered

**Symptom.** A profile loads with far fewer features than its
`profiles.toml` declares. Concretely on the current foundations checkout:

| Profile | Declared in `profiles.toml` | Loaded by validator | Dropped feature IDs |
|---|---|---|---|
| `Robot-Body-Runnable` v1.0.0 | 5 | **1** | `FET004_ROBOT_PHYSX` v0.2.0, `FET021_ROBOT_CORE_RUNNABLE` v0.2.0, `FET022_DRIVEN_JOINTS_PHYSX` v0.1.0, `FET024_BASE_ARTICULATION_PHYSX` v0.1.0 |
| `Robot-Body-Isaac` v1.0.0 | 6 | 2 | `FET004_ROBOT_PHYSX` v0.2.0, `FET021_ROBOT_CORE_ISAAC` v0.2.0, `FET022_DRIVEN_JOINTS_ISAAC` v0.1.0, `FET024_BASE_ARTICULATION_PHYSX` v0.1.0 |
| `Robot-Body-Neutral` v1.0.0 | 5 | 3 | `FET022_DRIVEN_JOINTS_NEUTRAL` v0.1.0, `FET024_BASE_ARTICULATION_NEUTRAL` v0.1.0 |
| `Prop-Robotics-Neutral` v2.0.0 | 6 | 5 | `FET006_BASE_MDL` v0.1.0 |

What's actually registered for the FET02x family (after `repo
usd_profiles_codegen`):

- `fet_021_robot_core` v0.2.0 (no variant suffix)
- `fet_022_driven_joints` v0.1.0
- `fet_023_robot_materials` v0.1.0
- `fet_024_base_articulation` v0.1.0

And for FET004/FET006 the only `_BASE_NEUTRAL` / `_BASE_USDPREVIEW`
forms are registered, not the `_ROBOT_PHYSX` / `_BASE_MDL` variants the
profiles ask for.

**Root cause.** Profile entries in
`<foundations>/nv_core/sr_specs/docs/profiles/profiles.toml` reference
variant feature IDs that the local foundations registry doesn't ship — e.g.
profiles ask for `FET021_ROBOT_CORE_RUNNABLE`, `FET022_DRIVEN_JOINTS_PHYSX`,
`FET004_ROBOT_PHYSX`, `FET024_BASE_ARTICULATION_NEUTRAL`. What is actually
registered (after `repo usd_profiles_codegen` runs against our checkout) is
only the unsuffixed `_BASE_NEUTRAL` / `_BASE_USDPREVIEW` / `_CORE` /
`_BASE_ISAACSIM` variants. The variant features the profiles ask for are
silently absent.

`omni.asset_validator.ProfileRegistry` resolves features by ID + version and
silently drops any it can't find. There's no warning emitted at profile
load time.

**Verified the gap is local, not universal.** Sample reports from another
SimReady environment (Fanuc content, sr9iar.usd.usd) show all five
declared `Robot-Body-Runnable` features actually loaded —
`FET001_BASE_NEUTRAL` (failed), plus `FET004_ROBOT_PHYSX`,
`FET021_ROBOT_CORE_RUNNABLE`, `FET022_DRIVEN_JOINTS_PHYSX`,
`FET024_BASE_ARTICULATION_PHYSX`, `FET024_BASE_ARTICULATION_NEUTRAL`
(passed). So the variants *are* registrable; our local codegen output
just doesn't include them.

**Where the variants live.** The foundation source ships JSON variant
definitions next to each feature's markdown. For FET004:

```
docs/features/FET_004-simulate_multi_body_physics.md           (md, _BASE_NEUTRAL only)
docs/features/FET_004_base_neutral-0.1.0-...json
docs/features/FET_004_base_physx-0.1.0-...json
docs/features/FET_004_base_physx-0.2.0-...json
docs/features/FET_004_robot_physx-0.1.0-...json    <- this is the one profiles ask for
docs/features/FET_004_robot_physx-0.2.0-...json
```

Each JSON declares an `id` (e.g. `FET004_ROBOT_PHYSX`), `version`, `path`,
and a `requirements` list. **Our `repo usd_profiles_codegen` run only
parses the `.md` files** — the generated
`_build/python/omni/capabilities/_features.py` contains zero
`_ROBOT_PHYSX` / `_BASE_PHYSX` / `_RUNNABLE` entries even though the JSON
sources are right there.

**How to address.** Three paths, in priority order:

1. **Codegen — pick up the JSON variant files.** The foundations
   `repo usd_profiles_codegen` tool needs to also enumerate
   `FET_*_*-*-...json` files in the features directory and emit Feature
   entries for each, not just the `Internal ID` declared in the `.md`.
   This closes the gap without changing source data — the variants
   already exist.
2. **Validator — emit a warning when a profile entry doesn't resolve.**
   `omni.asset_validator.ProfileRegistry` silently drops missing
   features. A `WARNING:` log at profile-load time would have made this
   bug visible from day one.
3. **Documentation** — once #1 lands, `repo.toml`'s codegen description
   should call out that JSON variants are part of the source of truth so
   downstream tooling doesn't repeat the mistake.

Open this with the SimReady foundations team
(`#omni-simready` / `#simready-next-support`); cite the discrepancy
between our local codegen output and the working Fanuc environment
sample.

**In-repo workaround.** The dashboard now surfaces the coverage gap
explicitly: when a profile declares more features than the validator
loaded, a banner at the top lists the missing IDs and tells the user which
ones got silently dropped. The validation result the report shows is
genuinely against the *loaded* feature set — running with
`Robot-Body-Runnable` does *not* check the four declared-but-missing
features.

**Status.** Local patch applied — see "Local patch applied" below.
Upstream codegen fix still needed. Banner workaround still in place
for environments without the patch.

### Local patch applied

A patch lives in
`plugins/simready-report/skills/simready-report/validate.py` —
function `_patch_register_json_variant_features()`. It runs at module
load (so worker processes pick it up too) right after
`import omni.asset_validator as oav`. It is loud about itself: every
run prints a `[PATCH P1] FeatureRegistry: +N JSON-variant feature(s)
... ProfileRegistry: rebuilt feature lists for M profile(s) ...`
line so it's never invisible.

**Concretely on the Yaskawa-local checkout** the patch lifts:
- FeatureRegistry: 11 features → 30 features (+19 from JSON variants).
- Robot-Body-Runnable v1.0.0: 1 feature / 8 requirements → **5 / 41**.
- 11 profiles got their `.features` lists rebuilt; +24 feature
  references appended in total.

**Exactly what the patch does**, step by step:

1. **Discover variant JSON files.** Globs
   `<foundations>/nv_core/sr_specs/docs/features/FET_*.json`. The
   filename styles in this repo are inconsistent
   (`FET_021-robot_core_isaac-0.1.0.json`,
   `FET_021_robot_core_runnable_0.2.0.json`,
   `FET_004_base_physx-0.2.0-simulate_multi_body_phyics.json`), so we
   ignore the filename and read the `id` + `version` fields from the
   JSON content.

2. **Build a Requirement code lookup.** `r.code -> r` for every
   `Requirement` in `omni.asset_validator.RequirementsRegistry()`.
   The JSON variants list dotted code strings like `"RB.COL.003"`,
   `"DJ.001"`, `"BA.002"` — those are looked up here.

3. **Construct a Feature-protocol object per JSON variant.**
   `omni.asset_validator._features.Feature` is a `typing.Protocol`
   declaring `id: str, version: str, path: str, requirements: list`.
   We use a frozen dataclass that satisfies the protocol structurally:

   ```python
   @dataclass(frozen=True)
   class _PatchedFeature:
       id: str
       version: str
       path: str
       requirements: list
   ```

4. **Skip duplicates and unresolvable variants.** If
   `FeatureRegistry.find(fid, fver)` already returns something, skip
   (don't double-register). If any of the JSON's requirement code
   strings doesn't resolve in the code lookup, skip the whole feature
   and record it in `unresolved_codes`. (Currently zero unresolved on
   our checkout — every JSON variant's required codes exist.)

5. **Register via `FeatureRegistry().add(feat)`.** The internal
   `create_key` uses `.id` + `.version` so our dataclass satisfies it.

6. **Re-resolve every profile's feature list — the critical step.**
   `Profile` is a frozen dataclass, so we can't reassign
   `profile.features`. But the field is a *mutable list*, and the
   freeze only prevents field reassignment, not mutation of held
   collections. So for every entry in `profiles.toml`'s declared
   features that isn't already in the live profile, we look it up in
   the now-fuller `FeatureRegistry` and `profile.features.append(...)`
   in place. **Without this step the patch is invisible** — adding to
   `FeatureRegistry` after profiles are already built doesn't
   retroactively join existing Profile objects.

**What the patch does NOT touch**: foundation source files, the
codegen tool, generated `_features.py`, profile definitions in
`profiles.toml`, or any rule code. The local patch is purely in our
validation runner.

**Remove the patch when**: the foundation `repo usd_profiles_codegen`
tool is updated to enumerate JSON variant files alongside the
markdown. Once `_features.py` carries every variant on its own,
delete `_patch_register_json_variant_features()` and its call site.

---

## P2 — Public ur10 sample asset fails current foundation specs

**Symptom.** Running `/simready-report` against the public ur10 sample
in
`simready_foundations/sample_content/common_assets/robots_general/ur10/`
(release `2026.04.0`, commit `805d2c5`) reports failures on every
variant when validated against its declared profile (or against
Robot-Body-Runnable cross-profile). Concretely, against each interface
USD's declared profile, with `--use-kit` so PhysX rules actually run:

| Variant interface | Declared profile | Result | Failing requirements |
|---|---|---|---|
| `simready_usd/ur10.usda` | Robot-Body-Neutral v1.0.0 | **PASS** | — |
| `simready_physx_usd/ur10.usda` | Robot-Body-Physx v1.0.0 | **FAIL** | (profile evaluates 0 features against the asset; `features_summary` empty in JSON output) |
| `simready_isaac_usd/ur10.usda` | Robot-Body-Isaac v1.0.0 | **FAIL** | `RC.005` (`VerifyRobotPhysicsAttributesSourceLayer`), `RC.008` (`robot-type`), `RC.009` (`root-joint-pinned`), `DJ.010`, `ISA.001` |

Under the cross-profile `Robot-Body-Runnable` run, Neutral and PhysX
variants additionally light up with `RC.007` (`robot-schema` — no
`IsaacRobotAPI` on the default prim) because neither of those variants
applies `IsaacRobotAPI`, only the Isaac variant does.

**Root cause.** Asset-side metadata gaps and layer-organization issues
in the shipped public sample, *not* validator or skill code. The asset
file is tracked by Git LFS at the same OID as the release (`b30f9c1b…`
for `simready_isaac_usd/payloads/base.usda`), so no in-tree edit
caused this. Each asset's `customLayerData.SimReady_Metadata.validation`
records `validated_features` against a snapshot dated **2026-01-09**;
between then and the current `805d2c5` foundation specs the rules in
question were tightened (or added), so the asset that passed in
January no longer passes today. Specifically:

- **`RC.008` (`robot-type`)** — `simready_isaac_usd/payloads/base.usda`
  lines 81-83 declare `token isaac:robotType` with **no value**. The
  current rule requires a valid schema-defined token (e.g.
  `"Manipulator"`, `"End Effector"`); the placeholder `"Default"` and
  the absent-value case both fail.
- **`RC.009` (`root-joint-pinned`)** — depends on `isaac:robotType`
  to decide whether the root joint must be pinned. The asset's
  `root_joint` *is* a `PhysicsFixedJoint` with empty `physics:body0`
  (pinned to world, correct for a Manipulator), but without a valid
  `robotType` value the rule can't evaluate and fails.
- **`RC.005` (`VerifyRobotPhysicsAttributesSourceLayer`)** — physics
  attributes (`physics:axis`, `physics:mass`, `physics:centerOfMass`,
  `physics:diagonalInertia`, `physics:principalAxes`) on each robot
  link are authored in *both* `payloads/base.usda` and
  `payloads/Physics/physics.usda`. The current rule wants them in the
  physics payload only.
- **PhysX variant empty `features_summary`** — Robot-Body-Physx v1.0.0
  evaluates zero features against the asset, suggesting the profile's
  declared feature set no longer matches what the asset has stamped
  (related to P1, but for a different feature family).

**How to address.** Asset fixes belong in
`github.com/NVIDIA/simready-foundation`, not here:

1. **`RC.008`/`RC.009`** — *verified one-liner*. In
   `simready_isaac_usd/payloads/base.usda` line 81, change
   `token isaac:robotType (…)` to
   `token isaac:robotType = "Manipulator" (…)`. Verified locally with
   `--use-kit`:
   - Against `Robot-Body-Isaac`, `FET021_ROBOT_CORE_ISAAC` failing
     codes went from `[RC.005, RC.008, RC.009]` →
     **`[RC.003, RC.005]`** (RC.008 and RC.009 cleared; RC.003 was
     previously cascade-suppressed and now surfaces).
   - Against `Robot-Body-Runnable` cross-profile, the **Isaac variant
     now PASSES** (`FET021_ROBOT_CORE_RUNNABLE` only requires
     RC.007/008/009 — all three OK once `isaac:robotType` is set,
     because IsaacRobotAPI is already applied with the correct
     joints/links relationships and the root joint is pinned via
     empty `physics:body0`). Cross-profile summary moved from
     `0 passed / 3 failed` → `1 passed / 2 failed` (Isaac passes,
     Neutral and PhysX still fail RC.007/008/009 because they don't
     apply IsaacRobotAPI at all).
2. **`RC.005`**: move every `physics:*` attribute currently authored
   on links in `payloads/base.usda` into `payloads/Physics/physics.usda`
   only (or refactor so `base.usda` carries no physics attributes at
   all). This is a meaningful layer-organization change, not a
   one-liner. Still failing after fix 1 (88 RC.005 issues remain).
3. **`RC.003` (`RobotNaming`)** — surfaced *only after* fix 1. The
   rule compares the **immediate folder name** (`simready_isaac_usd`)
   to the **interface USD stem** (`ur10`). Under the standard SimReady
   packaging convention these never match — every Isaac variant of
   every SimReady asset fails RC.003. Likely a rule bug (should
   probably look at the *bundle* parent, e.g. `ur10/`, not the
   variant folder) or a misalignment between rule and packaging spec.
   Worth raising upstream.
4. **`RC.007` (cross-profile only)**: if the intent is that the
   Neutral and PhysX variants should also satisfy Robot-Body-Runnable,
   apply `IsaacRobotAPI` to their default prims with the same
   `isaac:physics:robotJoints`/`robotLinks` relationships used in the
   Isaac variant. If the intent is that each variant only ever
   validates against its own declared profile, this is not needed.
5. **PhysX empty features**: align the variant USDs and Robot-Body-Physx
   profile's expected feature set — likely needs the assets to carry
   the variant feature IDs that the profile declares (intersect with
   P1's missing-feature list once that's resolved). Also: `validate.py`
   currently emits an empty `.reports/<…>/` dir when zero features
   match (no `index.html` / `results.json` written). Tracked
   separately as a `validate.py` robustness issue.

**In-repo workaround.** None — this is downstream of the validator. The
`simready-report` skill faithfully reports what the rules say. The
dashboard's "Failing requirements" panel surfaces the offending codes
so users can map each failure back to its requirement doc.

**Status.** Open against the foundations repo. Filed for tracking via
this MR; no action in `simready-explorer` is needed beyond surfacing
the issue here. Fix 1 above is **verified to work locally** (one-line
edit to `base.usda`) — upstream PR against `NVIDIA/simready-foundation`
is the right channel to land it. Fixes 2–5 still open. Re-validate
after the foundation team updates the sample assets, the rule code
(notably `RC.003 RobotNaming`), or relaxes the unmatched-feature
behavior.

---

## How to file a new entry here

Use this skeleton:

```
## P<N> — <one-line title>

**Symptom.** What the user/reader sees that's wrong.

**Root cause.** Where the bug actually lives (which repo / file / step).

**How to address.** Concrete fix path(s), upstream or local.

**In-repo workaround.** What this repo does today to mitigate, if anything.

**Status.** Open / mitigated / fixed (link to commit).
```

Bump the number; don't reuse retired ones — the git history is the
canonical record once an entry is resolved and removed.