data/analytical/SCHEMA.md

# Analytical Dataset Column Schema

Produced by `roverdevkit.surrogate.dataset.build_dataset` from
`LHSSample`s generated by `roverdevkit.surrogate.sampling.generate_samples`.
Each row is **one** `(design, scenario, soil)` triple evaluated by
`roverdevkit.mission.evaluator.evaluate_verbose`, flattened into a
single Parquet row.

- **Schema version:** `v9` (see `dataset.SCHEMA_VERSION`). Scientific
  payload is an explicit mission requirement, carried by the
  scenario-side inputs `scenario_payload_mass_kg` and
  `scenario_payload_power_w`. Payload mass enters the total vehicle
  mass as a line item outside the AIAA S-120A dry-mass growth margin
  (`m_total = m_dry + m_margin + m_payload`); payload power adds to the
  continuous ops-time electrical load (alongside avionics) and to the
  hot-case thermal dissipation. Both are sampled uniform and
  family-agnostic (`payload_mass_kg` in `[0, 30]`,
  `payload_power_w` in `[0, 30]`) so the entire webapp Mission-Inputs
  slider range is in-distribution. Payload lives on `MissionScenario`
  (a requirement set by the mission), not on `DesignVector` (a variable
  the designer trades); the design vector is 11-D. A per-call override
  on `evaluate` / `/evaluate` / `/predict` lets callers substitute a
  specific payload.
- **Fidelity level (this file):** `analytical` — the Bekker-Wong
  terramechanics path solved inside `traverse_sim.run_traverse`.
- **Canonical filename:** `lhs_v9.parquet` — the current training set,
  40k rows at 10k × 4 scenario families. Pilot (`lhs_pilot.parquet`)
  and challenge (`challenge_v1.parquet`) files are generated on demand
  from `scripts/build_dataset.py`; only the canonical training set is
  treated as a tracked artifact.

Dataset-level metadata is written to the Parquet file's schema footer;
use `read_parquet_metadata(path)` to recover it (seed, sampler version,
scenario families, val/test fractions, UTC build timestamp, evaluator
version, free-form notes).

## Column groups

Prefix conventions:

- `design_*` — inputs from the 11-D `DesignVector`.
- `scenario_*` — inputs from the `MissionScenario`, plus the sampler's
  jittered Bekker soil parameters (`scenario_soil_*`).
- `stat_*` — aggregate statistics (mean / p95 / max / final) reduced
  from the per-step `TraverseLog` time series.
- Unprefixed columns with physical units (e.g. `range_km`) — targets
  from `MissionMetrics`.
- Otherwise — dataset metadata.

### Dataset metadata (5 columns)

| Column | dtype | Description |
| --- | --- | --- |
| `sample_index` | int64 | Monotonic row id from the sampler. Stable across re-runs with the same seed. |
| `split` | category | `train` / `val` / `test`, assigned at sample time with a deterministic RNG independent of row ordering. |
| `stratum_id` | int | `0 = 4-wheel`, `1 = 6-wheel`. Matches `design_n_wheels`. |
| `fidelity` | category | `analytical` for this file — the Bekker-Wong terramechanics path. No separate fidelity tier is shipped. |
| `status` | category | `ok` if evaluator succeeded, else the exception class name (e.g. `ValueError`). Numeric target columns are NaN on non-`ok` rows; boolean targets are `False`. |

### Design vector (11 columns)

All `design_*` columns mirror the `DesignVector` pydantic schema.

| Column | dtype | Range | Description |
| --- | --- | --- | --- |
| `design_wheel_radius_m` | float64 | [0.05, 0.20] | Wheel radius R |
| `design_wheel_width_m` | float64 | [0.03, 0.20] | Wheel width W |
| `design_grouser_height_m` | float64 | [0.0, 0.020] | Grouser height |
| `design_grouser_count` | int64 | [0, 24] | Number of grousers per wheel |
| `design_n_wheels` | int64 | {4, 6} | Wheel count (kept in sync with architecture) |
| `design_mobility_architecture` | category | `rigid_4wheel`, `rocker_bogie_6wheel` | Primary mobility-architecture trade in the evaluator/optimizer. Not yet present in the shipped `lhs_v9.parquet` surrogate training set; the surrogate still keys off `design_n_wheels` until the dataset is rebuilt. |
| `design_chassis_mass_kg` | float64 | [0.5, 50.0] | Dry chassis mass (structural chassis only) |
| `design_wheelbase_m` | float64 | [0.3, 1.2] | Wheelbase |
| `design_solar_area_m2` | float64 | [0.1, 1.5] | Solar array area |
| `design_battery_capacity_wh` | float64 | [5.0, 500.0] | Usable battery energy |
| `design_avionics_power_w` | float64 | [5.0, 40.0] | Continuous avionics draw |
| `design_peak_wheel_torque_nm` | float64 | [0.05, 20.0] | Per-wheel hub torque capacity. Cruise speed is derived inside the evaluator from torque + slip + power balance (see `roverdevkit/drivetrain/motor.py::cruise_speed`). LHS is log-uniform around a per-row anchor rather than uniform on these bounds — see `roverdevkit/surrogate/sampling.py::_peak_wheel_torque_anchor_for_row`. |

### Scenario inputs (18 columns)

Family-fixed columns (`scenario_family`, `scenario_terrain_class`,
`scenario_soil_simulant`, `scenario_sun_geometry`,
`scenario_traverse_distance_m`) take one of four canonical values per
family. The remaining columns are jittered per sample.

| Column | dtype | Notes |
| --- | --- | --- |
| `scenario_family` | category | One of `equatorial_mare_traverse`, `polar_prospecting`, `highland_slope_capability`, `crater_rim_survey`. Use for per-scenario accuracy breakdown. |
| `scenario_name` | category | Mirrors `scenario_family` in this dataset (validation-only scenarios live elsewhere). |
| `scenario_latitude_deg` | float64 | Family-specific range; see `sampling.FAMILIES`. |
| `scenario_traverse_distance_m` | float64 | Family-fixed, non-binding — deliberately above the energy-/duty-limited reach so `range_km` stays a continuous signal instead of saturating at a distance cap. |
| `scenario_terrain_class` | category | `mare_nominal`, `mare_loose`, `highland_dense`, `polar_regolith`. |
| `scenario_soil_simulant` | category | Family nominal; the *actual* Bekker numbers used by the evaluator are the `scenario_soil_*` columns below. |
| `scenario_mission_duration_earth_days` | float64 | Family-specific range. |
| `scenario_max_slope_deg` | float64 | Family-specific range. |
| `scenario_operational_duty_cycle` | float64 | Drive duty cycle the rover would actually run on the ground — sets `δ_eff = clamp(δ_ops, 0, 0.6)` in the traverse loop. Sampled per row uniform on `[0, 0.6]` independently of family, so the surrogate keys off it as a true continuous input. The per-family default is kept on `ScenarioFamily` for canonical YAML / UI initial slider position. |
| `scenario_sun_geometry` | category | `continuous` / `diurnal` / `polar_intermittent`. |
| `scenario_soil_n` | float64 | Bekker sinkage exponent, jitter bounds [0.8, 1.2]. |
| `scenario_soil_k_c` | float64 | Cohesive modulus, [0.5, 2.0] kN/m^(n+1). |
| `scenario_soil_k_phi` | float64 | Frictional modulus, [400, 1200] kN/m^(n+2). |
| `scenario_soil_cohesion_kpa` | float64 | Soil cohesion, [0.1, 1.0] kPa. |
| `scenario_soil_friction_angle_deg` | float64 | Internal friction angle, [30, 50]°. |
| `scenario_soil_shear_modulus_k_m` | float64 | Janosi-Hanamoto K, [0.010, 0.025] m. |
| `scenario_payload_mass_kg` | float64 | Scientific-payload mass (mission requirement). Per-row LHS feature uniform on `[0, 30]` independently of family. Added to total vehicle mass as a line item outside the dry-mass growth margin; the per-scenario default is kept on the YAML / `ScenarioFamily` for canonical webapp slider position. |
| `scenario_payload_power_w` | float64 | Scientific-payload continuous ops-time power (mission requirement). Per-row LHS feature uniform on `[0, 30]`. Added to the continuous electrical load (alongside avionics) in the traverse budget and to the hot-case thermal dissipation. |
| `scenario_required_obstacle_height_m` | float64 | Minimum traversable obstacle/step height (m). Defaults to 0 on the canonical smooth-regolith scenarios. Evaluator-only today: obstacle metrics (`obstacle_capability_m`, `obstacle_margin_m`) are computed from `mobility_architecture` and wheel radius; the surrogate does not yet predict them. |

### Mission-metric targets (8 columns)

Mirror `MissionMetrics` fields. `range_km` and `energy_margin_raw_pct`
are the primary regression targets (no saturation); `*_pct` and the
boolean flag are secondary reporting/classification targets.
`thermal_survival` is **not** in this group: the evaluator still
computes it as a diagnostic, but the mass model treats RHU power and
MLI quality as free, so it reduces to a near-trivial gate with no real
design trade-off and the surrogate does not consume or predict it.

| Column | dtype | Notes |
| --- | --- | --- |
| `range_km` | float64 | Energy-feasible mission range. `run_traverse` applies an in-traverse throttle that drops effective duty when the battery floors and load exceeds solar. |
| `energy_margin_pct` | float64 | Clipped 0-100, SOC-based reporting metric. |
| `energy_margin_raw_pct` | float64 | Unclipped mission-integrated `(E_in - E_out)/E_out × 100`; primary surrogate target. |
| `slope_capability_deg` | float64 | Max climbable slope on this soil. |
| `total_mass_kg` | float64 | Mass-model output. |
| `peak_motor_torque_nm` | float64 | Observed peak wheel torque during traverse. |
| `sinkage_max_m` | float64 | Observed peak sinkage during traverse. |
| `stalled` | bool | Single feasibility classifier target (1 = infeasible). Captures whether the rover failed the slip-balance solve at any traverse step (Brent solver could not find a slip that satisfied force balance under the available drawbar pull and torque envelope). |

Evaluator-only architecture metrics (present on live `/evaluate` and optimizer outputs, not in the shipped `lhs_v9.parquet` targets):

| Column | dtype | Notes |
| --- | --- | --- |
| `obstacle_capability_m` | float64 | Estimated max traversable obstacle height from architecture proxy ($h_{\mathrm{obs}} = k_{\mathrm{arch}} R$). |
| `obstacle_margin_m` | float64 | Capability minus `scenario_required_obstacle_height_m`. |
| `obstacle_requirement_met` | bool | Whether `obstacle_margin_m \ge 0`. |
| `architecture_mass_kg` | float64 | Rocker-bogie suspension/linkage mass charged in the bottom-up mass model. |

### Traverse-log aggregate statistics (≥24 columns)

Reduced from the per-step `TraverseLog` time series. Used to measure
where the surrogate needs to be accurate (peak-load versus
steady-state regimes) and as auxiliary diagnostics for the baselines.

Numeric aggregates (mean / p95 / max over the whole traverse, absolute
value for signed quantities like slip and torque):

- `stat_power_in_{mean,p95,max}_w` — solar input power.
- `stat_power_out_{mean,p95,max}_w` — total electrical draw (mobility + avionics).
- `stat_mobility_power_{mean,p95,max}_w` — mobility subsystem draw alone.
- `stat_slip_{mean,p95,max}` — wheel slip magnitude in [0, ~0.95].
- `stat_sinkage_{mean,p95}_m` — peak is already `sinkage_max_m` above.
- `stat_wheel_torque_{mean,p95}_nm` — peak is already `peak_motor_torque_nm` above.
- `stat_sun_elevation_{mean,max}_deg` — degrees above horizon.
- `stat_soc_final` / `stat_soc_min` — end-of-mission and deepest SOC.

Boolean end-of-run flags:

- `stat_rover_stalled` — Brent slip solve failed at some step.
- `stat_battery_floored` — SOC hit the 15% DoD floor during the run.
- `stat_reached_distance` — the rover reached the scenario's (non-binding) distance budget.

Categorical:

- `stat_terminated_reason` — a free-form short string from the sim layer
  (e.g. `"mission_duration"`, `"evaluator_error"`). Use as a
  post-hoc diagnostic; not suitable as a model input.

## Layer-1 registry sanity scope

`roverdevkit.surrogate.baselines.predict_for_registry_rovers` produces
a `registry_sanity.csv` artifact with one row per `(rover, algorithm,
target)` tuple plus an `is_primary` boolean. The split is enforced
by the `LAYER1_PRIMARY_TARGETS` / `LAYER1_DIAGNOSTIC_TARGETS`
constants in `roverdevkit/surrogate/baselines.py`:

- **Primary (is_primary=True):** `total_mass_kg`,
  `slope_capability_deg`, `stalled`. Design-axis metrics where the LHS
  bounds put every flown / design-target rover inside the surrogate's
  training support. Treated as the main registry sanity set.
- **Diagnostic (is_primary=False):** `range_km`,
  `energy_margin_raw_pct`. Both are scenario-OOD for the registry:
  Pragyan ≈ 100 m, Yutu-2 ≈ 25 m / lunar day, MoonRanger and
  Rashid-1 ≈ 1 km published mission distances against LHS family
  budgets of 20–80 km (intentionally non-binding so `range_km` stays
  a continuous training signal). The relative error for these
  targets is dominated by the absolute-scale mismatch and reflects
  scenario-OOD rather than a surrogate-calibration failure. Reported
  for transparency only.

Slope MAPE on Pragyan and MoonRanger runs elevated relative to mass:
published rover slope-capability specs come from real-rover-specific
design choices the analytical Bekker-Wong kernel's feature space cannot
fully resolve.

## Column count sanity

Metadata (5) + design (11) + scenario (18) + metrics (8) + stats (≥24)
= ≥66 columns at `SCHEMA_VERSION = v9`. Future versions appending,
removing, or re-binding columns — *or* changing the LHS support so a
surrogate trained on one version would be OOD on the next — *must* bump
`SCHEMA_VERSION` so downstream code can detect a mismatch.