HOW_TO_READ.md

HOW TO READ A SCORE

Traceability. Every JSON record is keyed by video = a dataset-relative path, e.g. dense/single_arm/output/multiview/ctrlworld/makovian/episode_000255/pred_all_views.mp4. That is the exact path in the source dataset doanh25032004/video_gen_physics — join a score straight back to its clip by that string. No absolute/machine paths appear anywhere. traces.npz arrays are keyed <clip_id>/<metric>; traces_index.jsonl maps video → clip_id.

Two levels. Start at the top; drop down only when you want detail.

1. The readable score — scored.jsonl (one record per clip)

This is the file to read. Each line is one generated clip with a 1–5 severity and the exact frames that go wrong. You do not need to understand the 31 raw metrics for this.

{
  "episode": "episode_010407",
  "physics_severity": 5,          // 1 clean · 2 minor · 3 noticeable · 4 severe · 5 broken
  "verdict": "broken",
  "headline": "joint_limit",       // which failure dominates this clip
  "severity_by_metric": { "jerk": 4, "rigidity": 3, "joint_limit": 5, "teleport": 1 },
  "values": { "jerk_mps3": 31.5, "rigidity_worst_bone_mm": 17.8,
              "limit_violation_frac": 0.72, "no_teleport_frac": 0.0 },
  "n_violation_events": 23,
  "violations": [                  // the "tập violate" — [frameA, frameB] ranges
    {"metric": "jerk", "frames": [19, 31], "peak_frame": 28, "peak_val": 91.4, "severity": 5},
    {"metric": "rigidity", "frames": [40, 44], "peak_frame": 41, "peak_val": 26.0, "severity": 4}
  ]
}

Read it as: "clip is broken (5); the dominant problem is joint_limit (joints exceed the URDF limits in 72% of frames); on top of that there are 23 motion-glitch events — the worst is a jerk spike peaking at frame 28. Frames 19–31 and 40–44 are where it visibly goes wrong." That answers "which frames hallucinate, and how badly."

How severity is set (the "1..5, scale later" the ask called for — thresholds are in read_score.py, change them and re-run to rescale):

• jerk — ratio vs the same cell's GT median jerk (band_jerk_mps3_from_gt in the summary): <1×→1, <1.5×→2, <2×→3, <3×→4, ≥3×→5. GT is the healthy reference, so this is "how much jerkier than real".
• rigidity (worst-bone mm): <5→1, <10→2, <20→3, <35→4, ≥35→5 (arm stretching / rubber-band).
• joint_limit (fraction of frames a joint is past its URDF limit): 0→1, <0.05→2, <0.15→3, <0.30→4, ≥0.30→5.
• teleport (fraction of frames with a physically impossible jump): 0→1 … ≥0.1→5.
• physics_severity = max of the above (one bad axis is enough to break a clip).

2. The cell rollup — score_summary.json (one per cell)

{ "band_jerk_mps3_from_gt": 13.2, "n_clips": 128, "median_severity": 2,
  "severity_histogram": {"1":34,"2":52,"3":29,"4":12,"5":1}, "pct_with_violation": 99.2,
  "worst_5": [ {"episode":"episode_010407","severity":5,"headline":"joint_limit","n_violations":23}, ... ] }

severity_histogram = how many clips at each level; worst_5 = go look at these first.

3. Raw numbers, if you need them

• results.jsonl — all 31 metric scalars per clip (null = not measured, never 0). See METHODS.md.
• traces.npz — per-frame arrays keyed <clip_id>/<metric>; traces_index.jsonl maps clip path → clip_id. This is what violations are extracted from.

Quick start

import json
rows = [json.loads(l) for l in open("dense/single_arm/output/multiview/ctrlworld/makovian/pred/scored.jsonl")]
broken = [r for r in rows if r["physics_severity"] >= 4]           # the bad clips
for r in sorted(broken, key=lambda r: -r["physics_severity"]):
    print(r["episode"], r["verdict"], r["headline"], "frames:", [v["frames"] for v in r["violations"][:3]])