Upload evidence/LADDER-REPORT.md with huggingface_hub
Browse files- evidence/LADDER-REPORT.md +76 -0
evidence/LADDER-REPORT.md
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# A100 / current-upstream ladder (2026-08-20)
|
| 2 |
+
|
| 3 |
+
Rebuilt against the encoder Lightricks published on 2026-08-17 (commit
|
| 4 |
+
`1b92891c`, *"Aligns the published encoders with the LTX-2.5 model
|
| 5 |
+
checkpoints"*). Everything measured before that date used the superseded,
|
| 6 |
+
misaligned encoder and is not comparable to what is here.
|
| 7 |
+
|
| 8 |
+
Runtime: Colab A100-SXM4-40GB (cc 8.0), torch 2.11.0+cu128, transformers 5.14.1
|
| 9 |
+
pinned, plan `r45c`, calibration `calib-large.txt`, reference regenerated on this
|
| 10 |
+
card from the current source.
|
| 11 |
+
|
| 12 |
+
## Gate — A3 wins on every axis
|
| 13 |
+
|
| 14 |
+
| | A0 `legacy` | **A3 `bypass` + group bounded 12:1** |
|
| 15 |
+
|---|---:|---:|
|
| 16 |
+
| video mean relL2 | 0.06061 | **0.05204** (−14%) |
|
| 17 |
+
| audio mean relL2 | 0.06095 | **0.04827** (−21%) |
|
| 18 |
+
| video worst | 0.14733 | **0.09187** (−38%) |
|
| 19 |
+
| audio worst | 0.14440 | **0.10573** (−27%) |
|
| 20 |
+
| video cosine | 0.99763 | **0.99848** |
|
| 21 |
+
| `\|\|Q\|\|/\|\|W\|\|` | 302.654 | **4.216** (72x) |
|
| 22 |
+
| columns zeroed | 5 | **0** |
|
| 23 |
+
| max restore gain | 469.56 | **19.00** |
|
| 24 |
+
| damping escalation | 1x | 10x |
|
| 25 |
+
|
| 26 |
+
The trade this project spent two sessions quantifying is gone. On the
|
| 27 |
+
superseded source and a V100, the structural fix cost about 6% of conditioning
|
| 28 |
+
drift; here it costs nothing and improves the worst case most.
|
| 29 |
+
|
| 30 |
+
Both figures reproduced to five decimals across two independent A100 runtimes,
|
| 31 |
+
so builds are deterministic within an architecture even though they are not
|
| 32 |
+
across architectures.
|
| 33 |
+
|
| 34 |
+
## Render prompts, against a BF16 conditioning from the same card
|
| 35 |
+
|
| 36 |
+
| # | A0 | A3 | prompt |
|
| 37 |
+
|---:|---:|---:|---|
|
| 38 |
+
| 0 | 0.05870 | **0.05073** | humanoid robot, freight elevator |
|
| 39 |
+
| 1 | 0.07487 | **0.06596** | robotic hand, cracked ceramic cup |
|
| 40 |
+
| 2 | **0.05551** | 0.08213 | four-legged inspection robot |
|
| 41 |
+
| 3 | 0.05061 | **0.04489** | dozens of small wheeled robots |
|
| 42 |
+
| 4 | 0.08887 | **0.05286** | rescue robot in smoke |
|
| 43 |
+
| mean | 0.06571 | **0.05931** | |
|
| 44 |
+
| worst | 0.08887 | **0.08213** | |
|
| 45 |
+
|
| 46 |
+
A3 is better on four of five. Prompt 2 is the exception, and it was also the
|
| 47 |
+
exception on the superseded source with a V100 - the same prompt moving the same
|
| 48 |
+
way after both the weights and the architecture changed is a signal, not noise,
|
| 49 |
+
though five prompts cannot say what it means.
|
| 50 |
+
|
| 51 |
+
## Visual
|
| 52 |
+
|
| 53 |
+
`frames/cmp-*.png` puts BF16, A0 and A3 side by side - same card, same DiT, same
|
| 54 |
+
seeds, same prompts.
|
| 55 |
+
|
| 56 |
+
* **Prompt 0 now renders an actual robot.** Dome head, two amber eyes, segmented
|
| 57 |
+
neck, panelled torso, in all three. The superseded encoder produced a
|
| 58 |
+
human-faced android for the same prompt. That improvement comes from the
|
| 59 |
+
source alignment, not from anything this project did.
|
| 60 |
+
* BF16 and A3 track each other closely - same badge colour on prompt 0, glasses
|
| 61 |
+
present on prompt 1 - while A0 differs on both.
|
| 62 |
+
* Prompt 1 still fails to a human face in all three including BF16. Model limit,
|
| 63 |
+
not compression damage.
|
| 64 |
+
|
| 65 |
+
## Artifacts
|
| 66 |
+
|
| 67 |
+
On the Hub at `topabaem/LTX-2.5-TextEncoder-nvfp4-portable` (private), pushed by
|
| 68 |
+
the build worker as each stage finished rather than downloaded afterwards:
|
| 69 |
+
`A0.packed.safetensors`, `A3.packed.safetensors`, and `evidence/` carrying the
|
| 70 |
+
gate JSON, build logs with per-layer drift and `||Q||/||W||`, the A100 BF16
|
| 71 |
+
reference, all three conditionings and all fifteen clips.
|
| 72 |
+
|
| 73 |
+
That push-as-you-go arrangement exists because three Colab runtimes died
|
| 74 |
+
mid-build, twice destroying finished artifacts. The render stage then failed
|
| 75 |
+
three times on a missing `gguf` package - and cost only the render, because the
|
| 76 |
+
builds were already safe on the Hub.
|