resoajoe/loglens-longvideo-drift

NEW: what a monitoring camera writes down is not what the light did

Two cameras, same room, 30 paired hours. One with exposure and white balance locked (physical ground truth), one on auto (deployed-realistic).

	locked (true light)	auto (what the camera wrote)
luminance dynamic range	1.89x	1.11x
warmth (R/B) dynamic range	1.64x	1.07x
gamma, A ~ L^gamma	1.0 by definition	-0.045
log luminance range removed	—	82.9%
log warmth range removed	—	~86%

The auto camera's response is inverted, not merely flattened — it writes 124.6 at night and 115.4 at midday, brighter when it is darker. And auto white balance destroys colour temperature even more thoroughly than auto exposure destroys luminance: a clean 1.068 -> 1.735 step at the lighting changeover becomes 1.008 -> 1.075, essentially nothing.

This also confirms an earlier finding against ground truth for the first time: an original 220-hour auto-exposure capture measured a 1.1x luminance range, and this pair measures 1.11x for auto while showing the underlying light actually varied by 1.89x.

The fitted response is a rendering layer: synthetic footage with physically correct illumination can be passed through it to produce what a deployed camera would actually have recorded. Without it, synthetic monitoring footage is photometrically correct and unlike any real camera's output.

Full write-up: RESULTS_exposure.md. Reproduce: code/exposure_tf.py.

---

ALSO NEW: every instrument I built this week was wrong at least once

3 of 7 sealed predictions wrong. 3 measurement instruments wrong before they were right, every one caught by a control rather than by inspection. 1 published caveat that turned out backwards. 1 tidy statistic computed and discarded as invalid. 5 pipelines that exited zero and wrote garbage. 9 hours of "the GPU is slow" that were my own leftover processes.

A metric that scored the visibly broken image higher. A harness whose headline signal was a false positive on an empty bed. An instrument invariant to a 64x change in its own subject. A known-answer test that appeared to fail and was right — it had detected an event left inside the control window.

Full write-up: RESULTS_methods.md.

---

Generated video has no stillness

source	quiescent fraction	longest continuous stillness
real fixed-camera (ordinary, barely occupied room), 50 min	0.992	1726 s (28.8 min)
real, second window	0.988	1043 s
real, third window	0.991	849 s
clinical prior, hospital inpatient	0.674	minutes
generated, frontier model clip	0.320	4 s
generated, ours	0.175	1 s
generated, ours (longer run)	0.104	3 s

Real fixed-camera footage holds perfectly still for 28.8 minutes. No generated footage measured here stays still longer than 4 seconds. Video generators train on trimmed, action-centric clips; stillness is not in the training distribution.

Pre-registered before measuring (PREREGISTRATION_eventrate.md), and two of four sealed predictions were wrong — generated quiescent fraction came in at 0.10-0.32 against my stated bound of <0.10, and the primary discriminator reaches only 3.1x against the frontier clip, which is my own pre-registered weak-support band, not confirmation. Not rounded up. The run-length result is what survives every framing.

The instrument was rebuilt after the first one proved void. v1's threshold was scale-invariant and pinned every source to ~86% "active"; exposure-normalising the real footage changed the mean frame difference by 64x and moved the reported quiescent fraction by 0.002. v2 uses an absolute threshold (>=12 grey levels) plus a contiguous-area gate (>=0.15% of frame), constants fixed across all sources, and aborts unless it first passes a known-answer test on a provably-static control (reports duty 0.0000, max blob 0 px).

Full write-up: RESULTS_eventrate.md. Reproduce: code/eventrate2.py.

Real-footage rows are aggregate statistics only — no frames displayed, copied or published, source described only as an ordinary barely occupied room. Published with the capture owner's explicit consent.

---

Latent autoencoders implement an undeclared chroma subsampling

Blur only an image's chroma and 80.6% of its colour detail is destroyed — SSIM reports 0.9998. Standard SSIM is computed on luminance, so it is structurally blind; PSNR under-weights chroma by ~11 dB at matched damage. An autoencoder selected against these metrics faces no penalty for discarding chroma.

Measuring the consequence with 6-phase-averaged sinusoidal gratings through encode/decode — largest period at which chroma is >50% destroyed while luma survives intact:

autoencoder	type	chroma destroyed up to
LTX-Video-2B	video AE	8px
SDXL-VAE	image f8	16px
SD-1.5	image f8	24px

Universal, not a quirk of one model. At a 16px period SD-1.5 passes luma at 1.57 and chroma at 0.05 — a 31× disparity at identical spatial frequency. Notably the image autoencoders are worse than the video one, which rules out temporal compression as the cause.

Two things the curves show that a single cutoff number would have hidden:

• These are not low-pass filters. Deep nulls sit at exactly 4px and 8px — periods dividing the f8 lattice — with ~0.93 response either side, and they survive phase averaging. The curves are non-monotonic, so an "MTF50 cutoff" is meaningless here. We computed one first, got tidy 2.57×/3.74× chroma penalties, and discarded them as artifacts of a statistic the data does not support.
• Response above 1.0 is real. A passive optical system cannot amplify contrast; SD-1.5 reaches 1.57 (luma) and 1.73 (chroma). These decoders do not merely lose detail, they synthesise it — latent autoencoders are editorial, not simply lossy.

Consequence: anything carried by small colour features — wristbands, colour-coded labels and tubing, indicator LEDs, status lights, triage tags — does not survive generation, while the metrics used to certify the generator call the reconstruction essentially perfect.

CORRECTION (isoluminant control run): our original grating carried residual luma (17.25 vs 0.13 for a properly isoluminant one). We stated this made the measurement conservative. It was backwards — true isoluminant chroma scores higher at 6-8px (0.42/0.47 vs 0.34/0.21), so the deficit there is smaller than first reported. The headline is unchanged: chroma still >50% destroyed up to an 8px period while luma passes, now confirmed on both red-green and blue-yellow isoluminant axes independently. Scope now closed: the control was re-run on all three autoencoders on both isoluminant axes and every published period is confirmed unchanged — LTX 8px, SDXL 16px, SD-1.5 24px.

Full write-up: chroma/RESULTS_chroma.md. Reproduce: code/vae_mtf.py, code/mtf_crossmodel.py. Raw numbers: data/.

Related: identity has a per-resolution ceiling before any generation happens

Encoding an image to LTX latent space and decoding it — no transformer, no diffusion, no drift — already costs face identity, governed by face pixels, not frame resolution:

face px	identity retained (ArcFace)
54	0.514
75	0.729
95	0.858
110	0.910

But the ceiling is slack: generating at each rung, the full pipeline delivers only −0.03 / 0.20 / 0.13 — nowhere near its ceiling and below the 0.35 same-person threshold. The transformer, not the autoencoder, dominates identity loss. Raising resolution buys almost nothing, so "budget ~100px of face" is necessary but far from sufficient.

Consistent with this, an independently produced 25-second clip from a current frontier video generator (n=1, not ours) with ~77–112px faces scored 0.987 identity at frame 0 and 0.721 by the end — identity present where the pixel budget allows it, and drifting from there.

---

Long-horizon video generation from outside the model

Every published method for long-video consistency needs model internals — KV-cache surgery, custom attention, or retraining. Most practitioners only get black-box image conditioning. This is a measured study of what's actually achievable in that regime, run end-to-end on a single Jetson AGX Orin.

Seven runs, ~36,500 generated frames, four conditioning strategies, one metric harness.

Headline: anchoring frequency is a dial between two failure modes, not a fix for either

strategy	n	drift (end)	drift slope	jitter	spurious/1k
none — re-anchor every chunk	1200	0.092	0.0008	0.017	136
prev — never re-anchor	1200	0.279	0.0111	0.008	46
sink — prev + fixed seed anchor	1200	0.285	0.0011	0.022	222
log (naive) — 13 stacked anchors	1200	0.496	0.0100	0.026	86
tiered — re-anchor every 12	245*	0.205	0.0074	0.030	234

*shorter run; drift accumulates with length, so this row is not comparable on drift. Jitter and spurious/1k are length-normalised and are comparable. A matched-length run is in progress — we are not claiming a win on the unfair axis.

Re-anchor often → low drift, visible seams. Never → smooth but progressively wanders. You choose an operating point; you don't escape the tradeoff.

Four reproducible failure modes — failure is channel-dependent

1. Desaturation / blur (prev) — colour drains to grey-cyan; by chunk 140 the person in the scene has disappeared entirely.
2. Posterization (sink, tiered) — flat saturated colour, hard outlines; the scene becomes an illustration. Structure survives, photorealism doesn't.
3. Catastrophic collapse (log naive) — pure saturated green by ~chunk 40.
4. Jump-cutting (none) — no drift, but every chunk boundary is a cut.

A single scalar metric scores (1) and (4) similarly and misses that one is usable.

The over-conditioning collapse (a gotcha worth knowing)

Stacking N image anchors at the same frame_index with Σstrength = 2.53 drives the latents out of range and collapses to a flat colour field. It degrades with more history — i.e. worse precisely as the method does more of what it exists to do. Normalise Σstrength ≤ 1.0; prefer distributing anchors across frame indices.

Tiered conditioning: match each fact's persistence to a channel's drift-susceptibility

• INVARIANT ("static camera, no pan, bed centre-right, window left") → text prompt. Zero cost and structurally cannot drift. This worked — layout and camera stayed fixed.
• SLOW (appearance) → seed image, periodically re-anchored.
• FAST (motion) → previous frame.

At matched depth, tiered kept every object sharp and identifiable while naive chaining was already smearing faces. But text invariants hold semantic facts and not stylistic ones — output posterized despite "posterized, flat colours, cartoon, illustration" being explicitly in the negative prompt. Text carries what is true; images carry how it looks.

Practical recipe

LTX-Video 2B @ 704×448, 30 steps, CFG 3.0, seeded from a real photo of the target room; invariants in the prompt; Σstrength ≤ 1; hard re-anchor every 4–12 chunks depending on whether you prefer seams or wander. ~68 s per 49-frame chunk (≈2 s of video) on an Orin. For long footage, generate many independent 30–60 s segments from one seed rather than one continuous take — embarrassingly parallel and drift-bounded by construction.

Negative results (published rather than buried)

• 13B distilled loads, runs, and outputs noise. Distilled variants need their own timestep schedule, not linear_quadratic → sigmas.
• Disabling use_dynamic_shifting to silence a missing-mu error produces noise while appearing to succeed — correct exit code, plausible timings, files on disk.
• Negative prompts do not prevent progressive style drift.

Method note that cost the most

Three separate pipelines exited zero, wrote files, and produced garbage. Exit codes, timings, file counts, and even image mean/std all passed on pure noise. The only reliable detector was opening a JPEG. If you build an automated generation pipeline, put a human-visible sample check in the loop.

What's here

FINDINGS.md              full write-up
code/gen_tiered.py       tiered conditioning + periodic re-anchoring
code/gen_long.py         four-arm comparison harness (none/prev/sink/log)
code/drift_metric.py     CLIP-based drift, jitter, spurious-event metrics
results/                 the numbers behind the table

Not included

Seed images and generated frames are withheld: they derive from proprietary character and set assets. The code runs on any seed photo — point --anchor-image at your own. No real patients, real footage, or personal data are involved anywhere in this work; the scenes are entirely synthetic.

Limitations

One model family (LTX-Video), one scene, one resolution, single seeds per arm, and the tiered row is not length-matched. The metric is CLIP-embedding based and inherits CLIP's biases. Treat the ordering and the failure taxonomy as the contribution, not the absolute numbers.