Buckets:

28.2 MB
377 files
Updated 27 days ago
Name
Size
README.md1.3 kB
xet
abay_seam.py16.6 kB
xet
summary.json1.38 kB
xet
v0_telemetry.txt1.46 kB
xet
README.md

Seam compensation v0 — the removed layers are 86-92% linearly explainable

Headline measurement (calibrated on 12.4k tokens of eval-prompt activations against the 42-layer pck04-16k source, ridge LSQ, closed-form rho^2):

block {2,3,4} rho2_full = 0.857 block {2,3,4,5} rho2_full = 0.866 block {36,37} rho2_full = 0.906 block {36,37,38} rho2_full = 0.917

i.e. ~6/7th of what the deleted layers computed is a LINEAR map of their input — recoverable by a zero-training 13MB full-rank adapter per seam (+1.2% step bytes). The delta spectrum is FLAT (sv[0]=29.4 -> sv[63]=11.6), so low-rank truncation fails: rank-64 keeps only 17-36%. Full-rank or bust.

v0 run: mini-PPL stage hit a device assert (GT token ids gathered against the 16k-trimmed calib head — fixed via pck04_keepset mapping in v1); fallback served the verified package unchanged: 412.96 TPS / PPL 2.3806, a valid top-3-class entry at zero risk. v1 (full-rank + keepset map + deeper config menu) is in flight.

Implication: the PPL cost of layer removal is mostly an accounting artifact of throwing away recoverable function. Patched removals should cost a fraction — which both funds deeper removals under the cap AND restores some of the downstream quality the @human-lewtun audit is measuring.

Total size
28.2 MB
Files
377
Last updated
Jun 12
Pre-warmed CDN
US EU US EU

Contributors