Commit History
6 days ago
-
docs: LoRA branch engines for sa3-sm-music and sa3-sm-sfx (193 targets each)
Verified signature0b9ad3d -
tensorRT: add sa3-sm-sfx LoRA branch engine (sm_90, 193 targets)
Verified signature8acb4d0 -
tensorRT: add sa3-sm-music LoRA branch engine (sm_90, 193 targets)
Verified signature049c3f3 -
tensorRT/sm_90: add the sa3-m runtime LoRA engine (229 targets, rank<=512)
Verified signature46e266c -
tensorRT: document the runtime LoRA / DoRA engine
Verified signature1804a18
-
Delete the miscalibrated sm_120 SAME-S fp8 encoder
Verified signatureIts activation quantisers were calibrated with a percentile-of-percentile plus a 1e-4 floor, putting the clip points far below where the decoder's plain amax puts them. The equivalent sm_90 engine was deleted when that was found; this copy was missed and only surfaced when the deprecated engines were swept into legacy/. Keeping it under legacy/ would have left a trap: it is the one file there that is wrong rather than merely superseded, and anyone building sm_120 from it would inherit the bug. The recalibrated weights live in onnx/same-s/enc_fp8.onnx, which build_autoencoders.py uses to produce enc_fp8_chunkable.trt.197c684 -
Move deprecated sm_120 TRT engines under legacy/
Verified signatureThese are not reachable from the runtime registry: every tier resolves to the chunkable engines (dec_*_chunkable_limiter / enc_*_chunkable). Kept rather than deleted so older checkouts and reproductions still have them, but moved out of the way so the live set is unambiguous. Copies are server-side; the ONNX sources are untouched, and four of them (enc_dynamic_triton_swa.onnx, enc_dynamic_bf16.onnx, both enc_fp8.onnx) remain load-bearing for build_autoencoders.py -- 'chunkable' is a profile choice for the encoders, not a graph change.19ab22a -
Move deprecated sm_90 TRT engines under legacy/
Verified signatureThese are not reachable from the runtime registry: every tier resolves to the chunkable engines (dec_*_chunkable_limiter / enc_*_chunkable). Kept rather than deleted so older checkouts and reproductions still have them, but moved out of the way so the live set is unambiguous. Copies are server-side; the ONNX sources are untouched, and four of them (enc_dynamic_triton_swa.onnx, enc_dynamic_bf16.onnx, both enc_fp8.onnx) remain load-bearing for build_autoencoders.py -- 'chunkable' is a profile choice for the encoders, not a graph change.ceaecd4
-
retire fp8_fast; remove the superseded sm_90 fp8 engines (miscalibrated encoder, 8192-ceiling decoders)
Verified signature3963f0e -
retire fp8_fast; remove the superseded sm_90 fp8 engines (miscalibrated encoder, 8192-ceiling decoders)
Verified signaturef061405 -
retire fp8_fast; remove the superseded sm_90 fp8 engines (miscalibrated encoder, 8192-ceiling decoders)
Verified signature4b4cf7c -
retire fp8_fast; remove the superseded sm_90 fp8 engines (miscalibrated encoder, 8192-ceiling decoders)
Verified signature935685c -
retire fp8_fast; remove the superseded sm_90 fp8 engines (miscalibrated encoder, 8192-ceiling decoders)
Verified signature276f110 -
retire fp8_fast; remove the superseded sm_90 fp8 engines (miscalibrated encoder, 8192-ceiling decoders)
Verified signature0996c8a -
retire fp8_fast; remove the superseded sm_90 fp8 engines (miscalibrated encoder, 8192-ceiling decoders)
Verified signaturecf76999 -
SAME-L/SAME-S: bake the limiter ceiling so the chunkable decoders keep the ('latent','pcm') signature; add the SAME-S fp8 chunkable pair
Verified signature17258a7 -
SAME-L/SAME-S: bake the limiter ceiling so the chunkable decoders keep the ('latent','pcm') signature; add the SAME-S fp8 chunkable pair
Verified signature7ca4180 -
SAME-L/SAME-S: bake the limiter ceiling so the chunkable decoders keep the ('latent','pcm') signature; add the SAME-S fp8 chunkable pair
Verified signature775a981 -
SAME-L/SAME-S: bake the limiter ceiling so the chunkable decoders keep the ('latent','pcm') signature; add the SAME-S fp8 chunkable pair
Verified signatureaea1cf0 -
SAME-L/SAME-S: bake the limiter ceiling so the chunkable decoders keep the ('latent','pcm') signature; add the SAME-S fp8 chunkable pair
Verified signature9a01fee -
SAME-S: add the chunkable bf16 pair (two profiles, limiter)
Verified signaturebaa3cd7 -
SAME-S: add the chunkable bf16 pair (two profiles, limiter)
Verified signature7c73bf8 -
SAME-L encoders: low band 64 -> 256 (1.3-1.5x faster, cos 0.99831 -> 0.99969)
Verified signature1a8702f -
SAME-L encoders: low band 64 -> 256 (1.3-1.5x faster, cos 0.99831 -> 0.99969)
Verified signatured0d486b
-
SAME-L: remove miscalibrated enc_fp8.trt (activation scales 1.3-28x too small; superseded by enc_fp8_chunkable.trt on sm_90)
Verified signature2c8b02a -
SAME-L: remove miscalibrated enc_fp8.trt (activation scales 1.3-28x too small; superseded by enc_fp8_chunkable.trt on sm_90)
Verified signaturee00128c -
SAME-L: add enc_fp8_chunkable.trt (fp16, two-profile chunkable)
Verified signature7036ca2 -
SAME-L: add dec_fp8_chunkable_limiter.trt (fp16, two-profile chunkable, runtime limiter ceiling)
Verified signature093c4d6 -
SAME-L: add enc_fp16_chunkable.trt (fp16, two-profile chunkable)
Verified signatureeb343c9 -
SAME-L: add dec_fp16_chunkable_limiter.trt (fp16, two-profile chunkable, runtime limiter ceiling)
Verified signature5dba282
-
Delete old fp16mixed names + retired medium dit_bf16.trt (fp16 rename shipped)
Verified signature48e34e2 -
Rename fp16mixed->fp16 DiT/T5Gemma engines+onnx (copies; deletes follow after code merge)
Verified signature3f29675 -
Remove redundant w8_bf16 engines (int8 weights fold to bf16 at build → identical size/speed to the bf16 baseline)
Verified signature63cc70d -
Add sm_120 TRT engines (AOT SWA, wide profile) for the quantized tiers
Verified signature4e8cc21 -
Add sm_90 TRT engines (wide profile) for the quantized tiers
Verified signatured593f74
-
Add fp8 tier for sm-music / sm-sfx DiTs (clean fp8-linear graft on fp16mixed)
Verified signaturec3aa6e0 -
Add fp8 tier for sm-music / sm-sfx DiTs (clean fp8-linear graft on fp16mixed)
Verified signature1489a17 -
sm_120 sa3-m: add the fp8 medium-DiT tier (176 fp8 GEMM + 96 bf16 fused MHA, 1.43-1.95x vs fp16mixed) (#7)
- sm_120 sa3-m: add the fp8 medium-DiT tier (176 fp8 GEMM + 96 bf16 fused MHA, 1.43-1.95x vs fp16mixed) (af5680d9cca1dfc854e0dc95ea4e60470b33bc0f)f8e0a0f -
sa3-m fp8: ship calibrated-bakedmin engine (176 fp8 GEMM + 96 bf16 fused MHA + baked fp32 RoPE; calibration by @ryanontheinside #47)
Verified signature4697212 -
sm_120 SAME-L: AOT (graph-capturable) engines + tensorRT/README.md (#6)
- sm_120 SAME-L: AOT (graph-capturable) engines + tensorRT/README.md (1d2139821d87b6160099860d29a82d535269a33b)a05f2d4
-
Upload tensorRT/sm_90/sa3-m/dit_fp8.trt with huggingface_hub
Verified signaturefeddb8b -
Fix the bf16 medium DiT: bake RoPE's tables into the graph (clipping 3.112% -> 0.014%) (#5)
- Fix the bf16 medium DiT: bake RoPE's tables into the graph (clipping 3.112% -> 0.014%) (59bd967d4635c6e32b0b2a677d56a21984654b9d)6152b1d -
Rebuild medium DiT fp16-mixed engines (sm_90 + new sm_120): attention now fuses, 4.3x faster (#4)
- Rebuild medium DiT fp16-mixed engines (sm_90 + new sm_120): attention now fuses, 4.3x faster (f3263ec28281132dda00915ad3a83920f3b98730) - Complete the sm_120 engine set (mirror sm_90's 15 files) (dcca4a7a8676aa3846ca92ecf7607d572706d685)06debc3
-
Add bf16 medium DiT TRT engine (sm_90) — FMHA-fused, medium default (#1)
- Add bf16 medium DiT TRT engine (sm_90) — FMHA-fused, medium default (c8cb8ba88ddea67804bcacbbb9265afcbff03b6f) Co-authored-by: CJ Carr <cortexelus@users.noreply.huggingface.co>9a38efb
-
Add FP32 SAME-L + SAME-S TRT decoders
Cortexelus +1: Claude Opus 4.7 (1M context)Pair the existing FP32 DiT engines (sa3-{m,sm-music,sm-sfx}/dit_fp32.trt) with FP32 decoders so the consumer-side `--precision fp32` flag has a full FP32-throughout pipeline available, bit-equivalent to PyTorch eager. - same-l/dec_dynamic_fp32.trt — built by upcasting dec_dynamic_triton_swa.onnx initializers + value_info + Constant nodes + Cast.to from FP16 to FP32 (STRONGLY_TYPED). The Triton SWA plugin already runs on the FP32 side of the attention island, so the plugin contract is unchanged. ~1.7 GB. - same-s/dec_dynamic_fp32.trt — analogous FP32 engine for SAME-S. ~210 MB. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>773ac48 -
Decoder ONNX: clip+scale in FP32 before int32 cast
Cortexelus +1: Claude Opus 4.7 (1M context)The PCM-baked tail (audio → clamp → ×32767 → int32) ran in the engine's FP16/BF16 trunk, where 32767 is not representable (rounds up to 32768), causing the int32 output to exceed the int16 range. The consumer code then wrapped 32768 → -32768 on int16 downcast, producing audible clicks on loud transients. Fix: cast audio to FP32 before the clip/scale chain in both build_same_l_decoder.py and build_same_s_decoder.py producers. 32767 is exact in FP32. A trailing clamp(-32767, 32767) is added as belt-and-suspenders against any residual numerical drift. Affected: dec_dynamic_triton_swa.{onnx,trt} (SAME-L), dec_dynamic_bf16.{onnx,trt} (SAME-S). Verified: 0 samples outside [-32767, +32767] on real-music decode (was ~800 with the old engines). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>a429ffa
-
DiT engines: replace BF16 with FP16-mixed (FP32 islands) + add FP32 variants
Cortexelus +1: Claude Opus 4.7 (1M context)Drop dit_bf16.trt for all three SA3 DiTs (sm-music, sm-sfx, sa3-m). BF16 quantization error compounds over 8 pingpong sampling steps, drifting cos-sim vs PT FP32 from 0.99 (single step) to 0.81 (final latent) and producing audibly degraded output. Replace with dit_fp16mixed.trt (the canonical from now on): FP16 trunk with FP32 islands around every RMSNorm chain (Pow+ReduceMean+Sqrt+Mul), every attention Softmax, and the RoPE region (anything reachable from a Cast(to=FP32) feeding Cos/Sin/Einsum). 140 RMSNorms + 40 Softmaxes per sm-music block, more for medium. Built with STRONGLY_TYPED so TRT honors the explicit dtypes (no auto-promotion). Matches MLX's "FP16 with implicit FP32 reductions via fused kernels" recipe. Per-step cos-sim vs PT FP32: 0.99997 single-step, 0.998 over 8 steps. RMS-curve correlation: 0.998. Audio basically indistinguishable from FP32 PyTorch reference at 1/15 the wall time (43 ms vs 630 ms). Also adds dit_fp32.trt (1.8 GB sm-*, 5.8 GB medium) for users who wan…97651ef
-
DiT TRT engines: rebuild with profile min=1 (sm_90)
Cortexelus +1: Claude Opus 4.7 (1M context)Lowered the dynamic-shape lower bound on `x` and `local_add_cond` from 256 → 1 for all three DiT engines (sa3-m, sa3-sm-music, sa3-sm-sfx). This unlocks short-form output below 24 s (down to ~93 ms at L=1) at zero measured perf cost — TRT picks identical tactics across [1, 4096] when opt stays at 1292. Output quality below the trained range (L<256, ~23.8 s) is undefined but the engine runs. Also refreshes tokenizer.json content for the T5Gemma engine. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>5f27021
-
T5Gemma: FP16-mixed (FP32 attention island) — fixes BF16 numerical bug
CortexelusThe previous BF16 build had a numerical bug: cos similarity vs PyTorch FP32 reference dropped to 0.17 at specific tokens (the ' beautiful' token id 4964 output activations of magnitude 26 instead of 52). Some tokens silently produced bad conditioning for downstream DiT. Switch to FP16 trunk + FP32 attention island (STRONGLY_TYPED network), the same strategy that fixed SAME-L decoder's accuracy. Cos vs FP32 PyTorch is now 0.999998; the worst-token cos across our test prompts is 0.9987. Engine filename: t5gemma_bf16.trt → t5gemma_fp16mixed.trt. Engine size: 564 MB → 623 MB (+10%). Latency: 0.78 ms → 0.91 ms (negligible vs DiT's 50+ ms). ONNX: re-exported with mixed dtypes; STRONGLY_TYPED build respects them. Consumer code (sa3_trt.py, install.sh, build.py, build_from_onnx.py) updated in the github repo to point at the new filename.23f5c64 -
Decoder TRT engines: PCM-baked-in (sm_90)
CortexelusCompanion to the previous commit's ONNX update — the published sm_90 .trt engines for SAME-S and SAME-L decoders now have the same clamp+scale+ cast(int32)+permute postprocess baked into the engine graph. Drop-in compatible: sa3_trt.py auto-detects engine flavor by output tensor name (pcm vs audio) and skips Stage 5 clip/cast/transpose when pcm-baked. Stage 5 wall time drops from ~110ms to ~21ms on sm-music + same-s, 30s audio.106e6fd