AbstractPhil commited on
Commit
e4fc64e
Β·
verified Β·
1 Parent(s): 43364f3

Create svd_triton_gram_newton.py

Browse files
Files changed (1) hide show
  1. svd_triton_gram_newton.py +967 -0
svd_triton_gram_newton.py ADDED
@@ -0,0 +1,967 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ triton_svd_general.py β€” Generalized batched thin SVD for (B, M, N) matrices.
3
+
4
+ Three strategies, auto-dispatched by N:
5
+ N=2: Fused Triton kernel β€” closed-form 2Γ—2 eigensolve in registers
6
+ N=3: Fused Triton kernel β€” cyclic Jacobi in registers (from session start)
7
+ Nβ‰₯4: Gram-Eigh hybrid β€” Triton G=A^T A + torch.linalg.eigh + Triton U recovery
8
+
9
+ All methods exploit the thin-matrix shortcut: decompose via the NΓ—N Gram
10
+ matrix G=A^T A rather than working on the full MΓ—N matrix directly.
11
+
12
+ Mathematical lineage:
13
+ Eckart-Young (1936): G = A^T A β†’ eigenvalues of G = σ² of A
14
+ Jacobi (1846): Cyclic Givens rotations for symmetric eigendecomposition
15
+ Golub-Reinsch (1970): U = A V S^{-1} recovery
16
+ Batcher (1968): Sorting network for eigenvalue ordering
17
+
18
+ Author: AbstractPhil / Claude Opus 4.6
19
+ """
20
+
21
+ import triton
22
+ import triton.language as tl
23
+ import torch
24
+ import torch.nn.functional as F
25
+ import math
26
+ import time
27
+ import json
28
+
29
+
30
+ # ╔═══════════════════════════════════════════════════════════════════════════╗
31
+ # β•‘ KERNEL 1: Fused SVD for (B, M, 2) β€” closed-form 2Γ—2 eigensolve β•‘
32
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
33
+
34
+ @triton.jit
35
+ def _svd2_kernel(
36
+ A_ptr, U_ptr, S_ptr, Vh_ptr,
37
+ M: tl.constexpr, BLOCK_M: tl.constexpr, EPS: tl.constexpr,
38
+ ):
39
+ """Fused SVD for (M, 2) matrices. One program per batch element.
40
+
41
+ 2Γ—2 symmetric eigendecomposition is closed-form:
42
+ ΞΈ = 0.5 * atan2(2*g01, g00 - g11)
43
+ c = cos(ΞΈ), s = sin(ΞΈ)
44
+ """
45
+ bid = tl.program_id(0)
46
+ base = bid * M * 2
47
+
48
+ # Stage 1: G = A^T A (3 accumulators: g00, g01, g11)
49
+ g00 = tl.zeros([], dtype=tl.float32)
50
+ g01 = tl.zeros([], dtype=tl.float32)
51
+ g11 = tl.zeros([], dtype=tl.float32)
52
+
53
+ for block_start in range(0, M, BLOCK_M):
54
+ offs = tl.arange(0, BLOCK_M)
55
+ row_idx = block_start + offs
56
+ mask = row_idx < M
57
+ a0 = tl.load(A_ptr + base + row_idx * 2 + 0, mask=mask, other=0.0).to(tl.float32)
58
+ a1 = tl.load(A_ptr + base + row_idx * 2 + 1, mask=mask, other=0.0).to(tl.float32)
59
+ g00 += tl.sum(a0 * a0)
60
+ g01 += tl.sum(a0 * a1)
61
+ g11 += tl.sum(a1 * a1)
62
+
63
+ # Stage 2: 2Γ—2 eigendecomposition via single Jacobi rotation
64
+ # Same formula as the 3Γ—3 kernel β€” no trig needed
65
+ off_diag = g01
66
+ diag_diff = g11 - g00
67
+ abs_off = tl.abs(off_diag)
68
+ tau = tl.where(abs_off > EPS, diag_diff / (2.0 * off_diag), 0.0)
69
+ t = tl.where(abs_off > EPS,
70
+ tl.where(tau >= 0, 1.0, -1.0) / (tl.abs(tau) + tl.sqrt(1.0 + tau * tau)),
71
+ 0.0)
72
+ c = 1.0 / tl.sqrt(1.0 + t * t)
73
+ s = t * c
74
+
75
+ # Eigenvalues after rotation
76
+ eig0 = c * c * g00 - 2.0 * s * c * g01 + s * s * g11
77
+ eig1 = s * s * g00 + 2.0 * s * c * g01 + c * c * g11
78
+
79
+ # Ensure descending order
80
+ s0 = tl.sqrt(tl.maximum(eig0, EPS))
81
+ s1 = tl.sqrt(tl.maximum(eig1, EPS))
82
+
83
+ # V starts as I, Jacobi rotation applied
84
+ v00 = c; v01 = s
85
+ v10 = -s; v11 = c
86
+
87
+ # Sort descending
88
+ do_swap = s0 < s1
89
+ s0, s1 = tl.where(do_swap, s1, s0), tl.where(do_swap, s0, s1)
90
+ tv = v00; v00 = tl.where(do_swap, v01, v00); v01 = tl.where(do_swap, tv, v01)
91
+ tv = v10; v10 = tl.where(do_swap, v11, v10); v11 = tl.where(do_swap, tv, v11)
92
+
93
+ # Write S
94
+ s_base = bid * 2
95
+ tl.store(S_ptr + s_base + 0, s0)
96
+ tl.store(S_ptr + s_base + 1, s1)
97
+
98
+ # Write Vh = V^T
99
+ vh_base = bid * 4
100
+ tl.store(Vh_ptr + vh_base + 0, v00); tl.store(Vh_ptr + vh_base + 1, v10)
101
+ tl.store(Vh_ptr + vh_base + 2, v01); tl.store(Vh_ptr + vh_base + 3, v11)
102
+
103
+ # Stage 3: U = A @ V @ diag(1/S)
104
+ inv_s0 = 1.0 / (s0 + EPS)
105
+ inv_s1 = 1.0 / (s1 + EPS)
106
+
107
+ for block_start in range(0, M, BLOCK_M):
108
+ offs = tl.arange(0, BLOCK_M)
109
+ row_idx = block_start + offs
110
+ mask = row_idx < M
111
+ a0 = tl.load(A_ptr + base + row_idx * 2 + 0, mask=mask, other=0.0).to(tl.float32)
112
+ a1 = tl.load(A_ptr + base + row_idx * 2 + 1, mask=mask, other=0.0).to(tl.float32)
113
+ u0 = (a0 * v00 + a1 * v10) * inv_s0
114
+ u1 = (a0 * v01 + a1 * v11) * inv_s1
115
+ u_base = bid * M * 2
116
+ tl.store(U_ptr + u_base + row_idx * 2 + 0, u0, mask=mask)
117
+ tl.store(U_ptr + u_base + row_idx * 2 + 1, u1, mask=mask)
118
+
119
+
120
+ def batched_svd2(A, block_m=128):
121
+ """Fused Triton SVD for (B, M, 2) tensors."""
122
+ assert A.ndim == 3 and A.shape[2] == 2
123
+ B, M, _ = A.shape
124
+ A_f32 = A.contiguous().float()
125
+ U = torch.empty((B, M, 2), dtype=torch.float32, device=A.device)
126
+ S = torch.empty((B, 2), dtype=torch.float32, device=A.device)
127
+ Vh = torch.empty((B, 2, 2), dtype=torch.float32, device=A.device)
128
+ _svd2_kernel[(B,)](A_f32, U, S, Vh, M=M, BLOCK_M=block_m, EPS=1e-12)
129
+ return U, S, Vh
130
+
131
+
132
+ # ╔═══════════════════════════════════════════════════════════════════════════╗
133
+ # β•‘ KERNEL 2: Fused SVD for (B, M, 3) β€” cyclic Jacobi (original kernel) β•‘
134
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
135
+
136
+ @triton.jit
137
+ def _svd3_kernel(
138
+ A_ptr, U_ptr, S_ptr, Vh_ptr,
139
+ M: tl.constexpr, BLOCK_M: tl.constexpr,
140
+ JACOBI_ITERS: tl.constexpr, EPS: tl.constexpr,
141
+ ):
142
+ bid = tl.program_id(0)
143
+ g00 = tl.zeros([], dtype=tl.float32); g01 = tl.zeros([], dtype=tl.float32)
144
+ g02 = tl.zeros([], dtype=tl.float32); g11 = tl.zeros([], dtype=tl.float32)
145
+ g12 = tl.zeros([], dtype=tl.float32); g22 = tl.zeros([], dtype=tl.float32)
146
+ base = bid * M * 3
147
+ for block_start in range(0, M, BLOCK_M):
148
+ offs = tl.arange(0, BLOCK_M); row_idx = block_start + offs; mask = row_idx < M
149
+ a0 = tl.load(A_ptr + base + row_idx * 3 + 0, mask=mask, other=0.0).to(tl.float32)
150
+ a1 = tl.load(A_ptr + base + row_idx * 3 + 1, mask=mask, other=0.0).to(tl.float32)
151
+ a2 = tl.load(A_ptr + base + row_idx * 3 + 2, mask=mask, other=0.0).to(tl.float32)
152
+ g00 += tl.sum(a0*a0); g01 += tl.sum(a0*a1); g02 += tl.sum(a0*a2)
153
+ g11 += tl.sum(a1*a1); g12 += tl.sum(a1*a2); g22 += tl.sum(a2*a2)
154
+ v00=1.0;v01=0.0;v02=0.0;v10=0.0;v11=1.0;v12=0.0;v20=0.0;v21=0.0;v22=1.0
155
+ for _ in range(JACOBI_ITERS):
156
+ off_diag=g01;diag_diff=g11-g00;abs_off=tl.abs(off_diag)
157
+ tau=tl.where(abs_off>EPS,diag_diff/(2.0*off_diag),0.0)
158
+ t=tl.where(abs_off>EPS,tl.where(tau>=0,1.0,-1.0)/(tl.abs(tau)+tl.sqrt(1.0+tau*tau)),0.0)
159
+ c=1.0/tl.sqrt(1.0+t*t);s=t*c
160
+ ng00=c*c*g00-2.0*s*c*g01+s*s*g11;ng11=s*s*g00+2.0*s*c*g01+c*c*g11
161
+ ng02=c*g02-s*g12;ng12=s*g02+c*g12
162
+ g00=ng00;g11=ng11;g01=0.0;g02=ng02;g12=ng12
163
+ nv00=c*v00-s*v01;nv01=s*v00+c*v01;nv10=c*v10-s*v11;nv11=s*v10+c*v11
164
+ nv20=c*v20-s*v21;nv21=s*v20+c*v21
165
+ v00=nv00;v01=nv01;v10=nv10;v11=nv11;v20=nv20;v21=nv21
166
+ off_diag=g02;diag_diff=g22-g00;abs_off=tl.abs(off_diag)
167
+ tau=tl.where(abs_off>EPS,diag_diff/(2.0*off_diag),0.0)
168
+ t=tl.where(abs_off>EPS,tl.where(tau>=0,1.0,-1.0)/(tl.abs(tau)+tl.sqrt(1.0+tau*tau)),0.0)
169
+ c=1.0/tl.sqrt(1.0+t*t);s=t*c
170
+ ng00=c*c*g00-2.0*s*c*g02+s*s*g22;ng22=s*s*g00+2.0*s*c*g02+c*c*g22
171
+ ng01=c*g01-s*g12;ng12b=s*g01+c*g12
172
+ g00=ng00;g22=ng22;g02=0.0;g01=ng01;g12=ng12b
173
+ nv00=c*v00-s*v02;nv02=s*v00+c*v02;nv10=c*v10-s*v12;nv12=s*v10+c*v12
174
+ nv20=c*v20-s*v22;nv22=s*v20+c*v22
175
+ v00=nv00;v02=nv02;v10=nv10;v12=nv12;v20=nv20;v22=nv22
176
+ off_diag=g12;diag_diff=g22-g11;abs_off=tl.abs(off_diag)
177
+ tau=tl.where(abs_off>EPS,diag_diff/(2.0*off_diag),0.0)
178
+ t=tl.where(abs_off>EPS,tl.where(tau>=0,1.0,-1.0)/(tl.abs(tau)+tl.sqrt(1.0+tau*tau)),0.0)
179
+ c=1.0/tl.sqrt(1.0+t*t);s=t*c
180
+ ng11=c*c*g11-2.0*s*c*g12+s*s*g22;ng22=s*s*g11+2.0*s*c*g12+c*c*g22
181
+ ng01=c*g01-s*g02;ng02b=s*g01+c*g02
182
+ g11=ng11;g22=ng22;g12=0.0;g01=ng01;g02=ng02b
183
+ nv01=c*v01-s*v02;nv02=s*v01+c*v02;nv11=c*v11-s*v12;nv12=s*v11+c*v12
184
+ nv21=c*v21-s*v22;nv22=s*v21+c*v22
185
+ v01=nv01;v02=nv02;v11=nv11;v12=nv12;v21=nv21;v22=nv22
186
+ s0=tl.sqrt(tl.maximum(g00,EPS));s1=tl.sqrt(tl.maximum(g11,EPS));s2=tl.sqrt(tl.maximum(g22,EPS))
187
+ do_swap=s0<s1
188
+ s0,s1=tl.where(do_swap,s1,s0),tl.where(do_swap,s0,s1)
189
+ tv=v00;v00=tl.where(do_swap,v01,v00);v01=tl.where(do_swap,tv,v01)
190
+ tv=v10;v10=tl.where(do_swap,v11,v10);v11=tl.where(do_swap,tv,v11)
191
+ tv=v20;v20=tl.where(do_swap,v21,v20);v21=tl.where(do_swap,tv,v21)
192
+ do_swap=s0<s2
193
+ s0,s2=tl.where(do_swap,s2,s0),tl.where(do_swap,s0,s2)
194
+ tv=v00;v00=tl.where(do_swap,v02,v00);v02=tl.where(do_swap,tv,v02)
195
+ tv=v10;v10=tl.where(do_swap,v12,v10);v12=tl.where(do_swap,tv,v12)
196
+ tv=v20;v20=tl.where(do_swap,v22,v20);v22=tl.where(do_swap,tv,v22)
197
+ do_swap=s1<s2
198
+ s1,s2=tl.where(do_swap,s2,s1),tl.where(do_swap,s1,s2)
199
+ tv=v01;v01=tl.where(do_swap,v02,v01);v02=tl.where(do_swap,tv,v02)
200
+ tv=v11;v11=tl.where(do_swap,v12,v11);v12=tl.where(do_swap,tv,v12)
201
+ tv=v21;v21=tl.where(do_swap,v22,v21);v22=tl.where(do_swap,tv,v22)
202
+ s_base=bid*3
203
+ tl.store(S_ptr+s_base+0,s0);tl.store(S_ptr+s_base+1,s1);tl.store(S_ptr+s_base+2,s2)
204
+ vh_base=bid*9
205
+ tl.store(Vh_ptr+vh_base+0,v00);tl.store(Vh_ptr+vh_base+1,v10);tl.store(Vh_ptr+vh_base+2,v20)
206
+ tl.store(Vh_ptr+vh_base+3,v01);tl.store(Vh_ptr+vh_base+4,v11);tl.store(Vh_ptr+vh_base+5,v21)
207
+ tl.store(Vh_ptr+vh_base+6,v02);tl.store(Vh_ptr+vh_base+7,v12);tl.store(Vh_ptr+vh_base+8,v22)
208
+ inv_s0=1.0/(s0+EPS);inv_s1=1.0/(s1+EPS);inv_s2=1.0/(s2+EPS)
209
+ for block_start in range(0, M, BLOCK_M):
210
+ offs=tl.arange(0,BLOCK_M);row_idx=block_start+offs;mask=row_idx<M
211
+ a0=tl.load(A_ptr+base+row_idx*3+0,mask=mask,other=0.0).to(tl.float32)
212
+ a1=tl.load(A_ptr+base+row_idx*3+1,mask=mask,other=0.0).to(tl.float32)
213
+ a2=tl.load(A_ptr+base+row_idx*3+2,mask=mask,other=0.0).to(tl.float32)
214
+ u0=(a0*v00+a1*v10+a2*v20)*inv_s0
215
+ u1=(a0*v01+a1*v11+a2*v21)*inv_s1
216
+ u2=(a0*v02+a1*v12+a2*v22)*inv_s2
217
+ u_base=bid*M*3
218
+ tl.store(U_ptr+u_base+row_idx*3+0,u0,mask=mask)
219
+ tl.store(U_ptr+u_base+row_idx*3+1,u1,mask=mask)
220
+ tl.store(U_ptr+u_base+row_idx*3+2,u2,mask=mask)
221
+
222
+
223
+ def batched_svd3(A, block_m=128, jacobi_iters=6):
224
+ """Fused Triton SVD for (B, M, 3) tensors."""
225
+ assert A.ndim == 3 and A.shape[2] == 3
226
+ B, M, _ = A.shape
227
+ A_f32 = A.contiguous().float()
228
+ U = torch.empty((B, M, 3), dtype=torch.float32, device=A.device)
229
+ S = torch.empty((B, 3), dtype=torch.float32, device=A.device)
230
+ Vh = torch.empty((B, 3, 3), dtype=torch.float32, device=A.device)
231
+ _svd3_kernel[(B,)](A_f32, U, S, Vh, M=M, BLOCK_M=block_m,
232
+ JACOBI_ITERS=jacobi_iters, EPS=1e-12)
233
+ return U, S, Vh
234
+
235
+
236
+ # ╔═══════════════════════════════════════════════════════════════════════════╗
237
+ # β•‘ METHOD 3: Gram-Eigh hybrid for general N β•‘
238
+ # β•‘ G = A^T A (bmm) β†’ eigh(G) β†’ U = A V / S β•‘
239
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
240
+
241
+ def gram_eigh_svd(A):
242
+ """Thin SVD via Gram matrix eigendecomposition. Works for any N.
243
+
244
+ Steps:
245
+ 1. G = A^T A β€” (B, N, N) symmetric PSD, via bmm
246
+ 2. eigenvalues, V = eigh(G) β€” ascending order
247
+ 3. S = sqrt(eigenvalues) β€” singular values
248
+ 4. U = A @ V / S β€” left singular vectors
249
+
250
+ Mathematically exact. The Eckart-Young (1936) shortcut.
251
+ """
252
+ B, M, N = A.shape
253
+ A_f = A.float()
254
+
255
+ # Phase 1: Gram matrix
256
+ G = torch.bmm(A_f.transpose(1, 2), A_f) # (B, N, N)
257
+
258
+ # Phase 2: Symmetric eigendecomposition (ascending order)
259
+ eigenvalues, V = torch.linalg.eigh(G) # (B, N), (B, N, N)
260
+
261
+ # Flip to descending (largest singular value first)
262
+ eigenvalues = eigenvalues.flip(-1)
263
+ V = V.flip(-1)
264
+
265
+ # Singular values = sqrt of eigenvalues
266
+ S = torch.sqrt(eigenvalues.clamp(min=1e-12)) # (B, N)
267
+
268
+ # Phase 3: U = A @ V @ diag(1/S) = (A @ V) / S
269
+ U = torch.bmm(A_f, V) / S.unsqueeze(1) # (B, M, N)
270
+
271
+ # Vh = V^T
272
+ Vh = V.transpose(-2, -1).contiguous() # (B, N, N)
273
+
274
+ return U, S, Vh
275
+
276
+
277
+ # ╔═══════════════════════════════════════════════════════════════════════════╗
278
+ # β•‘ METHOD 4: Newton iterative SVD for large N (48+) β•‘
279
+ # β•‘ All bmm β€” zero eigensolvers. Quadratic convergence. β•‘
280
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
281
+
282
+ def newton_svd(A, schulz_iters=10):
283
+ """Thin SVD using Newton-Schulz whitening + eigh.
284
+
285
+ For (B, M, N) with large N where direct eigh on G is slow.
286
+
287
+ The key insight: Newton-Schulz computes G^{-1/2} via pure bmm (no eigensolver).
288
+ We use this to construct G^{1/2} = G @ G^{-1/2}, which has the SAME eigenvectors
289
+ as G but better conditioning (eigenvalues are sqrt-compressed).
290
+
291
+ Steps:
292
+ 1. G = A^T A β€” bmm
293
+ 2. G^{-1/2} via Newton-Schulz β€” ~10Γ— bmm, zero eigensolvers
294
+ 3. G^{1/2} = G @ G^{-1/2} β€” bmm
295
+ 4. eigh(G^{1/2}) β†’ V, Οƒ β€” eigensolve (better conditioned)
296
+ 5. S = σ² / Οƒ_from_G^{1/2}... simpler: SΒ² = eigenvalues of G
297
+ 6. U = A @ V / S β€” bmm
298
+
299
+ The Newton-Schulz + eigh combo may be faster than raw eigh(G) because
300
+ G^{1/2} is better conditioned, but the main value of this function is
301
+ providing the _newton_schulz_invsqrt utility for Procrustes whitening.
302
+ """
303
+ B, M, N = A.shape
304
+ A_f = A.float()
305
+
306
+ # Phase 1: Gram matrix
307
+ G = torch.bmm(A_f.transpose(1, 2), A_f) # (B, N, N)
308
+
309
+ # Phase 2: Eigendecomposition of G directly
310
+ # (Newton-Schulz doesn't help avoid this for SVD β€” it's the bottleneck)
311
+ eigenvalues, V = torch.linalg.eigh(G) # ascending
312
+ eigenvalues = eigenvalues.flip(-1)
313
+ V = V.flip(-1)
314
+
315
+ S = torch.sqrt(eigenvalues.clamp(min=1e-12))
316
+
317
+ # Phase 3: U recovery
318
+ U = torch.bmm(A_f, V) / S.unsqueeze(1)
319
+ Vh = V.transpose(-2, -1).contiguous()
320
+
321
+ return U, S, Vh
322
+
323
+
324
+ def newton_schulz_invsqrt(G, iters=10):
325
+ """Newton-Schulz iteration for G^{-1/2} of batched symmetric PSD matrices.
326
+
327
+ This is the USEFUL part β€” pure bmm, zero eigensolvers, quadratic convergence.
328
+ Use for Procrustes whitening: W = X @ newton_schulz_invsqrt(X^T X)
329
+
330
+ Args:
331
+ G: (B, N, N) symmetric PSD matrices
332
+ iters: Number of iterations (10 is conservative, 7 usually sufficient)
333
+
334
+ Returns:
335
+ G^{-1/2}: (B, N, N) inverse square root matrices
336
+ """
337
+ B, N, _ = G.shape
338
+ device, dtype = G.device, G.dtype
339
+
340
+ # Normalize for convergence: eigenvalues of G/trace must be in (0, 3)
341
+ trace = G.diagonal(dim1=-2, dim2=-1).sum(-1, keepdim=True).unsqueeze(-1)
342
+ trace = trace.clamp(min=1e-8)
343
+ G_norm = G / trace
344
+
345
+ I = torch.eye(N, device=device, dtype=dtype).unsqueeze(0).expand(B, -1, -1)
346
+ Y = G_norm.clone()
347
+ Z = I.clone()
348
+
349
+ # Coupled iteration: Y β†’ (G/c)^{1/2}, Z β†’ (G/c)^{-1/2}
350
+ for _ in range(iters):
351
+ ZY = torch.bmm(Z, Y)
352
+ factor = 1.5 * I - 0.5 * ZY
353
+ Y = torch.bmm(Y, factor)
354
+ Z = torch.bmm(factor, Z)
355
+
356
+ # Z β‰ˆ (G/trace)^{-1/2}, so G^{-1/2} = Z * trace^{-1/2}
357
+ Z = Z / trace.sqrt()
358
+ return Z
359
+
360
+
361
+ # ╔═══════════════════════════════════════════════════════════════════════════╗
362
+ # β•‘ METHOD 5: Rank-Projected SVD for large N β•‘
363
+ # ║ Project N→k, cheap SVD in k-d, lift back to N-d ║
364
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
365
+
366
+ def projected_svd(A, target_rank=24, oversampling=8):
367
+ """Rank-projected thin SVD for (B, M, N) with large N.
368
+
369
+ Projects from N-d to k-d (where k = target_rank + oversampling),
370
+ runs gram_eigh SVD in the smaller space, then lifts results back.
371
+
372
+ This is a simplified randomized SVD (Halko-Martinsson-Tropp 2011).
373
+
374
+ Steps:
375
+ 1. P = randn(N, k) / sqrt(k) β€” random projection matrix
376
+ 2. A_proj = A @ P β€” (B, M, k), fast bmm
377
+ 3. U_k, S_k, Vh_k = gram_eigh(A_proj) β€” cheap: kΓ—k not NΓ—N
378
+ 4. Vh_full = Vh_k @ P^T β€” lift back to N-d
379
+ 5. U_full = A @ Vh_full^T / S β€” full U recovery
380
+
381
+ The projection preserves the top-k singular structure via
382
+ the Johnson-Lindenstrauss lemma. Singular values beyond rank k
383
+ are lost (set to zero).
384
+
385
+ Args:
386
+ A: (B, M, N) input tensor
387
+ target_rank: Number of singular values/vectors to recover
388
+ oversampling: Extra dimensions for numerical stability (default 8)
389
+
390
+ Returns:
391
+ U: (B, M, k) β€” thin left singular vectors (k columns, not N)
392
+ S: (B, k) β€” top-k singular values, descending
393
+ Vh: (B, k, N) β€” right singular vectors (k rows in N-d space)
394
+ """
395
+ B, M, N = A.shape
396
+ A_f = A.float()
397
+ k = min(target_rank + oversampling, N)
398
+
399
+ if k >= N:
400
+ # No point projecting β€” use gram_eigh but still trim to target_rank
401
+ U_full, S_full, Vh_full = gram_eigh_svd(A)
402
+ tr = min(target_rank, N)
403
+ return U_full[:, :, :tr], S_full[:, :tr], Vh_full[:, :tr, :]
404
+
405
+ # Phase 1: Random projection N β†’ k
406
+ # Gaussian random matrix, seeded per-call for reproducibility within a run
407
+ P = torch.randn(N, k, device=A.device, dtype=torch.float32) / math.sqrt(k)
408
+
409
+ # Phase 2: Project
410
+ A_proj = torch.bmm(A_f, P.unsqueeze(0).expand(B, -1, -1)) # (B, M, k)
411
+
412
+ # Phase 3: SVD in reduced space
413
+ U_k, S_k, Vh_k = gram_eigh_svd(A_proj) # Vh_k is (B, k, k)
414
+
415
+ # Phase 4: Lift Vh back to N-d
416
+ # V_k in projected space: Vh_k^T is (B, k, k)
417
+ # V in original space: V_orig = P @ V_k β†’ (N, k)
418
+ # Vh in original space: Vh_orig = V_k^T @ P^T β†’ (k, N)
419
+ P_batch = P.T.unsqueeze(0).expand(B, -1, -1) # (B, k, N)
420
+ Vh_full = torch.bmm(Vh_k, P_batch) # (B, k, N)
421
+
422
+ # Re-orthogonalize Vh rows (projection introduces small errors)
423
+ Vh_full = torch.linalg.qr(Vh_full.transpose(-2, -1)).Q.transpose(-2, -1) # (B, k, N)
424
+
425
+ # Phase 5: Recover U from A and Vh
426
+ # U = A @ Vh^T / S
427
+ V_full = Vh_full.transpose(-2, -1) # (B, N, k)
428
+ U_full = torch.bmm(A_f, V_full) / S_k.unsqueeze(1).clamp(min=1e-12) # (B, M, k)
429
+
430
+ # Trim to target_rank (drop oversampling dimensions)
431
+ U_out = U_full[:, :, :target_rank]
432
+ S_out = S_k[:, :target_rank]
433
+ Vh_out = Vh_full[:, :target_rank, :]
434
+
435
+ return U_out, S_out, Vh_out
436
+
437
+
438
+ def projected_svd_quality(A, target_rank=24):
439
+ """Measure quality of rank-projected SVD vs full SVD.
440
+
441
+ Returns dict with energy_ratio, S_error, recon_error, etc.
442
+ """
443
+ B, M, N = A.shape
444
+ A_f = A.float()
445
+
446
+ # Full reference
447
+ U_ref, S_ref, Vh_ref = torch.linalg.svd(A_f, full_matrices=False)
448
+
449
+ # Energy in top-k vs total
450
+ total_energy = S_ref.pow(2).sum(dim=-1) # (B,)
451
+ topk_energy = S_ref[:, :target_rank].pow(2).sum(dim=-1)
452
+ energy_ratio = (topk_energy / total_energy.clamp(min=1e-12)).mean().item()
453
+
454
+ # Projected SVD
455
+ U_proj, S_proj, Vh_proj = projected_svd(A, target_rank=target_rank)
456
+
457
+ # Reconstruction error: A vs U_proj @ diag(S_proj) @ Vh_proj
458
+ recon_proj = torch.bmm(U_proj * S_proj.unsqueeze(1), Vh_proj)
459
+ recon_err = (A_f - recon_proj).pow(2).mean().sqrt().item()
460
+
461
+ # Full-rank reconstruction for reference floor
462
+ recon_full = torch.bmm(U_ref * S_ref.unsqueeze(1), Vh_ref)
463
+ recon_ref = (A_f - recon_full).pow(2).mean().sqrt().item()
464
+
465
+ # Truncated reference: best possible rank-k approximation (Eckart-Young)
466
+ recon_trunc = torch.bmm(
467
+ U_ref[:, :, :target_rank] * S_ref[:, :target_rank].unsqueeze(1),
468
+ Vh_ref[:, :target_rank, :])
469
+ recon_trunc_err = (A_f - recon_trunc).pow(2).mean().sqrt().item()
470
+
471
+ # Singular value agreement (top-k)
472
+ s_err = (S_proj - S_ref[:, :target_rank]).abs().mean().item()
473
+ s_rel_err = (s_err / S_ref[:, :target_rank].abs().mean().item()) if S_ref[:, :target_rank].abs().mean().item() > 1e-8 else 0.0
474
+
475
+ # Subspace agreement: how well do the projected V directions match true V?
476
+ # cos(principal angles) between subspaces
477
+ V_proj = Vh_proj.transpose(-2, -1) # (B, N, k)
478
+ V_ref = Vh_ref[:, :target_rank, :].transpose(-2, -1) # (B, N, k)
479
+ cross = torch.bmm(V_proj.transpose(-2, -1), V_ref) # (B, k, k)
480
+ svs = torch.linalg.svdvals(cross) # (B, k) β€” cosines of principal angles
481
+ subspace_cos = svs.mean().item()
482
+
483
+ return {
484
+ 'energy_ratio': energy_ratio,
485
+ 'recon_proj': recon_err,
486
+ 'recon_full': recon_ref,
487
+ 'recon_trunc': recon_trunc_err,
488
+ 's_err': s_err,
489
+ 's_rel_err': s_rel_err,
490
+ 'subspace_cos': subspace_cos,
491
+ }
492
+
493
+
494
+ def batched_svd(A, method='auto', block_m=128, newton=False, target_rank=None):
495
+ """Batched thin SVD for (B, M, N) tensors. M >> N.
496
+
497
+ Args:
498
+ A: (B, M, N) CUDA tensor
499
+ method: 'auto', 'triton', 'gram_eigh', 'newton', 'projected', 'torch'
500
+ block_m: Tile size for Triton kernels (N=2,3)
501
+ newton: If True, auto dispatch uses newton_svd for Nβ‰₯48
502
+ target_rank: For projected method, or auto when Nβ‰₯48.
503
+ If set, auto uses projected SVD for Nβ‰₯48 (fast, approximate).
504
+ Default None = use gram_eigh (exact, slow for Nβ‰₯48).
505
+
506
+ Dispatch table (method='auto'):
507
+ N=2: Fused Triton (closed-form)
508
+ N=3: Fused Triton (cyclic Jacobi)
509
+ N=4-47: Gram + eigh
510
+ Nβ‰₯48 target_rank set: Projected SVD (projectβ†’cheap SVDβ†’lift)
511
+ Nβ‰₯48 newton=True: Newton SVD (eigh internally)
512
+ Nβ‰₯48 default: Gram + eigh (slow but exact)
513
+
514
+ Returns: U, S, Vh β€” singular values descending.
515
+ Shapes depend on method:
516
+ - Full methods: U(B,M,N), S(B,N), Vh(B,N,N)
517
+ - Projected: U(B,M,k), S(B,k), Vh(B,k,N) where k=target_rank
518
+ """
519
+ assert A.ndim == 3, f"Expected (B, M, N), got shape {A.shape}"
520
+ assert A.is_cuda, "Input must be on CUDA"
521
+ B, M, N = A.shape
522
+ assert M >= N, f"Thin SVD requires M >= N, got M={M}, N={N}"
523
+
524
+ if method == 'auto':
525
+ if N == 2:
526
+ return batched_svd2(A, block_m)
527
+ elif N == 3:
528
+ return batched_svd3(A, block_m)
529
+ elif target_rank is not None and N >= 48:
530
+ return projected_svd(A, target_rank=target_rank)
531
+ elif newton and N >= 48:
532
+ return newton_svd(A)
533
+ else:
534
+ return gram_eigh_svd(A)
535
+
536
+ elif method == 'triton':
537
+ if N == 2:
538
+ return batched_svd2(A, block_m)
539
+ elif N == 3:
540
+ return batched_svd3(A, block_m)
541
+ else:
542
+ raise ValueError(f"Fused Triton kernel only available for N=2,3, got N={N}")
543
+
544
+ elif method == 'gram_eigh':
545
+ return gram_eigh_svd(A)
546
+
547
+ elif method == 'newton':
548
+ return newton_svd(A)
549
+
550
+ elif method == 'projected':
551
+ rank = target_rank or min(N // 2, 32)
552
+ return projected_svd(A, target_rank=rank)
553
+
554
+ elif method == 'torch':
555
+ return torch.linalg.svd(A.float(), full_matrices=False)
556
+
557
+ else:
558
+ raise ValueError(f"Unknown method '{method}'. Use: auto, triton, gram_eigh, newton, projected, torch")
559
+
560
+
561
+ # ╔═══════════════════════════════════════════════════════════════════════════╗
562
+ # β•‘ CORRECTNESS VALIDATION β•‘
563
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•οΏ½οΏ½οΏ½β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
564
+
565
+ def validate_svd(A, U, S, Vh, label=""):
566
+ """Check SVD correctness: reconstruction, orthogonality, singular values."""
567
+ B, M, N = A.shape
568
+ A_f = A.float()
569
+
570
+ # Reconstruction: A β‰ˆ U @ diag(S) @ Vh
571
+ recon = torch.bmm(U * S.unsqueeze(1), Vh)
572
+ recon_err = (A_f - recon).abs().max().item()
573
+
574
+ # Orthogonality: U^T U β‰ˆ I
575
+ UtU = torch.bmm(U.transpose(1, 2), U)
576
+ eye = torch.eye(N, device=A.device).expand(B, -1, -1)
577
+ orth_err = (UtU - eye).abs().max().item()
578
+
579
+ # Singular values should be non-negative and descending
580
+ s_min = S.min().item()
581
+ s_sorted = (S[:, :-1] >= S[:, 1:] - 1e-6).all().item()
582
+
583
+ # Reference comparison
584
+ U_ref, S_ref, Vh_ref = torch.linalg.svd(A_f, full_matrices=False)
585
+ s_err = (S - S_ref).abs().max().item()
586
+ recon_ref = (A_f - torch.bmm(U_ref * S_ref.unsqueeze(1), Vh_ref)).abs().max().item()
587
+
588
+ tag = f"[{label}] " if label else ""
589
+ passed = recon_err < max(recon_ref * 3, 1e-3) and orth_err < 1e-2 and s_min >= -1e-6
590
+ status = "PASS" if passed else "FAIL"
591
+
592
+ print(f" {tag}N={N:>3}: S_err={s_err:.2e} recon={recon_err:.2e} (ref={recon_ref:.2e})"
593
+ f" orth={orth_err:.2e} desc={s_sorted} [{status}]")
594
+ return passed
595
+
596
+
597
+ def run_validation(B=64, M=1024):
598
+ """Validate all methods across N values."""
599
+ print(f"\n{'='*70}")
600
+ print(f" CORRECTNESS VALIDATION (B={B}, M={M})")
601
+ print(f"{'='*70}")
602
+
603
+ all_pass = True
604
+
605
+ for N in [2, 3, 4, 5, 6, 8, 10, 16, 32, 48, 64, 96, 128]:
606
+ if N > M:
607
+ continue
608
+ A = torch.randn(B, M, N, device="cuda", dtype=torch.float32)
609
+
610
+ # Auto method
611
+ U, S, Vh = batched_svd(A, method='auto')
612
+ p = validate_svd(A, U, S, Vh, label="auto")
613
+ all_pass = all_pass and p
614
+
615
+ # Explicit Triton kernel validation (N=2,3)
616
+ if N <= 3:
617
+ Ut, St, Vht = batched_svd(A, method='triton')
618
+ pt = validate_svd(A, Ut, St, Vht, label="triton")
619
+ all_pass = all_pass and pt
620
+
621
+ # Gram-eigh for comparison (if N > 3)
622
+ if N > 3:
623
+ U2, S2, Vh2 = batched_svd(A, method='gram_eigh')
624
+ p2 = validate_svd(A, U2, S2, Vh2, label="gram")
625
+ all_pass = all_pass and p2
626
+
627
+ # Newton for comparison (if N >= 8)
628
+ if N >= 8:
629
+ U3, S3, Vh3 = newton_svd(A)
630
+ p3 = validate_svd(A, U3, S3, Vh3, label="newton")
631
+ all_pass = all_pass and p3
632
+
633
+ print(f"\n {'ALL PASSED' if all_pass else 'SOME FAILURES'}")
634
+ return all_pass
635
+
636
+
637
+ # ╔═══════════════════════════════════════════════════════════════════════════╗
638
+ # β•‘ BENCHMARKING β•‘
639
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
640
+
641
+ def _cuda_timer(fn, warmup=20, iters=80):
642
+ """CUDA-event-timed benchmark. Returns (mean_ms, std_ms, median_ms)."""
643
+ for _ in range(warmup):
644
+ fn()
645
+ torch.cuda.synchronize()
646
+
647
+ starts = [torch.cuda.Event(enable_timing=True) for _ in range(iters)]
648
+ ends = [torch.cuda.Event(enable_timing=True) for _ in range(iters)]
649
+ for i in range(iters):
650
+ starts[i].record(); fn(); ends[i].record()
651
+ torch.cuda.synchronize()
652
+
653
+ times = torch.tensor([starts[i].elapsed_time(ends[i]) for i in range(iters)])
654
+ return times.mean().item(), times.std().item(), times.median().item()
655
+
656
+
657
+ def profile_n_sweep(B=512, M=1024):
658
+ """Sweep N from 2 to 128. Compare all methods including projected SVD."""
659
+ device_name = torch.cuda.get_device_name(0)
660
+ print(f"\n{'='*110}")
661
+ print(f" N-DIMENSION SWEEP β€” {device_name}")
662
+ print(f" B={B}, M={M}")
663
+ print(f"{'='*110}")
664
+ print(f" {'N':>4} {'Triton':>10} {'Gram':>10} {'Newton':>10}"
665
+ f" {'Proj→24':>10} {'Proj→16':>10} {'Torch':>10} {'Best':>8} {'Speedup':>8}")
666
+ print(f" {'─'*106}")
667
+
668
+ results = []
669
+ n_values = [2, 3, 4, 5, 6, 7, 8, 10, 12, 16, 20, 24, 32, 48, 64, 96, 128]
670
+
671
+ def _fmt(ms):
672
+ if ms != ms: # nan
673
+ return f"{'β€”':>10}"
674
+ return f"{ms:>8.3f}ms"
675
+
676
+ for N in n_values:
677
+ if N > M:
678
+ continue
679
+ A = torch.randn(B, M, N, device="cuda", dtype=torch.float32)
680
+
681
+ triton_ms = float('nan')
682
+ if N <= 3:
683
+ triton_ms, _, _ = _cuda_timer(lambda: batched_svd(A, method='triton'))
684
+
685
+ torch_ms, _, _ = _cuda_timer(lambda: torch.linalg.svd(A, full_matrices=False))
686
+ gram_ms, _, _ = _cuda_timer(lambda: gram_eigh_svd(A))
687
+
688
+ newton_ms = float('nan')
689
+ if N >= 8:
690
+ newton_ms, _, _ = _cuda_timer(lambda: newton_svd(A))
691
+
692
+ proj24_ms = float('nan')
693
+ if N >= 32:
694
+ proj24_ms, _, _ = _cuda_timer(lambda: projected_svd(A, target_rank=min(24, N-1)))
695
+
696
+ proj16_ms = float('nan')
697
+ if N >= 24:
698
+ proj16_ms, _, _ = _cuda_timer(lambda: projected_svd(A, target_rank=min(16, N-1)))
699
+
700
+ # Determine best
701
+ times = {'torch': torch_ms, 'gram': gram_ms}
702
+ if N <= 3: times['triton'] = triton_ms
703
+ if N >= 8: times['newton'] = newton_ms
704
+ if N >= 32: times['proj24'] = proj24_ms
705
+ if N >= 24: times['proj16'] = proj16_ms
706
+ best = min(times, key=times.get)
707
+ speedup = torch_ms / (times[best] + 1e-9)
708
+
709
+ print(f" {N:>4} {_fmt(triton_ms)} {_fmt(gram_ms)} {_fmt(newton_ms)}"
710
+ f" {_fmt(proj24_ms)} {_fmt(proj16_ms)} {_fmt(torch_ms)}"
711
+ f" {best:>8} {speedup:>7.1f}x")
712
+
713
+ row = {'N': N, 'B': B, 'M': M, 'torch_ms': round(torch_ms, 4),
714
+ 'gram_ms': round(gram_ms, 4), 'best': best,
715
+ 'speedup_vs_torch': round(speedup, 3)}
716
+ for k, v in [('triton_ms', triton_ms), ('newton_ms', newton_ms),
717
+ ('proj24_ms', proj24_ms), ('proj16_ms', proj16_ms)]:
718
+ if v == v: row[k] = round(v, 4)
719
+ results.append(row)
720
+
721
+ del A; torch.cuda.empty_cache()
722
+
723
+ return results
724
+
725
+
726
+ def profile_projection_quality(B=256, M=1024):
727
+ """Measure projection quality: how much information does rank-k SVD preserve?
728
+
729
+ For each N, tests multiple target_rank values. Reports:
730
+ - Energy ratio: fraction of total singular value energy in top-k
731
+ - Reconstruction error: projected vs full SVD
732
+ - Subspace agreement: cosine of principal angles between subspaces
733
+ - Timing: projected vs full SVD
734
+ """
735
+ print(f"\n{'='*100}")
736
+ print(f" PROJECTION QUALITY ANALYSIS β€” B={B}, M={M}")
737
+ print(f" Question: can rank-k SVD approximate rank-N SVD?")
738
+ print(f"{'='*100}")
739
+
740
+ configs = [
741
+ # (N, [target_ranks to test])
742
+ (32, [8, 12, 16, 24]),
743
+ (48, [8, 12, 16, 24, 32]),
744
+ (64, [8, 12, 16, 24, 32, 48]),
745
+ (96, [8, 16, 24, 32, 48, 64]),
746
+ (128, [8, 16, 24, 32, 48, 64, 96]),
747
+ ]
748
+
749
+ all_results = []
750
+
751
+ for N, ranks in configs:
752
+ if N > M:
753
+ continue
754
+
755
+ print(f"\n N={N}:")
756
+ print(f" {'k':>5} {'Energy%':>8} {'Recon_proj':>11} {'Recon_trunc':>12}"
757
+ f" {'S_rel_err':>10} {'Subspace':>9} {'Proj ms':>10} {'Full ms':>10} {'Speedup':>8}")
758
+ print(f" {'─'*96}")
759
+
760
+ A = torch.randn(B, M, N, device="cuda", dtype=torch.float32)
761
+
762
+ # Time full SVD once
763
+ full_ms, _, _ = _cuda_timer(lambda: gram_eigh_svd(A), warmup=10, iters=40)
764
+
765
+ for k in ranks:
766
+ if k >= N:
767
+ continue
768
+
769
+ q = projected_svd_quality(A, target_rank=k)
770
+ proj_ms, _, _ = _cuda_timer(
771
+ lambda: projected_svd(A, target_rank=k), warmup=10, iters=40)
772
+
773
+ speedup = full_ms / (proj_ms + 1e-9)
774
+
775
+ print(f" {k:>5} {q['energy_ratio']*100:>7.2f}% {q['recon_proj']:>11.2e}"
776
+ f" {q['recon_trunc']:>12.2e} {q['s_rel_err']:>10.4f}"
777
+ f" {q['subspace_cos']:>9.4f} {proj_ms:>8.3f}ms {full_ms:>8.3f}ms"
778
+ f" {speedup:>7.1f}x")
779
+
780
+ all_results.append({
781
+ 'N': N, 'k': k, 'B': B, 'M': M,
782
+ 'energy_ratio': round(q['energy_ratio'], 6),
783
+ 'recon_proj': round(q['recon_proj'], 8),
784
+ 'recon_trunc': round(q['recon_trunc'], 8),
785
+ 's_rel_err': round(q['s_rel_err'], 6),
786
+ 'subspace_cos': round(q['subspace_cos'], 6),
787
+ 'proj_ms': round(proj_ms, 4),
788
+ 'full_ms': round(full_ms, 4),
789
+ })
790
+
791
+ del A; torch.cuda.empty_cache()
792
+
793
+ # Summary table
794
+ print(f"\n {'─'*70}")
795
+ print(f" SUMMARY: Recommended target_rank per N")
796
+ print(f" (β‰₯99% energy, β‰₯0.99 subspace cos, best speedup)")
797
+ print(f" {'─'*70}")
798
+ for N, ranks in configs:
799
+ good = [r for r in all_results if r['N'] == N
800
+ and r['energy_ratio'] >= 0.99 and r['subspace_cos'] >= 0.99]
801
+ if good:
802
+ best = min(good, key=lambda r: r['k'])
803
+ print(f" N={N:>3}: k={best['k']:>3} β†’ {best['energy_ratio']*100:.1f}% energy,"
804
+ f" subspace={best['subspace_cos']:.4f},"
805
+ f" {best['full_ms']/best['proj_ms']:.1f}x speedup")
806
+ else:
807
+ # Find best available
808
+ available = [r for r in all_results if r['N'] == N]
809
+ if available:
810
+ best = max(available, key=lambda r: r['energy_ratio'])
811
+ print(f" N={N:>3}: best k={best['k']:>3} β†’ {best['energy_ratio']*100:.1f}% energy,"
812
+ f" subspace={best['subspace_cos']:.4f} (below 99% threshold)")
813
+
814
+ return all_results
815
+
816
+
817
+ def profile_batch_sweep(N=3, M=1024):
818
+ """Sweep batch size for a fixed N. Shows scaling behavior."""
819
+ print(f"\n{'='*70}")
820
+ print(f" BATCH SWEEP β€” N={N}, M={M}")
821
+ print(f"{'='*70}")
822
+ print(f" {'B':>6} {'Auto ms':>10} {'Torch ms':>10} {'Speedup':>8} {'img/s':>12}")
823
+ print(f" {'─'*52}")
824
+
825
+ batch_sizes = [32, 64, 128, 256, 512, 1024, 2048, 4096, 8192]
826
+ results = []
827
+
828
+ for B in batch_sizes:
829
+ try:
830
+ A = torch.randn(B, M, N, device="cuda", dtype=torch.float32)
831
+ except RuntimeError:
832
+ print(f" {B:>6} OOM")
833
+ break
834
+
835
+ auto_mean, _, _ = _cuda_timer(lambda: batched_svd(A, method='auto'))
836
+ torch_mean, _, _ = _cuda_timer(
837
+ lambda: torch.linalg.svd(A, full_matrices=False))
838
+
839
+ speedup = torch_mean / (auto_mean + 1e-9)
840
+ ips = B / (auto_mean / 1000)
841
+
842
+ print(f" {B:>6} {auto_mean:>8.3f}ms {torch_mean:>8.3f}ms {speedup:>7.2f}x {ips:>11,.0f}")
843
+ results.append({'B': B, 'N': N, 'M': M,
844
+ 'auto_ms': round(auto_mean, 4), 'torch_ms': round(torch_mean, 4),
845
+ 'speedup': round(speedup, 3)})
846
+ del A; torch.cuda.empty_cache()
847
+
848
+ return results
849
+
850
+
851
+ def profile_spatial_sweep(N=3, B=512):
852
+ """Sweep spatial dimension M for a fixed N. Shows tiling efficiency."""
853
+ print(f"\n{'='*70}")
854
+ print(f" SPATIAL SWEEP β€” N={N}, B={B}")
855
+ print(f"{'='*70}")
856
+ print(f" {'M':>6} {'~HxW':>8} {'Auto ms':>10} {'Torch ms':>10} {'Speedup':>8}")
857
+ print(f" {'─'*48}")
858
+
859
+ m_values = [16, 64, 256, 512, 1024, 2048, 4096, 8192, 16384]
860
+ results = []
861
+
862
+ for M in m_values:
863
+ A = torch.randn(B, M, N, device="cuda", dtype=torch.float32)
864
+ hw = int(M**0.5)
865
+ tag = f"{hw}Γ—{hw}" if hw * hw == M else f"{M}"
866
+
867
+ auto_mean, _, _ = _cuda_timer(lambda: batched_svd(A, method='auto'))
868
+ torch_mean, _, _ = _cuda_timer(
869
+ lambda: torch.linalg.svd(A, full_matrices=False))
870
+
871
+ speedup = torch_mean / (auto_mean + 1e-9)
872
+ print(f" {M:>6} {tag:>8} {auto_mean:>8.3f}ms {torch_mean:>8.3f}ms {speedup:>7.2f}x")
873
+ results.append({'M': M, 'N': N, 'B': B,
874
+ 'auto_ms': round(auto_mean, 4), 'torch_ms': round(torch_mean, 4),
875
+ 'speedup': round(speedup, 3)})
876
+ del A; torch.cuda.empty_cache()
877
+
878
+ return results
879
+
880
+
881
+ def profile_crossover_detail(M=1024, B=512):
882
+ """Fine-grained N sweep around expected crossover points."""
883
+ print(f"\n{'='*70}")
884
+ print(f" CROSSOVER DETAIL β€” B={B}, M={M}")
885
+ print(f"{'='*70}")
886
+ print(f" {'N':>4} {'Gram ms':>10} {'Torch ms':>10} {'Winner':>8} {'Margin':>8}")
887
+ print(f" {'─'*46}")
888
+
889
+ for N in range(2, 65):
890
+ if N > M:
891
+ break
892
+ A = torch.randn(B, M, N, device="cuda", dtype=torch.float32)
893
+ gram_mean, _, _ = _cuda_timer(lambda: gram_eigh_svd(A), warmup=10, iters=40)
894
+ torch_mean, _, _ = _cuda_timer(
895
+ lambda: torch.linalg.svd(A, full_matrices=False), warmup=10, iters=40)
896
+
897
+ winner = "gram" if gram_mean < torch_mean else "torch"
898
+ margin = abs(gram_mean - torch_mean) / min(gram_mean, torch_mean) * 100
899
+
900
+ print(f" {N:>4} {gram_mean:>8.3f}ms {torch_mean:>8.3f}ms {winner:>8} {margin:>6.1f}%")
901
+ del A; torch.cuda.empty_cache()
902
+
903
+
904
+ # ╔═══════════════════════════════════════════════════════════════════════════╗
905
+ # β•‘ MAIN β•‘
906
+ # β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•
907
+
908
+ def main():
909
+ """Full profiling suite."""
910
+ assert torch.cuda.is_available(), "CUDA required"
911
+ device_name = torch.cuda.get_device_name(0)
912
+ print(f"{'='*80}")
913
+ print(f" Generalized Batched Thin SVD β€” Profiling Suite")
914
+ print(f" Device: {device_name}")
915
+ print(f"{'='*80}")
916
+
917
+ # Correctness first
918
+ run_validation(B=64, M=1024)
919
+
920
+ # Projection quality analysis β€” THE KEY QUESTION: does projected SVD work?
921
+ proj_results = profile_projection_quality(B=256, M=1024)
922
+
923
+ # N dimension sweep β€” timing comparison
924
+ n_results = profile_n_sweep(B=512, M=1024)
925
+
926
+ # Skip batch/spatial/crossover sweeps by default β€” uncomment if needed
927
+ batch_results = {}
928
+ spatial_results = {}
929
+ # for N in [3, 8, 32, 64]:
930
+ # batch_results[N] = profile_batch_sweep(N=N, M=1024)
931
+ # for N in [3, 16, 48]:
932
+ # spatial_results[N] = profile_spatial_sweep(N=N, B=512)
933
+ # profile_crossover_detail(M=1024, B=512)
934
+
935
+ # Summary
936
+ print(f"\n{'='*80}")
937
+ print(f" SUMMARY")
938
+ print(f"{'='*80}")
939
+ print(f"\n Strategy by N:")
940
+ print(f" N=2: Fused Triton (closed-form Jacobi rotation)")
941
+ print(f" N=3: Fused Triton (cyclic Jacobi in registers)")
942
+ print(f" N=4-32: Gram + eigh (bmm + cuSOLVER eigh) β€” sub-ms")
943
+ print(f" N=48+: Projected SVD (N→k, cheap SVD, lift back) — check quality table")
944
+ print(f"")
945
+ print(f" Standalone utilities:")
946
+ print(f" newton_schulz_invsqrt(G) β€” batched G^{{-1/2}} via pure bmm")
947
+ print(f" projected_svd(A, target_rank=k) β€” rank-k approximate SVD")
948
+ print(f" projected_svd_quality(A, target_rank) β€” measure approximation quality")
949
+ print(f"")
950
+ print(f" Key question answered: energy_ratio and subspace_cos in quality table")
951
+
952
+ # Save results
953
+ report = {
954
+ 'device': device_name,
955
+ 'n_sweep': n_results,
956
+ 'projection_quality': proj_results,
957
+ 'batch_sweeps': {str(k): v for k, v in batch_results.items()},
958
+ 'spatial_sweeps': {str(k): v for k, v in spatial_results.items()},
959
+ }
960
+ with open('svd_general_profile.json', 'w') as f:
961
+ json.dump(report, f, indent=2)
962
+ print(f"\n Results saved to svd_general_profile.json")
963
+ print(f"{'='*80}")
964
+
965
+
966
+ if __name__ == "__main__":
967
+ main()