| --- |
| license: apache-2.0 |
| --- |
| |
| I **could not** speed up algo **today**, however I found out something more crucial. |
|
|
| The cudasolver variation... is actually **quite inaccurate** compared to the traditional math results expectation. |
| It loses on many spectrums to regressive deflation, and I've organized a specific algorithm that supplements this exact |
| behavior for improved accuracy. |
|
|
| Faster, maybe not. Can't improve the speed. It's basically hardware optimized to the T until I hit the cuda kernels directly. |
|
|
| However, **accuracy**... in direct relation to the traditional math in a way that defeats the pytorch variation, we can do that. |
|
|
| So what we have here, is a more accurate fp64 variation of eigh THAT COMPILES. |
|
|
| Yes, full-graph compiles at around 75% speed, with improved... occasionally highly improved accuracy. |
|
|
| At a **FRACTION** of the vram cost. |
|
|
| ``` |
| ======================================================================== |
| FL Hybrid Eigh — Algebraic + Geometric Optimal |
| ======================================================================== |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | PyTorch 2.10.0+cu128 |
|
|
| ======================================================================== |
| MATHEMATICAL PURITY (no reference impl, only definitions) |
| ======================================================================== |
|
|
| n=3 B=2048: FL wins 12/12, cuSOLVER wins 0/12 |
| Property cuSOLVER FL |
| Eigenpair max 6.3e-07 1.8e-07 FL ◄ |
| Eigenpair mean 9.6e-08 3.5e-08 FL ◄ |
| Orthogonality max 1.1e-06 2.2e-07 FL ◄ |
| Orthogonality mean 3.6e-07 8.3e-08 FL ◄ |
| Reconstruction max 1.1e-06 3.1e-07 FL ◄ |
| Reconstruction mean 2.8e-07 9.3e-08 FL ◄ |
| Trace max 1.7e-06 8.9e-07 FL ◄ |
| Determinant max 5.2e-04 9.5e-05 FL ◄ |
| Char poly max 2.3e-05 1.7e-05 FL ◄ |
| Char poly mean 6.3e-07 3.3e-07 FL ◄ |
| |
| n=5 B=2048: FL wins 12/12, cuSOLVER wins 0/12 |
| Property cuSOLVER FL |
| Eigenpair max 5.7e-07 1.8e-07 FL ◄ |
| Eigenpair mean 1.2e-07 3.1e-08 FL ◄ |
| Orthogonality max 1.9e-06 5.7e-07 FL ◄ |
| Orthogonality mean 7.2e-07 1.3e-07 FL ◄ |
| Reconstruction max 1.1e-06 3.0e-07 FL ◄ |
| Reconstruction mean 4.3e-07 1.1e-07 FL ◄ |
| Trace max 2.5e-06 9.8e-07 FL ◄ |
| Determinant max 1.8e-03 8.8e-05 FL ◄ |
| Char poly max 1.1e-03 5.1e-04 FL ◄ |
| Char poly mean 1.0e-05 4.4e-06 FL ◄ |
| |
| n=6 B=2048: FL wins 12/12, cuSOLVER wins 0/12 |
| Property cuSOLVER FL |
| Eigenpair max 5.4e-07 1.8e-07 FL ◄ |
| Eigenpair mean 1.3e-07 2.9e-08 FL ◄ |
| Orthogonality max 2.1e-06 3.1e-07 FL ◄ |
| Orthogonality mean 9.0e-07 1.5e-07 FL ◄ |
| Reconstruction max 1.3e-06 3.0e-07 FL ◄ |
| Reconstruction mean 4.9e-07 1.1e-07 FL ◄ |
| Trace max 3.0e-06 1.2e-06 FL ◄ |
| Determinant max 3.4e-03 3.7e-04 FL ◄ |
| Char poly max 6.6e-03 2.1e-03 FL ◄ |
| Char poly mean 4.3e-05 1.9e-05 FL ◄ |
| |
| n=8 B=2048: FL wins 12/12, cuSOLVER wins 0/12 |
| Property cuSOLVER FL |
| Eigenpair max 6.7e-07 2.1e-07 FL ◄ |
| Eigenpair mean 1.3e-07 2.6e-08 FL ◄ |
| Orthogonality max 2.5e-06 2.0e-06 FL ◄ |
| Orthogonality mean 1.2e-06 1.9e-07 FL ◄ |
| Reconstruction max 1.6e-06 7.8e-07 FL ◄ |
| Reconstruction mean 5.9e-07 1.1e-07 FL ◄ |
| Trace max 3.8e-06 1.6e-06 FL ◄ |
| Determinant max 3.0e-03 2.6e-04 FL ◄ |
| Char poly max 2.7e-01 1.4e-01 FL ◄ |
| Char poly mean 1.2e-03 4.6e-04 FL ◄ |
| |
| n=10 B=1024: FL wins 12/12, cuSOLVER wins 0/12 |
| Property cuSOLVER FL |
| Eigenpair max 9.5e-07 1.4e-07 FL ◄ |
| Eigenpair mean 1.3e-07 2.5e-08 FL ◄ |
| Orthogonality max 2.8e-06 4.4e-07 FL ◄ |
| Orthogonality mean 1.4e-06 2.2e-07 FL ◄ |
| Reconstruction max 1.3e-06 3.0e-07 FL ◄ |
| Reconstruction mean 6.2e-07 1.2e-07 FL ◄ |
| Trace max 7.0e-06 1.7e-06 FL ◄ |
| Determinant max 2.3e-04 1.3e-05 FL ◄ |
| Char poly max 3.1e+01 6.5e+00 FL ◄ |
| Char poly mean 3.9e-02 1.4e-02 FL ◄ |
| |
| n=12 B=1024: FL wins 10/12, cuSOLVER wins 2/12 |
| Property cuSOLVER FL |
| Eigenpair max 8.5e-07 8.3e-07 FL ◄ |
| Eigenpair mean 1.2e-07 2.3e-08 FL ◄ |
| Orthogonality max 2.9e-06 4.5e-04 cuS ► |
| Orthogonality mean 1.6e-06 7.8e-07 FL ◄ |
| Reconstruction max 1.4e-06 1.5e-04 cuS ► |
| Reconstruction mean 6.3e-07 2.8e-07 FL ◄ |
| Trace max 9.0e-06 2.0e-06 FL ◄ |
| Determinant max 3.0e-04 5.5e-05 FL ◄ |
| Char poly max 9.9e+02 1.7e+02 FL ◄ |
| Char poly mean 1.3e+00 4.7e-01 FL ◄ |
| |
| ======================================================================== |
| ACCURACY PASS/FAIL |
| ======================================================================== |
| [OK] n= 3 val_diff=1.7e-06 align=1.000000 |
| [OK] n= 4 val_diff=1.9e-06 align=0.999999 |
| [OK] n= 5 val_diff=1.9e-06 align=0.999999 |
| [OK] n= 6 val_diff=2.4e-06 align=0.999998 |
| [OK] n= 8 val_diff=3.3e-06 align=0.999999 |
| [OK] n=10 val_diff=2.0e-05 align=0.999910 |
| [OK] n=12 val_diff=8.1e-06 align=0.999997 |
| [OK] n=16 val_diff=3.3e-05 align=0.999701 |
|
|
| ======================================================================== |
| THROUGHPUT (n=6 B=4096) |
| ======================================================================== |
| cuSOLVER: 243.4µs |
| FL eager: 8.11ms (0.03×) |
| Compiling... /usr/local/lib/python3.12/dist-packages/torch/_inductor/lowering.py:1904: FutureWarning: `torch._prims_common.check` is deprecated and will be removed in the future. Please use `torch._check*` functions instead. |
| check( |
| done. |
| FL compiled: 338.0µs (0.72×) |
| |
| MEMORY |
| cuSOLVER 1098.7MB |
| FL 32.3MB |
| |
| ======================================================================== |
| All pass: True |
| Compiled: 0.72× vs cuSOLVER |
| ======================================================================== |
| ``` |
| |
| |
| |
| Testing the barrage setup is reveals much information about parallel processing capacity. |
| |
| ``` |
| ============================================================================== |
| Diagnostic: Parallel Root-Finding |
| ============================================================================== |
| B=512 N=6 |
| True eigenvalue range: [-2.106, 2.099] |
| Diagonal init range: [-1.561, 1.722] |
| |
| --- Test 1: Pure Laguerre (no Aberth) --- |
| PurL it= 0 max_err=1.75e+00 min_gap=1.73e-06 |p(z)|=3.93e+00 |
| PurL it= 1 max_err=1.84e+00 min_gap=1.67e-16 |p(z)|=5.39e-01 |
| PurL it= 2 max_err=1.89e+00 min_gap=0.00e+00 |p(z)|=2.01e-02 |
| PurL it= 3 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=1.73e-04 |
| PurL it= 4 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=7.12e-07 |
| PurL it= 9 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=2.88e-15 |
| PurL it=14 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=2.88e-15 |
| PurL it=19 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=2.88e-15 |
| |
| --- Test 2: Laguerre + Aberth (full) --- |
| LA-F it= 0 max_err=1.03e+02 min_gap=5.69e-07 |p(z)|=1.33e+12 |
| LA-F it= 1 max_err=3.70e+02 min_gap=1.71e-06 |p(z)|=2.62e+15 |
| LA-F it= 2 max_err=4.63e+02 min_gap=5.12e-06 |p(z)|=1.00e+16 |
| LA-F it= 3 max_err=1.58e+03 min_gap=1.54e-05 |p(z)|=1.59e+19 |
| LA-F it= 4 max_err=1.98e+03 min_gap=4.61e-05 |p(z)|=6.05e+19 |
| LA-F it= 9 max_err=6.05e+03 min_gap=5.26e-04 |p(z)|=4.89e+22 |
| LA-F it=14 max_err=1.85e+04 min_gap=8.50e-04 |p(z)|=3.95e+25 |
| LA-F it=19 max_err=5.63e+04 min_gap=1.92e-02 |p(z)|=3.19e+28 |
| |
| --- Test 3: Laguerre + weak Aberth (0.1x) --- |
| LA.1 it= 0 max_err=2.89e+01 min_gap=5.69e-05 |p(z)|=7.50e+08 |
| LA.1 it= 1 max_err=2.09e+01 min_gap=2.84e-06 |p(z)|=1.23e+08 |
| LA.1 it= 2 max_err=1.35e+01 min_gap=6.74e-07 |p(z)|=1.06e+07 |
| LA.1 it= 3 max_err=6.44e+00 min_gap=4.80e-08 |p(z)|=2.25e+05 |
| LA.1 it= 4 max_err=1.89e+00 min_gap=4.16e-09 |p(z)|=1.99e+01 |
| LA.1 it= 9 max_err=1.90e+00 min_gap=1.45e-14 |p(z)|=4.90e-03 |
| LA.1 it=14 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=6.48e-04 |
| LA.1 it=19 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=8.53e-05 |
| |
| --- Test 4: Pure Laguerre + re-sort --- |
| PL+S it= 0 max_err=1.75e+00 min_gap=1.73e-06 |p(z)|=3.93e+00 |
| PL+S it= 1 max_err=1.84e+00 min_gap=1.67e-16 |p(z)|=5.39e-01 |
| PL+S it= 2 max_err=1.89e+00 min_gap=0.00e+00 |p(z)|=2.01e-02 |
| PL+S it= 3 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=1.73e-04 |
| PL+S it= 4 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=7.12e-07 |
| PL+S it= 9 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=2.88e-15 |
| PL+S it=14 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=2.88e-15 |
| PL+S it=19 max_err=1.90e+00 min_gap=0.00e+00 |p(z)|=2.88e-15 |
| |
| --- Test 5: Laguerre + Aberth damped (0.1 → 1.0) --- |
| LADa it= 0 max_err=2.89e+01 min_gap=5.69e-05 |p(z)|=7.50e+08 |
| LADa it= 1 max_err=3.72e+02 min_gap=3.82e-06 |p(z)|=2.69e+15 |
| LADa it= 2 max_err=1.13e+03 min_gap=4.82e-06 |p(z)|=2.08e+18 |
| LADa it= 3 max_err=2.26e+03 min_gap=1.90e-06 |p(z)|=1.34e+20 |
| LADa it= 4 max_err=3.77e+03 min_gap=9.02e-07 |p(z)|=2.88e+21 |
| LADa it= 9 max_err=1.70e+04 min_gap=2.34e-05 |p(z)|=2.38e+25 |
| LADa it=14 max_err=5.17e+04 min_gap=2.25e-03 |p(z)|=1.91e+28 |
| LADa it=19 max_err=1.58e+05 min_gap=8.72e-03 |p(z)|=1.54e+31 |
| |
| --- Test 6: Newton + Aberth --- |
| NwAb it= 0 max_err=4.35e+02 min_gap=3.29e-05 |p(z)|=6.91e+15 |
| NwAb it= 1 max_err=1.57e+01 min_gap=9.86e-05 |p(z)|=2.43e+07 |
| NwAb it= 2 max_err=5.28e+01 min_gap=1.70e-05 |p(z)|=2.54e+10 |
| NwAb it= 3 max_err=5.37e+01 min_gap=5.22e-05 |p(z)|=2.75e+10 |
| NwAb it= 4 max_err=3.34e+02 min_gap=1.91e-04 |p(z)|=1.41e+15 |
| NwAb it= 9 max_err=2.02e+00 min_gap=1.92e-02 |p(z)|=6.78e+02 |
| NwAb it=14 max_err=1.05e-06 min_gap=1.92e-02 |p(z)|=1.24e-14 |
| NwAb it=19 max_err=1.05e-06 min_gap=1.92e-02 |p(z)|=1.24e-14 |
| |
| --- Test 7: Pure Newton --- |
| PurN it= 0 max_err=3.51e+02 min_gap=6.24e-06 |p(z)|=1.93e+15 |
| PurN it= 1 max_err=2.93e+02 min_gap=1.69e-09 |p(z)|=6.46e+14 |
| PurN it= 2 max_err=2.44e+02 min_gap=0.00e+00 |p(z)|=2.16e+14 |
| PurN it= 3 max_err=2.03e+02 min_gap=0.00e+00 |p(z)|=7.25e+13 |
| PurN it= 4 max_err=1.69e+02 min_gap=0.00e+00 |p(z)|=2.43e+13 |
| PurN it= 9 max_err=6.69e+01 min_gap=0.00e+00 |p(z)|=1.02e+11 |
| PurN it=14 max_err=2.60e+01 min_gap=0.00e+00 |p(z)|=4.31e+08 |
| PurN it=19 max_err=9.63e+00 min_gap=0.00e+00 |p(z)|=1.81e+06 |
| ============================================================================== |
| |
| ``` |
