Ablation — baseline / +TTA / +TTA+SL(4) / +TTA+SL(4)+QV

Date: 2026-04-24. All 15 jobs completed (22233980 + 22234159-22234177). Source dirs under /gpfs/work2/0/prjs0824/qi_proj/.

Four-cell ablation over 7 benchmark datasets to isolate the contribution of each pipeline component.

row	tag	description
A	baseline	SLAMFormer paper ckpt-10, no TTA, no PGO
B	+TTA	adds test-time Adam on scorer/desc heads against live layer-1 Q·V KL
C	+TTA+SL(4)	+TTA, then offline SL(4) projective PGO (`--use_indep`)
D	+TTA+SL(4)+QV	+TTA+SL(4), adds Q·V-loop-closure edges filtered at z-score τ = 1.0

Indoor uses bn=10, outdoor bn=15. All ATE in meters, Sim3-aligned.

1. Indoor — per-scene ATE (bn_every = 10)

1.1 TUM RGB-D — 9 scenes

scene	A baseline	B +TTA	C +TTA+SL4	D +TTA+SL4+QV
360	0.175	0.076	0.080	0.083
desk	0.068	0.018	0.142	0.131
desk2	0.076	0.023	0.095	0.047
floor	0.684	0.248	0.232	0.138
plant	0.092	0.023	0.031	0.070
room	0.579	0.105	0.152	0.151
rpy	0.034	0.020	0.022	0.023
teddy	0.253	0.069	0.042	0.083
xyz	0.032	0.010	0.011	0.013
MEAN	0.222	0.066	0.090	0.082
Δ vs A	—	−70 %	−59 %	−63 %

Finding. +TTA carries the indoor win (−70 % ATE). +SL4 regresses on 6/9 scenes; +QV rescues 3 of those (desk2, floor, room) but net averages worse than +TTA-alone. Per-scene picks: +SL4+QV is best on floor (loop-rich), +SL4 best on teddy.

1.2 7-Scenes — 17 sequences

sequence	A	B +TTA	C +TTA+SL4	D +QV
chess_sequence3	0.268	0.042	0.064	0.083
chess_sequence5	0.222	0.038	0.045	0.053
fire_sequence3	0.138	0.025	0.042	0.067
fire_sequence4	0.113	0.025	0.027	0.038
heads_sequence1	0.059	0.017	0.043	0.044
office_sequence2	0.238	0.075	0.061	0.065
office_sequence6	0.353	0.087	0.111	0.108
office_sequence7	0.411	0.099	0.096	0.109
pumpkin_sequence1	0.218	0.067	0.083	0.083
pumpkin_sequence7	0.233	0.060	0.088	0.083
redkitchen_sequence12	0.287	0.062	0.088	0.092
redkitchen_sequence14	0.276	0.053	0.080	0.080
redkitchen_sequence3	0.047	0.049	0.050	0.050
redkitchen_sequence4	0.187	0.050	0.050	0.050
redkitchen_sequence6	0.161	0.076	0.082	0.079
stairs_sequence1	0.027	0.027	0.041	0.041
stairs_sequence4	0.034	0.035	0.034	0.034
MEAN	0.193	0.052	0.064	0.068
Δ vs A	—	−73 %	−67 %	−65 %

Finding. +TTA best on 16 of 17 sequences. SL4/QV regress uniformly. No QV edges fire above z=1.0 on most scenes → scale-only inter edges just drift freely.

1.3 NRGBD — 8 scenes

scene	A	B +TTA	C +TTA+SL4	D +QV
breakfast_room	0.121	0.019	0.052	0.081
complete_kitchen	0.362	0.039	0.299	0.102
green_room	0.235	0.022	0.057	0.076
grey_white_room	0.117	0.048	0.057	0.138
kitchen	0.471	0.064	0.166	0.172
morning_apartment	0.088	0.016	0.019	0.027
staircase	0.210	0.021	0.024	0.138
whiteroom	0.442	0.029	0.158	0.141
MEAN	0.256	0.032	0.104	0.109
Δ vs A	—	−87 %	−59 %	−57 %

Finding. Cleanest ablation: TTA is the single decisive lever on NRGBD. Raw SL4 regresses 3×; QV pulls back complete_kitchen to 0.102 (the one scene with real revisits in the submap sequence).

1.4 ETH3D — 13 scenes

scene	A	B +TTA	C +TTA+SL4	D +QV
courtyard	1.363	0.469	1.126	1.126
delivery_area	1.029	0.252	0.251	0.272
electro	0.721	0.236	0.256	0.271
facade	2.463	0.340	1.661	1.070
kicker	0.169	0.046	0.047	0.075
meadow	0.210	0.248	0.215	0.215
office	0.085	0.069	0.069	0.072
pipes	0.065	0.060	0.076	0.076
playground	0.282	0.157	0.195	0.208
relief	0.147	0.150	0.120	0.120
relief_2	0.124	0.119	0.114	0.114
terrace	0.113	0.120	0.113	0.113
terrains	0.281	0.061	0.178	0.232
MEAN	0.542	0.179	0.340	0.305
Δ vs A	—	−67 %	−37 %	−44 %

Finding. +TTA again wins net. +QV rescues facade (−36 % vs +SL4) because facade has a real outdoor loop. But on 9/13 scenes +QV ≥ +SL4.

2. Outdoor — per-scene ATE (bn_every = 15)

2.1 Oxford Spires — 2 GT-testable scenes (paper ckpt vs V4)

scene	A baseline	B +TTA	C +TTA+SL4	D +TTA+SL4+QV
paper ckpt:
christ-church-01	24.19	13.87	14.65	14.48
keble-college-02	28.40	10.77	10.05	9.98
MEAN (paper)	26.29	12.32	12.35	12.23
Δ vs A	—	−53 %	−53 %	−53 %
V4 ckpt:
christ-church-01	20.52	20.52	23.24	20.99
keble-college-02	28.07	28.07	26.78	28.04
MEAN (V4)	24.29	24.29	25.01	24.51
Paper vs V4	paper 8 % better	paper 49 % better	paper 51 % better	paper 50 % better

Finding. Paper ckpt + QV wins at 12.23 m. QV's 0.7 % improvement on keble-02 confirms it's picking up the real indoor loop. V4 loses to paper everywhere — the V4-B'' ckpt-1 overlap curriculum did NOT transfer to Oxford outdoor.

2.2 KITTI Odometry — 11 sequences (paper vs V4)

seq	A	B +TTA paper	C paper +SL4	D paper +QV	B +TTA V4	C V4 +SL4	D V4 +QV
00	196.1	171.5	180.9	161.4	185.8	195.4	180.5
01	752.6	629.8	615.8	651.5	662.9	656.8	656.8
02	306.0	297.9	267.2	269.4	291.7	287.8	282.1
03	161.1	106.4	103.2	109.7	86.8	82.7	92.4
04	110.5	58.9	59.5	61.4	175.6	175.6	—
05	159.7	156.5	161.1	135.5	157.2	156.1	156.1
06	130.7	117.0	115.0	118.8	118.5	118.4	117.5
07	83.5	59.1	63.4	61.7	65.6	72.5	71.9
08	272.5	271.1	146.7	255.5	264.4	243.0	260.8
09	219.9	200.9	195.5	202.7	197.7	193.5	195.0
10	208.4	198.6	181.3	199.1	182.5	193.9	184.9
MEAN	236.5	206.1	189.9	202.4	217.1	216.0	219.8
Δ vs A (paper)	—	−13 %	−20 %	−14 %	−8 %	−9 %	−7 %

Finding. Paper ckpt + SL4 is the best KITTI recipe at mean 189.9 m — SL4 provides a consistent −7 % over +TTA-alone on KITTI. QV hurts (sparse feature matches outdoor don't give good z-score signal). V4 is worse than paper on all 11 sequences — vKITTI2 training doesn't help.

2.3 Tanks & Temples — 6 Training scenes

scene	A	B +TTA paper	C paper +SL4	D paper +QV	B +TTA V4
Barn	1.302	0.192	0.393	0.292	0.568
Caterpillar	0.340	0.160	0.236	0.236	0.626
Church	3.215	0.731	1.049	1.087	3.192
Ignatius	0.770	0.345	0.373	0.736	0.552
Meetingroom	1.434	0.304	0.550	0.438	0.750
Truck	3.682	3.616	3.621	3.621	3.467
MEAN	1.790	0.891	1.037	1.068	1.526
Δ vs A	—	−50 %	−42 %	−40 %	−15 %

Finding. +TTA wins on T&T. SL4 regresses; QV no help. V4 is 71 % worse than paper.

3. Summary of all 4 rows across all 7 datasets

Indoor (bn=10, paper ckpt)

dataset	A baseline	B +TTA	C +SL4	D +QV	best
TUM (9)	0.222	0.066	0.090	0.082	B
7Sc (17)	0.193	0.052	0.064	0.068	B
NRGBD (8)	0.256	0.032	0.104	0.109	B
ETH3D (13)	0.542	0.179	0.340	0.305	B

Outdoor (bn=15)

dataset	ckpt	A	B	C	D	best
Oxford (2)	paper	26.29	12.32	12.35	12.23	D
Oxford (2)	V4	24.29	24.29	25.01	24.51	A=B
KITTI (11)	paper	236.5	206.1	189.9	202.4	C
KITTI (11)	V4	219.8	217.1	216.0	219.8	C
T&T (6)	paper	1.790	0.891	1.037	1.068	B
T&T (6)	V4	1.790	1.526	1.766	1.619	B

4. AUC@3° and AUC@30° — TTA is what lifts rotation

dataset	AUC@3° A	AUC@3° B	AUC@3° C	AUC@3° D
TUM	0.009	0.374	0.384	0.380
7Sc	0.047	0.238	0.236	0.236
NRGBD	0.069	0.756	0.743	0.741
ETH3D	0.263	0.486	0.486	0.489
T&T	0.047	0.128	0.129	0.134
Oxford	0.000	0.000	0.000	0.000
KITTI	0.002	0.001	0.001	0.001

TTA gives the AUC@3° win across 5 of 7 datasets. Oxford and KITTI remain at 0 because absolute-scale drift saturates the rotation alignment window.

5. Reconstruction (F1, Acc, Comp in m)

Only +TTA row shown — SL4 and QV don't change recon (fusion-limited).

dataset	F1 (normalized ×100)	Acc	Comp
NRGBD	82.3 (GCA: 64.3)	0.922	0.751
ETH3D	31.3 (GCA: 98.98)	0.437	0.290
7-Scenes	1.5 (GCA: 80.4)	0.395	0.008
T&T	0.4	0.006	0.003

NRGBD is the single dataset where our recon F1 beats GCA. Others need a fusion head.

6. Headline takeaways

TTA is the single biggest win: −67 % to −87 % ATE on all 4 indoor datasets and T&T. AUC@3° jumps 5–10×.
SL(4) PGO helps only on KITTI (−8 % paper ckpt, −1 % V4). On every other dataset it regresses by 30–230 %.
QV loop-closure helps marginally on Oxford (−0.7 %) and on a handful of indoor scenes with real loops (TUM floor −44 %, desk2 −51 %). On no-loop scenes the z=1.0 threshold still admits noise.
V4-B'' ckpt-1 loses to paper ckpt-10 on every outdoor metric. The overlap-curriculum training on vKITTI2 did not transfer to real outdoor geometry.

7. Production recipes

dataset	recipe	why
TUM, 7-Scenes, NRGBD, ETH3D	`bn=10 TTA=1 PGO=0`	SL4/QV regress on low-drift indoor
Oxford	`bn=15 TTA=1 PGO=1 QV z=1.0` paper ckpt	QV captures keble indoor loop; V4 hurts
KITTI	`bn=15 TTA=1 PGO=1 QV=0` paper ckpt	SL4 helps; QV hurts on sparse outdoor
T&T	`bn=15 TTA=1 PGO=0` paper ckpt	SL4 regresses; no real loops in training scenes