MaybeRichard commited on
Commit
377f288
·
verified ·
1 Parent(s): cf377f1

manuscript tex (7pp)

Browse files
Files changed (1) hide show
  1. results/manuscript/octflow_main.tex +23 -8
results/manuscript/octflow_main.tex CHANGED
@@ -52,9 +52,9 @@ Two abilities are out of reach for interpretation-only foundation models. The fi
52
  % 诚实结论(2026-06-10):纯合成跨中心=可用但不占优(每院 0.85-0.96,均<该院真实;best_epoch 三seed全=0,即立刻过拟合需早停);真正生成数据卖点是合成增广(mixed>real)。已去[待补]占位。
53
  % (c) FD-loss 提升生成→失败(已停,写作 negative ablation)。
54
 
55
- \subsection{Generalisation and external validation}\label{sec:validation}
56
  % [待补=用户资源] 多中心外部验证(未用过的公开集 + 1-2 家医院私有,不同设备/人群,免/少微调泛化)。
57
- % [待补=用户资源] 医生 reader study(生成图保真度 + 分割临床可用性)。
58
  % RIGOR: 这是冲 Nature/NBE 的硬门槛,目前完全缺,必须用户侧补。
59
 
60
  %% =========================== 3. DISCUSSION ===========================
@@ -69,12 +69,27 @@ Two forms of evidence remain to be added before clinical claims can be made: ext
69
 
70
  %% =========================== 4. METHODS ===========================
71
  \section{Methods}\label{sec:methods}
72
- % [详见 reports/EXPERIMENT_JOURNAL + CLAUDE.md]
73
- % 数据;SD3 域适配(diffusers 官方 SD3 loss,LR1e-4);channel-concat 16→32;指令微调+解码空间loss;
74
- % VAE 域微调;评测协议(冻结探针×3seed + lockbox + cluster bootstrap CI;分割 mIoU/Dice/逐类;
75
- % 生成 FD-RETFound+Inception-FID/KID/P/R;去噪 PSNR/SSIM/LPIPS@512);
76
- % 消融[数据已有]:域适配on/off、解码空间loss(v3a→v3b→v3b2)、多prompt(v2专才vs v3a通才)。
77
- % [negative ablation] FD-loss 后训练:lr 太低不动 / 太高退化 → 对本设置不奏效。
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
78
 
79
  %\bibliography{sn-bibliography}% RETFound, Vision-Banana, SD3, BUSGen, DDAE 等
80
 
 
52
  % 诚实结论(2026-06-10):纯合成跨中心=可用但不占优(每院 0.85-0.96,均<该院真实;best_epoch 三seed全=0,即立刻过拟合需早停);真正生成数据卖点是合成增广(mixed>real)。已去[待补]占位。
53
  % (c) FD-loss 提升生成→失败(已停,写作 negative ablation)。
54
 
55
+ % \subsection{Generalisation and external validation} % [待补=用户资源] 暂注释,无结果不显示空标题
56
  % [待补=用户资源] 多中心外部验证(未用过的公开集 + 1-2 家医院私有,不同设备/人群,免/少微调泛化)。
57
+ % [待补=用户资源] 医生 reader study(生成图保真度 + 分割临床可用性)。Discussion 末段已写为下一步。
58
  % RIGOR: 这是冲 Nature/NBE 的硬门槛,目前完全缺,必须用户侧补。
59
 
60
  %% =========================== 3. DISCUSSION ===========================
 
69
 
70
  %% =========================== 4. METHODS ===========================
71
  \section{Methods}\label{sec:methods}
72
+
73
+ \subsection{Data}
74
+ We assembled 1.1 million ophthalmic images: OCT B-scans (914{,}000), colour fundus photographs (177{,}000) and scanning laser ophthalmoscopy images (30{,}000). A paired subset of 30{,}734 Topcon DRI OCT Triton studies from a single centre and a single device provides, for each study, twelve radial OCT B-scans, one fundus image, one SLO image and a ten-layer retinal segmentation. Public OCT data (488{,}000 images across seven scanner vendors) and public fundus data (198{,}000 images across six cohorts) carry disease labels; about 510{,}000 segmentation masks are available, of which 370{,}000 are ten-layer OCT maps. Each image carries five machine-generated captions used as text conditioning. Images are routed to the correct store by a cohort field, and patient identifiers keep every training, validation and test split patient-disjoint.
75
+
76
+ \subsection{Backbone and domain adaptation}
77
+ The backbone is Stable Diffusion 3 medium, a latent diffusion model with a variational autoencoder that compresses images into sixteen-channel latents and a two-billion-parameter diffusion transformer, pre-trained on natural images. We adapt it to ophthalmic images by continued text-to-image training on the full corpus, reusing the published Stable Diffusion 3 training objective exactly (a flow-matching loss with logit-normal timestep sampling and a shifted noise schedule) at a learning rate of $10^{-4}$; a mismatched objective silently fails to train, so the loss is not re-derived. The domain-adapted base model is trained for 240{,}000 steps.
78
+
79
+ \subsection{Instruction tuning}
80
+ To bring image-to-image tasks such as segmentation and denoising into the same generative model, the reference image is supplied by channel concatenation: the patch-embedding convolution of the transformer is widened from sixteen to thirty-two input channels, the new channels are initialised to zero, and all weights are warm-started from the domain-adapted base. Instruction tuning then mixes text-to-image generation with image-to-map tasks. For each segmentation example an instruction is rendered on the fly from the integer label map, stating how many layers to produce (nine, five or three), which colour to assign to each layer, or a request to segment only a named structure, so the model must read the instruction to produce the correct map. A decoded-space objective (cross-entropy against the colour palette plus a soft Dice term, weighted inversely to layer frequency) sharpens the thinnest layers, and the final model additionally uses a decoder fine-tuned on colour layer maps. Because every instruction is generated from an integer label map, layer schemes with counts the model never saw in training (for example seven or four layers) can be evaluated zero-shot against exactly matched ground truth.
81
+
82
+ \subsection{Evaluation}
83
+ Disease understanding is measured by freezing the model's image features and training only a linear classifier, repeated over three random seeds, with patients held out between training and test and confidence intervals from a patient-level cluster bootstrap; we also report the accuracy gap across scanner brands and the accuracy at one per cent of labels. Every comparison model is evaluated under the identical protocol, and for segmentation and denoising each baseline encoder is paired with a trained decoder so the comparison is like-for-like. Segmentation is scored by mean and per-class overlap (intersection-over-union) and by boundary distances (the average symmetric surface distance and the 95th-percentile Hausdorff distance, computed from Euclidean distance transforms) over three seeds. Generation realism is measured by the Fréchet distance in the feature space of a retinal image model and with the standard Inception network, together with a kernel distance and precision/recall, on 5{,}000 images over three seeds. Denoising is scored by peak signal-to-noise ratio, structural similarity and a perceptual distance, all at a matched 512-pixel resolution. Zero-shot behaviour is probed along two axes: instruction generalisation to unseen layer counts and to unseen scanners and disease, and disease classification with no trained classifier, obtained by scoring how well the generative model reconstructs an image under each candidate diagnosis. The value of synthetic data is measured by training diagnostic models on real, synthetic or mixed data at four dataset sizes over three seeds.
84
+
85
+ \subsection{Privacy-preserving cross-centre experiment}
86
+ A separate copy of the generator was fine-tuned on each of three independent OCT datasets. From each fine-tuned generator we sampled labelled synthetic images, so that no real image left its source dataset, and trained a single normal-versus-abnormal classifier on the pooled synthetic images. The classifier was tested on held-out real images from each dataset, with three seeds and patient-disjoint splits. Because a classifier trained only on synthetic images overfits within a single epoch, the training epoch was selected on a small held-out real validation set rather than on the test set. Memorisation was assessed with SSCD copy-detection embeddings, reporting the distribution of each generated image's nearest-neighbour cosine similarity to the training set.
87
+
88
+ \subsection{Ablations and a negative result}
89
+ Controlled comparisons isolate the contribution of domain adaptation (present versus absent), of the decoded-space objective (across three successive refinements) and of instruction diversity (a single-scheme specialist versus the multi-scheme generalist). We also report a negative result: post-training the generator to minimise a differentiable Fréchet distance in a frozen feature space did not improve generation in our setting, where a low learning rate produced no change and a higher one degraded the generator, and it was therefore not adopted.
90
+
91
+ \subsection*{Data and code availability}
92
+ Domain-adapted and instruction-tuned model weights, the full results, and the pipeline code are available in a model repository [repository link]. Public datasets are available from their original sources; the single-centre paired dataset is available from the corresponding author under a data-use agreement, subject to patient-privacy constraints.
93
 
94
  %\bibliography{sn-bibliography}% RETFound, Vision-Banana, SD3, BUSGen, DDAE 等
95