# ReMAP-PET Paper and Experiment Plan ## Working Title **ReMAP-PET: Region-guided Metabolic Alignment with Partial-tuned PET Encoders** ## Central Claim FDG-PET should not be treated only as an image classification input. Its paired 120-region SUVR profile provides structured metabolic semantics. ReMAP-PET uses this region-level supervision to partially adapt a pretrained 3D medical/brain encoder into a PET metabolic encoder. The core claim is: > High-level partial tuning plus regional metabolic alignment produces a stronger FDG-PET representation than frozen probing, and it gives a usable bridge from 3D PET to structured metabolic language. ## Paper Structure ### 1. Introduction Motivation: - FDG-PET reflects glucose metabolism rather than anatomy. - Existing brain MRI foundation models are not directly optimized for PET metabolic semantics. - Current PET work often focuses on diagnosis labels, while our data provides richer region-level SUVR supervision. Main contribution: - Define PET-to-regional metabolic alignment. - Propose ReMAP-PET, a partial-tuned PET encoder trained with SUVR regression and PET-SUVR contrastive alignment. - Evaluate against frozen pretrained encoders and newer brain/neuroimaging foundation encoders. - Show interpretable region-level prediction and retrieval. ### 2. Related Work Groups to cover: - Brain MRI foundation models: BrainIAC, BrainFound, BrainDINO. - General clinical neuroimaging foundation models: NeuroVFM. - 3D MRI vision-language models: Decipher-MR. - Generative multimodal brain models: GenBrain. - FDG-PET and AD metabolic imaging. - Vision-language / CLIP-style contrastive alignment. ### 3. Dataset and Task Dataset: - 1015 paired FDG-PET volumes and SUVR CSV tables. - 120 real brain regions after excluding `Background`. - Fixed split: 710 train, 152 val, 153 test. Tasks: - PET-to-SUVR regression. - PET-SUVR bidirectional retrieval. - High/low metabolic region identification. - Controlled region-text alignment in the later stage. ### 4. Method ReMAP-PET Stage 1: ```text 3D FDG-PET -> pretrained 3D encoder -> partial tuning of high-level block -> PET embedding -> SUVR regression head -> contrastive alignment with SUVR embedding ``` Training loss: ```text L = lambda_reg * L_suvr + lambda_con * L_pet_suvr_contrastive ``` Main implementation: - PET encoder: MedicalNet 3D ResNet-50. - Partial tuning: only `layer4`. - Current best: `lambda_reg=1.0`, `lambda_con=0.2`. ### 5. Experiments #### Table 1: Dataset Summary Purpose: - Show what data we have and why the task is not ordinary classification. Columns: - Dataset - Samples - PET format - SUVR regions - Train / Val / Test - Labels available - Task supported Rows: - Our FDG-PET-SUVR dataset. - External ADNI/OASIS-3 if acquired later. #### Table 2: Main Stage 1 Internal Test Results Purpose: - Main quantitative result on the held-out test set. Rows: - MedicalNet frozen MLP. - BrainIAC frozen MLP. - ReMAP-PET / MedicalNet layer4 partial tuning. - BrainFound. - NeuroVFM encoder. - Decipher-MR. - BrainDINO. - GenBrain encoder. Metrics: - MAE - RMSE - Pearson - Spearman - PET->SUVR R@1 / R@5 - SUVR->PET R@1 / R@5 - MRR - Median Rank - Top5 High - Top5 Low #### Table 3: Modern Encoder Comparison Purpose: - Show whether newer brain/neuroimaging foundation encoders transfer better than the current ReMAP-PET backbone. Rows: - BrainFound. - NeuroVFM encoder. - Decipher-MR. - BrainDINO. - GenBrain encoder. - ReMAP-PET as reference. Settings: - Frozen probing. - Adapter tuning if feasible. - High-level partial tuning if the architecture supports it. Primary metrics: - MAE - Pearson - PET->SUVR R@1 - SUVR->PET R@1 - Top5 Low #### Table 4: ReMAP-PET Ablation Purpose: - Justify the ReMAP-PET training objective and show why the chosen partial-tuned alignment design works. Rows: - Regression only: `lambda_con=0.0`, `lambda_reg=1.0`. - Weak contrastive: `lambda_con=0.1`, `lambda_reg=1.0`. - Current setting: `lambda_con=0.2`, `lambda_reg=1.0`. - Strong contrastive: `lambda_con=0.5`, `lambda_reg=1.0`. - Contrastive-heavy: `lambda_con=1.0`, `lambda_reg=1.0`. Metrics: - MAE - RMSE - Pearson - Spearman - PET->SUVR R@1 - SUVR->PET R@1 - Top5 High - Top5 Low Expected interpretation: - Regression-only gives the best numeric SUVR prediction but fails at PET-SUVR retrieval. - Stronger contrastive weights greatly improve retrieval but gradually hurt numeric SUVR prediction. - The balanced ReMAP-PET setting should be selected by the intended Stage 1 target: metabolic prediction plus cross-modal alignment, not regression alone. Current test results are stored in: ```text docs/REMAP_PET_ABLATION_RESULTS.csv ``` Main observation: ```text lambda_con=0.0 -> MAE 0.0549, PET->SUVR R@1 0.0065 lambda_con=0.2 -> MAE 0.0700, PET->SUVR R@1 0.7778 lambda_con=0.5 -> MAE 0.0766, PET->SUVR R@1 0.8366 ``` This means pure regression is not enough for the paper goal. ReMAP-PET needs contrastive alignment because the paper claims a PET-to-regional-metabolic semantic space, not only SUVR curve fitting. #### Table 5: External Generalization Purpose: - Show the learned PET metabolic encoder is not only fitting our internal dataset. Candidate external datasets: - ADNI FDG-PET with PET Core SUVR summaries. - OASIS-3 FDG-PET with post-processed PET outputs. Rows: - Frozen MedicalNet. - ReMAP-PET. - Best modern encoder variant. Metrics: - If SUVR available: MAE, RMSE, Pearson, Spearman. - If diagnosis labels available: linear probe AUROC / balanced accuracy. - Retrieval if paired PET-SUVR is available. #### Table 6: Region Interpretability Purpose: - Demonstrate clinical interpretability at the brain-region level. Rows: - Model variants. Metrics: - Top5 Low overlap. - Top5 High overlap. - Region mention precision. - Region mention recall. - Example low-metabolism regions recovered. ### 6. Figures #### Figure 1: Method Overview Show: ```text 3D PET -> partial-tuned PET encoder -> PET embedding SUVR table -> region encoder -> SUVR embedding PET embedding <-> SUVR embedding contrastive alignment PET embedding -> 120-region SUVR prediction ``` #### Figure 2: Embedding Retrieval Visualization Show: - PET-SUVR similarity matrix. - Correct pairs on diagonal. - Compare frozen MedicalNet vs ReMAP-PET. #### Figure 3: Region-level Case Study Show: - Predicted vs true top low-metabolism regions. - Controlled text generated from true/predicted SUVR. - Example similar-case retrieval. #### Figure 4: Ablation Tradeoff Curve Show: - x-axis: contrastive weight. - y-axis 1: MAE. - y-axis 2: PET->SUVR Recall@1. ## Experiment Execution Order 1. Finish ReMAP-PET loss-weight ablation. 2. Evaluate each best checkpoint on the internal test set. 3. Integrate the five chosen modern encoders: - BrainFound. - NeuroVFM encoder. - Decipher-MR. - BrainDINO. - GenBrain encoder. 4. Run frozen probing first for every modern encoder. 5. Add adapter or high-level partial tuning only where the architecture and weights are stable. 6. Add external validation on ADNI or OASIS-3. ## What Not To Claim Do not claim: - We solve clinical AD diagnosis directly. - We train a full PET foundation model from scratch. - We generate free-form radiology reports. Safer claim: > ReMAP-PET learns a region-grounded FDG-PET metabolic representation that supports SUVR prediction, PET-SUVR retrieval, and interpretable metabolic region analysis.