carteeeltheboss/DPR_PFA4IADO

MedFusionNet — Hybrid Pneumonia Detection on Chest X-Rays

Original project: carteeeltheboss/DPR_PFA4IADO

ROC - Confusion Matrix - Benchmarks

MedFusionNet is a binary chest X-ray classifier for NORMAL vs PNEUMONIA using:

• DenseNet-121 for local patterns
• Swin Transformer for global context
• gated fusion
• Grad-CAM
• MC-Dropout

---

1) Global pipeline

$$\tilde{X}=T(X)\backslash tilde\; X\; =\; T(X)$$

$$F_c={\mathrm{\Phi }}_c(\tilde{X}),F_s={\mathrm{\Phi }}_s(\tilde{X})F\_c\; =\; \backslash Phi\_c(\backslash tilde\; X),\; \backslash qquad\; F\_s\; =\; \backslash Phi\_s(\backslash tilde\; X)$$

$$g=\sigma \text{\hspace{0.17em}⁣}(W_g[\mathrm{GAP}(F_c),\mathrm{GAP}(F_s)]+b_g)g\; =\; \backslash sigma\backslash !\backslash big(W\_g[\backslash operatorname\{GAP\}(F\_c),\; \backslash operatorname\{GAP\}(F\_s)]\; +\; b\_g\backslash big)$$

$$F=g\odot F_c+(1-g)\odot F_{s}F\; =\; g\; \backslash odot\; F\_c\; +\; (1-g)\backslash odot\; F\_s$$

$$\mathrm{\Psi }(F)=(p,L^c,u)\backslash Psi(F)\; =\; (p,\; L^c,\; u)$$

Short summary: preprocess the image, extract local and global features, fuse them adaptively, then output probability, heatmap, and uncertainty.

---

2) Input and preprocessing

$$X\in {\mathbb{R}}^{H\times W}X\; \backslash in\; \backslash mathbb\{R\}^\{H\; \backslash times\; W\}$$

$$\tilde{X}=T(X)=A(R(N(X)))\backslash tilde\; X\; =\; T(X)=A(R(N(X)))$$

$$X_{\text{norm}}=\frac{X-\mu }{\sigma }X\_\{\backslash text\{norm\}\}=\backslash frac\{X-\backslash mu\}\{\backslash sigma\}$$

$$X\in {\mathbb{R}}^{H\times W}\to \tilde{X}\in {\mathbb{R}}^{384\times 384}X\; \backslash in\; \backslash mathbb\{R\}^\{H\; \backslash times\; W\}\; \backslash rightarrow\; \backslash tilde\; X\; \backslash in\; \backslash mathbb\{R\}^\{384\; \backslash times\; 384\}$$

Short summary: normalize intensity, resize to a fixed resolution, and standardize the input domain.

---

3) Local branch: DenseNet-121

$$F_c={\mathrm{\Phi }}_c(\tilde{X})\in {\mathbb{R}}^{H^{\mathrm{\prime }}\times W^{\mathrm{\prime }}\times C_c}F\_c\; =\; \backslash Phi\_c(\backslash tilde\; X)\; \backslash in\; \backslash mathbb\{R\}^\{H\text{'}\; \backslash times\; W\text{'}\; \backslash times\; C\_c\}$$

$$F_{i,j,c^{\mathrm{\prime }}}^{(\mathrm{\ell })}=\sum _{u,v,c}{W}_{u,v,c,c^{\mathrm{\prime }}}^{(\mathrm{\ell })}{F}_{i+u,j+v,c}^{(\mathrm{\ell }-1)}+b_{c^{\mathrm{\prime }}}^{(\mathrm{\ell })}F^\{(\backslash ell)\}\_\{i,j,c\text{'}\}=\; \backslash sum\_\{u,v,c\}\; W^\{(\backslash ell)\}\_\{u,v,c,c\text{'}\}F^\{(\backslash ell-1)\}\_\{i+u,j+v,c\}+b^\{(\backslash ell)\}\_\{c\text{'}\}$$

Short summary: captures local opacities, fine textures, subtle consolidations, and small radiographic details.

---

4) Global branch: Swin Transformer

$$z_i=E{x}_i+e_i^{pos}z\_i\; =\; E\; x\_i\; +\; e\_i^\{pos\}$$

$$F_s={\mathrm{\Phi }}_s(\tilde{X})\in {\mathbb{R}}^{H^{\mathrm{\prime }}\times W^{\mathrm{\prime }}\times C_s}F\_s=\backslash Phi\_s(\backslash tilde\; X)\backslash in\backslash mathbb\{R\}^\{H\text{'}\; \backslash times\; W\text{'}\; \backslash times\; C\_s\}$$

$$Q=X{W}_Q,K=X{W}_K,V=X{W}_{V}Q=XW\_Q,\backslash qquad\; K=XW\_K,\backslash qquad\; V=XW\_V$$

$$\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\text{\hspace{0.17em}⁣}\left(\frac{Q{K}^{\mathrm{\top }}}{\sqrt{d_k}}\right)V\backslash operatorname\{Attention\}(Q,K,V)=\; \backslash operatorname\{softmax\}\backslash !\backslash left(\backslash frac\{QK^\backslash top\}\{\backslash sqrt\{d\_k\}\}\backslash right)V$$

Short summary: captures long-range thoracic structure and contextual anatomical dependencies.

---

5) Adaptive gated fusion

$${\bar{F}}_c=\mathrm{GAP}(F_c),{\bar{F}}_s=\mathrm{GAP}(F_s)\backslash bar\; F\_c=\backslash operatorname\{GAP\}(F\_c),\; \backslash qquad\; \backslash bar\; F\_s=\backslash operatorname\{GAP\}(F\_s)$$

$$g=\sigma \text{\hspace{0.17em}⁣}(W_g[{\bar{F}}_c,{\bar{F}}_s]+b_g)g=\backslash sigma\backslash !\backslash left(W\_g[\backslash bar\; F\_c,\backslash bar\; F\_s]+b\_g\backslash right)$$

$$F=g\odot F_c+(1-g)\odot F_{s}F=g\backslash odot\; F\_c\; +\; (1-g)\backslash odot\; F\_s$$

Short summary: learns how much the model should trust local evidence versus global context for each image.

---

6) Classification head

$$z=\mathrm{GAP}(F)z=\backslash operatorname\{GAP\}(F)$$

$$p(y=1\mid X)=\sigma (w^{\mathrm{\top }}z+b)p(y=1\backslash mid\; X)=\backslash sigma(w^\backslash top\; z+b)$$

Short summary: converts fused features into a binary pneumonia probability.

---

7) Grad-CAM

$${\alpha }_k^c=\frac{1}{Z}\sum _i\sum _j\frac{\mathrm{\partial }{y}^c}{\mathrm{\partial }{A}_{ij}^k}\backslash alpha\_k^c\; =\; \backslash frac\{1\}\{Z\}\backslash sum\_i\backslash sum\_j\; \backslash frac\{\backslash partial\; y^c\}\{\backslash partial\; A^k\_\{ij\}\}$$

$$L^c=\mathrm{ReLU}\text{\hspace{0.17em}⁣}\left(\sum _k{\alpha }_k^c{A}^k\right)L^c=\backslash operatorname\{ReLU\}\backslash !\backslash left(\backslash sum\_k\; \backslash alpha\_k^c\; A^k\backslash right)$$

Short summary: highlights the image regions that support the predicted class.

---

8) MC-Dropout uncertainty

$$u={\mathrm{Var}}_t(p_t)u=\backslash operatorname\{Var\}\_t(p\_t)$$

Short summary: repeated stochastic forward passes estimate prediction stability.

---

9) Training objective

$$\mathcal{L}={\mathcal{L}}_{cls}+{\lambda }_1{\mathcal{L}}_{loc}+{\lambda }_2{\mathcal{L}}_{cons}+{\lambda }_3{\mathcal{L}}_{cal}\backslash mathcal\{L\}\; =\; \backslash mathcal\{L\}\_\{cls\}\; +\; \backslash lambda\_1\backslash mathcal\{L\}\_\{loc\}\; +\; \backslash lambda\_2\backslash mathcal\{L\}\_\{cons\}\; +\; \backslash lambda\_3\backslash mathcal\{L\}\_\{cal\}$$

$${\mathcal{L}}_{focal}=-\alpha (1-p_t{)}^{\gamma }\log (p_t)\backslash mathcal\{L\}\_\{focal\}\; =\; -\backslash alpha\; (1-p\_t)^\backslash gamma\; \backslash log(p\_t)$$

Short summary: the objective combines classification, localization regularization, consistency, and calibration.

---

10) Dataset

Dataset: Paul Mooney Chest X-Ray Images (Pneumonia)
Classes: NORMAL, PNEUMONIA

data/
├── train/
│   ├── NORMAL/
│   └── PNEUMONIA/
├── val/
│   ├── NORMAL/
│   └── PNEUMONIA/
└── test/
    ├── NORMAL/
    └── PNEUMONIA/

Short summary: binary chest X-ray classification with strong class imbalance and noisy acquisition conditions.

---

11) Final output

$$(p(y=1\mid X),L^c,u)\backslash big(p(y=1\backslash mid\; X),\backslash ,L^c,\backslash ,u\backslash big)$$

Short summary: final prediction = probability + explanation map + uncertainty estimate.

---

12) Original project

GitHub: https://github.com/carteeeltheboss/DPR_PFA4IADO