Add pipeline tag and links to paper/code
#1
by nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,9 +1,11 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
| 3 |
---
|
|
|
|
| 4 |
# Model Card for AdaLoRA-QAT
|
| 5 |
|
| 6 |
-
AdaLoRA-QAT is an efficient, compact foundation model variant designed for accurate chest X-ray (CXR) lung segmentation.It adapts the Segment Anything Model (SAM) to meet strict clinical computational constraints by combining adaptive low-rank parameter fine-tuning with quantization-aware training.
|
| 7 |
|
| 8 |
## Model Details
|
| 9 |
|
|
@@ -19,8 +21,9 @@ AdaLoRA-QAT introduces a two-stage fine-tuning framework for medical image segme
|
|
| 19 |
|
| 20 |
### Model Sources
|
| 21 |
|
| 22 |
-
- **Repository:** https://prantik-pdeb
|
| 23 |
-
- **
|
|
|
|
| 24 |
|
| 25 |
## Uses
|
| 26 |
|
|
@@ -31,11 +34,24 @@ AdaLoRA-QAT introduces a two-stage fine-tuning framework for medical image segme
|
|
| 31 |
* Improving the reliability of computer-aided diagnosis (CAD) systems.
|
| 32 |
* Enabling deployable foundation models on resource-constrained clinical hardware.
|
| 33 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
## Bias, Risks, and Limitations
|
| 35 |
|
| 36 |
* Robust generalization across deep learning models remains challenging due to anatomical variability.
|
| 37 |
* Generalization is also challenged by pathological distortions and imaging artifacts.
|
| 38 |
-
* The Structural Similarity Index (
|
| 39 |
|
| 40 |
## Training Details
|
| 41 |
|
|
@@ -55,7 +71,7 @@ AdaLoRA-QAT introduces a two-stage fine-tuning framework for medical image segme
|
|
| 55 |
|
| 56 |
- **Training regime:** Stage 1 uses FP32 precision. Stage 2 uses a selective mixed-precision strategy. Encoder feed-forward layers, the decoder, and the prompt encoder are quantized to INT8. Attention QKV projections and AdaLoRA parameters (P, Q, A) remain in FP32.
|
| 57 |
- **Batch Size:** 16 during Stage 1.
|
| 58 |
-
- **Learning Rates:** In Stage 1, 5e-5 for the encoder and 2e-5 for the decoder.In Stage 2, singular values are fine-tuned at 1e-6.
|
| 59 |
|
| 60 |
#### Speeds, Sizes, Times
|
| 61 |
|
|
@@ -75,7 +91,7 @@ AdaLoRA-QAT introduces a two-stage fine-tuning framework for medical image segme
|
|
| 75 |
* Dice Score (DSC).
|
| 76 |
* Intersection over Union (IOU).
|
| 77 |
* Normalized Surface Distance (NSD).
|
| 78 |
-
* Structural Similarity Index (SSIM)
|
| 79 |
* Wilcoxon signed-rank test for statistical significance assessment.
|
| 80 |
|
| 81 |
### Results
|
|
@@ -85,10 +101,6 @@ AdaLoRA-QAT introduces a two-stage fine-tuning framework for medical image segme
|
|
| 85 |
* Statistical analysis confirms that full INT8 quantization preserves segmentation accuracy without significant degradation.
|
| 86 |
* SSIM analysis exhibits strong structural agreement along lung boundaries and vascular regions.
|
| 87 |
|
| 88 |
-
#### Summary
|
| 89 |
-
|
| 90 |
-
AdaLoRA-QAT effectively balances accuracy, efficiency, and structural trustworthiness. It establishes a proof of concept for substantially compressing foundation models for scalable AI-assisted diagnosis without compromising diagnostic accuracy.
|
| 91 |
-
|
| 92 |
## Model Examination
|
| 93 |
|
| 94 |
* Quantization error analysis shows that FP32-INT8 quantization noise follows an approximately zero-mean Gaussian distribution.
|
|
@@ -104,6 +116,17 @@ AdaLoRA-QAT effectively balances accuracy, efficiency, and structural trustworth
|
|
| 104 |
|
| 105 |
* NVIDIA RTX A6000 GPUs (48 GB).
|
| 106 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
## Model Card Authors
|
| 108 |
|
| 109 |
Prantik Deb, Srimanth Dhondy, N. Ramakrishna, Anu Kapoor, Raju S. Bapi, Tapabrata Chakraborti.
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
pipeline_tag: image-segmentation
|
| 4 |
---
|
| 5 |
+
|
| 6 |
# Model Card for AdaLoRA-QAT
|
| 7 |
|
| 8 |
+
AdaLoRA-QAT is an efficient, compact foundation model variant designed for accurate chest X-ray (CXR) lung segmentation. It adapts the Segment Anything Model (SAM) to meet strict clinical computational constraints by combining adaptive low-rank parameter fine-tuning with quantization-aware training.
|
| 9 |
|
| 10 |
## Model Details
|
| 11 |
|
|
|
|
| 21 |
|
| 22 |
### Model Sources
|
| 23 |
|
| 24 |
+
- **Repository:** [https://github.com/prantik-pdeb/ADALORA-QAT](https://github.com/prantik-pdeb/ADALORA-QAT)
|
| 25 |
+
- **Project Page:** [https://prantik-pdeb.github.io/adaloraqat.github.io/](https://prantik-pdeb.github.io/adaloraqat.github.io/)
|
| 26 |
+
- **Paper:** [AdaLoRA-QAT: Adaptive Low-Rank and Quantization-Aware Segmentation](https://huggingface.co/papers/2604.01167)
|
| 27 |
|
| 28 |
## Uses
|
| 29 |
|
|
|
|
| 34 |
* Improving the reliability of computer-aided diagnosis (CAD) systems.
|
| 35 |
* Enabling deployable foundation models on resource-constrained clinical hardware.
|
| 36 |
|
| 37 |
+
## Sample Usage
|
| 38 |
+
|
| 39 |
+
To run inference using the provided scripts in the repository:
|
| 40 |
+
|
| 41 |
+
```bash
|
| 42 |
+
python -u inference/inference.py \
|
| 43 |
+
--image_path sample_data/images/C19RD_COVID-29.png \
|
| 44 |
+
--checkpoint_path "best_model_stage2_int8.pth" \
|
| 45 |
+
--bbox 0 0 511 511 --save_mask --visualize \
|
| 46 |
+
--output_mask_path ./inf_res.png \
|
| 47 |
+
--save_overlay ./overlay
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
## Bias, Risks, and Limitations
|
| 51 |
|
| 52 |
* Robust generalization across deep learning models remains challenging due to anatomical variability.
|
| 53 |
* Generalization is also challenged by pathological distortions and imaging artifacts.
|
| 54 |
+
* The Structural Similarity Index (SSIM) map indicates minor degradations primarily associated with severe motion artifacts or extreme pathologies.
|
| 55 |
|
| 56 |
## Training Details
|
| 57 |
|
|
|
|
| 71 |
|
| 72 |
- **Training regime:** Stage 1 uses FP32 precision. Stage 2 uses a selective mixed-precision strategy. Encoder feed-forward layers, the decoder, and the prompt encoder are quantized to INT8. Attention QKV projections and AdaLoRA parameters (P, Q, A) remain in FP32.
|
| 73 |
- **Batch Size:** 16 during Stage 1.
|
| 74 |
+
- **Learning Rates:** In Stage 1, 5e-5 for the encoder and 2e-5 for the decoder. In Stage 2, singular values are fine-tuned at 1e-6.
|
| 75 |
|
| 76 |
#### Speeds, Sizes, Times
|
| 77 |
|
|
|
|
| 91 |
* Dice Score (DSC).
|
| 92 |
* Intersection over Union (IOU).
|
| 93 |
* Normalized Surface Distance (NSD).
|
| 94 |
+
* Structural Similarity Index (SSIM).
|
| 95 |
* Wilcoxon signed-rank test for statistical significance assessment.
|
| 96 |
|
| 97 |
### Results
|
|
|
|
| 101 |
* Statistical analysis confirms that full INT8 quantization preserves segmentation accuracy without significant degradation.
|
| 102 |
* SSIM analysis exhibits strong structural agreement along lung boundaries and vascular regions.
|
| 103 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
## Model Examination
|
| 105 |
|
| 106 |
* Quantization error analysis shows that FP32-INT8 quantization noise follows an approximately zero-mean Gaussian distribution.
|
|
|
|
| 116 |
|
| 117 |
* NVIDIA RTX A6000 GPUs (48 GB).
|
| 118 |
|
| 119 |
+
## Citation
|
| 120 |
+
|
| 121 |
+
```bibtex
|
| 122 |
+
@article{deb2025adaloraqat,
|
| 123 |
+
title={AdaLoRA-QAT: Adaptive Low-Rank and Quantization-Aware Segmentation},
|
| 124 |
+
author={Deb, Prantik and Dhondy, Srimanth and Ramakrishna, N. and Kapoor, Anu and Bapi, Raju S. and Chakraborti, Tapabrata},
|
| 125 |
+
journal={arXiv preprint arXiv:2604.01167},
|
| 126 |
+
year={2025}
|
| 127 |
+
}
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
## Model Card Authors
|
| 131 |
|
| 132 |
Prantik Deb, Srimanth Dhondy, N. Ramakrishna, Anu Kapoor, Raju S. Bapi, Tapabrata Chakraborti.
|