Hugging Face's logo

manu02
/
LAnA-MIMIC

Image-to-Text
Transformers
Safetensors
lana_radgen
feature-extraction
medical-ai
radiology
chest-xray
report-generation
segmentation
anatomical-attention
custom_code
Model card Files
xet
 Community
manu02 commited on
Commit
188c2e7
·
verified ·
1 Parent(s): 40ec82f

Upload MIMIC test evaluation results

Browse files
Files changed (7) hide show
  1. README.md +36 -49
  2. evaluations/mimic_test_findings_only_metrics.json +21 -21
  3. evaluations/mimic_test_findings_only_predictions.csv +0 -0
  4. evaluations/mimic_test_metrics.json +63 -63
  5. evaluations/mimic_test_predictions.csv +0 -0
  6. model.safetensors +1 -1
  7. run_summary.json +77 -77
README.md CHANGED
@@ -98,56 +98,43 @@ print(report)
98
 
99
  Frontal-only evaluation using `PA/AP` studies only.
100
 
101
- ### Current Checkpoint Results
 
 
102
 
103
  ### All Frontal Test Studies
104
 
105
  | Metric | Value |
106
  | --- | --- |
107
  | Number of studies | `3041` |
108
- | ROUGE-L | `0.1645` |
109
- | BLEU-1 | `0.1938` |
110
- | BLEU-4 | `0.0394` |
111
- | METEOR | `0.2192` |
112
- | RadGraph F1 | `0.0945` |
113
- | RadGraph entity F1 | `0.1497` |
114
- | RadGraph relation F1 | `0.1350` |
115
- | CheXpert F1 14-micro | `0.1571` |
116
- | CheXpert F1 5-micro | `0.1573` |
117
- | CheXpert F1 14-macro | `0.0762` |
118
- | CheXpert F1 5-macro | `0.1068` |
119
 
120
  ### Findings-Only Frontal Test Studies
121
 
122
  | Metric | Value |
123
  | --- | --- |
124
  | Number of studies | `2210` |
125
- | ROUGE-L | `0.1714` |
126
- | BLEU-1 | `0.2021` |
127
- | BLEU-4 | `0.0459` |
128
- | METEOR | `0.2339` |
129
- | RadGraph F1 | `0.1026` |
130
- | RadGraph entity F1 | `0.1605` |
131
- | RadGraph relation F1 | `0.1434` |
132
- | CheXpert F1 14-micro | `0.1450` |
133
- | CheXpert F1 5-micro | `0.1747` |
134
- | CheXpert F1 14-macro | `0.0749` |
135
- | CheXpert F1 5-macro | `0.1182` |
136
-
137
- ### Final Completed Training Results
138
-
139
- The final table will be populated when the planned training run is completed. Until then, final-report metrics remain `TBD`.
140
-
141
- | Metric | Value |
142
- | --- | --- |
143
- | Number of studies | TBD |
144
- | RadGraph F1 | TBD |
145
- | RadGraph entity F1 | TBD |
146
- | RadGraph relation F1 | TBD |
147
- | CheXpert F1 14-micro | TBD |
148
- | CheXpert F1 5-micro | TBD |
149
- | CheXpert F1 14-macro | TBD |
150
- | CheXpert F1 5-macro | TBD |
151
 
152
  ## Data
153
 
@@ -163,7 +150,7 @@ The final table will be populated when the planned training run is completed. Un
163
  ## Training Snapshot
164
 
165
  - Run: `mimic only`
166
- - This section describes the current public checkpoint, not the final completed project.
167
  - Method: `lora_adamw`
168
  - Vision encoder: `facebook/dinov3-vits16-pretrain-lvd1689m`
169
  - Text decoder: `gpt2`
@@ -174,21 +161,21 @@ The final table will be populated when the planned training run is completed. Un
174
  - Scheduler: `cosine`
175
  - Warmup steps: `2636`
176
  - Weight decay: `0.01`
177
- - Steps completed: `46866`
178
  - Planned total steps: `52716`
179
- - Images seen: `374957`
180
- - Total training time: `7.0001` hours
181
  - Hardware: `NVIDIA GeForce RTX 5070`
182
- - Final train loss: `2.1600`
183
- - Validation loss: `1.7540`
184
 
185
  ## Status
186
 
187
- - Project status: `Training in progress`
188
- - Release status: `Research preview checkpoint`
189
- - Current checkpoint status: `Not final`
190
- - Training completion toward planned run: `88.91%` (`3` / `3` epochs)
191
- - Current published metrics are intermediate and will change as training continues.
192
 
193
  ## Notes
194
 
 
98
 
99
  Frontal-only evaluation using `PA/AP` studies only.
100
 
101
+ ### Final Completed Training Results
102
+
103
+ These final-report metrics correspond to the completed training run.
104
 
105
  ### All Frontal Test Studies
106
 
107
  | Metric | Value |
108
  | --- | --- |
109
  | Number of studies | `3041` |
110
+ | ROUGE-L | `0.1653` |
111
+ | BLEU-1 | `0.1916` |
112
+ | BLEU-4 | `0.0386` |
113
+ | METEOR | `0.2202` |
114
+ | RadGraph F1 | `0.0921` |
115
+ | RadGraph entity F1 | `0.1459` |
116
+ | RadGraph relation F1 | `0.1322` |
117
+ | CheXpert F1 14-micro | `0.1565` |
118
+ | CheXpert F1 5-micro | `0.1530` |
119
+ | CheXpert F1 14-macro | `0.0713` |
120
+ | CheXpert F1 5-macro | `0.1007` |
121
 
122
  ### Findings-Only Frontal Test Studies
123
 
124
  | Metric | Value |
125
  | --- | --- |
126
  | Number of studies | `2210` |
127
+ | ROUGE-L | `0.1720` |
128
+ | BLEU-1 | `0.2003` |
129
+ | BLEU-4 | `0.0449` |
130
+ | METEOR | `0.2347` |
131
+ | RadGraph F1 | `0.1000` |
132
+ | RadGraph entity F1 | `0.1577` |
133
+ | RadGraph relation F1 | `0.1413` |
134
+ | CheXpert F1 14-micro | `0.1442` |
135
+ | CheXpert F1 5-micro | `0.1716` |
136
+ | CheXpert F1 14-macro | `0.0700` |
137
+ | CheXpert F1 5-macro | `0.1112` |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
138
 
139
  ## Data
140
 
 
150
  ## Training Snapshot
151
 
152
  - Run: `mimic only`
153
+ - This section describes the completed public training run.
154
  - Method: `lora_adamw`
155
  - Vision encoder: `facebook/dinov3-vits16-pretrain-lvd1689m`
156
  - Text decoder: `gpt2`
 
161
  - Scheduler: `cosine`
162
  - Warmup steps: `2636`
163
  - Weight decay: `0.01`
164
+ - Steps completed: `52709`
165
  - Planned total steps: `52716`
166
+ - Images seen: `421707`
167
+ - Total training time: `7.8770` hours
168
  - Hardware: `NVIDIA GeForce RTX 5070`
169
+ - Final train loss: `1.9587`
170
+ - Validation loss: `1.7493`
171
 
172
  ## Status
173
 
174
+ - Project status: `Training completed`
175
+ - Release status: `Completed training run`
176
+ - Current checkpoint status: `Final completed run`
177
+ - Training completion toward planned run: `100.00%` (`3` / `3` epochs)
178
+ - Current published metrics correspond to the completed training run.
179
 
180
  ## Notes
181
 
evaluations/mimic_test_findings_only_metrics.json CHANGED
@@ -4,35 +4,35 @@
4
  "dataset": "mimic-cxr",
5
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
6
  "num_examples": 2210,
7
- "bleu_1": 0.20210908304941103,
8
- "bleu_4": 0.04590488103429025,
9
- "meteor": 0.23388109986749062,
10
- "rouge_l": 0.1714484670889256,
11
- "chexpert_f1_14_micro": 0.14500057267208796,
12
- "chexpert_f1_5_micro": 0.17467652495378927,
13
- "chexpert_f1_14_macro": 0.07493938867200726,
14
- "chexpert_f1_5_macro": 0.11821241603707926,
15
- "chexpert_f1_micro": 0.14500057267208796,
16
- "chexpert_f1_macro": 0.07493938867200726,
17
  "chexpert_per_label_f1": {
18
  "Enlarged Cardiomediastinum": 0.0,
19
- "Cardiomegaly": 0.04140786749482402,
20
  "Lung Opacity": 0.0,
21
  "Lung Lesion": 0.0,
22
- "Edema": 0.1845018450184502,
23
- "Consolidation": 0.08130081300813008,
24
- "Pneumonia": 0.018433179723502308,
25
  "Atelectasis": 0.0,
26
- "Pneumothorax": 0.031665611146295125,
27
- "Pleural Effusion": 0.283851554663992,
28
  "Pleural Other": 0.0,
29
- "Fracture": 0.020618556701030927,
30
- "Support Devices": 0.3873720136518771,
31
  "No Finding": 0.0
32
  },
33
- "radgraph_f1": 0.10264445141069471,
34
- "radgraph_f1_entity": 0.16049110076991005,
35
- "radgraph_f1_relation": 0.1433593428807833,
36
  "radgraph_available": true,
37
  "radgraph_error": null
38
  }
 
4
  "dataset": "mimic-cxr",
5
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
6
  "num_examples": 2210,
7
+ "bleu_1": 0.2002922421466921,
8
+ "bleu_4": 0.04489491191876499,
9
+ "meteor": 0.2347130721675525,
10
+ "rouge_l": 0.1720439436054897,
11
+ "chexpert_f1_14_micro": 0.14424186475882758,
12
+ "chexpert_f1_5_micro": 0.1716295427901524,
13
+ "chexpert_f1_14_macro": 0.06995745688790377,
14
+ "chexpert_f1_5_macro": 0.11117899627949375,
15
+ "chexpert_f1_micro": 0.14424186475882758,
16
+ "chexpert_f1_macro": 0.06995745688790377,
17
  "chexpert_per_label_f1": {
18
  "Enlarged Cardiomediastinum": 0.0,
19
+ "Cardiomegaly": 0.020790020790020795,
20
  "Lung Opacity": 0.0,
21
  "Lung Lesion": 0.0,
22
+ "Edema": 0.16448598130841122,
23
+ "Consolidation": 0.08403361344537814,
24
+ "Pneumonia": 0.009090909090909092,
25
  "Atelectasis": 0.0,
26
+ "Pneumothorax": 0.03612903225806451,
27
+ "Pleural Effusion": 0.28658536585365857,
28
  "Pleural Other": 0.0,
29
+ "Fracture": 0.0,
30
+ "Support Devices": 0.3782894736842105,
31
  "No Finding": 0.0
32
  },
33
+ "radgraph_f1": 0.10000876408490725,
34
+ "radgraph_f1_entity": 0.15770684059564277,
35
+ "radgraph_f1_relation": 0.14126690975537282,
36
  "radgraph_available": true,
37
  "radgraph_error": null
38
  }
evaluations/mimic_test_findings_only_predictions.csv CHANGED
The diff for this file is too large to render. See raw diff
 
evaluations/mimic_test_metrics.json CHANGED
@@ -4,35 +4,35 @@
4
  "dataset": "mimic-cxr",
5
  "view_filter": "frontal-only (PA/AP)",
6
  "num_examples": 3041,
7
- "bleu_1": 0.1938387766203937,
8
- "bleu_4": 0.039402443288372446,
9
- "meteor": 0.21919553279400095,
10
- "rouge_l": 0.16453907670771373,
11
- "chexpert_f1_14_micro": 0.15713704758031719,
12
- "chexpert_f1_5_micro": 0.15734035549703754,
13
- "chexpert_f1_14_macro": 0.07622868830082337,
14
- "chexpert_f1_5_macro": 0.10675159700014243,
15
- "chexpert_f1_micro": 0.15713704758031719,
16
- "chexpert_f1_macro": 0.07622868830082337,
17
  "chexpert_per_label_f1": {
18
  "Enlarged Cardiomediastinum": 0.0,
19
- "Cardiomegaly": 0.02650602409638554,
20
  "Lung Opacity": 0.0,
21
  "Lung Lesion": 0.0,
22
- "Edema": 0.15193026151930264,
23
- "Consolidation": 0.08468468468468468,
24
- "Pneumonia": 0.012539184952978058,
25
  "Atelectasis": 0.0,
26
- "Pneumothorax": 0.03817177297840281,
27
- "Pleural Effusion": 0.27063701470033924,
28
  "Pleural Other": 0.0,
29
- "Fracture": 0.032,
30
- "Support Devices": 0.45073269327943405,
31
  "No Finding": 0.0
32
  },
33
- "radgraph_f1": 0.09451862388311026,
34
- "radgraph_f1_entity": 0.1497185319208198,
35
- "radgraph_f1_relation": 0.13500246910571428,
36
  "radgraph_available": true,
37
  "radgraph_error": null,
38
  "evaluation_suite": "mimic_test_dual",
@@ -42,35 +42,35 @@
42
  "dataset": "mimic-cxr",
43
  "view_filter": "frontal-only (PA/AP)",
44
  "num_examples": 3041,
45
- "bleu_1": 0.1938387766203937,
46
- "bleu_4": 0.039402443288372446,
47
- "meteor": 0.21919553279400095,
48
- "rouge_l": 0.16453907670771373,
49
- "chexpert_f1_14_micro": 0.15713704758031719,
50
- "chexpert_f1_5_micro": 0.15734035549703754,
51
- "chexpert_f1_14_macro": 0.07622868830082337,
52
- "chexpert_f1_5_macro": 0.10675159700014243,
53
- "chexpert_f1_micro": 0.15713704758031719,
54
- "chexpert_f1_macro": 0.07622868830082337,
55
  "chexpert_per_label_f1": {
56
  "Enlarged Cardiomediastinum": 0.0,
57
- "Cardiomegaly": 0.02650602409638554,
58
  "Lung Opacity": 0.0,
59
  "Lung Lesion": 0.0,
60
- "Edema": 0.15193026151930264,
61
- "Consolidation": 0.08468468468468468,
62
- "Pneumonia": 0.012539184952978058,
63
  "Atelectasis": 0.0,
64
- "Pneumothorax": 0.03817177297840281,
65
- "Pleural Effusion": 0.27063701470033924,
66
  "Pleural Other": 0.0,
67
- "Fracture": 0.032,
68
- "Support Devices": 0.45073269327943405,
69
  "No Finding": 0.0
70
  },
71
- "radgraph_f1": 0.09451862388311026,
72
- "radgraph_f1_entity": 0.1497185319208198,
73
- "radgraph_f1_relation": 0.13500246910571428,
74
  "radgraph_available": true,
75
  "radgraph_error": null
76
  },
@@ -80,35 +80,35 @@
80
  "dataset": "mimic-cxr",
81
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
82
  "num_examples": 2210,
83
- "bleu_1": 0.20210908304941103,
84
- "bleu_4": 0.04590488103429025,
85
- "meteor": 0.23388109986749062,
86
- "rouge_l": 0.1714484670889256,
87
- "chexpert_f1_14_micro": 0.14500057267208796,
88
- "chexpert_f1_5_micro": 0.17467652495378927,
89
- "chexpert_f1_14_macro": 0.07493938867200726,
90
- "chexpert_f1_5_macro": 0.11821241603707926,
91
- "chexpert_f1_micro": 0.14500057267208796,
92
- "chexpert_f1_macro": 0.07493938867200726,
93
  "chexpert_per_label_f1": {
94
  "Enlarged Cardiomediastinum": 0.0,
95
- "Cardiomegaly": 0.04140786749482402,
96
  "Lung Opacity": 0.0,
97
  "Lung Lesion": 0.0,
98
- "Edema": 0.1845018450184502,
99
- "Consolidation": 0.08130081300813008,
100
- "Pneumonia": 0.018433179723502308,
101
  "Atelectasis": 0.0,
102
- "Pneumothorax": 0.031665611146295125,
103
- "Pleural Effusion": 0.283851554663992,
104
  "Pleural Other": 0.0,
105
- "Fracture": 0.020618556701030927,
106
- "Support Devices": 0.3873720136518771,
107
  "No Finding": 0.0
108
  },
109
- "radgraph_f1": 0.10264445141069471,
110
- "radgraph_f1_entity": 0.16049110076991005,
111
- "radgraph_f1_relation": 0.1433593428807833,
112
  "radgraph_available": true,
113
  "radgraph_error": null
114
  }
 
4
  "dataset": "mimic-cxr",
5
  "view_filter": "frontal-only (PA/AP)",
6
  "num_examples": 3041,
7
+ "bleu_1": 0.19157722199566865,
8
+ "bleu_4": 0.038562318137225,
9
+ "meteor": 0.2201968843785929,
10
+ "rouge_l": 0.16527670443755577,
11
+ "chexpert_f1_14_micro": 0.15648041946583646,
12
+ "chexpert_f1_5_micro": 0.15297261189044756,
13
+ "chexpert_f1_14_macro": 0.07133711095756078,
14
+ "chexpert_f1_5_macro": 0.10070328760806402,
15
+ "chexpert_f1_micro": 0.15648041946583646,
16
+ "chexpert_f1_macro": 0.07133711095756078,
17
  "chexpert_per_label_f1": {
18
  "Enlarged Cardiomediastinum": 0.0,
19
+ "Cardiomegaly": 0.012135922330097087,
20
  "Lung Opacity": 0.0,
21
  "Lung Lesion": 0.0,
22
+ "Edema": 0.1355081555834379,
23
+ "Consolidation": 0.0855813953488372,
24
+ "Pneumonia": 0.006153846153846154,
25
  "Atelectasis": 0.0,
26
+ "Pneumothorax": 0.04209445585215606,
27
+ "Pleural Effusion": 0.27029096477794795,
28
  "Pleural Other": 0.0,
29
+ "Fracture": 0.0,
30
+ "Support Devices": 0.44695481335952847,
31
  "No Finding": 0.0
32
  },
33
+ "radgraph_f1": 0.09208450105647495,
34
+ "radgraph_f1_entity": 0.14592416918723933,
35
+ "radgraph_f1_relation": 0.13218202095291967,
36
  "radgraph_available": true,
37
  "radgraph_error": null,
38
  "evaluation_suite": "mimic_test_dual",
 
42
  "dataset": "mimic-cxr",
43
  "view_filter": "frontal-only (PA/AP)",
44
  "num_examples": 3041,
45
+ "bleu_1": 0.19157722199566865,
46
+ "bleu_4": 0.038562318137225,
47
+ "meteor": 0.2201968843785929,
48
+ "rouge_l": 0.16527670443755577,
49
+ "chexpert_f1_14_micro": 0.15648041946583646,
50
+ "chexpert_f1_5_micro": 0.15297261189044756,
51
+ "chexpert_f1_14_macro": 0.07133711095756078,
52
+ "chexpert_f1_5_macro": 0.10070328760806402,
53
+ "chexpert_f1_micro": 0.15648041946583646,
54
+ "chexpert_f1_macro": 0.07133711095756078,
55
  "chexpert_per_label_f1": {
56
  "Enlarged Cardiomediastinum": 0.0,
57
+ "Cardiomegaly": 0.012135922330097087,
58
  "Lung Opacity": 0.0,
59
  "Lung Lesion": 0.0,
60
+ "Edema": 0.1355081555834379,
61
+ "Consolidation": 0.0855813953488372,
62
+ "Pneumonia": 0.006153846153846154,
63
  "Atelectasis": 0.0,
64
+ "Pneumothorax": 0.04209445585215606,
65
+ "Pleural Effusion": 0.27029096477794795,
66
  "Pleural Other": 0.0,
67
+ "Fracture": 0.0,
68
+ "Support Devices": 0.44695481335952847,
69
  "No Finding": 0.0
70
  },
71
+ "radgraph_f1": 0.09208450105647495,
72
+ "radgraph_f1_entity": 0.14592416918723933,
73
+ "radgraph_f1_relation": 0.13218202095291967,
74
  "radgraph_available": true,
75
  "radgraph_error": null
76
  },
 
80
  "dataset": "mimic-cxr",
81
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
82
  "num_examples": 2210,
83
+ "bleu_1": 0.2002922421466921,
84
+ "bleu_4": 0.04489491191876499,
85
+ "meteor": 0.2347130721675525,
86
+ "rouge_l": 0.1720439436054897,
87
+ "chexpert_f1_14_micro": 0.14424186475882758,
88
+ "chexpert_f1_5_micro": 0.1716295427901524,
89
+ "chexpert_f1_14_macro": 0.06995745688790377,
90
+ "chexpert_f1_5_macro": 0.11117899627949375,
91
+ "chexpert_f1_micro": 0.14424186475882758,
92
+ "chexpert_f1_macro": 0.06995745688790377,
93
  "chexpert_per_label_f1": {
94
  "Enlarged Cardiomediastinum": 0.0,
95
+ "Cardiomegaly": 0.020790020790020795,
96
  "Lung Opacity": 0.0,
97
  "Lung Lesion": 0.0,
98
+ "Edema": 0.16448598130841122,
99
+ "Consolidation": 0.08403361344537814,
100
+ "Pneumonia": 0.009090909090909092,
101
  "Atelectasis": 0.0,
102
+ "Pneumothorax": 0.03612903225806451,
103
+ "Pleural Effusion": 0.28658536585365857,
104
  "Pleural Other": 0.0,
105
+ "Fracture": 0.0,
106
+ "Support Devices": 0.3782894736842105,
107
  "No Finding": 0.0
108
  },
109
+ "radgraph_f1": 0.10000876408490725,
110
+ "radgraph_f1_entity": 0.15770684059564277,
111
+ "radgraph_f1_relation": 0.14126690975537282,
112
  "radgraph_available": true,
113
  "radgraph_error": null
114
  }
evaluations/mimic_test_predictions.csv CHANGED
The diff for this file is too large to render. See raw diff
 
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e71c53db281afe04ceb0e6c2a1ad401ce0294ac93cc031227fcda484621a09b9
3
  size 1159628024
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:98095132398151c23e62d922442131d9b0d4607ee40207bbc4b95a5e7fa543a0
3
  size 1159628024
run_summary.json CHANGED
@@ -1,18 +1,18 @@
1
  {
2
  "method": "lora_adamw",
3
  "run_name": "mimic only",
4
- "steps": 46866,
5
- "epochs_completed": 2,
6
- "epoch_index": 2,
7
  "target_epochs": 3,
8
- "progress_epochs": 2.6674231160497692,
9
- "training_completion_percent": 88.91410386832564,
10
- "elapsed_seconds": 25200.2360115,
11
- "images_seen": 374957,
12
- "train_loss_last": 2.1600470542907715,
13
- "train_loss_mean": 1.799472624994218,
14
- "val_loss": 1.7539901971817016,
15
- "images_per_second": 14.879106680941014,
16
  "trainable_params": 2878464,
17
  "vision_model_name": "facebook/dinov3-vits16-pretrain-lvd1689m",
18
  "text_model_name": "gpt2",
@@ -36,9 +36,9 @@
36
  "seed": 42,
37
  "resume_supported": true,
38
  "checkpoint_every_n_steps": 1000,
39
- "cumulative_loss_sum": 674724.8570499569,
40
- "cumulative_loss_count": 374957,
41
- "completed": false,
42
  "target_duration_seconds": 3600,
43
  "target_duration_mode": "per_invocation",
44
  "train_datasets": "MIMIC-CXR (findings-only)",
@@ -49,35 +49,35 @@
49
  "dataset": "mimic-cxr",
50
  "view_filter": "frontal-only (PA/AP)",
51
  "num_examples": 3041,
52
- "bleu_1": 0.1938387766203937,
53
- "bleu_4": 0.039402443288372446,
54
- "meteor": 0.21919553279400095,
55
- "rouge_l": 0.16453907670771373,
56
- "chexpert_f1_14_micro": 0.15713704758031719,
57
- "chexpert_f1_5_micro": 0.15734035549703754,
58
- "chexpert_f1_14_macro": 0.07622868830082337,
59
- "chexpert_f1_5_macro": 0.10675159700014243,
60
- "chexpert_f1_micro": 0.15713704758031719,
61
- "chexpert_f1_macro": 0.07622868830082337,
62
  "chexpert_per_label_f1": {
63
  "Enlarged Cardiomediastinum": 0.0,
64
- "Cardiomegaly": 0.02650602409638554,
65
  "Lung Opacity": 0.0,
66
  "Lung Lesion": 0.0,
67
- "Edema": 0.15193026151930264,
68
- "Consolidation": 0.08468468468468468,
69
- "Pneumonia": 0.012539184952978058,
70
  "Atelectasis": 0.0,
71
- "Pneumothorax": 0.03817177297840281,
72
- "Pleural Effusion": 0.27063701470033924,
73
  "Pleural Other": 0.0,
74
- "Fracture": 0.032,
75
- "Support Devices": 0.45073269327943405,
76
  "No Finding": 0.0
77
  },
78
- "radgraph_f1": 0.09451862388311026,
79
- "radgraph_f1_entity": 0.1497185319208198,
80
- "radgraph_f1_relation": 0.13500246910571428,
81
  "radgraph_available": true,
82
  "radgraph_error": null
83
  },
@@ -88,35 +88,35 @@
88
  "dataset": "mimic-cxr",
89
  "view_filter": "frontal-only (PA/AP)",
90
  "num_examples": 3041,
91
- "bleu_1": 0.1938387766203937,
92
- "bleu_4": 0.039402443288372446,
93
- "meteor": 0.21919553279400095,
94
- "rouge_l": 0.16453907670771373,
95
- "chexpert_f1_14_micro": 0.15713704758031719,
96
- "chexpert_f1_5_micro": 0.15734035549703754,
97
- "chexpert_f1_14_macro": 0.07622868830082337,
98
- "chexpert_f1_5_macro": 0.10675159700014243,
99
- "chexpert_f1_micro": 0.15713704758031719,
100
- "chexpert_f1_macro": 0.07622868830082337,
101
  "chexpert_per_label_f1": {
102
  "Enlarged Cardiomediastinum": 0.0,
103
- "Cardiomegaly": 0.02650602409638554,
104
  "Lung Opacity": 0.0,
105
  "Lung Lesion": 0.0,
106
- "Edema": 0.15193026151930264,
107
- "Consolidation": 0.08468468468468468,
108
- "Pneumonia": 0.012539184952978058,
109
  "Atelectasis": 0.0,
110
- "Pneumothorax": 0.03817177297840281,
111
- "Pleural Effusion": 0.27063701470033924,
112
  "Pleural Other": 0.0,
113
- "Fracture": 0.032,
114
- "Support Devices": 0.45073269327943405,
115
  "No Finding": 0.0
116
  },
117
- "radgraph_f1": 0.09451862388311026,
118
- "radgraph_f1_entity": 0.1497185319208198,
119
- "radgraph_f1_relation": 0.13500246910571428,
120
  "radgraph_available": true,
121
  "radgraph_error": null
122
  },
@@ -126,35 +126,35 @@
126
  "dataset": "mimic-cxr",
127
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
128
  "num_examples": 2210,
129
- "bleu_1": 0.20210908304941103,
130
- "bleu_4": 0.04590488103429025,
131
- "meteor": 0.23388109986749062,
132
- "rouge_l": 0.1714484670889256,
133
- "chexpert_f1_14_micro": 0.14500057267208796,
134
- "chexpert_f1_5_micro": 0.17467652495378927,
135
- "chexpert_f1_14_macro": 0.07493938867200726,
136
- "chexpert_f1_5_macro": 0.11821241603707926,
137
- "chexpert_f1_micro": 0.14500057267208796,
138
- "chexpert_f1_macro": 0.07493938867200726,
139
  "chexpert_per_label_f1": {
140
  "Enlarged Cardiomediastinum": 0.0,
141
- "Cardiomegaly": 0.04140786749482402,
142
  "Lung Opacity": 0.0,
143
  "Lung Lesion": 0.0,
144
- "Edema": 0.1845018450184502,
145
- "Consolidation": 0.08130081300813008,
146
- "Pneumonia": 0.018433179723502308,
147
  "Atelectasis": 0.0,
148
- "Pneumothorax": 0.031665611146295125,
149
- "Pleural Effusion": 0.283851554663992,
150
  "Pleural Other": 0.0,
151
- "Fracture": 0.020618556701030927,
152
- "Support Devices": 0.3873720136518771,
153
  "No Finding": 0.0
154
  },
155
- "radgraph_f1": 0.10264445141069471,
156
- "radgraph_f1_entity": 0.16049110076991005,
157
- "radgraph_f1_relation": 0.1433593428807833,
158
  "radgraph_available": true,
159
  "radgraph_error": null
160
  }
 
1
  {
2
  "method": "lora_adamw",
3
  "run_name": "mimic only",
4
+ "steps": 52709,
5
+ "epochs_completed": 3,
6
+ "epoch_index": 3,
7
  "target_epochs": 3,
8
+ "progress_epochs": 4.0,
9
+ "training_completion_percent": 100.0,
10
+ "elapsed_seconds": 28357.192767200002,
11
+ "images_seen": 421707,
12
+ "train_loss_last": 1.9586591720581055,
13
+ "train_loss_mean": 1.7669410426096004,
14
+ "val_loss": 1.749265193939209,
15
+ "images_per_second": 14.871253422792156,
16
  "trainable_params": 2878464,
17
  "vision_model_name": "facebook/dinov3-vits16-pretrain-lvd1689m",
18
  "text_model_name": "gpt2",
 
36
  "seed": 42,
37
  "resume_supported": true,
38
  "checkpoint_every_n_steps": 1000,
39
+ "cumulative_loss_sum": 745131.4062557667,
40
+ "cumulative_loss_count": 421707,
41
+ "completed": true,
42
  "target_duration_seconds": 3600,
43
  "target_duration_mode": "per_invocation",
44
  "train_datasets": "MIMIC-CXR (findings-only)",
 
49
  "dataset": "mimic-cxr",
50
  "view_filter": "frontal-only (PA/AP)",
51
  "num_examples": 3041,
52
+ "bleu_1": 0.19157722199566865,
53
+ "bleu_4": 0.038562318137225,
54
+ "meteor": 0.2201968843785929,
55
+ "rouge_l": 0.16527670443755577,
56
+ "chexpert_f1_14_micro": 0.15648041946583646,
57
+ "chexpert_f1_5_micro": 0.15297261189044756,
58
+ "chexpert_f1_14_macro": 0.07133711095756078,
59
+ "chexpert_f1_5_macro": 0.10070328760806402,
60
+ "chexpert_f1_micro": 0.15648041946583646,
61
+ "chexpert_f1_macro": 0.07133711095756078,
62
  "chexpert_per_label_f1": {
63
  "Enlarged Cardiomediastinum": 0.0,
64
+ "Cardiomegaly": 0.012135922330097087,
65
  "Lung Opacity": 0.0,
66
  "Lung Lesion": 0.0,
67
+ "Edema": 0.1355081555834379,
68
+ "Consolidation": 0.0855813953488372,
69
+ "Pneumonia": 0.006153846153846154,
70
  "Atelectasis": 0.0,
71
+ "Pneumothorax": 0.04209445585215606,
72
+ "Pleural Effusion": 0.27029096477794795,
73
  "Pleural Other": 0.0,
74
+ "Fracture": 0.0,
75
+ "Support Devices": 0.44695481335952847,
76
  "No Finding": 0.0
77
  },
78
+ "radgraph_f1": 0.09208450105647495,
79
+ "radgraph_f1_entity": 0.14592416918723933,
80
+ "radgraph_f1_relation": 0.13218202095291967,
81
  "radgraph_available": true,
82
  "radgraph_error": null
83
  },
 
88
  "dataset": "mimic-cxr",
89
  "view_filter": "frontal-only (PA/AP)",
90
  "num_examples": 3041,
91
+ "bleu_1": 0.19157722199566865,
92
+ "bleu_4": 0.038562318137225,
93
+ "meteor": 0.2201968843785929,
94
+ "rouge_l": 0.16527670443755577,
95
+ "chexpert_f1_14_micro": 0.15648041946583646,
96
+ "chexpert_f1_5_micro": 0.15297261189044756,
97
+ "chexpert_f1_14_macro": 0.07133711095756078,
98
+ "chexpert_f1_5_macro": 0.10070328760806402,
99
+ "chexpert_f1_micro": 0.15648041946583646,
100
+ "chexpert_f1_macro": 0.07133711095756078,
101
  "chexpert_per_label_f1": {
102
  "Enlarged Cardiomediastinum": 0.0,
103
+ "Cardiomegaly": 0.012135922330097087,
104
  "Lung Opacity": 0.0,
105
  "Lung Lesion": 0.0,
106
+ "Edema": 0.1355081555834379,
107
+ "Consolidation": 0.0855813953488372,
108
+ "Pneumonia": 0.006153846153846154,
109
  "Atelectasis": 0.0,
110
+ "Pneumothorax": 0.04209445585215606,
111
+ "Pleural Effusion": 0.27029096477794795,
112
  "Pleural Other": 0.0,
113
+ "Fracture": 0.0,
114
+ "Support Devices": 0.44695481335952847,
115
  "No Finding": 0.0
116
  },
117
+ "radgraph_f1": 0.09208450105647495,
118
+ "radgraph_f1_entity": 0.14592416918723933,
119
+ "radgraph_f1_relation": 0.13218202095291967,
120
  "radgraph_available": true,
121
  "radgraph_error": null
122
  },
 
126
  "dataset": "mimic-cxr",
127
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
128
  "num_examples": 2210,
129
+ "bleu_1": 0.2002922421466921,
130
+ "bleu_4": 0.04489491191876499,
131
+ "meteor": 0.2347130721675525,
132
+ "rouge_l": 0.1720439436054897,
133
+ "chexpert_f1_14_micro": 0.14424186475882758,
134
+ "chexpert_f1_5_micro": 0.1716295427901524,
135
+ "chexpert_f1_14_macro": 0.06995745688790377,
136
+ "chexpert_f1_5_macro": 0.11117899627949375,
137
+ "chexpert_f1_micro": 0.14424186475882758,
138
+ "chexpert_f1_macro": 0.06995745688790377,
139
  "chexpert_per_label_f1": {
140
  "Enlarged Cardiomediastinum": 0.0,
141
+ "Cardiomegaly": 0.020790020790020795,
142
  "Lung Opacity": 0.0,
143
  "Lung Lesion": 0.0,
144
+ "Edema": 0.16448598130841122,
145
+ "Consolidation": 0.08403361344537814,
146
+ "Pneumonia": 0.009090909090909092,
147
  "Atelectasis": 0.0,
148
+ "Pneumothorax": 0.03612903225806451,
149
+ "Pleural Effusion": 0.28658536585365857,
150
  "Pleural Other": 0.0,
151
+ "Fracture": 0.0,
152
+ "Support Devices": 0.3782894736842105,
153
  "No Finding": 0.0
154
  },
155
+ "radgraph_f1": 0.10000876408490725,
156
+ "radgraph_f1_entity": 0.15770684059564277,
157
+ "radgraph_f1_relation": 0.14126690975537282,
158
  "radgraph_available": true,
159
  "radgraph_error": null
160
  }