Hugging Face's logo

manu02
/
LAnA-v2

Image-to-Text
Transformers
Safetensors
lana_radgen
feature-extraction
medical-ai
radiology
chest-xray
report-generation
segmentation
anatomical-attention
custom_code
Model card Files
xet
 Community
manu02 commited on
Commit
5b76451
·
verified ·
1 Parent(s): 45c97b1

Refresh LAnA collection comparison tables

Browse files
Files changed (7) hide show
  1. README.md +32 -32
  2. evaluations/mimic_test_findings_only_metrics.json +17 -17
  3. evaluations/mimic_test_findings_only_predictions.csv +0 -0
  4. evaluations/mimic_test_metrics.json +51 -51
  5. evaluations/mimic_test_predictions.csv +0 -0
  6. model.safetensors +1 -1
  7. run_summary.json +65 -65
README.md CHANGED
@@ -105,34 +105,34 @@ These comparison tables are refreshed across the full LAnA collection whenever a
105
  | Metric | LAnA | LAnA-MIMIC | LAnA-MIMIC-CHEXPERT | LAnA-v2 |
106
  | --- | --- | --- | --- | --- |
107
  | Number of studies | `3041` | `3041` | `3041` | `3041` |
108
- | ROUGE-L | `0.1686` | `0.1653` | `0.1513` | `0.1602` |
109
- | BLEU-1 | `0.2091` | `0.1916` | `0.1707` | `0.2056` |
110
- | BLEU-4 | `0.0417` | `0.0386` | `0.0357` | `0.0343` |
111
- | METEOR | `0.2298` | `0.2202` | `0.2079` | `0.1777` |
112
- | RadGraph F1 | `0.1024` | `0.0921` | `0.0918` | `0.1002` |
113
- | RadGraph entity F1 | `0.1587` | `0.1459` | `0.1399` | `0.1507` |
114
- | RadGraph relation F1 | `0.1443` | `0.1322` | `0.1246` | `0.1430` |
115
- | CheXpert F1 14-micro | `0.2116` | `0.1565` | `0.1829` | `0.1483` |
116
- | CheXpert F1 5-micro | `0.2512` | `0.1530` | `0.2183` | `0.2384` |
117
- | CheXpert F1 14-macro | `0.1095` | `0.0713` | `0.1095` | `0.0532` |
118
- | CheXpert F1 5-macro | `0.1644` | `0.1007` | `0.1634` | `0.1370` |
119
 
120
  ### Cross-Model Comparison: Findings-Only Frontal Test Studies
121
 
122
  | Metric | LAnA | LAnA-MIMIC | LAnA-MIMIC-CHEXPERT | LAnA-v2 |
123
  | --- | --- | --- | --- | --- |
124
  | Number of studies | `2210` | `2210` | `2210` | `2210` |
125
- | ROUGE-L | `0.1771` | `0.1720` | `0.1576` | `0.1694` |
126
- | BLEU-1 | `0.2177` | `0.2003` | `0.1754` | `0.2211` |
127
- | BLEU-4 | `0.0484` | `0.0449` | `0.0405` | `0.0410` |
128
- | METEOR | `0.2466` | `0.2347` | `0.2207` | `0.1914` |
129
- | RadGraph F1 | `0.1119` | `0.1000` | `0.1010` | `0.1092` |
130
- | RadGraph entity F1 | `0.1713` | `0.1577` | `0.1517` | `0.1655` |
131
- | RadGraph relation F1 | `0.1549` | `0.1413` | `0.1347` | `0.1564` |
132
- | CheXpert F1 14-micro | `0.1907` | `0.1442` | `0.1651` | `0.1318` |
133
- | CheXpert F1 5-micro | `0.2415` | `0.1716` | `0.2152` | `0.2110` |
134
- | CheXpert F1 14-macro | `0.1039` | `0.0700` | `0.1047` | `0.0456` |
135
- | CheXpert F1 5-macro | `0.1578` | `0.1112` | `0.1611` | `0.1163` |
136
 
137
  ## Data
138
 
@@ -148,7 +148,7 @@ These comparison tables are refreshed across the full LAnA collection whenever a
148
  ## Training Snapshot
149
 
150
  - Run: `LAnA-v2`
151
- - This section describes the current public checkpoint, not the final completed project.
152
  - Method: `full_adamw`
153
  - Vision encoder: `facebook/dinov3-vits16-pretrain-lvd1689m`
154
  - Text decoder: `gpt2`
@@ -159,20 +159,20 @@ These comparison tables are refreshed across the full LAnA collection whenever a
159
  - Scheduler: `cosine`
160
  - Warmup steps: `165`
161
  - Weight decay: `0.01`
162
- - Steps completed: `1192`
163
  - Planned total steps: `3297`
164
- - Images seen: `152727`
165
- - Total training time: `3.0000` hours
166
  - Hardware: `NVIDIA GeForce RTX 5070`
167
- - Final train loss: `1.3767`
168
- - Validation loss: `2.2491`
169
 
170
  ## Status
171
 
172
- - Project status: `Training in progress`
173
- - Release status: `Research preview checkpoint`
174
- - Current checkpoint status: `Not final`
175
- - Training completion toward planned run: `36.22%` (`1` / `3` epochs)
176
  - Current published metrics are intermediate and will change as training continues.
177
 
178
  ## Notes
 
105
  | Metric | LAnA | LAnA-MIMIC | LAnA-MIMIC-CHEXPERT | LAnA-v2 |
106
  | --- | --- | --- | --- | --- |
107
  | Number of studies | `3041` | `3041` | `3041` | `3041` |
108
+ | ROUGE-L | `0.1686` | `0.1653` | `0.1513` | `0.1670` |
109
+ | BLEU-1 | `0.2091` | `0.1916` | `0.1707` | `0.2174` |
110
+ | BLEU-4 | `0.0417` | `0.0386` | `0.0357` | `0.0417` |
111
+ | METEOR | `0.2298` | `0.2202` | `0.2079` | `0.2063` |
112
+ | RadGraph F1 | `0.1024` | `0.0921` | `0.0918` | `0.1057` |
113
+ | RadGraph entity F1 | `0.1587` | `0.1459` | `0.1399` | `0.1569` |
114
+ | RadGraph relation F1 | `0.1443` | `0.1322` | `0.1246` | `0.1474` |
115
+ | CheXpert F1 14-micro | `0.2116` | `0.1565` | `0.1829` | `0.1401` |
116
+ | CheXpert F1 5-micro | `0.2512` | `0.1530` | `0.2183` | `0.2506` |
117
+ | CheXpert F1 14-macro | `0.1095` | `0.0713` | `0.1095` | `0.0401` |
118
+ | CheXpert F1 5-macro | `0.1644` | `0.1007` | `0.1634` | `0.1004` |
119
 
120
  ### Cross-Model Comparison: Findings-Only Frontal Test Studies
121
 
122
  | Metric | LAnA | LAnA-MIMIC | LAnA-MIMIC-CHEXPERT | LAnA-v2 |
123
  | --- | --- | --- | --- | --- |
124
  | Number of studies | `2210` | `2210` | `2210` | `2210` |
125
+ | ROUGE-L | `0.1771` | `0.1720` | `0.1576` | `0.1771` |
126
+ | BLEU-1 | `0.2177` | `0.2003` | `0.1754` | `0.2263` |
127
+ | BLEU-4 | `0.0484` | `0.0449` | `0.0405` | `0.0487` |
128
+ | METEOR | `0.2466` | `0.2347` | `0.2207` | `0.2240` |
129
+ | RadGraph F1 | `0.1119` | `0.1000` | `0.1010` | `0.1181` |
130
+ | RadGraph entity F1 | `0.1713` | `0.1577` | `0.1517` | `0.1739` |
131
+ | RadGraph relation F1 | `0.1549` | `0.1413` | `0.1347` | `0.1628` |
132
+ | CheXpert F1 14-micro | `0.1907` | `0.1442` | `0.1651` | `0.1365` |
133
+ | CheXpert F1 5-micro | `0.2415` | `0.1716` | `0.2152` | `0.2455` |
134
+ | CheXpert F1 14-macro | `0.1039` | `0.0700` | `0.1047` | `0.0381` |
135
+ | CheXpert F1 5-macro | `0.1578` | `0.1112` | `0.1611` | `0.0952` |
136
 
137
  ## Data
138
 
 
148
  ## Training Snapshot
149
 
150
  - Run: `LAnA-v2`
151
+ - This section describes the completed public training run.
152
  - Method: `full_adamw`
153
  - Vision encoder: `facebook/dinov3-vits16-pretrain-lvd1689m`
154
  - Text decoder: `gpt2`
 
159
  - Scheduler: `cosine`
160
  - Warmup steps: `165`
161
  - Weight decay: `0.01`
162
+ - Steps completed: `3291`
163
  - Planned total steps: `3297`
164
+ - Images seen: `421707`
165
+ - Total training time: `8.2334` hours
166
  - Hardware: `NVIDIA GeForce RTX 5070`
167
+ - Final train loss: `2.0342`
168
+ - Validation loss: `1.8935`
169
 
170
  ## Status
171
 
172
+ - Project status: `Training completed`
173
+ - Release status: `Completed training run`
174
+ - Current checkpoint status: `Final completed run`
175
+ - Training completion toward planned run: `100.00%` (`3` / `3` epochs)
176
  - Current published metrics are intermediate and will change as training continues.
177
 
178
  ## Notes
evaluations/mimic_test_findings_only_metrics.json CHANGED
@@ -4,35 +4,35 @@
4
  "dataset": "mimic-cxr",
5
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
6
  "num_examples": 2210,
7
- "bleu_1": 0.22113242366581598,
8
- "bleu_4": 0.041034597753022414,
9
- "meteor": 0.1913622592810065,
10
- "rouge_l": 0.16940848109876366,
11
- "chexpert_f1_14_micro": 0.13183812774256462,
12
- "chexpert_f1_5_micro": 0.21098066298342544,
13
- "chexpert_f1_14_macro": 0.04563254512484167,
14
- "chexpert_f1_5_macro": 0.11634255492098525,
15
- "chexpert_f1_micro": 0.13183812774256462,
16
- "chexpert_f1_macro": 0.04563254512484167,
17
  "chexpert_per_label_f1": {
18
  "Enlarged Cardiomediastinum": 0.0,
19
  "Cardiomegaly": 0.0,
20
  "Lung Opacity": 0.0,
21
  "Lung Lesion": 0.0,
22
- "Edema": 0.06382978723404256,
23
- "Consolidation": 0.08139534883720931,
24
  "Pneumonia": 0.0,
25
  "Atelectasis": 0.0,
26
- "Pneumothorax": 0.05714285714285715,
27
- "Pleural Effusion": 0.43648763853367434,
28
  "Pleural Other": 0.0,
29
  "Fracture": 0.0,
30
  "Support Devices": 0.0,
31
  "No Finding": 0.0
32
  },
33
- "radgraph_f1": 0.1092071535341682,
34
- "radgraph_f1_entity": 0.16554743220060325,
35
- "radgraph_f1_relation": 0.15638716446592138,
36
  "radgraph_available": true,
37
  "radgraph_error": null
38
  }
 
4
  "dataset": "mimic-cxr",
5
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
6
  "num_examples": 2210,
7
+ "bleu_1": 0.22630609621984024,
8
+ "bleu_4": 0.04866563061068937,
9
+ "meteor": 0.22402634017445067,
10
+ "rouge_l": 0.177079798127793,
11
+ "chexpert_f1_14_micro": 0.13649310698416317,
12
+ "chexpert_f1_5_micro": 0.24546914429915118,
13
+ "chexpert_f1_14_macro": 0.0380912817648837,
14
+ "chexpert_f1_5_macro": 0.09524230316369844,
15
+ "chexpert_f1_micro": 0.13649310698416317,
16
+ "chexpert_f1_macro": 0.0380912817648837,
17
  "chexpert_per_label_f1": {
18
  "Enlarged Cardiomediastinum": 0.0,
19
  "Cardiomegaly": 0.0,
20
  "Lung Opacity": 0.0,
21
  "Lung Lesion": 0.0,
22
+ "Edema": 0.004608294930875575,
23
+ "Consolidation": 0.06896551724137931,
24
  "Pneumonia": 0.0,
25
  "Atelectasis": 0.0,
26
+ "Pneumothorax": 0.05706642888987963,
27
+ "Pleural Effusion": 0.40263770364623735,
28
  "Pleural Other": 0.0,
29
  "Fracture": 0.0,
30
  "Support Devices": 0.0,
31
  "No Finding": 0.0
32
  },
33
+ "radgraph_f1": 0.11810302465847898,
34
+ "radgraph_f1_entity": 0.17388228759554128,
35
+ "radgraph_f1_relation": 0.16282894612926332,
36
  "radgraph_available": true,
37
  "radgraph_error": null
38
  }
evaluations/mimic_test_findings_only_predictions.csv CHANGED
The diff for this file is too large to render. See raw diff
 
evaluations/mimic_test_metrics.json CHANGED
@@ -4,35 +4,35 @@
4
  "dataset": "mimic-cxr",
5
  "view_filter": "frontal-only (PA/AP)",
6
  "num_examples": 3041,
7
- "bleu_1": 0.20564157881166142,
8
- "bleu_4": 0.03427664002701497,
9
- "meteor": 0.17768676174073475,
10
- "rouge_l": 0.16016357197091377,
11
- "chexpert_f1_14_micro": 0.14833243679397526,
12
- "chexpert_f1_5_micro": 0.23839338452451267,
13
- "chexpert_f1_14_macro": 0.05322713592072648,
14
- "chexpert_f1_5_macro": 0.1370423601633611,
15
- "chexpert_f1_micro": 0.14833243679397526,
16
- "chexpert_f1_macro": 0.05322713592072648,
17
  "chexpert_per_label_f1": {
18
  "Enlarged Cardiomediastinum": 0.0,
19
  "Cardiomegaly": 0.0,
20
  "Lung Opacity": 0.0,
21
  "Lung Lesion": 0.0,
22
- "Edema": 0.08355795148247978,
23
- "Consolidation": 0.11724137931034483,
24
  "Pneumonia": 0.0,
25
  "Atelectasis": 0.0,
26
- "Pneumothorax": 0.059968102073365226,
27
- "Pleural Effusion": 0.4844124700239809,
28
  "Pleural Other": 0.0,
29
  "Fracture": 0.0,
30
  "Support Devices": 0.0,
31
  "No Finding": 0.0
32
  },
33
- "radgraph_f1": 0.10023178019933149,
34
- "radgraph_f1_entity": 0.15074529626474276,
35
- "radgraph_f1_relation": 0.14297091453365654,
36
  "radgraph_available": true,
37
  "radgraph_error": null,
38
  "evaluation_suite": "mimic_test_dual",
@@ -42,35 +42,35 @@
42
  "dataset": "mimic-cxr",
43
  "view_filter": "frontal-only (PA/AP)",
44
  "num_examples": 3041,
45
- "bleu_1": 0.20564157881166142,
46
- "bleu_4": 0.03427664002701497,
47
- "meteor": 0.17768676174073475,
48
- "rouge_l": 0.16016357197091377,
49
- "chexpert_f1_14_micro": 0.14833243679397526,
50
- "chexpert_f1_5_micro": 0.23839338452451267,
51
- "chexpert_f1_14_macro": 0.05322713592072648,
52
- "chexpert_f1_5_macro": 0.1370423601633611,
53
- "chexpert_f1_micro": 0.14833243679397526,
54
- "chexpert_f1_macro": 0.05322713592072648,
55
  "chexpert_per_label_f1": {
56
  "Enlarged Cardiomediastinum": 0.0,
57
  "Cardiomegaly": 0.0,
58
  "Lung Opacity": 0.0,
59
  "Lung Lesion": 0.0,
60
- "Edema": 0.08355795148247978,
61
- "Consolidation": 0.11724137931034483,
62
  "Pneumonia": 0.0,
63
  "Atelectasis": 0.0,
64
- "Pneumothorax": 0.059968102073365226,
65
- "Pleural Effusion": 0.4844124700239809,
66
  "Pleural Other": 0.0,
67
  "Fracture": 0.0,
68
  "Support Devices": 0.0,
69
  "No Finding": 0.0
70
  },
71
- "radgraph_f1": 0.10023178019933149,
72
- "radgraph_f1_entity": 0.15074529626474276,
73
- "radgraph_f1_relation": 0.14297091453365654,
74
  "radgraph_available": true,
75
  "radgraph_error": null
76
  },
@@ -80,35 +80,35 @@
80
  "dataset": "mimic-cxr",
81
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
82
  "num_examples": 2210,
83
- "bleu_1": 0.22113242366581598,
84
- "bleu_4": 0.041034597753022414,
85
- "meteor": 0.1913622592810065,
86
- "rouge_l": 0.16940848109876366,
87
- "chexpert_f1_14_micro": 0.13183812774256462,
88
- "chexpert_f1_5_micro": 0.21098066298342544,
89
- "chexpert_f1_14_macro": 0.04563254512484167,
90
- "chexpert_f1_5_macro": 0.11634255492098525,
91
- "chexpert_f1_micro": 0.13183812774256462,
92
- "chexpert_f1_macro": 0.04563254512484167,
93
  "chexpert_per_label_f1": {
94
  "Enlarged Cardiomediastinum": 0.0,
95
  "Cardiomegaly": 0.0,
96
  "Lung Opacity": 0.0,
97
  "Lung Lesion": 0.0,
98
- "Edema": 0.06382978723404256,
99
- "Consolidation": 0.08139534883720931,
100
  "Pneumonia": 0.0,
101
  "Atelectasis": 0.0,
102
- "Pneumothorax": 0.05714285714285715,
103
- "Pleural Effusion": 0.43648763853367434,
104
  "Pleural Other": 0.0,
105
  "Fracture": 0.0,
106
  "Support Devices": 0.0,
107
  "No Finding": 0.0
108
  },
109
- "radgraph_f1": 0.1092071535341682,
110
- "radgraph_f1_entity": 0.16554743220060325,
111
- "radgraph_f1_relation": 0.15638716446592138,
112
  "radgraph_available": true,
113
  "radgraph_error": null
114
  }
 
4
  "dataset": "mimic-cxr",
5
  "view_filter": "frontal-only (PA/AP)",
6
  "num_examples": 3041,
7
+ "bleu_1": 0.21742288322695155,
8
+ "bleu_4": 0.04173214056804489,
9
+ "meteor": 0.20634783846307375,
10
+ "rouge_l": 0.1670005312845011,
11
+ "chexpert_f1_14_micro": 0.14014644516179825,
12
+ "chexpert_f1_5_micro": 0.2505880508075898,
13
+ "chexpert_f1_14_macro": 0.040103904892629286,
14
+ "chexpert_f1_5_macro": 0.10043811702726824,
15
+ "chexpert_f1_micro": 0.14014644516179825,
16
+ "chexpert_f1_macro": 0.040103904892629286,
17
  "chexpert_per_label_f1": {
18
  "Enlarged Cardiomediastinum": 0.0,
19
  "Cardiomegaly": 0.0,
20
  "Lung Opacity": 0.0,
21
  "Lung Lesion": 0.0,
22
+ "Edema": 0.008915304606240713,
23
+ "Consolidation": 0.061488673139158574,
24
  "Pneumonia": 0.0,
25
  "Atelectasis": 0.0,
26
+ "Pneumothorax": 0.0592640833604689,
27
+ "Pleural Effusion": 0.4317866073909419,
28
  "Pleural Other": 0.0,
29
  "Fracture": 0.0,
30
  "Support Devices": 0.0,
31
  "No Finding": 0.0
32
  },
33
+ "radgraph_f1": 0.10566925514565507,
34
+ "radgraph_f1_entity": 0.15694271054583397,
35
+ "radgraph_f1_relation": 0.14738487269906847,
36
  "radgraph_available": true,
37
  "radgraph_error": null,
38
  "evaluation_suite": "mimic_test_dual",
 
42
  "dataset": "mimic-cxr",
43
  "view_filter": "frontal-only (PA/AP)",
44
  "num_examples": 3041,
45
+ "bleu_1": 0.21742288322695155,
46
+ "bleu_4": 0.04173214056804489,
47
+ "meteor": 0.20634783846307375,
48
+ "rouge_l": 0.1670005312845011,
49
+ "chexpert_f1_14_micro": 0.14014644516179825,
50
+ "chexpert_f1_5_micro": 0.2505880508075898,
51
+ "chexpert_f1_14_macro": 0.040103904892629286,
52
+ "chexpert_f1_5_macro": 0.10043811702726824,
53
+ "chexpert_f1_micro": 0.14014644516179825,
54
+ "chexpert_f1_macro": 0.040103904892629286,
55
  "chexpert_per_label_f1": {
56
  "Enlarged Cardiomediastinum": 0.0,
57
  "Cardiomegaly": 0.0,
58
  "Lung Opacity": 0.0,
59
  "Lung Lesion": 0.0,
60
+ "Edema": 0.008915304606240713,
61
+ "Consolidation": 0.061488673139158574,
62
  "Pneumonia": 0.0,
63
  "Atelectasis": 0.0,
64
+ "Pneumothorax": 0.0592640833604689,
65
+ "Pleural Effusion": 0.4317866073909419,
66
  "Pleural Other": 0.0,
67
  "Fracture": 0.0,
68
  "Support Devices": 0.0,
69
  "No Finding": 0.0
70
  },
71
+ "radgraph_f1": 0.10566925514565507,
72
+ "radgraph_f1_entity": 0.15694271054583397,
73
+ "radgraph_f1_relation": 0.14738487269906847,
74
  "radgraph_available": true,
75
  "radgraph_error": null
76
  },
 
80
  "dataset": "mimic-cxr",
81
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
82
  "num_examples": 2210,
83
+ "bleu_1": 0.22630609621984024,
84
+ "bleu_4": 0.04866563061068937,
85
+ "meteor": 0.22402634017445067,
86
+ "rouge_l": 0.177079798127793,
87
+ "chexpert_f1_14_micro": 0.13649310698416317,
88
+ "chexpert_f1_5_micro": 0.24546914429915118,
89
+ "chexpert_f1_14_macro": 0.0380912817648837,
90
+ "chexpert_f1_5_macro": 0.09524230316369844,
91
+ "chexpert_f1_micro": 0.13649310698416317,
92
+ "chexpert_f1_macro": 0.0380912817648837,
93
  "chexpert_per_label_f1": {
94
  "Enlarged Cardiomediastinum": 0.0,
95
  "Cardiomegaly": 0.0,
96
  "Lung Opacity": 0.0,
97
  "Lung Lesion": 0.0,
98
+ "Edema": 0.004608294930875575,
99
+ "Consolidation": 0.06896551724137931,
100
  "Pneumonia": 0.0,
101
  "Atelectasis": 0.0,
102
+ "Pneumothorax": 0.05706642888987963,
103
+ "Pleural Effusion": 0.40263770364623735,
104
  "Pleural Other": 0.0,
105
  "Fracture": 0.0,
106
  "Support Devices": 0.0,
107
  "No Finding": 0.0
108
  },
109
+ "radgraph_f1": 0.11810302465847898,
110
+ "radgraph_f1_entity": 0.17388228759554128,
111
+ "radgraph_f1_relation": 0.16282894612926332,
112
  "radgraph_available": true,
113
  "radgraph_error": null
114
  }
evaluations/mimic_test_predictions.csv CHANGED
The diff for this file is too large to render. See raw diff
 
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7a4935365de9b1e7d6eb9d43a837a4547dff809b493e73c238848add471cf947
3
  size 1159628024
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1eab62beaf106b8a979ada16f463a6affcda40816ded81ce20ebeaf8c4e3e06
3
  size 1159628024
run_summary.json CHANGED
@@ -1,18 +1,18 @@
1
  {
2
  "method": "full_adamw",
3
  "run_name": "LAnA-v2",
4
- "steps": 1192,
5
- "epochs_completed": 1,
6
- "epoch_index": 1,
7
  "target_epochs": 3,
8
- "progress_epochs": 1.086491331659185,
9
- "training_completion_percent": 36.21637772197283,
10
- "elapsed_seconds": 10800.1367957,
11
- "images_seen": 152727,
12
- "train_loss_last": 1.3767017126083374,
13
- "train_loss_mean": 3.2542089584969984,
14
- "val_loss": 2.2491145610809324,
15
- "images_per_second": 14.141209772528732,
16
  "trainable_params": 127293696,
17
  "vision_model_name": "facebook/dinov3-vits16-pretrain-lvd1689m",
18
  "text_model_name": "gpt2",
@@ -36,9 +36,9 @@
36
  "seed": 42,
37
  "resume_supported": true,
38
  "checkpoint_every_n_steps": 1000,
39
- "cumulative_loss_sum": 497005.57160437107,
40
- "cumulative_loss_count": 152727,
41
- "completed": false,
42
  "target_duration_seconds": 3600,
43
  "target_duration_mode": "per_invocation",
44
  "train_datasets": "MIMIC-CXR (findings-only)",
@@ -49,35 +49,35 @@
49
  "dataset": "mimic-cxr",
50
  "view_filter": "frontal-only (PA/AP)",
51
  "num_examples": 3041,
52
- "bleu_1": 0.20564157881166142,
53
- "bleu_4": 0.03427664002701497,
54
- "meteor": 0.17768676174073475,
55
- "rouge_l": 0.16016357197091377,
56
- "chexpert_f1_14_micro": 0.14833243679397526,
57
- "chexpert_f1_5_micro": 0.23839338452451267,
58
- "chexpert_f1_14_macro": 0.05322713592072648,
59
- "chexpert_f1_5_macro": 0.1370423601633611,
60
- "chexpert_f1_micro": 0.14833243679397526,
61
- "chexpert_f1_macro": 0.05322713592072648,
62
  "chexpert_per_label_f1": {
63
  "Enlarged Cardiomediastinum": 0.0,
64
  "Cardiomegaly": 0.0,
65
  "Lung Opacity": 0.0,
66
  "Lung Lesion": 0.0,
67
- "Edema": 0.08355795148247978,
68
- "Consolidation": 0.11724137931034483,
69
  "Pneumonia": 0.0,
70
  "Atelectasis": 0.0,
71
- "Pneumothorax": 0.059968102073365226,
72
- "Pleural Effusion": 0.4844124700239809,
73
  "Pleural Other": 0.0,
74
  "Fracture": 0.0,
75
  "Support Devices": 0.0,
76
  "No Finding": 0.0
77
  },
78
- "radgraph_f1": 0.10023178019933149,
79
- "radgraph_f1_entity": 0.15074529626474276,
80
- "radgraph_f1_relation": 0.14297091453365654,
81
  "radgraph_available": true,
82
  "radgraph_error": null
83
  },
@@ -88,35 +88,35 @@
88
  "dataset": "mimic-cxr",
89
  "view_filter": "frontal-only (PA/AP)",
90
  "num_examples": 3041,
91
- "bleu_1": 0.20564157881166142,
92
- "bleu_4": 0.03427664002701497,
93
- "meteor": 0.17768676174073475,
94
- "rouge_l": 0.16016357197091377,
95
- "chexpert_f1_14_micro": 0.14833243679397526,
96
- "chexpert_f1_5_micro": 0.23839338452451267,
97
- "chexpert_f1_14_macro": 0.05322713592072648,
98
- "chexpert_f1_5_macro": 0.1370423601633611,
99
- "chexpert_f1_micro": 0.14833243679397526,
100
- "chexpert_f1_macro": 0.05322713592072648,
101
  "chexpert_per_label_f1": {
102
  "Enlarged Cardiomediastinum": 0.0,
103
  "Cardiomegaly": 0.0,
104
  "Lung Opacity": 0.0,
105
  "Lung Lesion": 0.0,
106
- "Edema": 0.08355795148247978,
107
- "Consolidation": 0.11724137931034483,
108
  "Pneumonia": 0.0,
109
  "Atelectasis": 0.0,
110
- "Pneumothorax": 0.059968102073365226,
111
- "Pleural Effusion": 0.4844124700239809,
112
  "Pleural Other": 0.0,
113
  "Fracture": 0.0,
114
  "Support Devices": 0.0,
115
  "No Finding": 0.0
116
  },
117
- "radgraph_f1": 0.10023178019933149,
118
- "radgraph_f1_entity": 0.15074529626474276,
119
- "radgraph_f1_relation": 0.14297091453365654,
120
  "radgraph_available": true,
121
  "radgraph_error": null
122
  },
@@ -126,35 +126,35 @@
126
  "dataset": "mimic-cxr",
127
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
128
  "num_examples": 2210,
129
- "bleu_1": 0.22113242366581598,
130
- "bleu_4": 0.041034597753022414,
131
- "meteor": 0.1913622592810065,
132
- "rouge_l": 0.16940848109876366,
133
- "chexpert_f1_14_micro": 0.13183812774256462,
134
- "chexpert_f1_5_micro": 0.21098066298342544,
135
- "chexpert_f1_14_macro": 0.04563254512484167,
136
- "chexpert_f1_5_macro": 0.11634255492098525,
137
- "chexpert_f1_micro": 0.13183812774256462,
138
- "chexpert_f1_macro": 0.04563254512484167,
139
  "chexpert_per_label_f1": {
140
  "Enlarged Cardiomediastinum": 0.0,
141
  "Cardiomegaly": 0.0,
142
  "Lung Opacity": 0.0,
143
  "Lung Lesion": 0.0,
144
- "Edema": 0.06382978723404256,
145
- "Consolidation": 0.08139534883720931,
146
  "Pneumonia": 0.0,
147
  "Atelectasis": 0.0,
148
- "Pneumothorax": 0.05714285714285715,
149
- "Pleural Effusion": 0.43648763853367434,
150
  "Pleural Other": 0.0,
151
  "Fracture": 0.0,
152
  "Support Devices": 0.0,
153
  "No Finding": 0.0
154
  },
155
- "radgraph_f1": 0.1092071535341682,
156
- "radgraph_f1_entity": 0.16554743220060325,
157
- "radgraph_f1_relation": 0.15638716446592138,
158
  "radgraph_available": true,
159
  "radgraph_error": null
160
  }
 
1
  {
2
  "method": "full_adamw",
3
  "run_name": "LAnA-v2",
4
+ "steps": 3291,
5
+ "epochs_completed": 3,
6
+ "epoch_index": 3,
7
  "target_epochs": 3,
8
+ "progress_epochs": 4.0,
9
+ "training_completion_percent": 100.0,
10
+ "elapsed_seconds": 29640.131814300014,
11
+ "images_seen": 421707,
12
+ "train_loss_last": 2.034162998199463,
13
+ "train_loss_mean": 2.34121817688727,
14
+ "val_loss": 1.893471932411194,
15
+ "images_per_second": 14.227568306445438,
16
  "trainable_params": 127293696,
17
  "vision_model_name": "facebook/dinov3-vits16-pretrain-lvd1689m",
18
  "text_model_name": "gpt2",
 
36
  "seed": 42,
37
  "resume_supported": true,
38
  "checkpoint_every_n_steps": 1000,
39
+ "cumulative_loss_sum": 987308.0937206,
40
+ "cumulative_loss_count": 421707,
41
+ "completed": true,
42
  "target_duration_seconds": 3600,
43
  "target_duration_mode": "per_invocation",
44
  "train_datasets": "MIMIC-CXR (findings-only)",
 
49
  "dataset": "mimic-cxr",
50
  "view_filter": "frontal-only (PA/AP)",
51
  "num_examples": 3041,
52
+ "bleu_1": 0.21742288322695155,
53
+ "bleu_4": 0.04173214056804489,
54
+ "meteor": 0.20634783846307375,
55
+ "rouge_l": 0.1670005312845011,
56
+ "chexpert_f1_14_micro": 0.14014644516179825,
57
+ "chexpert_f1_5_micro": 0.2505880508075898,
58
+ "chexpert_f1_14_macro": 0.040103904892629286,
59
+ "chexpert_f1_5_macro": 0.10043811702726824,
60
+ "chexpert_f1_micro": 0.14014644516179825,
61
+ "chexpert_f1_macro": 0.040103904892629286,
62
  "chexpert_per_label_f1": {
63
  "Enlarged Cardiomediastinum": 0.0,
64
  "Cardiomegaly": 0.0,
65
  "Lung Opacity": 0.0,
66
  "Lung Lesion": 0.0,
67
+ "Edema": 0.008915304606240713,
68
+ "Consolidation": 0.061488673139158574,
69
  "Pneumonia": 0.0,
70
  "Atelectasis": 0.0,
71
+ "Pneumothorax": 0.0592640833604689,
72
+ "Pleural Effusion": 0.4317866073909419,
73
  "Pleural Other": 0.0,
74
  "Fracture": 0.0,
75
  "Support Devices": 0.0,
76
  "No Finding": 0.0
77
  },
78
+ "radgraph_f1": 0.10566925514565507,
79
+ "radgraph_f1_entity": 0.15694271054583397,
80
+ "radgraph_f1_relation": 0.14738487269906847,
81
  "radgraph_available": true,
82
  "radgraph_error": null
83
  },
 
88
  "dataset": "mimic-cxr",
89
  "view_filter": "frontal-only (PA/AP)",
90
  "num_examples": 3041,
91
+ "bleu_1": 0.21742288322695155,
92
+ "bleu_4": 0.04173214056804489,
93
+ "meteor": 0.20634783846307375,
94
+ "rouge_l": 0.1670005312845011,
95
+ "chexpert_f1_14_micro": 0.14014644516179825,
96
+ "chexpert_f1_5_micro": 0.2505880508075898,
97
+ "chexpert_f1_14_macro": 0.040103904892629286,
98
+ "chexpert_f1_5_macro": 0.10043811702726824,
99
+ "chexpert_f1_micro": 0.14014644516179825,
100
+ "chexpert_f1_macro": 0.040103904892629286,
101
  "chexpert_per_label_f1": {
102
  "Enlarged Cardiomediastinum": 0.0,
103
  "Cardiomegaly": 0.0,
104
  "Lung Opacity": 0.0,
105
  "Lung Lesion": 0.0,
106
+ "Edema": 0.008915304606240713,
107
+ "Consolidation": 0.061488673139158574,
108
  "Pneumonia": 0.0,
109
  "Atelectasis": 0.0,
110
+ "Pneumothorax": 0.0592640833604689,
111
+ "Pleural Effusion": 0.4317866073909419,
112
  "Pleural Other": 0.0,
113
  "Fracture": 0.0,
114
  "Support Devices": 0.0,
115
  "No Finding": 0.0
116
  },
117
+ "radgraph_f1": 0.10566925514565507,
118
+ "radgraph_f1_entity": 0.15694271054583397,
119
+ "radgraph_f1_relation": 0.14738487269906847,
120
  "radgraph_available": true,
121
  "radgraph_error": null
122
  },
 
126
  "dataset": "mimic-cxr",
127
  "view_filter": "frontal-only (PA/AP), structured Findings section only",
128
  "num_examples": 2210,
129
+ "bleu_1": 0.22630609621984024,
130
+ "bleu_4": 0.04866563061068937,
131
+ "meteor": 0.22402634017445067,
132
+ "rouge_l": 0.177079798127793,
133
+ "chexpert_f1_14_micro": 0.13649310698416317,
134
+ "chexpert_f1_5_micro": 0.24546914429915118,
135
+ "chexpert_f1_14_macro": 0.0380912817648837,
136
+ "chexpert_f1_5_macro": 0.09524230316369844,
137
+ "chexpert_f1_micro": 0.13649310698416317,
138
+ "chexpert_f1_macro": 0.0380912817648837,
139
  "chexpert_per_label_f1": {
140
  "Enlarged Cardiomediastinum": 0.0,
141
  "Cardiomegaly": 0.0,
142
  "Lung Opacity": 0.0,
143
  "Lung Lesion": 0.0,
144
+ "Edema": 0.004608294930875575,
145
+ "Consolidation": 0.06896551724137931,
146
  "Pneumonia": 0.0,
147
  "Atelectasis": 0.0,
148
+ "Pneumothorax": 0.05706642888987963,
149
+ "Pleural Effusion": 0.40263770364623735,
150
  "Pleural Other": 0.0,
151
  "Fracture": 0.0,
152
  "Support Devices": 0.0,
153
  "No Finding": 0.0
154
  },
155
+ "radgraph_f1": 0.11810302465847898,
156
+ "radgraph_f1_entity": 0.17388228759554128,
157
+ "radgraph_f1_relation": 0.16282894612926332,
158
  "radgraph_available": true,
159
  "radgraph_error": null
160
  }