Numan Saeed commited on
Commit
a874986
·
1 Parent(s): d0dbc48

Upgrade to React + FastAPI (Docker-based)

Browse files

- Replace Gradio with modern React frontend
- Add FastAPI backend with proper API
- Docker-based deployment for HF Spaces
- Professional UI with NVIDIA-inspired theme
- DICOM file support with full preprocessing
- Better performance and UX

This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. Dockerfile +66 -0
  2. README.md +63 -9
  3. app.py +0 -320
  4. assets/FetalCLIP_config.json +15 -15
  5. assets/prompt_fetal_view.json +93 -92
  6. backend/app/__init__.py +2 -0
  7. backend/app/main.py +96 -0
  8. backend/app/routes/__init__.py +5 -0
  9. backend/app/routes/classification.py +89 -0
  10. backend/app/routes/gestational_age.py +41 -0
  11. backend/app/services/__init__.py +4 -0
  12. backend/app/services/model.py +267 -0
  13. backend/app/services/preprocessing.py +514 -0
  14. backend/requirements.txt +22 -0
  15. examples/Fetal_abdomen_1.png +0 -3
  16. examples/Fetal_abdomen_2.png +0 -3
  17. examples/Fetal_brain_1.png +0 -3
  18. examples/Fetal_brain_2.png +0 -3
  19. examples/Fetal_femur_1.png +0 -3
  20. examples/Fetal_femur_2.png +0 -3
  21. examples/Fetal_orbit_1 copy.jpg +0 -3
  22. examples/Fetal_orbit_1.jpg +0 -3
  23. examples/Fetal_orbit_2.png +0 -3
  24. examples/Fetal_profile_1 copy.jpg +0 -3
  25. examples/Fetal_profile_1.jpg +0 -3
  26. examples/Fetal_profile_2.png +0 -3
  27. examples/Fetal_thorax_1.png +0 -3
  28. examples/Fetal_thorax_2.png +0 -3
  29. examples/Maternal_cervix_1.png +0 -3
  30. examples/Maternal_cervix_2.png +0 -3
  31. examples/ga_333_HC.png +0 -3
  32. examples/ga_351_HC.png +0 -3
  33. examples/ga_385_HC.png +0 -3
  34. examples/ga_584_HC.png +0 -3
  35. examples/ga_615_HC.png +0 -3
  36. examples/ga_notes.txt +0 -6
  37. frontend/index.html +17 -0
  38. frontend/package-lock.json +0 -0
  39. frontend/package.json +32 -0
  40. frontend/postcss.config.js +7 -0
  41. frontend/public/favicon.svg +6 -0
  42. frontend/src/App.tsx +69 -0
  43. frontend/src/components/Button.tsx +46 -0
  44. frontend/src/components/FileUpload.tsx +106 -0
  45. frontend/src/components/GAResultsCard.tsx +83 -0
  46. frontend/src/components/Header.tsx +50 -0
  47. frontend/src/components/ImageUpload.tsx +77 -0
  48. frontend/src/components/NumberInput.tsx +50 -0
  49. frontend/src/components/Panel.tsx +20 -0
  50. frontend/src/components/PreprocessingBadge.tsx +125 -0
Dockerfile ADDED
@@ -0,0 +1,66 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ============================================
2
+ # FetalCLIP - Hugging Face Spaces Docker Image
3
+ # ============================================
4
+ # This Dockerfile creates a container that runs:
5
+ # - FastAPI backend on port 7860 (HF Spaces requirement)
6
+ # - Serves React frontend as static files
7
+ #
8
+ # Deploy to: https://huggingface.co/spaces
9
+
10
+ FROM python:3.10-slim
11
+
12
+ # Set working directory
13
+ WORKDIR /app
14
+
15
+ # Install system dependencies
16
+ RUN apt-get update && apt-get install -y \
17
+ build-essential \
18
+ curl \
19
+ git \
20
+ libgl1-mesa-glx \
21
+ libglib2.0-0 \
22
+ && rm -rf /var/lib/apt/lists/*
23
+
24
+ # Install Node.js for building frontend
25
+ RUN curl -fsSL https://deb.nodesource.com/setup_18.x | bash - \
26
+ && apt-get install -y nodejs \
27
+ && rm -rf /var/lib/apt/lists/*
28
+
29
+ # Copy backend requirements first (for Docker caching)
30
+ COPY backend/requirements.txt /app/backend/requirements.txt
31
+ RUN pip install --no-cache-dir -r /app/backend/requirements.txt
32
+
33
+ # Copy assets
34
+ COPY assets /app/assets
35
+
36
+ # Copy backend code
37
+ COPY backend/app /app/backend/app
38
+
39
+ # Copy frontend and build
40
+ COPY frontend/package*.json /app/frontend/
41
+ WORKDIR /app/frontend
42
+ RUN npm install
43
+
44
+ COPY frontend /app/frontend
45
+ RUN npm run build
46
+
47
+ # Move built frontend to backend for serving
48
+ RUN mkdir -p /app/backend/static && cp -r /app/frontend/dist/* /app/backend/static/
49
+
50
+ WORKDIR /app
51
+
52
+ # Copy the HF Spaces specific server
53
+ COPY huggingface-spaces/server.py /app/server.py
54
+
55
+ # Expose port 7860 (Hugging Face Spaces requirement)
56
+ EXPOSE 7860
57
+
58
+ # Set environment variables
59
+ ENV PYTHONUNBUFFERED=1
60
+ ENV HF_HOME=/app/.cache
61
+
62
+ # Create cache directory
63
+ RUN mkdir -p /app/.cache
64
+
65
+ # Run the server
66
+ CMD ["python", "server.py"]
README.md CHANGED
@@ -1,14 +1,68 @@
1
  ---
2
  title: FetalCLIP
3
- emoji: 🏆
4
- colorFrom: indigo
5
- colorTo: indigo
6
- sdk: gradio
7
- sdk_version: 5.28.0
8
- app_file: app.py
9
  pinned: false
10
- license: cc-by-nc-4.0
11
- short_description: ' A Visual-Language Foundation Model for Fetal Ultrasound Ima'
12
  ---
13
 
14
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: FetalCLIP
3
+ emoji: 👶
4
+ colorFrom: green
5
+ colorTo: blue
6
+ sdk: docker
 
 
7
  pinned: false
8
+ license: apache-2.0
 
9
  ---
10
 
11
+ # FetalCLIP - Fetal Ultrasound Analysis
12
+
13
+ **Foundation Model for Zero-Shot Fetal Ultrasound Analysis**
14
+
15
+ ## Features
16
+
17
+ - 🔬 **View Classification**: Classify ultrasound images into 13 anatomical views
18
+ - 📅 **Gestational Age Estimation**: Estimate gestational age from fetal brain ultrasounds
19
+ - 🏥 **DICOM Support**: Full preprocessing pipeline for medical DICOM files
20
+ - 🖼️ **PNG/JPEG Support**: Basic preprocessing for standard image files
21
+
22
+ ## How to Use
23
+
24
+ 1. Upload a fetal ultrasound image (PNG, JPEG, or DICOM)
25
+ 2. Click "Classify View" to identify the anatomical plane
26
+ 3. View the top predictions with confidence scores
27
+
28
+ ## Model
29
+
30
+ This demo uses the FetalCLIP model, a vision-language foundation model trained on fetal ultrasound images.
31
+
32
+ - **Model**: [numansaeed/fetalclip-model](https://huggingface.co/numansaeed/fetalclip-model)
33
+ - **Architecture**: ViT-L/14 based CLIP model
34
+ - **Training**: Contrastive learning on fetal ultrasound-text pairs
35
+
36
+ ## Supported Views
37
+
38
+ 1. Fetal abdomen
39
+ 2. Fetal brain (transventricular)
40
+ 3. Fetal brain (transthalamic)
41
+ 4. Fetal brain (transcerebellar)
42
+ 5. Fetal femur
43
+ 6. Fetal heart (4-chamber)
44
+ 7. Fetal heart (LVOT)
45
+ 8. Fetal heart (RVOT)
46
+ 9. Fetal heart (3VV)
47
+ 10. Fetal kidney
48
+ 11. Fetal face (lips)
49
+ 12. Fetal spine (coronal)
50
+ 13. Fetal spine (sagittal)
51
+
52
+ ## Citation
53
+
54
+ If you use this model, please cite:
55
+
56
+ ```bibtex
57
+ @article{fetalclip2024,
58
+ title={FetalCLIP: A Foundation Model for Fetal Ultrasound Analysis},
59
+ author={...},
60
+ year={2024}
61
+ }
62
+ ```
63
+
64
+ ## Links
65
+
66
+ - 📦 [Model Hub](https://huggingface.co/numansaeed/fetalclip-model)
67
+ - 📄 [Paper](#)
68
+ - 💻 [GitHub](#)
app.py DELETED
@@ -1,320 +0,0 @@
1
- import gradio as gr
2
- import torch
3
- from huggingface_hub import hf_hub_download
4
- import open_clip
5
- from PIL import Image
6
- import json
7
- import os
8
- from utils import make_image_square_with_zero_padding
9
- from tqdm import tqdm
10
- import plotly.graph_objects as go
11
- import numpy as np
12
-
13
- # Constants and Configuration
14
- ASSETS_DIR = "assets"
15
- EXAMPLES_DIR = "examples"
16
- PATH_TEXT_PROMPTS = os.path.join(ASSETS_DIR, "prompt_fetal_view.json")
17
- PATH_FETALCLIP_CONFIG = os.path.join(ASSETS_DIR, "FetalCLIP_config.json")
18
- MODEL_NAME = "numansaeed/fetalclip-model"
19
-
20
- # Helper functions for gestational age estimation
21
- INPUT_SIZE = 224
22
- TEXT_PROMPTS = [
23
- "Ultrasound image at {weeks} weeks and {day} days gestation focusing on the fetal brain, highlighting anatomical structures with a pixel spacing of {pixel_spacing} mm/pixel.",
24
- "Fetal ultrasound image at {weeks} weeks, {day} days of gestation, focusing on the developing brain, with a pixel spacing of {pixel_spacing} mm/pixel, highlighting the structures of the fetal brain.",
25
- "Fetal ultrasound image at {weeks} weeks and {day} days gestational age, highlighting the developing brain structures with a pixel spacing of {pixel_spacing} mm/pixel, providing important visual insights for ongoing prenatal assessments.",
26
- "Ultrasound image at {weeks} weeks and {day} days gestation, highlighting the fetal brain structures with a pixel spacing of {pixel_spacing} mm/pixel.",
27
- "Fetal ultrasound at {weeks} weeks and {day} days, showing a clear view of the developing brain, with an image pixel spacing of {pixel_spacing} mm/pixel."
28
- ]
29
- list_ga_in_days = [weeks * 7 + days for weeks in range(14, 39) for days in range(0, 7)]
30
- assert sorted(list_ga_in_days) == list_ga_in_days
31
- TOP_N_PROBS = 15
32
-
33
- tokenizer = None # Make tokenizer global
34
-
35
- def load_model():
36
- global model, preprocess_test, text_features, list_plane, device, tokenizer
37
- device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
38
-
39
- # Load and register model configuration
40
- with open(PATH_FETALCLIP_CONFIG, "r") as file:
41
- config_fetalclip = json.load(file)
42
- open_clip.factory._MODEL_CONFIGS["FetalCLIP"] = config_fetalclip
43
-
44
- # Download model weights from Hugging Face Hub
45
- weights_path = hf_hub_download(
46
- repo_id=MODEL_NAME,
47
- filename="FetalCLIP_weights.pt"
48
- )
49
-
50
- # Load the FetalCLIP model and preprocessing transforms
51
- model, _, preprocess_test = open_clip.create_model_and_transforms(
52
- "FetalCLIP",
53
- pretrained=weights_path
54
- )
55
- tokenizer = open_clip.get_tokenizer("FetalCLIP")
56
-
57
- model = model.float()
58
- model.eval()
59
- model.to(device)
60
-
61
- # Load text prompts
62
- with open(PATH_TEXT_PROMPTS, 'r') as json_file:
63
- text_prompts = json.load(json_file)
64
-
65
- # Extract text features
66
- list_text_features = []
67
- list_plane = []
68
- with torch.no_grad():
69
- for plane, prompts in tqdm(text_prompts.items()):
70
- list_plane.append(plane)
71
-
72
- prompts = tokenizer(prompts).to(device)
73
- text_features = model.encode_text(prompts)
74
- text_features /= text_features.norm(dim=-1, keepdim=True)
75
-
76
- text_features = text_features.mean(dim=0).unsqueeze(0)
77
- text_features /= text_features.norm(dim=-1, keepdim=True)
78
-
79
- list_text_features.append(text_features)
80
- text_features = torch.stack(list_text_features)[:,0]
81
-
82
- return model, preprocess_test, text_features, list_plane, device
83
-
84
- # Load model and text features at startup
85
- model, preprocess_test, text_features, list_plane, device = load_model()
86
-
87
- def process_image(image, top_k):
88
- if image is None:
89
- return None
90
-
91
- try:
92
- # Convert top_k to integer and ensure it's within valid range
93
- top_k = min(int(top_k), 13) # Ensure we don't exceed the number of possible classes
94
-
95
- # Preprocess image
96
- img = make_image_square_with_zero_padding(Image.fromarray(image))
97
- img = preprocess_test(img).unsqueeze(0).to(device)
98
-
99
- # Get image features
100
- with torch.no_grad():
101
- image_features = model.encode_image(img)
102
- image_features /= image_features.norm(dim=-1, keepdim=True)
103
-
104
- # Calculate similarity scores
105
- similarity = (99.2198 * image_features @ text_features.T).softmax(dim=-1) #model.logit_scal.exp() = 99.2198
106
- values, indices = similarity[0].topk(top_k)
107
-
108
- # Create bar chart
109
- labels = [list_plane[idx] for idx in indices]
110
- values = [value.item() * 100 for value in values] # Convert to percentage
111
-
112
- # Reverse the order of labels and values to show highest probability at top
113
- labels = labels[::-1]
114
- values = values[::-1]
115
-
116
- fig = go.Figure(data=[
117
- go.Bar(
118
- x=values,
119
- y=labels,
120
- orientation='h',
121
- text=[f'{v:.1f}%' for v in values],
122
- textposition='auto',
123
- )
124
- ])
125
-
126
- fig.update_layout(
127
- title="Classification Results",
128
- xaxis_title="Confidence (%)",
129
- yaxis_title="Fetal View",
130
- xaxis=dict(range=[0, 100]),
131
- height=max(300, 50 * len(labels)),
132
- margin=dict(l=20, r=20, t=40, b=20)
133
- )
134
-
135
- return fig
136
- except Exception as e:
137
- print(f"Error in process_image: {str(e)}") # Add error logging
138
- return None
139
-
140
- def get_text_prompts(template, pixel_spacing, tokenizer, model, device):
141
- prompts = []
142
- for weeks in range(14, 39):
143
- for days in range(0, 7):
144
- prompt = template.replace("{weeks}", str(weeks))
145
- prompt = prompt.replace("{day}", str(days))
146
- prompt = prompt.replace("{pixel_spacing}", f"{pixel_spacing:.2f}")
147
- prompts.append(prompt)
148
- with torch.no_grad():
149
- prompts = tokenizer(prompts).to(device)
150
- text_features = model.encode_text(prompts)
151
- text_features /= text_features.norm(dim=-1, keepdim=True) # (n_days, 768)
152
- return text_features
153
-
154
- def get_unnormalized_dot_products(image_features, list_text_features):
155
- text_features = torch.cat(list_text_features, dim=0) # (n_days * n_prompts, 768)
156
- text_dot_prods = (100.0 * image_features @ text_features.T)
157
- n_prompts = len(list_text_features) # 5 --> 5 text prompts for each day
158
- n_days = len(list_text_features[0])
159
- text_dot_prods = text_dot_prods.view(image_features.shape[0], n_prompts, n_days)
160
- text_dot_prods = text_dot_prods.mean(dim=1)
161
- return text_dot_prods
162
-
163
- def find_median_from_top_n(text_dot_prods, n):
164
- assert len(text_dot_prods.shape) == 1
165
- tmp = [[i, t] for i, t in enumerate(text_dot_prods)]
166
- tmp = sorted(tmp, key=lambda x: x[1], reverse=True)[:n]
167
- tmp = sorted(tmp, key=lambda x: x[0])
168
- median_ind = tmp[n // 2][0]
169
- return median_ind
170
-
171
- def get_hc_from_days(t, quartile='0.5'):
172
- t = t / 7
173
- dict_params = {
174
- '0.025': [1.59317517131532e+0, 2.9459800552433e-1, -7.3860372566707e-3, 6.56951770216148e-5, 0e+0],
175
- '0.500': [2.09924879247164e+0, 2.53373656106037e-1, -6.05647816678282e-3, 5.14256072059917e-5, 0e+0],
176
- '0.975': [2.50074069629423e+0, 2.20067854715719e-1, -4.93623111462443e-3, 3.89066000946519e-5, 0e+0],
177
- }
178
- b0, b1, b2, b3, b4 = dict_params[quartile]
179
- hc_q50 = np.exp(
180
- b0 + b1*t + b2*t**2 + b3*t**3 + b4*t**4
181
- )
182
- return hc_q50
183
-
184
- def estimate_gestational_age(image, pixel_size):
185
- try:
186
- if image is None or pixel_size is None:
187
- return "Please upload an image and enter pixel size.", "--"
188
- # Convert image to PIL and preprocess
189
- img = Image.fromarray(image)
190
- # Calculate effective pixel spacing after resizing
191
- pixel_spacing = max(img.size) / INPUT_SIZE * float(pixel_size)
192
- img = make_image_square_with_zero_padding(img)
193
- img = preprocess_test(img)
194
- img = img.unsqueeze(0)
195
- img = img.to(device)
196
- # Compute image features
197
- with torch.no_grad():
198
- image_features = model.encode_image(img)
199
- image_features /= image_features.norm(dim=-1, keepdim=True)
200
- # Compute text features for all prompts
201
- values = [get_text_prompts(val, pixel_spacing, tokenizer, model, device) for val in TEXT_PROMPTS]
202
- # Compute dot products
203
- text_dot_prods = get_unnormalized_dot_products(image_features, values) # (1, n_days)
204
- # Compute the GA prediction
205
- text_dot_prod = text_dot_prods.detach().cpu().numpy()[0] # (n_days)
206
- med_indices = find_median_from_top_n(text_dot_prod, TOP_N_PROBS)
207
- pred_day = list_ga_in_days[med_indices]
208
- pred_weeks = pred_day // 7
209
- pred_days = pred_day % 7
210
- # Compute HC interval
211
- q025 = get_hc_from_days(pred_day, '0.025')
212
- q500 = get_hc_from_days(pred_day, '0.500')
213
- q975 = get_hc_from_days(pred_day, '0.975')
214
- # Format outputs
215
- ga_str = f"Predicted: {pred_weeks} weeks, {pred_days} days"
216
- hc_str = f"HC: {q025:.2f} mm [2.5%], {q500:.2f} mm [50%], {q975:.2f} mm [97.5%]"
217
- return ga_str, hc_str
218
- except Exception as e:
219
- print(f"Error in estimate_gestational_age: {str(e)}")
220
- return "Error in estimation.", "--"
221
-
222
- # Create the Gradio interface
223
- with gr.Blocks(title="Fetal View Classification") as demo:
224
- with gr.Tab("Fetal View Classification"):
225
- gr.Markdown("""
226
- # Zero-shot Fetal View Classification
227
-
228
- Upload an ultrasound image to classify the fetal view. The model will predict the most likely views from 13 possible categories:
229
- abdomen, brain, femur, heart, kidney, lips_nose, profile_patient, spine, cervix, cord, diaphragm, feet, orbit
230
- """)
231
-
232
- with gr.Row():
233
- with gr.Column(scale=1):
234
- # Input controls
235
- image_input = gr.Image(
236
- label="Upload Ultrasound Image",
237
- type="numpy",
238
- height=400
239
- )
240
- submit_btn = gr.Button("Classify View", variant="primary")
241
-
242
- with gr.Column(scale=1):
243
- # Output controls and display
244
- top_k = gr.Slider(
245
- minimum=1,
246
- maximum=13,
247
- value=5,
248
- step=1,
249
- label="Number of top predictions to show",
250
- info="Adjust how many top predictions to display"
251
- )
252
- plot_output = gr.Plot(label="Classification Results")
253
-
254
- # Example images section
255
- gr.Examples(
256
- examples=[
257
- [os.path.join(EXAMPLES_DIR, "Fetal_abdomen_1.png"), 5],
258
- [os.path.join(EXAMPLES_DIR, "Fetal_abdomen_2.png"), 5],
259
- [os.path.join(EXAMPLES_DIR, "Fetal_brain_1.png"), 5],
260
- [os.path.join(EXAMPLES_DIR, "Fetal_brain_2.png"), 5],
261
- [os.path.join(EXAMPLES_DIR, "Fetal_femur_1.png"), 5],
262
- [os.path.join(EXAMPLES_DIR, "Fetal_femur_2.png"), 5],
263
- [os.path.join(EXAMPLES_DIR, "Fetal_orbit_2.png"), 5],
264
- [os.path.join(EXAMPLES_DIR, "Fetal_profile_2.png"), 5],
265
- [os.path.join(EXAMPLES_DIR, "Fetal_thorax_1.png"), 5],
266
- [os.path.join(EXAMPLES_DIR, "Fetal_thorax_2.png"), 5],
267
- ],
268
- inputs=[image_input, top_k],
269
- outputs=plot_output,
270
- fn=process_image,
271
- cache_examples=True,
272
- )
273
-
274
- # Set up event handler
275
- submit_btn.click(
276
- fn=process_image,
277
- inputs=[image_input, top_k],
278
- outputs=plot_output
279
- )
280
-
281
- top_k.change(
282
- fn=process_image,
283
- inputs=[image_input, top_k],
284
- outputs=plot_output
285
- )
286
- image_input.change(
287
- fn=process_image,
288
- inputs=[image_input, top_k],
289
- outputs=plot_output
290
- )
291
-
292
- with gr.Tab("Gestational Age Estimation"):
293
- gr.Markdown("""
294
- # Zero-shot Gestational Age Estimation
295
-
296
- Upload a fetal brain ultrasound image and enter the pixel size (mm/pixel) to estimate gestational age and head circumference percentiles.
297
- """)
298
- with gr.Row():
299
- with gr.Column(scale=1):
300
- ga_image_input = gr.Image(
301
- label="Upload Gestational Age Sample Image",
302
- type="numpy",
303
- height=400
304
- )
305
- pixel_size_input = gr.Number(
306
- label="Pixel Size (mm/pixel)",
307
- value=0.1
308
- )
309
- ga_submit_btn = gr.Button("Estimate Gestational Age", variant="primary")
310
- with gr.Column(scale=1):
311
- ga_output = gr.Textbox(label="Predicted Gestational Age (weeks + days)")
312
- hc_output = gr.Textbox(label="Head Circumference (mm) [2.5, 50, 97.5 percentiles]")
313
- ga_submit_btn.click(
314
- fn=estimate_gestational_age,
315
- inputs=[ga_image_input, pixel_size_input],
316
- outputs=[ga_output, hc_output]
317
- )
318
-
319
- if __name__ == "__main__":
320
- demo.launch()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
assets/FetalCLIP_config.json CHANGED
@@ -1,16 +1,16 @@
1
  {
2
- "embed_dim": 768,
3
- "vision_cfg": {
4
- "image_size": 224,
5
- "layers": 24,
6
- "width": 1024,
7
- "patch_size": 14
8
- },
9
- "text_cfg": {
10
- "context_length": 117,
11
- "vocab_size": 49408,
12
- "width": 768,
13
- "heads": 12,
14
- "layers": 12
15
- }
16
- }
 
1
  {
2
+ "embed_dim": 768,
3
+ "vision_cfg": {
4
+ "image_size": 224,
5
+ "layers": 24,
6
+ "width": 1024,
7
+ "patch_size": 14
8
+ },
9
+ "text_cfg": {
10
+ "context_length": 117,
11
+ "vocab_size": 49408,
12
+ "width": 768,
13
+ "heads": 12,
14
+ "layers": 12
15
+ }
16
+ }
assets/prompt_fetal_view.json CHANGED
@@ -1,93 +1,94 @@
1
  {
2
- "abdomen": [
3
- "Ultrasound image focusing on the fetal abdominal area, highlighting structural development.",
4
- "Detailed ultrasound highlighting the fetal abdomen, emphasizing anatomical structures.",
5
- "Ultrasound scan of the fetal abdomen, showcasing structural details.",
6
- "Focused ultrasound image highlighting the development of the fetal abdominal structures.",
7
- "Clear ultrasound of the fetal abdomen, emphasizing its anatomical development."
8
- ],
9
- "brain": [
10
- "Ultrasound image focusing on the fetal brain, highlighting key anatomical features.",
11
- "Detailed ultrasound scan of the developing fetal brain, showcasing structural highlights.",
12
- "Ultrasound highlighting the fetal brain structures with detailed visualization.",
13
- "Focused ultrasound showing the fetal brain and its developing anatomical structures.",
14
- "Clear ultrasound of the fetal brain, emphasizing its structural development."
15
- ],
16
- "femur": [
17
- "Ultrasound image focusing on the developing fetal femur, highlighting bone length and structure.",
18
- "Detailed ultrasound showcasing the fetal femur, providing a view of skeletal development.",
19
- "Ultrasound scan focusing on the fetal femur, emphasizing structural highlights.",
20
- "Clear ultrasound image highlighting the fetal femur and its bone development.",
21
- "Focused ultrasound showcasing the fetal femur, emphasizing skeletal details."
22
- ],
23
- "heart": [
24
- "Fetal ultrasound image focusing on the heart, highlighting detailed cardiac structures.",
25
- "Ultrasound scan showcasing the fetal heart and its developing anatomy.",
26
- "Clear ultrasound of the fetal heart, emphasizing detailed structural highlights.",
27
- "Detailed ultrasound image highlighting the fetal heart and its development.",
28
- "Focused ultrasound scan showing the fetal heart and its anatomical features."
29
- ],
30
- "kidney": [
31
- "Fetal ultrasound focusing on the kidney, showcasing structural details and development.",
32
- "Detailed ultrasound scan of the fetal kidney, emphasizing its anatomical position.",
33
- "Focused ultrasound highlighting the fetal kidney and its structural characteristics.",
34
- "Clear ultrasound image showing the fetal kidney, emphasizing its development.",
35
- "Ultrasound scan focusing on the fetal kidney, showcasing anatomical highlights."
36
- ],
37
- "lips_nose": [
38
- "Ultrasound image focusing on the lips and nose, highlighting facial development.",
39
- "Detailed ultrasound scan showcasing the fetal lips and nose structures.",
40
- "Clear ultrasound image of the fetal lips and nose, emphasizing anatomical features.",
41
- "Focused ultrasound highlighting the development of the fetal lips and nose.",
42
- "Ultrasound scan showcasing the fetal lips and nose, emphasizing structural details."
43
- ],
44
- "profile_patient": [
45
- "Ultrasound image showing the fetal profile, with clear visualization of facial features.",
46
- "Detailed ultrasound scan of the fetal profile, emphasizing facial development.",
47
- "Focused ultrasound highlighting the fetal profile and its anatomical details.",
48
- "Clear ultrasound image showcasing the fetal profile and facial structure.",
49
- "Ultrasound scan emphasizing the fetal profile, highlighting facial features."
50
- ],
51
- "spine": [
52
- "Ultrasound image focusing on the fetal spine, highlighting vertebral alignment.",
53
- "Detailed ultrasound scan showcasing the fetal spine and its anatomical structures.",
54
- "Focused ultrasound image emphasizing the fetal spine and vertebral development.",
55
- "Clear ultrasound showing the fetal spine and its structural highlights.",
56
- "Ultrasound scan highlighting the fetal spine, showcasing vertebral anatomy."
57
- ],
58
- "cervix": [
59
- "Ultrasound image highlighting the cervix, showcasing its structure and position.",
60
- "Detailed ultrasound scan of the cervix, emphasizing its length and anatomical features.",
61
- "Clear ultrasound image focusing on the cervix, providing structural insights.",
62
- "Focused ultrasound showcasing the cervical region and its anatomical details.",
63
- "Ultrasound image highlighting the cervix, emphasizing its structure and appearance."
64
- ],
65
- "cord": [
66
- "Ultrasound image focusing on the umbilical cord, highlighting its structure and position.",
67
- "Detailed ultrasound scan showcasing the umbilical cord in relation to the fetus.",
68
- "Focused ultrasound image highlighting the umbilical cord and its anatomical features.",
69
- "Clear ultrasound scan emphasizing the umbilical cord structure.",
70
- "Ultrasound image highlighting the umbilical cord and its placement near the fetus."
71
- ],
72
- "diaphragm": [
73
- "Ultrasound image focusing on the fetal diaphragm, highlighting its anatomical structure.",
74
- "Detailed ultrasound scan showcasing the fetal diaphragm and surrounding anatomy.",
75
- "Focused ultrasound image emphasizing the diaphragm in fetal development.",
76
- "Clear ultrasound highlighting the fetal diaphragm and its structure.",
77
- "Ultrasound scan showcasing the fetal diaphragm with detailed visualization."
78
- ],
79
- "feet": [
80
- "Ultrasound image focusing on the fetal feet, highlighting their development and position.",
81
- "Detailed ultrasound scan showcasing the fetal feet and structural features.",
82
- "Clear ultrasound image highlighting the fetal feet and their anatomical details.",
83
- "Focused ultrasound showing the development of the fetal feet.",
84
- "Ultrasound scan emphasizing the fetal feet and their structure."
85
- ],
86
- "orbit": [
87
- "Ultrasound image focusing on the fetal orbit, highlighting ocular structures.",
88
- "Detailed ultrasound scan showcasing the fetal orbit and eye development.",
89
- "Focused ultrasound highlighting the fetal orbital region and structural features.",
90
- "Clear ultrasound image emphasizing the fetal orbit and ocular anatomy.",
91
- "Ultrasound scan highlighting the development of the fetal orbit."
92
- ]
93
- }
 
 
1
  {
2
+ "abdomen": [
3
+ "Ultrasound image focusing on the fetal abdominal area, highlighting structural development.",
4
+ "Detailed ultrasound highlighting the fetal abdomen, emphasizing anatomical structures.",
5
+ "Ultrasound scan of the fetal abdomen, showcasing structural details.",
6
+ "Focused ultrasound image highlighting the development of the fetal abdominal structures.",
7
+ "Clear ultrasound of the fetal abdomen, emphasizing its anatomical development."
8
+ ],
9
+ "brain": [
10
+ "Ultrasound image focusing on the fetal brain, highlighting key anatomical features.",
11
+ "Detailed ultrasound scan of the developing fetal brain, showcasing structural highlights.",
12
+ "Ultrasound highlighting the fetal brain structures with detailed visualization.",
13
+ "Focused ultrasound showing the fetal brain and its developing anatomical structures.",
14
+ "Clear ultrasound of the fetal brain, emphasizing its structural development."
15
+ ],
16
+ "femur": [
17
+ "Ultrasound image focusing on the developing fetal femur, highlighting bone length and structure.",
18
+ "Detailed ultrasound showcasing the fetal femur, providing a view of skeletal development.",
19
+ "Ultrasound scan focusing on the fetal femur, emphasizing structural highlights.",
20
+ "Clear ultrasound image highlighting the fetal femur and its bone development.",
21
+ "Focused ultrasound showcasing the fetal femur, emphasizing skeletal details."
22
+ ],
23
+ "heart": [
24
+ "Fetal ultrasound image focusing on the heart, highlighting detailed cardiac structures.",
25
+ "Ultrasound scan showcasing the fetal heart and its developing anatomy.",
26
+ "Clear ultrasound of the fetal heart, emphasizing detailed structural highlights.",
27
+ "Detailed ultrasound image highlighting the fetal heart and its development.",
28
+ "Focused ultrasound scan showing the fetal heart and its anatomical features."
29
+ ],
30
+ "kidney": [
31
+ "Fetal ultrasound focusing on the kidney, showcasing structural details and development.",
32
+ "Detailed ultrasound scan of the fetal kidney, emphasizing its anatomical position.",
33
+ "Focused ultrasound highlighting the fetal kidney and its structural characteristics.",
34
+ "Clear ultrasound image showing the fetal kidney, emphasizing its development.",
35
+ "Ultrasound scan focusing on the fetal kidney, showcasing anatomical highlights."
36
+ ],
37
+ "lips_nose": [
38
+ "Ultrasound image focusing on the lips and nose, highlighting facial development.",
39
+ "Detailed ultrasound scan showcasing the fetal lips and nose structures.",
40
+ "Clear ultrasound image of the fetal lips and nose, emphasizing anatomical features.",
41
+ "Focused ultrasound highlighting the development of the fetal lips and nose.",
42
+ "Ultrasound scan showcasing the fetal lips and nose, emphasizing structural details."
43
+ ],
44
+ "profile_patient": [
45
+ "Ultrasound image showing the fetal profile, with clear visualization of facial features.",
46
+ "Detailed ultrasound scan of the fetal profile, emphasizing facial development.",
47
+ "Focused ultrasound highlighting the fetal profile and its anatomical details.",
48
+ "Clear ultrasound image showcasing the fetal profile and facial structure.",
49
+ "Ultrasound scan emphasizing the fetal profile, highlighting facial features."
50
+ ],
51
+ "spine": [
52
+ "Ultrasound image focusing on the fetal spine, highlighting vertebral alignment.",
53
+ "Detailed ultrasound scan showcasing the fetal spine and its anatomical structures.",
54
+ "Focused ultrasound image emphasizing the fetal spine and vertebral development.",
55
+ "Clear ultrasound showing the fetal spine and its structural highlights.",
56
+ "Ultrasound scan highlighting the fetal spine, showcasing vertebral anatomy."
57
+ ],
58
+ "cervix": [
59
+ "Ultrasound image highlighting the cervix, showcasing its structure and position.",
60
+ "Detailed ultrasound scan of the cervix, emphasizing its length and anatomical features.",
61
+ "Clear ultrasound image focusing on the cervix, providing structural insights.",
62
+ "Focused ultrasound showcasing the cervical region and its anatomical details.",
63
+ "Ultrasound image highlighting the cervix, emphasizing its structure and appearance."
64
+ ],
65
+ "cord": [
66
+ "Ultrasound image focusing on the umbilical cord, highlighting its structure and position.",
67
+ "Detailed ultrasound scan showcasing the umbilical cord in relation to the fetus.",
68
+ "Focused ultrasound image highlighting the umbilical cord and its anatomical features.",
69
+ "Clear ultrasound scan emphasizing the umbilical cord structure.",
70
+ "Ultrasound image highlighting the umbilical cord and its placement near the fetus."
71
+ ],
72
+ "diaphragm": [
73
+ "Ultrasound image focusing on the fetal diaphragm, highlighting its anatomical structure.",
74
+ "Detailed ultrasound scan showcasing the fetal diaphragm and surrounding anatomy.",
75
+ "Focused ultrasound image emphasizing the diaphragm in fetal development.",
76
+ "Clear ultrasound highlighting the fetal diaphragm and its structure.",
77
+ "Ultrasound scan showcasing the fetal diaphragm with detailed visualization."
78
+ ],
79
+ "feet": [
80
+ "Ultrasound image focusing on the fetal feet, highlighting their development and position.",
81
+ "Detailed ultrasound scan showcasing the fetal feet and structural features.",
82
+ "Clear ultrasound image highlighting the fetal feet and their anatomical details.",
83
+ "Focused ultrasound showing the development of the fetal feet.",
84
+ "Ultrasound scan emphasizing the fetal feet and their structure."
85
+ ],
86
+ "orbit": [
87
+ "Ultrasound image focusing on the fetal orbit, highlighting ocular structures.",
88
+ "Detailed ultrasound scan showcasing the fetal orbit and eye development.",
89
+ "Focused ultrasound highlighting the fetal orbital region and structural features.",
90
+ "Clear ultrasound image emphasizing the fetal orbit and ocular anatomy.",
91
+ "Ultrasound scan highlighting the development of the fetal orbit."
92
+ ]
93
+ }
94
+
backend/app/__init__.py ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ # FetalCLIP Backend
2
+
backend/app/main.py ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from fastapi import FastAPI
2
+ from fastapi.middleware.cors import CORSMiddleware
3
+ from fastapi.responses import JSONResponse
4
+ from contextlib import asynccontextmanager
5
+ from pathlib import Path
6
+ import sys
7
+
8
+ from .routes import classification_router, gestational_age_router
9
+ from .services.model import model_service
10
+
11
+ # Get assets directory - handle both development and PyInstaller frozen modes
12
+ def get_assets_dir() -> Path:
13
+ """Get the assets directory, works in both development and frozen (PyInstaller) modes."""
14
+ if getattr(sys, 'frozen', False):
15
+ # Running as PyInstaller bundle - assets are in _MEIPASS/assets
16
+ base_path = Path(sys._MEIPASS)
17
+ assets = base_path / "assets"
18
+ print(f"[Frozen Mode] Base path: {base_path}")
19
+ print(f"[Frozen Mode] Assets path: {assets}")
20
+ return assets
21
+ else:
22
+ # Development mode - assets are in project root
23
+ assets = Path(__file__).parent.parent.parent / "assets"
24
+ print(f"[Dev Mode] Assets path: {assets}")
25
+ return assets
26
+
27
+ ASSETS_DIR = get_assets_dir()
28
+
29
+
30
+ @asynccontextmanager
31
+ async def lifespan(app: FastAPI):
32
+ """Load model on startup, cleanup on shutdown."""
33
+ print("🚀 Starting FetalCLIP API...")
34
+ model_service.load_model(ASSETS_DIR)
35
+ yield
36
+ print("👋 Shutting down FetalCLIP API...")
37
+
38
+
39
+ app = FastAPI(
40
+ title="FetalCLIP API",
41
+ description="""
42
+ ## FetalCLIP - Foundation Model for Fetal Ultrasound Analysis
43
+
44
+ This API provides two main capabilities:
45
+
46
+ ### 1. Fetal View Classification
47
+ Classify ultrasound images into 13 anatomical view categories using zero-shot learning.
48
+
49
+ ### 2. Gestational Age Estimation
50
+ Estimate gestational age from fetal brain ultrasounds with head circumference percentiles.
51
+
52
+ ---
53
+
54
+ Built with ❤️ using PyTorch and OpenCLIP
55
+ """,
56
+ version="1.0.0",
57
+ lifespan=lifespan,
58
+ docs_url="/docs",
59
+ redoc_url="/redoc"
60
+ )
61
+
62
+ # CORS middleware for frontend
63
+ app.add_middleware(
64
+ CORSMiddleware,
65
+ allow_origins=["http://localhost:5173", "http://localhost:3000", "http://127.0.0.1:5173", "tauri://localhost"],
66
+ allow_credentials=True,
67
+ allow_methods=["*"],
68
+ allow_headers=["*"],
69
+ )
70
+
71
+ # Include routers
72
+ app.include_router(classification_router)
73
+ app.include_router(gestational_age_router)
74
+
75
+
76
+ @app.get("/", tags=["Health"])
77
+ async def root():
78
+ """API root - health check."""
79
+ return JSONResponse(content={
80
+ "name": "FetalCLIP API",
81
+ "version": "1.0.0",
82
+ "status": "healthy",
83
+ "docs": "/docs"
84
+ })
85
+
86
+
87
+ @app.get("/health", tags=["Health"])
88
+ async def health_check():
89
+ """Detailed health check."""
90
+ return JSONResponse(content={
91
+ "status": "healthy",
92
+ "model_loaded": model_service.model is not None,
93
+ "device": str(model_service.device),
94
+ "available_views": len(model_service.list_plane)
95
+ })
96
+
backend/app/routes/__init__.py ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ from .classification import router as classification_router
2
+ from .gestational_age import router as gestational_age_router
3
+
4
+ __all__ = ["classification_router", "gestational_age_router"]
5
+
backend/app/routes/classification.py ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from fastapi import APIRouter, UploadFile, File, Query, HTTPException
2
+ from fastapi.responses import JSONResponse
3
+ from ..services.model import model_service
4
+ from ..services.preprocessing import get_dicom_preview, is_dicom_file
5
+
6
+ router = APIRouter(prefix="/api/v1/classify", tags=["Classification"])
7
+
8
+
9
+ @router.post("/preview")
10
+ async def get_file_preview(
11
+ file: UploadFile = File(..., description="File to preview (DICOM or image)")
12
+ ):
13
+ """
14
+ Get a preview image from a file.
15
+ For DICOM files, extracts the raw pixel data.
16
+ For images, returns as base64.
17
+ """
18
+ try:
19
+ contents = await file.read()
20
+ filename = file.filename or "unknown"
21
+
22
+ if is_dicom_file(contents, filename):
23
+ preview_base64 = get_dicom_preview(contents)
24
+ return JSONResponse(content={
25
+ "success": True,
26
+ "preview": preview_base64,
27
+ "type": "dicom"
28
+ })
29
+ else:
30
+ # For regular images, just encode as base64
31
+ import base64
32
+ preview_base64 = base64.b64encode(contents).decode('utf-8')
33
+ # Determine mime type
34
+ content_type = file.content_type or "image/png"
35
+ return JSONResponse(content={
36
+ "success": True,
37
+ "preview": preview_base64,
38
+ "type": "image",
39
+ "mime_type": content_type
40
+ })
41
+ except Exception as e:
42
+ raise HTTPException(status_code=500, detail=str(e))
43
+
44
+
45
+ @router.post("/")
46
+ async def classify_fetal_view(
47
+ file: UploadFile = File(..., description="Ultrasound file (DICOM or image)"),
48
+ top_k: int = Query(default=5, ge=1, le=13, description="Number of top predictions")
49
+ ):
50
+ """
51
+ Classify fetal ultrasound view.
52
+
53
+ Supports both DICOM files (full preprocessing) and image files (basic preprocessing).
54
+
55
+ Returns the top-k most likely anatomical views with confidence scores,
56
+ plus information about the preprocessing applied.
57
+
58
+ Supported views:
59
+ - abdomen, brain, femur, heart, kidney, lips_nose
60
+ - profile_patient, spine, cervix, cord, diaphragm, feet, orbit
61
+ """
62
+ try:
63
+ # Read file bytes
64
+ contents = await file.read()
65
+ filename = file.filename or "unknown"
66
+
67
+ # Classify with automatic preprocessing
68
+ predictions, preprocessing_info = model_service.classify_from_file(
69
+ contents, filename, top_k=top_k
70
+ )
71
+
72
+ return JSONResponse(content={
73
+ "success": True,
74
+ "predictions": predictions,
75
+ "top_prediction": predictions[0] if predictions else None,
76
+ "preprocessing": preprocessing_info
77
+ })
78
+
79
+ except Exception as e:
80
+ raise HTTPException(status_code=500, detail=str(e))
81
+
82
+
83
+ @router.get("/views")
84
+ async def get_available_views():
85
+ """Get list of all classifiable fetal views."""
86
+ return JSONResponse(content={
87
+ "views": model_service.list_plane,
88
+ "count": len(model_service.list_plane)
89
+ })
backend/app/routes/gestational_age.py ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from fastapi import APIRouter, UploadFile, File, Query, HTTPException
2
+ from fastapi.responses import JSONResponse
3
+ from ..services.model import model_service
4
+
5
+ router = APIRouter(prefix="/api/v1/gestational-age", tags=["Gestational Age"])
6
+
7
+
8
+ @router.post("/")
9
+ async def estimate_gestational_age(
10
+ file: UploadFile = File(..., description="Fetal brain ultrasound file (DICOM or image)"),
11
+ pixel_size: float = Query(default=0.1, ge=0.01, le=1.0, description="Pixel size in mm/pixel")
12
+ ):
13
+ """
14
+ Estimate gestational age from fetal brain ultrasound.
15
+
16
+ Supports both DICOM files (full preprocessing, auto pixel spacing)
17
+ and image files (basic preprocessing, manual pixel size).
18
+
19
+ For DICOM files, pixel spacing is automatically extracted from metadata.
20
+ For image files, you must provide the pixel_size parameter.
21
+
22
+ Returns estimated gestational age and head circumference percentiles.
23
+ """
24
+ try:
25
+ # Read file bytes
26
+ contents = await file.read()
27
+ filename = file.filename or "unknown"
28
+
29
+ # Estimate GA with automatic preprocessing
30
+ ga_results, preprocessing_info = model_service.estimate_ga_from_file(
31
+ contents, filename, pixel_size=pixel_size
32
+ )
33
+
34
+ return JSONResponse(content={
35
+ "success": True,
36
+ **ga_results,
37
+ "preprocessing": preprocessing_info
38
+ })
39
+
40
+ except Exception as e:
41
+ raise HTTPException(status_code=500, detail=str(e))
backend/app/services/__init__.py ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ from .model import FetalCLIPService
2
+
3
+ __all__ = ["FetalCLIPService"]
4
+
backend/app/services/model.py ADDED
@@ -0,0 +1,267 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import torch
2
+ import open_clip
3
+ import json
4
+ import numpy as np
5
+ from PIL import Image
6
+ from pathlib import Path
7
+ from huggingface_hub import hf_hub_download
8
+ from typing import List, Dict, Tuple, Optional
9
+
10
+ from .preprocessing import preprocess_file, preprocess_image
11
+
12
+ # Constants
13
+ MODEL_NAME = "numansaeed/fetalclip-model"
14
+ INPUT_SIZE = 224
15
+ TOP_N_PROBS = 15
16
+
17
+ # GA Text prompts
18
+ GA_TEXT_PROMPTS = [
19
+ "Ultrasound image at {weeks} weeks and {day} days gestation focusing on the fetal brain, highlighting anatomical structures with a pixel spacing of {pixel_spacing} mm/pixel.",
20
+ "Fetal ultrasound image at {weeks} weeks, {day} days of gestation, focusing on the developing brain, with a pixel spacing of {pixel_spacing} mm/pixel, highlighting the structures of the fetal brain.",
21
+ "Fetal ultrasound image at {weeks} weeks and {day} days gestational age, highlighting the developing brain structures with a pixel spacing of {pixel_spacing} mm/pixel, providing important visual insights for ongoing prenatal assessments.",
22
+ "Ultrasound image at {weeks} weeks and {day} days gestation, highlighting the fetal brain structures with a pixel spacing of {pixel_spacing} mm/pixel.",
23
+ "Fetal ultrasound at {weeks} weeks and {day} days, showing a clear view of the developing brain, with an image pixel spacing of {pixel_spacing} mm/pixel."
24
+ ]
25
+
26
+ LIST_GA_IN_DAYS = [weeks * 7 + days for weeks in range(14, 39) for days in range(0, 7)]
27
+
28
+
29
+ class FetalCLIPService:
30
+ _instance = None
31
+ _initialized = False
32
+
33
+ def __new__(cls):
34
+ if cls._instance is None:
35
+ cls._instance = super().__new__(cls)
36
+ return cls._instance
37
+
38
+ def __init__(self):
39
+ if FetalCLIPService._initialized:
40
+ return
41
+
42
+ self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
43
+ self.model = None
44
+ self.preprocess = None
45
+ self.tokenizer = None
46
+ self.text_features = None
47
+ self.list_plane = []
48
+
49
+ FetalCLIPService._initialized = True
50
+
51
+ def load_model(self, assets_dir: Path):
52
+ """Load the FetalCLIP model and precompute text features."""
53
+ config_path = assets_dir / "FetalCLIP_config.json"
54
+ prompts_path = assets_dir / "prompt_fetal_view.json"
55
+
56
+ # Load config
57
+ with open(config_path, "r") as f:
58
+ config = json.load(f)
59
+ open_clip.factory._MODEL_CONFIGS["FetalCLIP"] = config
60
+
61
+ # Download weights
62
+ weights_path = hf_hub_download(
63
+ repo_id=MODEL_NAME,
64
+ filename="FetalCLIP_weights.pt"
65
+ )
66
+
67
+ # Create model
68
+ self.model, _, self.preprocess = open_clip.create_model_and_transforms(
69
+ "FetalCLIP",
70
+ pretrained=weights_path
71
+ )
72
+ self.tokenizer = open_clip.get_tokenizer("FetalCLIP")
73
+
74
+ self.model = self.model.float()
75
+ self.model.eval()
76
+ self.model.to(self.device)
77
+
78
+ # Load text prompts and compute features
79
+ with open(prompts_path, 'r') as f:
80
+ text_prompts = json.load(f)
81
+
82
+ list_text_features = []
83
+ self.list_plane = []
84
+
85
+ with torch.no_grad():
86
+ for plane, prompts in text_prompts.items():
87
+ self.list_plane.append(plane)
88
+
89
+ tokens = self.tokenizer(prompts).to(self.device)
90
+ features = self.model.encode_text(tokens)
91
+ features /= features.norm(dim=-1, keepdim=True)
92
+
93
+ features = features.mean(dim=0).unsqueeze(0)
94
+ features /= features.norm(dim=-1, keepdim=True)
95
+
96
+ list_text_features.append(features)
97
+
98
+ self.text_features = torch.stack(list_text_features)[:, 0]
99
+
100
+ print(f"✓ FetalCLIP model loaded on {self.device}")
101
+ return True
102
+
103
+ def classify_view(self, image: Image.Image, top_k: int = 5) -> List[Dict]:
104
+ """Classify fetal ultrasound view from preprocessed image."""
105
+ if self.model is None:
106
+ raise RuntimeError("Model not loaded. Call load_model() first.")
107
+
108
+ top_k = min(top_k, len(self.list_plane))
109
+
110
+ # Apply model preprocessing (resize to 224, normalize)
111
+ img_tensor = self.preprocess(image).unsqueeze(0).to(self.device)
112
+
113
+ # Inference
114
+ with torch.no_grad():
115
+ image_features = self.model.encode_image(img_tensor)
116
+ image_features /= image_features.norm(dim=-1, keepdim=True)
117
+
118
+ # Compute similarity
119
+ similarity = (99.2198 * image_features @ self.text_features.T).softmax(dim=-1)
120
+ values, indices = similarity[0].topk(top_k)
121
+
122
+ results = []
123
+ for idx, val in zip(indices, values):
124
+ results.append({
125
+ "label": self.list_plane[idx],
126
+ "confidence": round(val.item() * 100, 2)
127
+ })
128
+
129
+ return results
130
+
131
+ def classify_from_file(self, file_bytes: bytes, filename: str, top_k: int = 5) -> Tuple[List[Dict], Dict]:
132
+ """
133
+ Classify from raw file bytes with automatic preprocessing.
134
+
135
+ Returns:
136
+ Tuple of (predictions, preprocessing_info)
137
+ """
138
+ # Preprocess based on file type
139
+ processed_image, preprocessing_info = preprocess_file(file_bytes, filename)
140
+
141
+ # Classify
142
+ predictions = self.classify_view(processed_image, top_k)
143
+
144
+ return predictions, preprocessing_info
145
+
146
+ def _get_ga_text_features(self, template: str, pixel_spacing: float) -> torch.Tensor:
147
+ """Generate text features for GA estimation."""
148
+ prompts = []
149
+ for weeks in range(14, 39):
150
+ for days in range(0, 7):
151
+ prompt = template.format(
152
+ weeks=weeks,
153
+ day=days,
154
+ pixel_spacing=f"{pixel_spacing:.2f}"
155
+ )
156
+ prompts.append(prompt)
157
+
158
+ with torch.no_grad():
159
+ tokens = self.tokenizer(prompts).to(self.device)
160
+ features = self.model.encode_text(tokens)
161
+ features /= features.norm(dim=-1, keepdim=True)
162
+
163
+ return features
164
+
165
+ def _get_unnormalized_dot_products(self, image_features: torch.Tensor, list_text_features: List[torch.Tensor]) -> torch.Tensor:
166
+ """Compute dot products between image and text features."""
167
+ text_features = torch.cat(list_text_features, dim=0)
168
+ text_dot_prods = (100.0 * image_features @ text_features.T)
169
+
170
+ n_prompts = len(list_text_features)
171
+ n_days = len(list_text_features[0])
172
+
173
+ text_dot_prods = text_dot_prods.view(image_features.shape[0], n_prompts, n_days)
174
+ text_dot_prods = text_dot_prods.mean(dim=1)
175
+
176
+ return text_dot_prods
177
+
178
+ def _find_median_from_top_n(self, text_dot_prods: np.ndarray, n: int) -> int:
179
+ """Find median index from top N predictions."""
180
+ tmp = [[i, t] for i, t in enumerate(text_dot_prods)]
181
+ tmp = sorted(tmp, key=lambda x: x[1], reverse=True)[:n]
182
+ tmp = sorted(tmp, key=lambda x: x[0])
183
+ return tmp[n // 2][0]
184
+
185
+ def _get_hc_from_days(self, t: int, quartile: str = '0.5') -> float:
186
+ """Calculate head circumference from gestational age."""
187
+ t = t / 7
188
+ params = {
189
+ '0.025': [1.59317517131532e+0, 2.9459800552433e-1, -7.3860372566707e-3, 6.56951770216148e-5, 0e+0],
190
+ '0.500': [2.09924879247164e+0, 2.53373656106037e-1, -6.05647816678282e-3, 5.14256072059917e-5, 0e+0],
191
+ '0.975': [2.50074069629423e+0, 2.20067854715719e-1, -4.93623111462443e-3, 3.89066000946519e-5, 0e+0],
192
+ }
193
+ b0, b1, b2, b3, b4 = params[quartile]
194
+ return np.exp(b0 + b1*t + b2*t**2 + b3*t**3 + b4*t**4)
195
+
196
+ def estimate_gestational_age(self, image: Image.Image, pixel_size: float) -> Dict:
197
+ """Estimate gestational age from preprocessed fetal brain ultrasound."""
198
+ if self.model is None:
199
+ raise RuntimeError("Model not loaded. Call load_model() first.")
200
+
201
+ # Calculate effective pixel spacing
202
+ pixel_spacing = max(image.size) / INPUT_SIZE * pixel_size
203
+
204
+ # Apply model preprocessing
205
+ img_tensor = self.preprocess(image).unsqueeze(0).to(self.device)
206
+
207
+ # Inference
208
+ with torch.no_grad():
209
+ image_features = self.model.encode_image(img_tensor)
210
+ image_features /= image_features.norm(dim=-1, keepdim=True)
211
+
212
+ # Get text features for all prompts
213
+ text_features_list = [
214
+ self._get_ga_text_features(template, pixel_spacing)
215
+ for template in GA_TEXT_PROMPTS
216
+ ]
217
+
218
+ text_dot_prods = self._get_unnormalized_dot_products(image_features, text_features_list)
219
+
220
+ # Compute prediction
221
+ text_dot_prod = text_dot_prods.detach().cpu().numpy()[0]
222
+ med_idx = self._find_median_from_top_n(text_dot_prod, TOP_N_PROBS)
223
+ pred_day = LIST_GA_IN_DAYS[med_idx]
224
+
225
+ pred_weeks = pred_day // 7
226
+ pred_days = pred_day % 7
227
+
228
+ # Compute HC percentiles
229
+ q025 = self._get_hc_from_days(pred_day, '0.025')
230
+ q500 = self._get_hc_from_days(pred_day, '0.500')
231
+ q975 = self._get_hc_from_days(pred_day, '0.975')
232
+
233
+ return {
234
+ "gestational_age": {
235
+ "weeks": pred_weeks,
236
+ "days": pred_days,
237
+ "total_days": pred_day
238
+ },
239
+ "head_circumference": {
240
+ "p2_5": round(q025, 2),
241
+ "p50": round(q500, 2),
242
+ "p97_5": round(q975, 2)
243
+ }
244
+ }
245
+
246
+ def estimate_ga_from_file(self, file_bytes: bytes, filename: str, pixel_size: float) -> Tuple[Dict, Dict]:
247
+ """
248
+ Estimate GA from raw file bytes with automatic preprocessing.
249
+
250
+ Returns:
251
+ Tuple of (ga_results, preprocessing_info)
252
+ """
253
+ # Preprocess based on file type
254
+ processed_image, preprocessing_info = preprocess_file(file_bytes, filename)
255
+
256
+ # Use pixel spacing from DICOM if available
257
+ if preprocessing_info["type"] == "dicom":
258
+ pixel_size = preprocessing_info["metadata"].get("pixel_spacing", pixel_size)
259
+
260
+ # Estimate GA
261
+ ga_results = self.estimate_gestational_age(processed_image, pixel_size)
262
+
263
+ return ga_results, preprocessing_info
264
+
265
+
266
+ # Singleton instance
267
+ model_service = FetalCLIPService()
backend/app/services/preprocessing.py ADDED
@@ -0,0 +1,514 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Preprocessing module for FetalCLIP.
3
+
4
+ Supports two pipelines:
5
+ 1. DICOM (Full): US region extraction, fan isolation, text removal, denoising
6
+ 2. Image (Basic): Square padding, resize
7
+ """
8
+
9
+ import cv2
10
+ import copy
11
+ import numpy as np
12
+ from PIL import Image
13
+ from typing import Tuple, Dict, List, Optional
14
+ from io import BytesIO
15
+
16
+ # Try importing DICOM-specific libraries
17
+ try:
18
+ from pydicom import dcmread
19
+ from pydicom.pixel_data_handlers import convert_color_space
20
+ DICOM_AVAILABLE = True
21
+ except ImportError:
22
+ DICOM_AVAILABLE = False
23
+
24
+ try:
25
+ from skimage.restoration import denoise_nl_means, estimate_sigma
26
+ SKIMAGE_AVAILABLE = True
27
+ except ImportError:
28
+ SKIMAGE_AVAILABLE = False
29
+
30
+ try:
31
+ import albumentations as A
32
+ ALBUMENTATIONS_AVAILABLE = True
33
+ except ImportError:
34
+ ALBUMENTATIONS_AVAILABLE = False
35
+
36
+
37
+ # ============================================================================
38
+ # CONSTANTS
39
+ # ============================================================================
40
+
41
+ TARGET_SIZE = (512, 512)
42
+ INTERPOLATION = cv2.INTER_LANCZOS4
43
+
44
+ INTENSITY_THRESHOLD = 0
45
+ SMALL_VIEW_MARGIN_CROP_Y = 1
46
+
47
+ YELLOW_BOX_BACKGROUND_PIXEL = np.array([57, 57, 57])
48
+ MIN_YELLOW_BOX_RECT_AREA = 2_000
49
+
50
+ MASK_INPAINTING_DILATE_KERNEL = np.ones((9, 9), np.uint8)
51
+ DENOISE_NL_MEANS_PATCH_KW = dict(
52
+ patch_size=7,
53
+ patch_distance=6,
54
+ channel_axis=-1,
55
+ )
56
+ INPAINT_RADIUS = 5
57
+
58
+
59
+ # ============================================================================
60
+ # TEXT DETECTION UTILITIES (from utils_husain.py)
61
+ # ============================================================================
62
+
63
+ def rgb2gray(rgb: np.ndarray) -> np.ndarray:
64
+ """Convert RGB to grayscale while keeping 3 channels."""
65
+ r, g, b = rgb[:, :, 0], rgb[:, :, 1], rgb[:, :, 2]
66
+ gray = 0.299 * r + 0.5870 * g + 0.1140 * b
67
+ rgb_grey = rgb.copy()
68
+ rgb_grey[:, :, 0] = gray
69
+ rgb_grey[:, :, 1] = gray
70
+ rgb_grey[:, :, 2] = gray
71
+ return rgb_grey
72
+
73
+
74
+ def mask_filter(image: np.ndarray, grey_threshold: int) -> np.ndarray:
75
+ """Create binary mask for pixels above threshold."""
76
+ img = image.copy()
77
+ grey_img = rgb2gray(img)
78
+ convert = np.zeros((img.shape[0], img.shape[1], 3))
79
+ idxs = np.where(
80
+ (grey_img[:, :, 0] > grey_threshold)
81
+ & (grey_img[:, :, 1] > grey_threshold)
82
+ & (grey_img[:, :, 2] > grey_threshold)
83
+ )
84
+ convert[idxs] = [255, 255, 255]
85
+ return np.uint8(convert)
86
+
87
+
88
+ def maximize_contrast(img_grayscale: np.ndarray) -> np.ndarray:
89
+ """Enhance contrast using morphological operations."""
90
+ height, width = img_grayscale.shape
91
+ structuring_element = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3))
92
+
93
+ img_top_hat = cv2.morphologyEx(img_grayscale, cv2.MORPH_TOPHAT, structuring_element)
94
+ img_black_hat = cv2.morphologyEx(img_grayscale, cv2.MORPH_BLACKHAT, structuring_element)
95
+
96
+ img_plus_top_hat = cv2.add(img_grayscale, img_top_hat)
97
+ result = cv2.subtract(img_plus_top_hat, img_black_hat)
98
+
99
+ return result
100
+
101
+
102
+ def detect_white_annotation(img: np.ndarray) -> np.ndarray:
103
+ """Detect white text/annotations."""
104
+ img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
105
+ dif = maximize_contrast(img_gray)
106
+ dif_rgb = cv2.cvtColor(dif, cv2.COLOR_GRAY2BGR)
107
+ masked_img = mask_filter(dif_rgb, 254)
108
+ dilation = cv2.dilate(masked_img, np.ones((3, 3), np.uint8), iterations=1)
109
+ mask = cv2.cvtColor(dilation, cv2.COLOR_BGR2GRAY)
110
+ return mask
111
+
112
+
113
+ def detect_cyan(img: np.ndarray) -> np.ndarray:
114
+ """Detect cyan colored text."""
115
+ image_hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
116
+ lowers = np.uint8([85, 150, 20])
117
+ uppers = np.uint8([95, 255, 255])
118
+ mask = np.array(cv2.inRange(image_hsv, lowers, uppers))
119
+ return mask
120
+
121
+
122
+ def detect_purple_text(img: np.ndarray) -> np.ndarray:
123
+ """Detect purple colored text."""
124
+ image_hsv = cv2.cvtColor(img, cv2.COLOR_RGB2HSV)
125
+ lowers = np.uint8([110, 100, 50])
126
+ uppers = np.uint8([130, 255, 255])
127
+ mask = np.array(cv2.inRange(image_hsv, lowers, uppers))
128
+ return mask
129
+
130
+
131
+ def detect_orange_text(img: np.ndarray) -> np.ndarray:
132
+ """Detect orange colored text."""
133
+ image_hsv = cv2.cvtColor(img, cv2.COLOR_RGB2HSV)
134
+ lowers = np.uint8([12, 150, 100])
135
+ uppers = np.uint8([27, 255, 255])
136
+ mask = np.array(cv2.inRange(image_hsv, lowers, uppers))
137
+ return mask
138
+
139
+
140
+ def detect_green_text(img: np.ndarray) -> np.ndarray:
141
+ """Detect green colored text."""
142
+ image_hsv = cv2.cvtColor(img, cv2.COLOR_RGB2HSV)
143
+ lowers = np.uint8([50, 100, 50])
144
+ uppers = np.uint8([70, 255, 255])
145
+ mask = np.array(cv2.inRange(image_hsv, lowers, uppers))
146
+ return mask
147
+
148
+
149
+ def detect_annotation(img: np.ndarray) -> np.ndarray:
150
+ """Detect all text annotations (white, cyan, orange, purple, green)."""
151
+ d1 = (detect_white_annotation(img) >= 127).astype(np.float32)
152
+ d2 = (detect_cyan(img) >= 127).astype(np.float32)
153
+ d3 = (detect_orange_text(img) >= 127).astype(np.float32)
154
+ d4 = (detect_purple_text(img) >= 127).astype(np.float32)
155
+ d5 = (detect_green_text(img) >= 127).astype(np.float32)
156
+
157
+ inpaint_mask = d1 + d2 + d3 + d4 + d5
158
+ inpaint_mask = (inpaint_mask > 0).astype(np.uint8) * 255
159
+
160
+ inpaint_mask = maximize_contrast(inpaint_mask)
161
+ blur = cv2.GaussianBlur(inpaint_mask, (5, 5), 0)
162
+ ret3, th3 = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
163
+ th3 = cv2.bitwise_or(th3, inpaint_mask)
164
+ return th3
165
+
166
+
167
+ # ============================================================================
168
+ # DICOM UTILITIES (from utils_adam.py)
169
+ # ============================================================================
170
+
171
+ def remove_text_box(im: np.ndarray, box_background_pixel: np.ndarray, min_rect_area: int = 2000) -> np.ndarray:
172
+ """Remove yellow/gray text boxes from image."""
173
+ binary = np.all(im == box_background_pixel, axis=-1).astype(np.uint8)
174
+ binary = binary * 255
175
+ contours, hierarchy = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
176
+
177
+ if len(contours) == 0:
178
+ return im
179
+
180
+ contour = max(contours, key=cv2.contourArea)
181
+ x, y, w, h = cv2.boundingRect(contour)
182
+
183
+ if w * h >= min_rect_area:
184
+ im[y:y+h, x:x+w] = 0
185
+
186
+ return im
187
+
188
+
189
+ def pad_to_square(im: np.ndarray) -> np.ndarray:
190
+ """Pad image to square using black padding."""
191
+ if ALBUMENTATIONS_AVAILABLE:
192
+ target_size = max(im.shape[:2])
193
+ return A.PadIfNeeded(min_height=target_size, min_width=target_size,
194
+ border_mode=0, value=(0, 0, 0))(image=im)["image"]
195
+ else:
196
+ # Fallback without albumentations
197
+ height, width = im.shape[:2]
198
+ max_side = max(height, width)
199
+
200
+ if len(im.shape) == 3:
201
+ result = np.zeros((max_side, max_side, im.shape[2]), dtype=im.dtype)
202
+ else:
203
+ result = np.zeros((max_side, max_side), dtype=im.dtype)
204
+
205
+ y_offset = (max_side - height) // 2
206
+ x_offset = (max_side - width) // 2
207
+ result[y_offset:y_offset+height, x_offset:x_offset+width] = im
208
+
209
+ return result
210
+
211
+
212
+ def get_fan_region(im: np.ndarray, threshold: int = 1) -> np.ndarray:
213
+ """Extract the ultrasound fan/cone region."""
214
+ imgray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
215
+
216
+ ret, thresh = cv2.threshold(imgray, threshold, 255, 0)
217
+ contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
218
+
219
+ if len(contours) == 0:
220
+ return im
221
+
222
+ contour = max(contours, key=cv2.contourArea)
223
+
224
+ # Create mask
225
+ filled_image = np.zeros_like(im)
226
+ cv2.drawContours(filled_image, [contour], -1, (255, 255, 255), thickness=cv2.FILLED)
227
+
228
+ # Crop to bounding box
229
+ x, y, w, h = cv2.boundingRect(contour)
230
+ cropped_image = im[y:y+h, x:x+w]
231
+ filled_image = filled_image[y:y+h, x:x+w]
232
+
233
+ # Apply mask
234
+ masked_image = cv2.bitwise_and(cropped_image, filled_image)
235
+
236
+ return masked_image
237
+
238
+
239
+ def get_us_region_from_dcm(us, sv_mc_y: int = 1) -> np.ndarray:
240
+ """Extract ultrasound region from DICOM using metadata."""
241
+ # Initialize default coordinates
242
+ x0_f, x1_f, y0_f, y1_f = None, None, None, None
243
+ x0, x1, y0, y1 = 0, us.pixel_array.shape[1], 0, us.pixel_array.shape[0]
244
+
245
+ # Check for ultrasound regions metadata
246
+ if hasattr(us, 'SequenceOfUltrasoundRegions') and len(us.SequenceOfUltrasoundRegions) > 0:
247
+ regions = us.SequenceOfUltrasoundRegions
248
+
249
+ if len(regions) == 2 and int(regions[0].RegionDataType) == 1 and int(regions[1].RegionDataType) == 1:
250
+ # Image with small view (picture-in-picture)
251
+ x0_f = np.min([regions[0].RegionLocationMinX0, regions[1].RegionLocationMinX0])
252
+ x1_f = np.max([regions[0].RegionLocationMinX0, regions[1].RegionLocationMinX0])
253
+ y0_f = np.max([regions[0].RegionLocationMinY0, regions[1].RegionLocationMinY0])
254
+ y1_f = np.max([regions[0].RegionLocationMaxY1, regions[1].RegionLocationMaxY1])
255
+
256
+ x0 = min(regions[0].RegionLocationMinX0, regions[1].RegionLocationMinX0)
257
+ x1 = max(regions[0].RegionLocationMaxX1, regions[1].RegionLocationMaxX1)
258
+ y0 = min(regions[0].RegionLocationMinY0, regions[1].RegionLocationMinY0)
259
+ y1 = max(regions[0].RegionLocationMaxY1, regions[1].RegionLocationMaxY1)
260
+
261
+ elif len(regions) >= 1 and int(regions[0].RegionDataType) == 1:
262
+ x0 = regions[0].RegionLocationMinX0
263
+ x1 = regions[0].RegionLocationMaxX1
264
+ y0 = regions[0].RegionLocationMinY0
265
+ y1 = regions[0].RegionLocationMaxY1
266
+
267
+ ds = copy.deepcopy(us.pixel_array)
268
+
269
+ # Handle color space conversion
270
+ if hasattr(us, 'PhotometricInterpretation'):
271
+ if 'ybr_full' in us.PhotometricInterpretation.lower():
272
+ ds = convert_color_space(ds, "YBR_FULL", "RGB", per_frame=True)
273
+
274
+ # Remove small view if present
275
+ if x0_f is not None:
276
+ ds[y0_f - sv_mc_y:y1_f, x0_f:x1_f, :] = 0
277
+
278
+ # Crop to ultrasound region
279
+ ds = ds[y0:y1, x0:x1, :]
280
+
281
+ return ds
282
+
283
+
284
+ # ============================================================================
285
+ # MAIN PREPROCESSING FUNCTIONS
286
+ # ============================================================================
287
+
288
+ def preprocess_dicom(file_bytes: bytes) -> Tuple[Image.Image, Dict]:
289
+ """
290
+ Full DICOM preprocessing pipeline.
291
+
292
+ Steps:
293
+ 1. Parse DICOM file
294
+ 2. Extract ultrasound region from metadata
295
+ 3. Remove yellow text boxes
296
+ 4. Extract fan/cone region
297
+ 5. Detect text annotations
298
+ 6. Inpaint to remove text
299
+ 7. Denoise using non-local means
300
+ 8. Pad to square
301
+ 9. Resize to target size
302
+
303
+ Returns:
304
+ Tuple of (PIL Image, metadata dict)
305
+ """
306
+ if not DICOM_AVAILABLE:
307
+ raise RuntimeError("pydicom not installed. Install with: pip install pydicom")
308
+
309
+ # Parse DICOM
310
+ us = dcmread(BytesIO(file_bytes))
311
+
312
+ # Extract ultrasound region
313
+ ds = get_us_region_from_dcm(us, sv_mc_y=SMALL_VIEW_MARGIN_CROP_Y)
314
+
315
+ # Remove text box
316
+ img = remove_text_box(ds.copy(), box_background_pixel=YELLOW_BOX_BACKGROUND_PIXEL,
317
+ min_rect_area=MIN_YELLOW_BOX_RECT_AREA)
318
+
319
+ # Extract fan region
320
+ fan = get_fan_region(img, threshold=INTENSITY_THRESHOLD)
321
+
322
+ # Detect annotations
323
+ image_grey = fan.copy()
324
+ mask_inpaint = detect_annotation(fan)
325
+ mask_inpaint = cv2.dilate(mask_inpaint, MASK_INPAINTING_DILATE_KERNEL)
326
+
327
+ # Inpaint to remove text
328
+ dst = cv2.inpaint(image_grey, mask_inpaint, INPAINT_RADIUS, cv2.INPAINT_TELEA)
329
+ dst = dst / np.max(dst) if np.max(dst) > 0 else dst
330
+
331
+ # Denoise
332
+ if SKIMAGE_AVAILABLE:
333
+ sigma = estimate_sigma(dst, channel_axis=-1, average_sigmas=True)
334
+ median = denoise_nl_means(dst, h=0.8 * sigma, fast_mode=True, **DENOISE_NL_MEANS_PATCH_KW)
335
+ median = np.clip(median * 255, 0, 255).astype(np.uint8)
336
+ else:
337
+ median = np.clip(dst * 255, 0, 255).astype(np.uint8)
338
+
339
+ # Pad to square
340
+ img = pad_to_square(median)
341
+
342
+ # Resize
343
+ img = cv2.resize(img, TARGET_SIZE, interpolation=INTERPOLATION)
344
+
345
+ # Extract metadata
346
+ try:
347
+ rows = getattr(us, 'Rows', fan.shape[0])
348
+ columns = getattr(us, 'Columns', fan.shape[1])
349
+
350
+ if hasattr(us, 'PixelSpacing') and us.PixelSpacing is not None:
351
+ orig_pixel_spacing = [float(sp) for sp in us.PixelSpacing]
352
+ else:
353
+ orig_pixel_spacing = [1.0, 1.0]
354
+ except Exception:
355
+ rows = fan.shape[0]
356
+ columns = fan.shape[1]
357
+ orig_pixel_spacing = [1.0, 1.0]
358
+
359
+ metadata = {
360
+ 'original_size': (rows, columns),
361
+ 'original_pixel_spacing': orig_pixel_spacing,
362
+ 'fan_size': (fan.shape[0], fan.shape[1]),
363
+ 'pixel_spacing': orig_pixel_spacing[0] if orig_pixel_spacing else 1.0,
364
+ 'processed_size': TARGET_SIZE,
365
+ }
366
+
367
+ # Convert to PIL
368
+ if len(img.shape) == 2:
369
+ img_rgb = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)
370
+ else:
371
+ img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
372
+
373
+ img_pil = Image.fromarray(img_rgb)
374
+
375
+ steps_applied = [
376
+ "dicom_parsing",
377
+ "us_region_extraction",
378
+ "text_box_removal",
379
+ "fan_extraction",
380
+ "annotation_detection",
381
+ "inpainting",
382
+ "denoising" if SKIMAGE_AVAILABLE else "normalization",
383
+ "square_padding",
384
+ "resize_512"
385
+ ]
386
+
387
+ return img_pil, {
388
+ "type": "dicom",
389
+ "pipeline": "full",
390
+ "steps_applied": steps_applied,
391
+ "metadata": metadata
392
+ }
393
+
394
+
395
+ def preprocess_image(image: Image.Image) -> Tuple[Image.Image, Dict]:
396
+ """
397
+ Basic image preprocessing pipeline.
398
+
399
+ Steps:
400
+ 1. Convert to RGB if needed
401
+ 2. Pad to square
402
+ 3. (Model will resize to 224)
403
+
404
+ Returns:
405
+ Tuple of (PIL Image, preprocessing info dict)
406
+ """
407
+ # Convert to RGB
408
+ if image.mode not in ('RGB', 'L'):
409
+ image = image.convert('RGB')
410
+
411
+ width, height = image.size
412
+ max_side = max(width, height)
413
+
414
+ # Create square image with black padding
415
+ padding_color = (0, 0, 0) if image.mode == "RGB" else 0
416
+ new_image = Image.new(image.mode, (max_side, max_side), padding_color)
417
+
418
+ # Center the original
419
+ padding_left = (max_side - width) // 2
420
+ padding_top = (max_side - height) // 2
421
+ new_image.paste(image, (padding_left, padding_top))
422
+
423
+ # Ensure RGB
424
+ if new_image.mode == 'L':
425
+ new_image = new_image.convert('RGB')
426
+
427
+ steps_applied = [
428
+ "rgb_conversion",
429
+ "square_padding",
430
+ ]
431
+
432
+ return new_image, {
433
+ "type": "image",
434
+ "pipeline": "basic",
435
+ "steps_applied": steps_applied,
436
+ "metadata": {
437
+ "original_size": (height, width),
438
+ "processed_size": (max_side, max_side),
439
+ }
440
+ }
441
+
442
+
443
+ def is_dicom_file(file_bytes: bytes, filename: str) -> bool:
444
+ """Check if file is a DICOM file."""
445
+ # Check by extension
446
+ lower_name = filename.lower()
447
+ if lower_name.endswith('.dcm') or lower_name.endswith('.dicom'):
448
+ return True
449
+
450
+ # Check DICOM magic number (DICM at offset 128)
451
+ if len(file_bytes) > 132:
452
+ if file_bytes[128:132] == b'DICM':
453
+ return True
454
+
455
+ return False
456
+
457
+
458
+ def image_to_base64(image: Image.Image) -> str:
459
+ """Convert PIL Image to base64 string."""
460
+ buffered = BytesIO()
461
+ image.save(buffered, format="PNG")
462
+ import base64
463
+ return base64.b64encode(buffered.getvalue()).decode('utf-8')
464
+
465
+
466
+ def get_dicom_preview(file_bytes: bytes) -> str:
467
+ """Extract raw image from DICOM for preview (no preprocessing)."""
468
+ if not DICOM_AVAILABLE:
469
+ raise RuntimeError("pydicom not installed")
470
+
471
+ us = dcmread(BytesIO(file_bytes))
472
+ ds = us.pixel_array
473
+
474
+ # Handle color space
475
+ if hasattr(us, 'PhotometricInterpretation'):
476
+ if 'ybr_full' in us.PhotometricInterpretation.lower():
477
+ ds = convert_color_space(ds, "YBR_FULL", "RGB", per_frame=True)
478
+
479
+ # Handle video (take first frame)
480
+ if len(ds.shape) == 4:
481
+ ds = ds[0]
482
+
483
+ # Normalize to 0-255
484
+ if ds.max() > 255:
485
+ ds = ((ds - ds.min()) / (ds.max() - ds.min()) * 255).astype(np.uint8)
486
+
487
+ # Convert to RGB if grayscale
488
+ if len(ds.shape) == 2:
489
+ ds = cv2.cvtColor(ds, cv2.COLOR_GRAY2RGB)
490
+ elif ds.shape[2] == 3:
491
+ ds = cv2.cvtColor(ds, cv2.COLOR_BGR2RGB)
492
+
493
+ img_pil = Image.fromarray(ds)
494
+ return image_to_base64(img_pil)
495
+
496
+
497
+ def preprocess_file(file_bytes: bytes, filename: str) -> Tuple[Image.Image, Dict]:
498
+ """
499
+ Automatically detect file type and apply appropriate preprocessing.
500
+
501
+ Returns:
502
+ Tuple of (PIL Image, preprocessing info dict with base64 image)
503
+ """
504
+ if is_dicom_file(file_bytes, filename):
505
+ processed_image, info = preprocess_dicom(file_bytes)
506
+ # Add base64 encoded image for frontend display
507
+ info["processed_image_base64"] = image_to_base64(processed_image)
508
+ return processed_image, info
509
+ else:
510
+ # Regular image
511
+ image = Image.open(BytesIO(file_bytes))
512
+ processed_image, info = preprocess_image(image)
513
+ info["processed_image_base64"] = image_to_base64(processed_image)
514
+ return processed_image, info
backend/requirements.txt ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Core
2
+ fastapi>=0.104.0
3
+ uvicorn[standard]>=0.24.0
4
+ python-multipart>=0.0.6
5
+
6
+ # ML & Deep Learning
7
+ torch>=2.0.0
8
+ open-clip-torch>=2.24.0
9
+ huggingface-hub>=0.19.0
10
+
11
+ # Image Processing
12
+ Pillow>=10.0.0
13
+ numpy>=1.24.0
14
+ opencv-python>=4.8.0
15
+ scikit-image>=0.21.0
16
+ albumentations>=1.3.0
17
+
18
+ # DICOM
19
+ pydicom>=2.4.0
20
+
21
+ # Utilities
22
+ pydantic>=2.0.0
examples/Fetal_abdomen_1.png DELETED

Git LFS Details

  • SHA256: c0cd8944a9286a3ab0100bd7b0758702d0d708f2a6aad06d87d10ea4b1625d06
  • Pointer size: 131 Bytes
  • Size of remote file: 196 kB
examples/Fetal_abdomen_2.png DELETED

Git LFS Details

  • SHA256: 0eae449d206ffd6aa57a041bf02696f49e39721e5588ddfbbabf76bece42cb43
  • Pointer size: 131 Bytes
  • Size of remote file: 276 kB
examples/Fetal_brain_1.png DELETED

Git LFS Details

  • SHA256: d08ef4f4a460b1110388dd2d0ef6e1e107da366ff6d8f5084bafe13118d42248
  • Pointer size: 131 Bytes
  • Size of remote file: 183 kB
examples/Fetal_brain_2.png DELETED

Git LFS Details

  • SHA256: 7da269eef8a7cf19878d0b159efe4d4e14133c9f4198ad625501b1bd576f12b8
  • Pointer size: 131 Bytes
  • Size of remote file: 261 kB
examples/Fetal_femur_1.png DELETED

Git LFS Details

  • SHA256: f83639a118fc983649168adaf6eaafb422b1d8964642d542b63b40aa7047f2e0
  • Pointer size: 131 Bytes
  • Size of remote file: 218 kB
examples/Fetal_femur_2.png DELETED

Git LFS Details

  • SHA256: 5947bb4b2e773f6f9b49be76fc22bdb8b5854bbd87d5337d497dd1290cef8563
  • Pointer size: 131 Bytes
  • Size of remote file: 210 kB
examples/Fetal_orbit_1 copy.jpg DELETED

Git LFS Details

  • SHA256: d93e60a226e7514504b76fbf860c13b9f717028aafd92a7f853a6ca12030a1ab
  • Pointer size: 130 Bytes
  • Size of remote file: 41.6 kB
examples/Fetal_orbit_1.jpg DELETED

Git LFS Details

  • SHA256: d93e60a226e7514504b76fbf860c13b9f717028aafd92a7f853a6ca12030a1ab
  • Pointer size: 130 Bytes
  • Size of remote file: 41.6 kB
examples/Fetal_orbit_2.png DELETED

Git LFS Details

  • SHA256: aa3d1f2d4869b33cd4d525ffe30d3fede422f6e23b4ed08499f3abb4440d1ea6
  • Pointer size: 131 Bytes
  • Size of remote file: 137 kB
examples/Fetal_profile_1 copy.jpg DELETED

Git LFS Details

  • SHA256: d5bffb090a9ca2161828b971af01bae0b3af883c84bd50ed755596d5f0c9517f
  • Pointer size: 130 Bytes
  • Size of remote file: 14.8 kB
examples/Fetal_profile_1.jpg DELETED

Git LFS Details

  • SHA256: d5bffb090a9ca2161828b971af01bae0b3af883c84bd50ed755596d5f0c9517f
  • Pointer size: 130 Bytes
  • Size of remote file: 14.8 kB
examples/Fetal_profile_2.png DELETED

Git LFS Details

  • SHA256: b831b37fe4266008859fbfa3ead99b3b6f98c9cc2e2c5a92d6e5336c9291f165
  • Pointer size: 131 Bytes
  • Size of remote file: 932 kB
examples/Fetal_thorax_1.png DELETED

Git LFS Details

  • SHA256: 65af18ce27e45c1d8a9f925f285bc2f9a54abae614d6577b868818217a5a3b25
  • Pointer size: 131 Bytes
  • Size of remote file: 224 kB
examples/Fetal_thorax_2.png DELETED

Git LFS Details

  • SHA256: 7cb38f233cdc087296718ba896eee429a350743610f8993b32b0eef59fe3394c
  • Pointer size: 131 Bytes
  • Size of remote file: 173 kB
examples/Maternal_cervix_1.png DELETED

Git LFS Details

  • SHA256: b5cf5a2ae1eb700afe3f0cf3d7c125a097ad0d1ea2b8e9d4c1ffc4434d6f6d33
  • Pointer size: 131 Bytes
  • Size of remote file: 208 kB
examples/Maternal_cervix_2.png DELETED

Git LFS Details

  • SHA256: 7908342d714fdfbba61c0b2d48fbb7af19921de353568ac114efda82bdfbf536
  • Pointer size: 131 Bytes
  • Size of remote file: 187 kB
examples/ga_333_HC.png DELETED

Git LFS Details

  • SHA256: 49bedf156880c50b798b111286df899eb3cc438d7c6c4e7c3383e0f5bd0fa35d
  • Pointer size: 131 Bytes
  • Size of remote file: 141 kB
examples/ga_351_HC.png DELETED

Git LFS Details

  • SHA256: fc293a681cacbdc92439e8483b71dc2d93bbcb0910354b3f59836fff6c45c2dd
  • Pointer size: 131 Bytes
  • Size of remote file: 131 kB
examples/ga_385_HC.png DELETED

Git LFS Details

  • SHA256: fe3dc9f347ab7762acc333c8ac013bdec9753df8c2d69d81c69870fd2c677af2
  • Pointer size: 131 Bytes
  • Size of remote file: 112 kB
examples/ga_584_HC.png DELETED

Git LFS Details

  • SHA256: 1079948afee5ca29bfee63a92e37d0f20e2733717181174d5c92e7aea8beb9e4
  • Pointer size: 131 Bytes
  • Size of remote file: 131 kB
examples/ga_615_HC.png DELETED

Git LFS Details

  • SHA256: 2017c0846a1358346b981a7c62ad9a2ff55b63227afdfa1fbef4ae6c2418046e
  • Pointer size: 131 Bytes
  • Size of remote file: 133 kB
examples/ga_notes.txt DELETED
@@ -1,6 +0,0 @@
1
- filename pixel size(mm) head circumference (mm)
2
- 431 351_HC.png 0.144119 172.10
3
- 759 615_HC.png 0.111789 178.15
4
- 411 333_HC.png 0.106052 166.40
5
- 474 385_HC.png 0.133191 171.12
6
- 722 584_HC.png 0.202031 185.60
 
 
 
 
 
 
 
frontend/index.html ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="UTF-8" />
5
+ <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
6
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7
+ <title>FetalCLIP - Fetal Ultrasound Analysis</title>
8
+ <link rel="preconnect" href="https://fonts.googleapis.com">
9
+ <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
10
+ <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
11
+ </head>
12
+ <body>
13
+ <div id="root"></div>
14
+ <script type="module" src="/src/main.tsx"></script>
15
+ </body>
16
+ </html>
17
+
frontend/package-lock.json ADDED
The diff for this file is too large to render. See raw diff
 
frontend/package.json ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "fetalclip-frontend",
3
+ "private": true,
4
+ "version": "1.0.0",
5
+ "type": "module",
6
+ "scripts": {
7
+ "dev": "vite",
8
+ "build": "tsc && vite build",
9
+ "preview": "vite preview",
10
+ "tauri": "tauri"
11
+ },
12
+ "dependencies": {
13
+ "react": "^18.2.0",
14
+ "react-dom": "^18.2.0",
15
+ "react-dropzone": "^14.2.3",
16
+ "lucide-react": "^0.309.0",
17
+ "clsx": "^2.1.0",
18
+ "tailwind-merge": "^2.2.0"
19
+ },
20
+ "devDependencies": {
21
+ "@tauri-apps/cli": "^1.5.0",
22
+ "@types/react": "^18.2.43",
23
+ "@types/react-dom": "^18.2.17",
24
+ "@vitejs/plugin-react": "^4.2.1",
25
+ "autoprefixer": "^10.4.16",
26
+ "postcss": "^8.4.33",
27
+ "tailwindcss": "^3.4.1",
28
+ "typescript": "^5.3.3",
29
+ "vite": "^5.0.10"
30
+ }
31
+ }
32
+
frontend/postcss.config.js ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ export default {
2
+ plugins: {
3
+ tailwindcss: {},
4
+ autoprefixer: {},
5
+ },
6
+ }
7
+
frontend/public/favicon.svg ADDED
frontend/src/App.tsx ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { useState, useEffect } from 'react';
2
+ import { Scan, Calendar } from 'lucide-react';
3
+ import { Header } from './components/Header';
4
+ import { Tabs } from './components/Tabs';
5
+ import { ClassificationPage } from './pages/ClassificationPage';
6
+ import { GestationalAgePage } from './pages/GestationalAgePage';
7
+ import { checkHealth } from './lib/api';
8
+
9
+ const tabs = [
10
+ { id: 'classification', label: 'View Classification', icon: <Scan className="w-4 h-4" /> },
11
+ { id: 'gestational-age', label: 'Gestational Age', icon: <Calendar className="w-4 h-4" /> },
12
+ ];
13
+
14
+ function App() {
15
+ const [activeTab, setActiveTab] = useState('classification');
16
+ const [isConnected, setIsConnected] = useState(false);
17
+
18
+ useEffect(() => {
19
+ const checkConnection = async () => {
20
+ const healthy = await checkHealth();
21
+ setIsConnected(healthy);
22
+ };
23
+
24
+ checkConnection();
25
+ const interval = setInterval(checkConnection, 10000);
26
+ return () => clearInterval(interval);
27
+ }, []);
28
+
29
+ return (
30
+ <div className="h-screen flex flex-col bg-dark-bg overflow-hidden">
31
+ {/* Header - fixed height */}
32
+ <Header isConnected={isConnected} />
33
+
34
+ {/* Tabs - fixed height */}
35
+ <Tabs tabs={tabs} activeTab={activeTab} onChange={setActiveTab} />
36
+
37
+ {/* Main content - fills remaining space */}
38
+ <main className="flex-1 flex min-h-0 overflow-hidden">
39
+ {activeTab === 'classification' && <ClassificationPage />}
40
+ {activeTab === 'gestational-age' && <GestationalAgePage />}
41
+ </main>
42
+
43
+ {/* Footer - fixed height, always visible */}
44
+ <footer className="flex-shrink-0 px-6 py-3 border-t border-dark-border bg-white">
45
+ <div className="flex items-center justify-between text-xs">
46
+ <span className="text-text-secondary">FetalCLIP • Foundation Model for Fetal Ultrasound Analysis</span>
47
+ <div className="flex items-center gap-4">
48
+ <a
49
+ href="https://huggingface.co/numansaeed/fetalclip-model"
50
+ target="_blank"
51
+ rel="noopener noreferrer"
52
+ className="text-accent-blue hover:text-accent-blue-hover transition-colors font-medium"
53
+ >
54
+ 🤗 Model Hub
55
+ </a>
56
+ <a
57
+ href="#"
58
+ className="text-accent-blue hover:text-accent-blue-hover transition-colors font-medium"
59
+ >
60
+ 📄 Paper
61
+ </a>
62
+ </div>
63
+ </div>
64
+ </footer>
65
+ </div>
66
+ );
67
+ }
68
+
69
+ export default App;
frontend/src/components/Button.tsx ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { cn } from '../lib/utils';
2
+ import { Loader2 } from 'lucide-react';
3
+
4
+ interface ButtonProps extends React.ButtonHTMLAttributes<HTMLButtonElement> {
5
+ variant?: 'primary' | 'secondary';
6
+ isLoading?: boolean;
7
+ icon?: React.ReactNode;
8
+ }
9
+
10
+ export function Button({
11
+ children,
12
+ variant = 'primary',
13
+ isLoading = false,
14
+ icon,
15
+ className,
16
+ disabled,
17
+ ...props
18
+ }: ButtonProps) {
19
+ return (
20
+ <button
21
+ className={cn(
22
+ 'flex items-center justify-center gap-2 px-6 py-3 rounded-lg font-semibold text-sm transition-all duration-200',
23
+ 'disabled:opacity-50 disabled:cursor-not-allowed shadow-card',
24
+ variant === 'primary' && [
25
+ 'bg-nvidia-green text-white',
26
+ 'hover:bg-nvidia-green-hover hover:-translate-y-0.5 hover:shadow-card-hover',
27
+ 'active:translate-y-0',
28
+ ],
29
+ variant === 'secondary' && [
30
+ 'bg-white text-nvidia-green border-2 border-nvidia-green',
31
+ 'hover:bg-nvidia-green/5',
32
+ ],
33
+ className
34
+ )}
35
+ disabled={disabled || isLoading}
36
+ {...props}
37
+ >
38
+ {isLoading ? (
39
+ <Loader2 className="w-4 h-4 animate-spin" />
40
+ ) : (
41
+ icon
42
+ )}
43
+ {children}
44
+ </button>
45
+ );
46
+ }
frontend/src/components/FileUpload.tsx ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { useCallback } from 'react';
2
+ import { useDropzone } from 'react-dropzone';
3
+ import { Upload, FileImage, FileText, Loader2 } from 'lucide-react';
4
+ import { cn } from '../lib/utils';
5
+ import { isDicomFile } from '../lib/api';
6
+
7
+ interface FileUploadProps {
8
+ onUpload: (file: File) => void;
9
+ preview: string | null;
10
+ currentFile: File | null;
11
+ isLoading?: boolean;
12
+ }
13
+
14
+ export function FileUpload({ onUpload, preview, currentFile, isLoading = false }: FileUploadProps) {
15
+ const onDrop = useCallback(
16
+ (acceptedFiles: File[]) => {
17
+ if (acceptedFiles.length > 0) {
18
+ onUpload(acceptedFiles[0]);
19
+ }
20
+ },
21
+ [onUpload]
22
+ );
23
+
24
+ const { getRootProps, getInputProps, isDragActive } = useDropzone({
25
+ onDrop,
26
+ accept: {
27
+ 'image/*': ['.png', '.jpg', '.jpeg', '.gif', '.bmp', '.webp'],
28
+ 'application/dicom': ['.dcm', '.dicom'],
29
+ 'application/octet-stream': ['.dcm', '.dicom'],
30
+ },
31
+ maxFiles: 1,
32
+ });
33
+
34
+ const isDicom = currentFile ? isDicomFile(currentFile.name) : false;
35
+
36
+ return (
37
+ <div className="h-full w-full flex flex-col">
38
+ <div
39
+ {...getRootProps()}
40
+ className={cn(
41
+ 'flex-1 relative border-2 border-dashed rounded-xl transition-all duration-200 cursor-pointer overflow-hidden',
42
+ 'hover:border-nvidia-green hover:bg-nvidia-green/5',
43
+ isDragActive
44
+ ? 'border-nvidia-green bg-nvidia-green/10'
45
+ : 'border-dark-border bg-dark-input'
46
+ )}
47
+ >
48
+ <input {...getInputProps()} />
49
+
50
+ {isLoading ? (
51
+ // Loading state for DICOM preview
52
+ <div className="absolute inset-0 flex flex-col items-center justify-center gap-3 bg-slate-900">
53
+ <Loader2 className="w-8 h-8 text-nvidia-green animate-spin" />
54
+ <p className="text-white text-sm">Loading DICOM preview...</p>
55
+ </div>
56
+ ) : preview ? (
57
+ // Show preview image - dark background for medical images
58
+ <div className="absolute inset-0 flex items-center justify-center bg-slate-900 rounded-lg">
59
+ <img
60
+ src={preview}
61
+ alt="Preview"
62
+ className="max-w-full max-h-full w-full h-full object-contain"
63
+ />
64
+ <div className="absolute inset-0 bg-black/60 opacity-0 hover:opacity-100 transition-opacity flex items-center justify-center rounded-lg">
65
+ <p className="text-white text-sm font-medium">Click or drop to replace</p>
66
+ </div>
67
+ {/* File type badge */}
68
+ <div className={cn(
69
+ 'absolute top-3 right-3 px-2.5 py-1 rounded-full text-xs font-semibold shadow-lg',
70
+ isDicom ? 'bg-nvidia-green text-white' : 'bg-accent-blue text-white'
71
+ )}>
72
+ {isDicom ? 'DICOM' : 'IMAGE'}
73
+ </div>
74
+ </div>
75
+ ) : (
76
+ // Empty state / upload prompt
77
+ <div className="absolute inset-0 flex flex-col items-center justify-center gap-4 p-4">
78
+ <div className="p-4 rounded-full bg-white border border-dark-border shadow-card">
79
+ <Upload className="w-8 h-8 text-text-muted" />
80
+ </div>
81
+ <div className="text-center">
82
+ <p className="text-text-primary font-medium text-sm mb-1">
83
+ {isDragActive ? 'Drop file here' : 'Drop or click to upload'}
84
+ </p>
85
+ <p className="text-text-muted text-xs">
86
+ Supports DICOM and image files
87
+ </p>
88
+ </div>
89
+ {/* Format hints */}
90
+ <div className="flex gap-3 mt-2">
91
+ <div className="flex items-center gap-1.5 px-3 py-1.5 rounded-full bg-nvidia-green/10 border border-nvidia-green/30">
92
+ <FileText className="w-3.5 h-3.5 text-nvidia-green" />
93
+ <span className="text-xs font-medium text-nvidia-green">DICOM</span>
94
+ <span className="text-[10px] px-1.5 py-0.5 rounded-full bg-nvidia-green text-white">Best</span>
95
+ </div>
96
+ <div className="flex items-center gap-1.5 px-3 py-1.5 rounded-full bg-accent-blue/10 border border-accent-blue/30">
97
+ <FileImage className="w-3.5 h-3.5 text-accent-blue" />
98
+ <span className="text-xs font-medium text-accent-blue">PNG/JPEG</span>
99
+ </div>
100
+ </div>
101
+ </div>
102
+ )}
103
+ </div>
104
+ </div>
105
+ );
106
+ }
frontend/src/components/GAResultsCard.tsx ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import type { GestationalAgeResponse } from '../lib/api';
2
+
3
+ interface GAResultsCardProps {
4
+ results: GestationalAgeResponse | null;
5
+ isLoading: boolean;
6
+ }
7
+
8
+ export function GAResultsCard({ results, isLoading }: GAResultsCardProps) {
9
+ if (isLoading) {
10
+ return (
11
+ <div className="space-y-3 animate-pulse">
12
+ <div className="bg-white border border-dark-border rounded-xl p-4 shadow-card">
13
+ <div className="h-3 w-24 bg-dark-input rounded mb-2" />
14
+ <div className="h-8 w-40 bg-dark-input rounded" />
15
+ </div>
16
+ <div className="bg-white border border-dark-border rounded-xl p-4 shadow-card">
17
+ <div className="h-3 w-40 bg-dark-input rounded mb-3" />
18
+ <div className="grid grid-cols-3 gap-2">
19
+ {[...Array(3)].map((_, i) => (
20
+ <div key={i} className="h-16 bg-dark-input rounded-lg" />
21
+ ))}
22
+ </div>
23
+ </div>
24
+ </div>
25
+ );
26
+ }
27
+
28
+ if (!results) {
29
+ return (
30
+ <div className="bg-white border border-dark-border rounded-xl p-8 text-center shadow-card">
31
+ <p className="text-text-muted text-sm">
32
+ Upload a fetal brain ultrasound and click "Estimate Age"
33
+ </p>
34
+ </div>
35
+ );
36
+ }
37
+
38
+ const { gestational_age, head_circumference } = results;
39
+
40
+ return (
41
+ <div className="space-y-3 animate-fade-in">
42
+ {/* Gestational Age */}
43
+ <div className="bg-gradient-to-r from-nvidia-green/10 to-nvidia-green/5 border border-nvidia-green/20 rounded-xl p-4 shadow-card">
44
+ <p className="text-[10px] uppercase tracking-wider text-text-muted mb-1">
45
+ Gestational Age
46
+ </p>
47
+ <div className="text-2xl font-bold text-nvidia-green">
48
+ {gestational_age.weeks} weeks, {gestational_age.days} days
49
+ </div>
50
+ <p className="text-text-muted text-xs mt-1">
51
+ Total: {gestational_age.total_days} days
52
+ </p>
53
+ </div>
54
+
55
+ {/* Head Circumference Percentiles */}
56
+ <div className="bg-white border border-dark-border rounded-xl p-4 shadow-card">
57
+ <p className="text-[10px] uppercase tracking-wider text-text-muted mb-3">
58
+ Head Circumference Percentiles
59
+ </p>
60
+ <div className="grid grid-cols-3 gap-3">
61
+ <div className="bg-dark-input rounded-xl p-3 text-center">
62
+ <p className="text-[10px] text-text-muted mb-1">2.5th</p>
63
+ <p className="text-base font-semibold text-text-primary">
64
+ {head_circumference.p2_5} mm
65
+ </p>
66
+ </div>
67
+ <div className="bg-nvidia-green/10 rounded-xl p-3 text-center border-2 border-nvidia-green">
68
+ <p className="text-[10px] text-nvidia-green mb-1 font-medium">50th</p>
69
+ <p className="text-base font-bold text-nvidia-green">
70
+ {head_circumference.p50} mm
71
+ </p>
72
+ </div>
73
+ <div className="bg-dark-input rounded-xl p-3 text-center">
74
+ <p className="text-[10px] text-text-muted mb-1">97.5th</p>
75
+ <p className="text-base font-semibold text-text-primary">
76
+ {head_circumference.p97_5} mm
77
+ </p>
78
+ </div>
79
+ </div>
80
+ </div>
81
+ </div>
82
+ );
83
+ }
frontend/src/components/Header.tsx ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { Zap } from 'lucide-react';
2
+
3
+ interface HeaderProps {
4
+ isConnected: boolean;
5
+ }
6
+
7
+ export function Header({ isConnected }: HeaderProps) {
8
+ return (
9
+ <header className="bg-white border-b border-dark-border px-8 py-4 shadow-sm">
10
+ <div className="flex items-center justify-between">
11
+ <div className="flex items-center gap-3">
12
+ <div className="p-2 rounded-lg bg-nvidia-green/10 border border-nvidia-green/20">
13
+ <Zap className="w-5 h-5 text-nvidia-green" />
14
+ </div>
15
+ <div>
16
+ <h1 className="text-xl font-semibold text-text-primary tracking-tight">
17
+ Fetal<span className="text-nvidia-green">CLIP</span>
18
+ </h1>
19
+ <p className="text-text-muted text-xs">
20
+ Foundation model for zero-shot fetal ultrasound analysis
21
+ </p>
22
+ </div>
23
+ </div>
24
+ <div className="flex items-center gap-5">
25
+ <div className="flex items-center gap-2 px-3 py-1.5 rounded-full bg-dark-input border border-dark-border">
26
+ <div
27
+ className={`w-2 h-2 rounded-full transition-colors ${
28
+ isConnected
29
+ ? 'bg-nvidia-green shadow-[0_0_8px_rgba(118,185,0,0.5)]'
30
+ : 'bg-red-500 animate-pulse'
31
+ }`}
32
+ />
33
+ <span className="text-xs text-text-secondary">
34
+ {isConnected ? 'Model Ready' : 'Connecting...'}
35
+ </span>
36
+ </div>
37
+ <a
38
+ href="https://huggingface.co/numansaeed/fetalclip-model"
39
+ target="_blank"
40
+ rel="noopener noreferrer"
41
+ className="flex items-center gap-1.5 text-sm text-accent-blue hover:text-accent-blue-hover transition-colors"
42
+ >
43
+ <span>🤗</span>
44
+ <span>Model Hub</span>
45
+ </a>
46
+ </div>
47
+ </div>
48
+ </header>
49
+ );
50
+ }
frontend/src/components/ImageUpload.tsx ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { useCallback } from 'react';
2
+ import { useDropzone } from 'react-dropzone';
3
+ import { Upload, Image as ImageIcon } from 'lucide-react';
4
+ import { cn } from '../lib/utils';
5
+
6
+ interface ImageUploadProps {
7
+ onUpload: (file: File) => void;
8
+ preview: string | null;
9
+ label: string;
10
+ description: string;
11
+ }
12
+
13
+ export function ImageUpload({ onUpload, preview, label, description }: ImageUploadProps) {
14
+ const onDrop = useCallback(
15
+ (acceptedFiles: File[]) => {
16
+ if (acceptedFiles.length > 0) {
17
+ onUpload(acceptedFiles[0]);
18
+ }
19
+ },
20
+ [onUpload]
21
+ );
22
+
23
+ const { getRootProps, getInputProps, isDragActive } = useDropzone({
24
+ onDrop,
25
+ accept: {
26
+ 'image/*': ['.png', '.jpg', '.jpeg', '.gif', '.bmp', '.webp'],
27
+ },
28
+ maxFiles: 1,
29
+ });
30
+
31
+ return (
32
+ <div className="h-full flex flex-col gap-1">
33
+ <label className="flex-shrink-0 text-xs font-medium text-white flex items-center gap-2">
34
+ <ImageIcon className="w-3 h-3 text-text-secondary" />
35
+ {label}
36
+ </label>
37
+ <div
38
+ {...getRootProps()}
39
+ className={cn(
40
+ 'flex-1 relative border-2 border-dashed rounded-lg transition-all duration-200 cursor-pointer',
41
+ 'hover:border-nvidia-green hover:bg-nvidia-green/5',
42
+ isDragActive
43
+ ? 'border-nvidia-green bg-nvidia-green/10'
44
+ : 'border-dark-border bg-dark-input',
45
+ preview ? 'p-1' : 'p-4'
46
+ )}
47
+ >
48
+ <input {...getInputProps()} />
49
+
50
+ {preview ? (
51
+ <div className="relative w-full h-full overflow-hidden rounded">
52
+ <img
53
+ src={preview}
54
+ alt="Uploaded preview"
55
+ className="w-full h-full object-contain bg-black"
56
+ />
57
+ <div className="absolute inset-0 bg-black/50 opacity-0 hover:opacity-100 transition-opacity flex items-center justify-center">
58
+ <p className="text-white text-sm">Click or drop to replace</p>
59
+ </div>
60
+ </div>
61
+ ) : (
62
+ <div className="h-full flex flex-col items-center justify-center gap-3">
63
+ <div className="p-3 rounded-full bg-dark-card">
64
+ <Upload className="w-6 h-6 text-text-muted" />
65
+ </div>
66
+ <div className="text-center">
67
+ <p className="text-white font-medium text-sm">
68
+ {isDragActive ? 'Drop image here' : 'Drop or click to upload'}
69
+ </p>
70
+ <p className="text-text-muted text-xs mt-1">{description}</p>
71
+ </div>
72
+ </div>
73
+ )}
74
+ </div>
75
+ </div>
76
+ );
77
+ }
frontend/src/components/NumberInput.tsx ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ interface NumberInputProps {
2
+ label: string;
3
+ value: number;
4
+ onChange: (value: number) => void;
5
+ min?: number;
6
+ max?: number;
7
+ step?: number;
8
+ info?: string;
9
+ unit?: string;
10
+ compact?: boolean;
11
+ }
12
+
13
+ export function NumberInput({
14
+ label,
15
+ value,
16
+ onChange,
17
+ min,
18
+ max,
19
+ step = 0.01,
20
+ info,
21
+ unit,
22
+ compact = false,
23
+ }: NumberInputProps) {
24
+ return (
25
+ <div className="space-y-1.5">
26
+ <label className={`font-semibold text-text-primary ${compact ? 'text-xs' : 'text-sm'}`}>{label}</label>
27
+ {info && <p className={`text-text-muted ${compact ? 'text-[10px]' : 'text-xs'}`}>{info}</p>}
28
+ <div className="relative">
29
+ <input
30
+ type="number"
31
+ value={value}
32
+ onChange={(e) => onChange(Number(e.target.value))}
33
+ min={min}
34
+ max={max}
35
+ step={step}
36
+ className={`w-full bg-white border border-dark-border rounded-lg text-text-primary focus:border-nvidia-green focus:outline-none focus:ring-2 focus:ring-nvidia-green/20 transition-all ${
37
+ compact ? 'px-3 py-2 text-sm' : 'px-4 py-3 text-base'
38
+ }`}
39
+ />
40
+ {unit && (
41
+ <span className={`absolute right-3 top-1/2 -translate-y-1/2 text-text-muted ${
42
+ compact ? 'text-xs' : 'text-sm'
43
+ }`}>
44
+ {unit}
45
+ </span>
46
+ )}
47
+ </div>
48
+ </div>
49
+ );
50
+ }
frontend/src/components/Panel.tsx ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { cn } from '../lib/utils';
2
+
3
+ interface PanelProps {
4
+ title: string;
5
+ action?: React.ReactNode;
6
+ children: React.ReactNode;
7
+ className?: string;
8
+ }
9
+
10
+ export function Panel({ title, action, children, className }: PanelProps) {
11
+ return (
12
+ <div className={cn('flex flex-col', className)}>
13
+ <div className="flex-shrink-0 flex items-center justify-between px-4 py-2.5 border-b border-dark-border bg-white">
14
+ <h2 className="text-sm font-semibold text-text-primary">{title}</h2>
15
+ {action}
16
+ </div>
17
+ <div className="flex-1 p-3 overflow-hidden">{children}</div>
18
+ </div>
19
+ );
20
+ }
frontend/src/components/PreprocessingBadge.tsx ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import { CheckCircle, AlertCircle, Info } from 'lucide-react';
2
+ import { useState } from 'react';
3
+ import type { PreprocessingInfo } from '../lib/api';
4
+
5
+ interface PreprocessingBadgeProps {
6
+ info: PreprocessingInfo | null;
7
+ fileType?: 'dicom' | 'image' | null;
8
+ compact?: boolean;
9
+ }
10
+
11
+ const STEP_LABELS: Record<string, string> = {
12
+ dicom_parsing: 'DICOM Parsing',
13
+ us_region_extraction: 'US Region Extraction',
14
+ text_box_removal: 'Text Box Removal',
15
+ fan_extraction: 'Fan Extraction',
16
+ annotation_detection: 'Annotation Detection',
17
+ inpainting: 'Inpainting',
18
+ denoising: 'Denoising',
19
+ normalization: 'Normalization',
20
+ square_padding: 'Square Padding',
21
+ resize_512: 'Resize to 512×512',
22
+ rgb_conversion: 'RGB Conversion',
23
+ };
24
+
25
+ export function PreprocessingBadge({ info, fileType, compact = false }: PreprocessingBadgeProps) {
26
+ const [expanded, setExpanded] = useState(false);
27
+
28
+ // Show pending state when file is selected but not yet processed
29
+ if (!info && fileType) {
30
+ const isDicom = fileType === 'dicom';
31
+ return (
32
+ <div className={`rounded-xl border shadow-card ${isDicom ? 'border-nvidia-green/30 bg-nvidia-green/5' : 'border-accent-blue/30 bg-accent-blue/5'} ${compact ? 'px-3 py-2' : 'px-4 py-3'}`}>
33
+ <div className="flex items-center gap-2">
34
+ <div className={`w-2 h-2 rounded-full ${isDicom ? 'bg-nvidia-green' : 'bg-accent-blue'}`} />
35
+ <span className={`font-semibold ${isDicom ? 'text-nvidia-green' : 'text-accent-blue'} ${compact ? 'text-xs' : 'text-sm'}`}>
36
+ {isDicom ? 'DICOM' : 'PNG/JPEG'}
37
+ </span>
38
+ <span className={`text-text-secondary ${compact ? 'text-xs' : 'text-sm'}`}>
39
+ • {isDicom ? 'Full Pipeline' : 'Basic Pipeline'}
40
+ </span>
41
+ </div>
42
+ {!compact && (
43
+ <p className="text-xs text-text-muted mt-1">
44
+ {isDicom
45
+ ? 'Will apply: Fan extraction, text removal, denoising'
46
+ : 'Will apply: Square padding only. For best accuracy, use DICOM files.'
47
+ }
48
+ </p>
49
+ )}
50
+ </div>
51
+ );
52
+ }
53
+
54
+ if (!info) return null;
55
+
56
+ const isDicom = info.type === 'dicom';
57
+ const isFull = info.pipeline === 'full';
58
+
59
+ return (
60
+ <div className={`rounded-xl border shadow-card ${isFull ? 'border-nvidia-green/30 bg-nvidia-green/5' : 'border-amber-500/30 bg-amber-500/5'} ${compact ? 'px-3 py-2' : 'px-4 py-3'}`}>
61
+ {/* Header */}
62
+ <button
63
+ onClick={() => setExpanded(!expanded)}
64
+ className="w-full flex items-center justify-between"
65
+ >
66
+ <div className="flex items-center gap-2">
67
+ <div className={`w-2 h-2 rounded-full ${isFull ? 'bg-nvidia-green' : 'bg-amber-500'}`} />
68
+ <span className={`font-semibold ${isFull ? 'text-nvidia-green' : 'text-amber-600'} ${compact ? 'text-xs' : 'text-sm'}`}>
69
+ {isDicom ? 'DICOM' : 'PNG/JPEG'}
70
+ </span>
71
+ <span className={`text-text-secondary ${compact ? 'text-xs' : 'text-sm'}`}>
72
+ • {isFull ? 'Full Pipeline' : 'Basic Pipeline'}
73
+ </span>
74
+ </div>
75
+ <Info className={`text-text-muted ${compact ? 'w-3 h-3' : 'w-4 h-4'}`} />
76
+ </button>
77
+
78
+ {/* Expanded Details */}
79
+ {expanded && (
80
+ <div className="mt-3 pt-3 border-t border-dark-border">
81
+ <p className="text-xs text-text-muted mb-2 font-medium">Steps Applied:</p>
82
+ <div className="space-y-1.5">
83
+ {info.steps_applied.map((step) => (
84
+ <div key={step} className="flex items-center gap-2">
85
+ <CheckCircle className="w-3.5 h-3.5 text-nvidia-green" />
86
+ <span className="text-xs text-text-primary">
87
+ {STEP_LABELS[step] || step}
88
+ </span>
89
+ </div>
90
+ ))}
91
+ </div>
92
+
93
+ {/* Missing steps for basic pipeline */}
94
+ {!isFull && (
95
+ <div className="mt-3 pt-3 border-t border-dark-border">
96
+ <p className="text-xs text-text-muted mb-2 font-medium">Not Applied:</p>
97
+ <div className="space-y-1.5">
98
+ {['fan_extraction', 'annotation_detection', 'inpainting', 'denoising'].map((step) => (
99
+ <div key={step} className="flex items-center gap-2">
100
+ <AlertCircle className="w-3.5 h-3.5 text-amber-500" />
101
+ <span className="text-xs text-text-muted">
102
+ {STEP_LABELS[step] || step}
103
+ </span>
104
+ </div>
105
+ ))}
106
+ </div>
107
+ <p className="text-xs text-amber-600 mt-2 font-medium">
108
+ ⚠️ For best accuracy, use DICOM files from the ultrasound machine.
109
+ </p>
110
+ </div>
111
+ )}
112
+
113
+ {/* Metadata */}
114
+ {info.metadata.pixel_spacing && (
115
+ <div className="mt-3 pt-3 border-t border-dark-border">
116
+ <p className="text-xs text-text-muted">
117
+ Pixel Spacing: <span className="text-text-primary font-medium">{info.metadata.pixel_spacing.toFixed(3)} mm/px</span>
118
+ </p>
119
+ </div>
120
+ )}
121
+ </div>
122
+ )}
123
+ </div>
124
+ );
125
+ }