VibecoderMcSwaggins commited on
Commit
bc1d8e8
·
unverified ·
1 Parent(s): 4b42170

fix(ui): use js_on_load for NiiVue viewer instead of script tags (#20)

Browse files

* docs: add bug spec for NiiVue 3D viewer black screen (P2)

Document the NiiVue WebGL viewer rendering black on HF Spaces.
Core inference and 2D visualization work correctly.

Hypotheses:
1. Base64 data URL size (~65MB) exceeding limits
2. CDN accessibility issues
3. WebGL2 context limitations
4. Gradio HTML script execution

Proposed solutions ranked by complexity.
See: docs/specs/10-bug-niivue-viewer-black-screen.md

* docs: comprehensive root cause analysis for NiiVue black screen

ROOT CAUSE CONFIRMED: gr.HTML does not execute <script> tags.
Gradio intentionally ignores embedded JavaScript for security.

Key findings:
- js_on_load parameter is the correct Gradio 6 approach
- Base64 payload (~65MB) is a secondary concern
- Gradio 6 moved js/head params from Blocks() to launch()

Added 5 ranked solutions with code examples.
References: Gradio docs, HF forums, GitHub issues.

See: docs/specs/10-bug-niivue-viewer-black-screen.md

* docs: add external validation results to NiiVue bug spec

Agent claimed js_on_load doesn't exist - REFUTED by Gradio docs.
Parameter exists with default value, added in PR #12098.

Validated findings:
- js_on_load IS valid in Gradio 6.x gr.HTML
- Base64 payload concern is legitimate (use file URLs)
- CSP headers may need testing on HF Spaces

Added Step 0 to test async/await in js_on_load before implementing fix.

* fix(ui): use js_on_load for NiiVue viewer instead of script tags

ROOT CAUSE: Gradio's gr.HTML strips <script> tags for XSS security.
Our NiiVue initialization code was embedded in <script type="module">
tags, which Gradio ignored → black screen.

FIX: Use Gradio's js_on_load parameter instead:
- viewer.py: Removed script tags, added NIIVUE_JS_ON_LOAD constant
- HTML now uses data-* attributes for volume URLs
- JavaScript reads URLs from element.dataset
- components.py: Added js_on_load=NIIVUE_JS_ON_LOAD to gr.HTML
- Added test script for js_on_load validation

All 130 tests pass.

Fixes: #10 NiiVue 3D viewer black screen on HF Spaces
See: docs/specs/10-bug-niivue-viewer-black-screen.md

* fix: address CodeRabbit review feedback

- viewer.py: Use textContent instead of innerHTML to prevent XSS
- test_js_on_load.py: Add null coalescing for props.value
- docs: Add language identifier to code block

docs/specs/10-bug-niivue-viewer-black-screen.md ADDED
@@ -0,0 +1,408 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Bug #10: NiiVue 3D Viewer Renders Black Screen on HF Spaces
2
+
3
+ ## Status: FIXED (pending HF Spaces verification)
4
+
5
+ **Date:** 2025-12-09
6
+ **Branch:** `fix/niivue-js-on-load`
7
+ **Discovered:** After fixing Bug #9 (DeepISLES subprocess bridge)
8
+
9
+ ### Fix Applied (2025-12-09)
10
+
11
+ Implemented `js_on_load` approach (Solution 1 from this spec):
12
+
13
+ 1. **`viewer.py`**: Removed `<script>` tags, added `NIIVUE_JS_ON_LOAD` constant
14
+ 2. **`components.py`**: Added `js_on_load=NIIVUE_JS_ON_LOAD` to gr.HTML
15
+ 3. **All 130 tests pass locally**
16
+
17
+ The HTML now uses `data-*` attributes to pass volume URLs, and JavaScript
18
+ executes via `js_on_load` instead of inline `<script>` tags.
19
+
20
+ ---
21
+
22
+ ## TL;DR - ROOT CAUSE
23
+
24
+ **Gradio's `gr.HTML` component does NOT execute `<script>` tags (including `type="module"`).**
25
+
26
+ Our code embeds NiiVue initialization JavaScript inside `<script type="module">` tags within the HTML value. Gradio intentionally ignores these for security reasons. The canvas renders but NiiVue never initializes → black screen.
27
+
28
+ ---
29
+
30
+ ## Symptom
31
+
32
+ After successful DeepISLES inference on HF Spaces, the NiiVue 3D viewer component (top-right panel) renders as a completely black rectangle. No brain scan or mask overlay is visible.
33
+
34
+ **What IS working:**
35
+ - DeepISLES inference completes successfully (~32 seconds)
36
+ - Slice Comparison (matplotlib 2D view) renders correctly
37
+ - Metrics JSON displays correctly
38
+ - Download button provides the prediction mask
39
+ - Ground truth overlay in Slice Comparison works
40
+
41
+ **What is NOT working:**
42
+ - NiiVue WebGL 3D viewer shows black screen
43
+ - No error message displayed in the viewer area
44
+ - No visible WebGL error fallback message
45
+
46
+ **What SHOULD appear:**
47
+ - Multi-planar view (axial/coronal/sagittal slices)
48
+ - Optional 3D volume rendering
49
+ - Interactive crosshairs for navigation
50
+ - DWI volume as grayscale background
51
+ - Prediction mask as semi-transparent red overlay
52
+
53
+ ---
54
+
55
+ ## Root Cause Analysis
56
+
57
+ ### Evidence Chain
58
+
59
+ 1. **[HuggingFace Forum](https://discuss.huggingface.co/t/gradio-html-component-with-javascript-code-dont-work/37316)**:
60
+ > "You can't load scripts via `gr.HTML`"
61
+
62
+ 2. **[Gradio Official Docs](https://www.gradio.app/docs/gradio/html)**:
63
+ > "Only static HTML is rendered (e.g., no JavaScript). To render JavaScript, use the `js` or `head` parameters"
64
+
65
+ 3. **[Gradio 6 Migration Guide](https://www.gradio.app/main/guides/gradio-6-migration-guide)**:
66
+ > The `js` and `head` parameters moved from `gr.Blocks()` to `launch()` in Gradio 6
67
+
68
+ 4. **[GitHub Issue #10250](https://github.com/gradio-app/gradio/issues/10250)**:
69
+ > Known issue with JavaScript in `head` param not executing reliably
70
+
71
+ ### Our Code (BROKEN)
72
+
73
+ ```python
74
+ # viewer.py:324-385 - Returns HTML with embedded script tags
75
+ def create_niivue_html(volume_url, mask_url, height=400) -> str:
76
+ return f"""
77
+ <div id="{container_id}" style="...">
78
+ <canvas id="{canvas_id}" style="..."></canvas>
79
+ </div>
80
+ <script type="module">
81
+ // THIS ENTIRE BLOCK IS IGNORED BY GRADIO!
82
+ (async function() {{
83
+ const niivueModule = await import('{NIIVUE_CDN_URL}');
84
+ const Niivue = niivueModule.Niivue;
85
+ const nv = new Niivue({{...}});
86
+ await nv.attachToCanvas(document.getElementById('{canvas_id}'));
87
+ await nv.loadVolumes([{{ url: {volume_url_js} }}]);
88
+ // ... more initialization
89
+ }})();
90
+ </script>
91
+ """
92
+
93
+ # components.py:42 - Basic HTML component without js_on_load
94
+ niivue_viewer = gr.HTML(label="Interactive 3D Viewer") # No js_on_load!
95
+ ```
96
+
97
+ ### Why It Fails
98
+
99
+ 1. `gr.HTML` receives our HTML string as `value`
100
+ 2. Gradio renders the `<div>` and `<canvas>` elements (static HTML)
101
+ 3. Gradio **strips or ignores** the `<script>` tags for security
102
+ 4. NiiVue JavaScript never executes
103
+ 5. Canvas remains empty → black screen
104
+ 6. Our try/catch error handling never runs (script doesn't execute at all)
105
+
106
+ ---
107
+
108
+ ## Secondary Issues
109
+
110
+ ### Issue 1: Base64 Payload Size (~65MB)
111
+
112
+ Even if JavaScript executed, we're passing massive base64-encoded NIfTI data:
113
+
114
+ | File | Raw Size | Base64 Size |
115
+ |------|----------|-------------|
116
+ | DWI | 30.1 MB | ~40 MB |
117
+ | ADC | 17.7 MB | ~24 MB |
118
+ | **Total** | ~48 MB | **~65 MB** |
119
+
120
+ This could cause:
121
+ - Browser memory issues
122
+ - Gradio payload limits
123
+ - Slow/failed rendering
124
+
125
+ ### Issue 2: Gradio 6 Breaking Changes
126
+
127
+ Our code uses Gradio 5.x patterns. In Gradio 6.x:
128
+ - `js`, `head`, `head_paths` moved from `gr.Blocks()` to `launch()`
129
+ - `padding` default changed from `True` to `False`
130
+ - `js_on_load` is now the proper way for component-level JavaScript
131
+
132
+ ### Issue 3: No Error Visibility
133
+
134
+ Our JavaScript has try/catch that should display errors in the container, but since the script never executes, the error handling never runs. The canvas just stays black with no feedback to the user.
135
+
136
+ ---
137
+
138
+ ## Code Locations
139
+
140
+ | File | Lines | Description |
141
+ |------|-------|-------------|
142
+ | `src/stroke_deepisles_demo/ui/viewer.py` | 277-385 | `create_niivue_html()` - generates broken HTML |
143
+ | `src/stroke_deepisles_demo/ui/viewer.py` | 34-51 | `nifti_to_data_url()` - base64 encoding |
144
+ | `src/stroke_deepisles_demo/ui/app.py` | 101-117 | NiiVue HTML generation in `run_segmentation()` |
145
+ | `src/stroke_deepisles_demo/ui/components.py` | 41-42 | `gr.HTML` component creation (missing js_on_load) |
146
+
147
+ ---
148
+
149
+ ## External Validation (2025-12-09)
150
+
151
+ An external agent review claimed `js_on_load` does not exist. **This claim was REFUTED.**
152
+
153
+ ### Verification Results
154
+
155
+ | Claim | Status | Evidence |
156
+ |-------|--------|----------|
157
+ | "gr.HTML does NOT have js_on_load parameter" | ❌ **REFUTED** | [Gradio Docs](https://www.gradio.app/docs/gradio/html) show `js_on_load` with default value |
158
+ | "js_on_load was added in PR #12098" | ✅ Confirmed | Part of "gr.HTML custom components" feature |
159
+ | "Base64 payload (~65MB) is a risk" | ✅ Confirmed | Valid concern, should use file URLs |
160
+ | "CSP headers may block CDN" | ⚠️ Possible | HF Spaces typically allows unpkg.com, but worth testing |
161
+
162
+ ### Validated `js_on_load` Signature
163
+
164
+ ```python
165
+ js_on_load: str | None = "element.addEventListener('click', function() { trigger('click') });"
166
+ ```
167
+
168
+ **Available in js_on_load context:**
169
+ - `element` - The HTML DOM element
170
+ - `trigger(event_name)` - Fire Gradio events
171
+ - `props` - Access component props including `props.value`
172
+
173
+ **Untested (needs verification):**
174
+ - Async/await patterns
175
+ - Dynamic `import()` for CDN modules
176
+ - Error propagation to Gradio
177
+
178
+ ---
179
+
180
+ ## Proposed Solutions (Ranked)
181
+
182
+ ### Solution 1: Use `js_on_load` Parameter (Recommended)
183
+
184
+ Gradio 6's `gr.HTML` supports `js_on_load` for component-level JavaScript (added in PR #12098):
185
+
186
+ ```python
187
+ def create_niivue_component(volume_url, mask_url, height=400):
188
+ container_id = f"nv-{uuid.uuid4().hex[:8]}"
189
+
190
+ html_content = f'<div id="{container_id}" style="height:{height}px;background:#000;"><canvas></canvas></div>'
191
+
192
+ js_code = f"""
193
+ (async () => {{
194
+ try {{
195
+ const {{ Niivue }} = await import('https://unpkg.com/@niivue/niivue@0.65.0/dist/index.js');
196
+ const nv = new Niivue({{ logging: false, backColor: [0,0,0,1] }});
197
+ await nv.attachToCanvas(element.querySelector('canvas'));
198
+ await nv.loadVolumes([{{ url: {json.dumps(volume_url)} }}]);
199
+ nv.setSliceType(nv.sliceTypeMultiplanar);
200
+ }} catch (e) {{
201
+ element.innerHTML = '<div style="color:#fff;padding:20px;">Error: ' + e.message + '</div>';
202
+ }}
203
+ }})();
204
+ """
205
+
206
+ return gr.HTML(
207
+ value=html_content,
208
+ js_on_load=js_code,
209
+ label="Interactive 3D Viewer"
210
+ )
211
+ ```
212
+
213
+ **Pros:** Native Gradio 6 approach, component-scoped
214
+ **Cons:** May have issues with dynamic import in js_on_load context
215
+
216
+ ### Solution 2: Use `head` Parameter in `launch()`
217
+
218
+ Load NiiVue globally via the `head` parameter:
219
+
220
+ ```python
221
+ # app.py
222
+ NIIVUE_HEAD = '''
223
+ <script type="module">
224
+ import { Niivue } from 'https://unpkg.com/@niivue/niivue@0.65.0/dist/index.js';
225
+ window.Niivue = Niivue;
226
+ </script>
227
+ '''
228
+
229
+ demo.launch(
230
+ head=NIIVUE_HEAD,
231
+ server_name="0.0.0.0",
232
+ server_port=7860
233
+ )
234
+ ```
235
+
236
+ **Pros:** Loads library once, available globally
237
+ **Cons:** GitHub Issue #10250 reports unreliable execution
238
+
239
+ ### Solution 3: Server-Side File Serving
240
+
241
+ Instead of base64 data URLs, serve NIfTI files via Gradio's file system:
242
+
243
+ ```python
244
+ # Use Gradio's file URL instead of data URLs
245
+ from gradio import FileData
246
+ file_data = FileData(path=str(dwi_path))
247
+ # Pass file_data.url to NiiVue instead of base64
248
+ ```
249
+
250
+ **Pros:** Avoids 65MB payload, better memory efficiency
251
+ **Cons:** Requires refactoring data flow, CORS considerations
252
+
253
+ ### Solution 4: Custom Gradio Component
254
+
255
+ Build a proper `gradio_niivue` package:
256
+
257
+ ```bash
258
+ gradio cc create NiiVue --template HTML
259
+ # Implement Svelte frontend with NiiVue
260
+ # Publish to PyPI
261
+ ```
262
+
263
+ **Pros:** Most robust, reusable, proper architecture
264
+ **Cons:** Significant development effort
265
+
266
+ ### Solution 5: Enhanced 2D Fallback (Simplest)
267
+
268
+ Remove NiiVue entirely, enhance matplotlib visualization:
269
+
270
+ ```python
271
+ def create_results_display():
272
+ with gr.Group():
273
+ # Remove: niivue_viewer = gr.HTML(...)
274
+
275
+ # Enhanced 2D visualization
276
+ slice_plot = gr.Plot(label="Multi-View Comparison")
277
+ slice_slider = gr.Slider(label="Slice", minimum=0, maximum=100)
278
+
279
+ # Add orthogonal views
280
+ with gr.Row():
281
+ axial_plot = gr.Plot(label="Axial")
282
+ coronal_plot = gr.Plot(label="Coronal")
283
+ sagittal_plot = gr.Plot(label="Sagittal")
284
+ ```
285
+
286
+ **Pros:** Eliminates WebGL complexity, works reliably
287
+ **Cons:** Loses 3D interactivity, less impressive demo
288
+
289
+ ---
290
+
291
+ ## Investigation Steps
292
+
293
+ ### Step 0: Test Async/Await in js_on_load (CRITICAL)
294
+ Before implementing Solution 1, verify async works:
295
+ ```python
296
+ import gradio as gr
297
+
298
+ with gr.Blocks() as demo:
299
+ html = gr.HTML(
300
+ value="<div>Testing async...</div>",
301
+ js_on_load="""
302
+ (async () => {
303
+ element.innerText = 'Async started...';
304
+ await new Promise(r => setTimeout(r, 1000));
305
+ element.innerText = 'Async works!';
306
+ element.style.background = 'green';
307
+ })();
308
+ """
309
+ )
310
+
311
+ demo.launch()
312
+ ```
313
+
314
+ If this shows "Async works!" with green background after 1 second, async is supported.
315
+
316
+ ### Step 1: Verify js_on_load Works (Basic)
317
+ Create minimal test:
318
+ ```python
319
+ import gradio as gr
320
+
321
+ with gr.Blocks() as demo:
322
+ html = gr.HTML(
323
+ value="<div id='test'>Loading...</div>",
324
+ js_on_load="element.style.background='green'; element.innerText='JS Works!';"
325
+ )
326
+
327
+ demo.launch()
328
+ ```
329
+
330
+ ### Step 2: Test Dynamic Import in js_on_load
331
+ ```python
332
+ js_on_load="""
333
+ (async () => {
334
+ const mod = await import('https://unpkg.com/@niivue/niivue@0.65.0/dist/index.js');
335
+ console.log('NiiVue loaded:', mod);
336
+ element.innerText = 'Import succeeded!';
337
+ })();
338
+ """
339
+ ```
340
+
341
+ ### Step 3: Check Browser Console
342
+ 1. Open HF Spaces demo
343
+ 2. Open DevTools (F12) → Console
344
+ 3. Look for errors related to:
345
+ - Module loading failures
346
+ - WebGL context issues
347
+ - CORS errors
348
+ - Memory errors
349
+
350
+ ### Step 4: Test with Smaller Files
351
+ Create downsampled test NIfTI (~1MB) to isolate size vs JS issues.
352
+
353
+ ---
354
+
355
+ ## Related Issues
356
+
357
+ - **Bug #9**: DeepISLES modules not found (FIXED - subprocess bridge)
358
+ - **Bug #8**: HF Spaces streaming hang (FIXED)
359
+ - **Technical Debt**: NiiVue memory overhead (P2)
360
+ - **[Gradio #4511](https://github.com/gradio-app/gradio/issues/4511)**: 3D medical image support request (closed, not planned)
361
+ - **[Gradio #10250](https://github.com/gradio-app/gradio/issues/10250)**: JS in head param issues (open)
362
+
363
+ ---
364
+
365
+ ## Priority Assessment
366
+
367
+ **Severity:** P2 (Medium)
368
+ - Core inference pipeline works correctly
369
+ - 2D visualization provides adequate fallback
370
+ - No data loss or security impact
371
+ - Demo is functional for evaluation purposes
372
+
373
+ **Impact:**
374
+ - Less impressive without 3D viewer
375
+ - Users can still evaluate predictions via 2D slices
376
+ - Download functionality unaffected
377
+
378
+ **Recommendation:**
379
+ 1. First, validate inference accuracy across multiple cases
380
+ 2. Then attempt Solution 1 (js_on_load) as quick fix
381
+ 3. If that fails, implement Solution 5 (enhanced 2D) for reliability
382
+ 4. Consider Solution 4 (custom component) for future enhancement
383
+
384
+ ---
385
+
386
+ ## References
387
+
388
+ - [Gradio HTML Docs](https://www.gradio.app/docs/gradio/html)
389
+ - [Gradio Custom HTML Components Guide](https://www.gradio.app/guides/custom_HTML_components)
390
+ - [Gradio 6 Migration Guide](https://www.gradio.app/main/guides/gradio-6-migration-guide)
391
+ - [HuggingFace Forum: JS doesn't work in gr.HTML](https://discuss.huggingface.co/t/gradio-html-component-with-javascript-code-dont-work/37316)
392
+ - [GitHub Issue #10250: JS in head param](https://github.com/gradio-app/gradio/issues/10250)
393
+ - [GitHub Issue #4511: 3D Medical Images](https://github.com/gradio-app/gradio/issues/4511)
394
+ - [NiiVue GitHub](https://github.com/niivue/niivue)
395
+ - [ipyniivue (Jupyter Widget)](https://github.com/niivue/ipyniivue)
396
+ - [Gradio 6 Announcement](https://alternativeto.net/news/2025/11/gradio-6-released-with-faster-performance-for-creating-machine-learning-apps-in-python/)
397
+
398
+ ---
399
+
400
+ ## Appendix: HF Spaces Logs
401
+
402
+ ```text
403
+ INFO: Running segmentation for sub-stroke0002
404
+ INFO: Case sub-stroke0002 ready: DWI=20.9MB, ADC=12.6MB
405
+ INFO: DeepISLES subprocess completed in 30.88s
406
+ ```
407
+
408
+ Note: No JavaScript errors visible in server logs (client-side only).
scripts/test_js_on_load.py ADDED
@@ -0,0 +1,149 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python
2
+ """Test script to verify js_on_load + async + dynamic import works in Gradio.
3
+
4
+ This tests the core mechanism needed to fix the NiiVue black screen bug.
5
+
6
+ Run:
7
+ uv run python scripts/test_js_on_load.py
8
+
9
+ Then open http://localhost:7860 and check if the tests pass.
10
+ """
11
+
12
+ import gradio as gr
13
+
14
+ # Test 1: Basic js_on_load execution
15
+ TEST1_HTML = '<div id="test1" style="padding:20px;background:#333;color:#fff;margin:10px;border-radius:8px;">Test 1: Waiting...</div>'
16
+ TEST1_JS = """
17
+ element.innerText = 'Test 1: PASS - js_on_load executed!';
18
+ element.style.background = '#228B22';
19
+ """
20
+
21
+ # Test 2: Async IIFE pattern
22
+ TEST2_HTML = '<div id="test2" style="padding:20px;background:#333;color:#fff;margin:10px;border-radius:8px;">Test 2: Waiting...</div>'
23
+ TEST2_JS = """
24
+ (async () => {
25
+ element.innerText = 'Test 2: Async started...';
26
+ await new Promise(r => setTimeout(r, 500));
27
+ element.innerText = 'Test 2: PASS - Async/await works!';
28
+ element.style.background = '#228B22';
29
+ })();
30
+ """
31
+
32
+ # Test 3: Dynamic import from CDN
33
+ TEST3_HTML = '<div id="test3" style="padding:20px;background:#333;color:#fff;margin:10px;border-radius:8px;">Test 3: Waiting...</div>'
34
+ TEST3_JS = """
35
+ (async () => {
36
+ element.innerText = 'Test 3: Loading NiiVue from CDN...';
37
+ try {
38
+ const mod = await import('https://unpkg.com/@niivue/niivue@0.65.0/dist/index.js');
39
+ if (mod.Niivue) {
40
+ element.innerText = 'Test 3: PASS - NiiVue loaded! Niivue class available.';
41
+ element.style.background = '#228B22';
42
+ } else {
43
+ element.innerText = 'Test 3: PARTIAL - Module loaded but no Niivue class';
44
+ element.style.background = '#FFA500';
45
+ }
46
+ } catch(e) {
47
+ element.innerText = 'Test 3: FAIL - ' + e.message;
48
+ element.style.background = '#DC143C';
49
+ }
50
+ })();
51
+ """
52
+
53
+ # Test 4: Canvas + WebGL2 check
54
+ TEST4_HTML = """
55
+ <div id="test4-container" style="padding:20px;background:#333;color:#fff;margin:10px;border-radius:8px;">
56
+ <div id="test4-status">Test 4: Waiting...</div>
57
+ <canvas id="test4-canvas" style="width:200px;height:100px;background:#000;margin-top:10px;"></canvas>
58
+ </div>
59
+ """
60
+ TEST4_JS = """
61
+ (async () => {
62
+ const status = element.querySelector('#test4-status');
63
+ const canvas = element.querySelector('#test4-canvas');
64
+
65
+ status.innerText = 'Test 4: Checking WebGL2...';
66
+
67
+ const gl = canvas.getContext('webgl2');
68
+ if (!gl) {
69
+ status.innerText = 'Test 4: FAIL - WebGL2 not supported';
70
+ element.style.background = '#DC143C';
71
+ return;
72
+ }
73
+
74
+ status.innerText = 'Test 4: Loading NiiVue...';
75
+ try {
76
+ const { Niivue } = await import('https://unpkg.com/@niivue/niivue@0.65.0/dist/index.js');
77
+ const nv = new Niivue({ logging: false, backColor: [0.2, 0.2, 0.3, 1] });
78
+ await nv.attachToCanvas(canvas);
79
+ nv.drawScene();
80
+
81
+ status.innerText = 'Test 4: PASS - NiiVue attached to canvas!';
82
+ status.style.color = '#90EE90';
83
+ } catch(e) {
84
+ status.innerText = 'Test 4: FAIL - ' + e.message;
85
+ element.style.background = '#DC143C';
86
+ }
87
+ })();
88
+ """
89
+
90
+ # Test 5: Full integration with props.value
91
+ TEST5_HTML = """
92
+ <div id="test5-container" style="padding:20px;background:#333;color:#fff;margin:10px;border-radius:8px;">
93
+ <div id="test5-status">Test 5: Waiting...</div>
94
+ <div id="test5-value" style="font-family:monospace;font-size:12px;margin-top:10px;"></div>
95
+ </div>
96
+ """
97
+ TEST5_JS = """
98
+ const status = element.querySelector('#test5-status');
99
+ const valueDiv = element.querySelector('#test5-value');
100
+
101
+ // Check if we can access props
102
+ if (typeof props !== 'undefined') {
103
+ status.innerText = 'Test 5: PASS - props object accessible!';
104
+ status.style.color = '#90EE90';
105
+ valueDiv.innerText = 'props.value = ' + JSON.stringify(props.value ?? null).substring(0, 100) + '...';
106
+ } else {
107
+ status.innerText = 'Test 5: FAIL - props not defined';
108
+ element.style.background = '#DC143C';
109
+ }
110
+ """
111
+
112
+
113
+ with gr.Blocks(title="js_on_load Test Suite") as demo:
114
+ gr.Markdown("""
115
+ # NiiVue js_on_load Test Suite
116
+
117
+ Testing if `js_on_load` supports the patterns we need for NiiVue:
118
+
119
+ 1. **Basic execution** - Does js_on_load run at all?
120
+ 2. **Async IIFE** - Does `(async () => { await ... })()` work?
121
+ 3. **Dynamic import** - Can we `await import()` from CDN?
122
+ 4. **Canvas + NiiVue** - Can we attach NiiVue to a canvas?
123
+ 5. **Props access** - Can we read `props.value`?
124
+
125
+ All tests should show green "PASS" if our fix will work.
126
+ """)
127
+
128
+ with gr.Row():
129
+ with gr.Column():
130
+ gr.HTML(value=TEST1_HTML, js_on_load=TEST1_JS)
131
+ gr.HTML(value=TEST2_HTML, js_on_load=TEST2_JS)
132
+ gr.HTML(value=TEST3_HTML, js_on_load=TEST3_JS)
133
+
134
+ with gr.Column():
135
+ gr.HTML(value=TEST4_HTML, js_on_load=TEST4_JS)
136
+ gr.HTML(value=TEST5_HTML, js_on_load=TEST5_JS)
137
+
138
+ gr.Markdown("""
139
+ ---
140
+ **If all tests pass:** We can implement the `js_on_load` fix for NiiVue viewer.
141
+
142
+ **If tests fail:** We'll need alternative approaches (gradio-iframe or enhanced 2D).
143
+ """)
144
+
145
+
146
+ if __name__ == "__main__":
147
+ print("Starting js_on_load test server...")
148
+ print("Open http://localhost:7860 to see test results")
149
+ demo.launch(server_port=7860)
src/stroke_deepisles_demo/ui/components.py CHANGED
@@ -6,6 +6,7 @@ import gradio as gr
6
 
7
  from stroke_deepisles_demo.core.config import get_settings
8
  from stroke_deepisles_demo.core.logging import get_logger
 
9
 
10
  logger = get_logger(__name__)
11
 
@@ -38,8 +39,14 @@ def create_results_display() -> dict[str, gr.components.Component]:
38
  """
39
  # Using gr.Group to group them visually
40
  with gr.Group():
41
- # NiiVue visualization uses HTML
42
- niivue_viewer = gr.HTML(label="Interactive 3D Viewer")
 
 
 
 
 
 
43
 
44
  # Slice comparisons (Matplotlib)
45
  slice_plot = gr.Plot(label="Slice Comparison")
 
6
 
7
  from stroke_deepisles_demo.core.config import get_settings
8
  from stroke_deepisles_demo.core.logging import get_logger
9
+ from stroke_deepisles_demo.ui.viewer import NIIVUE_JS_ON_LOAD
10
 
11
  logger = get_logger(__name__)
12
 
 
39
  """
40
  # Using gr.Group to group them visually
41
  with gr.Group():
42
+ # NiiVue visualization uses HTML with js_on_load for JavaScript execution
43
+ # Note: Gradio strips <script> tags from HTML value for security,
44
+ # so we must use js_on_load to run our NiiVue initialization code.
45
+ # The HTML value contains data-* attributes with volume URLs.
46
+ niivue_viewer = gr.HTML(
47
+ label="Interactive 3D Viewer",
48
+ js_on_load=NIIVUE_JS_ON_LOAD,
49
+ )
50
 
51
  # Slice comparisons (Matplotlib)
52
  slice_plot = gr.Plot(label="Slice Comparison")
src/stroke_deepisles_demo/ui/viewer.py CHANGED
@@ -281,11 +281,14 @@ def create_niivue_html(
281
  height: int = 400,
282
  ) -> str:
283
  """
284
- Create HTML/JS for NiiVue viewer.
285
 
286
- This function generates an HTML snippet with embedded JavaScript for
287
- NiiVue WebGL-based neuroimaging visualization. Each invocation creates
288
- a unique canvas ID to avoid conflicts when multiple viewers are rendered.
 
 
 
289
 
290
  Args:
291
  volume_url: Data URL or URL to volume NIfTI file
@@ -293,93 +296,117 @@ def create_niivue_html(
293
  height: Viewer height in pixels
294
 
295
  Returns:
296
- HTML string with embedded NiiVue viewer
297
 
298
  Note:
299
- The JavaScript uses dynamic import() which works in modern browsers
300
- and Gradio's HTML component. Each viewer gets a unique ID to support
301
- multiple simultaneous viewers.
302
  """
303
  # Generate unique ID for this viewer instance
304
  viewer_id = uuid.uuid4().hex[:8]
305
- canvas_id = f"niivue-canvas-{viewer_id}"
306
- container_id = f"niivue-container-{viewer_id}"
307
-
308
- # Safely serialize URLs for JavaScript (prevents XSS)
309
- volume_url_js = json.dumps(volume_url)
310
-
311
- # Build mask volume configuration if provided
312
- mask_js = ""
313
- if mask_url:
314
- mask_url_js = json.dumps(mask_url)
315
- mask_js = f"""
316
- volumes.push({{
317
- url: {mask_url_js},
318
- colorMap: 'red',
319
- opacity: 0.5
320
- }});"""
321
-
322
- # JavaScript that initializes NiiVue
323
- # Using an IIFE pattern that works better in Gradio's HTML component
324
- return f"""
325
- <div id="{container_id}" style="width:100%; height:{height}px; background:#000; border-radius:8px; position:relative;">
326
- <canvas id="{canvas_id}" style="width:100%; height:100%;"></canvas>
327
  </div>
328
- <script type="module">
329
- // NiiVue initialization for viewer {viewer_id}
330
- (async function() {{
331
- try {{
332
- // Check if browser supports WebGL2
333
- const testCanvas = document.createElement('canvas');
334
- const gl = testCanvas.getContext('webgl2');
335
- if (!gl) {{
336
- document.getElementById('{container_id}').innerHTML =
337
- '<div style="color:#fff;padding:20px;text-align:center;">' +
338
- 'WebGL2 not supported. Please use a modern browser.</div>';
339
- return;
340
- }}
341
-
342
- // Dynamically import NiiVue
343
- const niivueModule = await import('{NIIVUE_CDN_URL}');
344
- const Niivue = niivueModule.Niivue;
345
-
346
- // Initialize NiiVue with options
347
- const nv = new Niivue({{
348
- logging: false,
349
- show3Dcrosshair: true,
350
- textHeight: 0.04,
351
- backColor: [0, 0, 0, 1],
352
- crosshairColor: [0.2, 0.8, 0.2, 1]
353
- }});
354
-
355
- // Attach to canvas
356
- await nv.attachToCanvas(document.getElementById('{canvas_id}'));
357
-
358
- // Prepare volumes
359
- const volumes = [{{
360
- url: {volume_url_js},
361
- name: 'input.nii.gz'
362
- }}];{mask_js}
363
-
364
- // Load volumes
365
- await nv.loadVolumes(volumes);
366
-
367
- // Configure view: multiplanar + 3D
368
- nv.setSliceType(nv.sliceTypeMultiplanar);
369
- if (typeof nv.setMultiplanarLayout === 'function') {{
370
- nv.setMultiplanarLayout(2);
371
- }}
372
- nv.opts.show3Dcrosshair = true;
373
- nv.setRenderAzimuthElevation(120, 10);
374
- nv.drawScene();
375
-
376
- console.log('NiiVue viewer {viewer_id} initialized successfully');
377
- }} catch (error) {{
378
- console.error('NiiVue initialization error:', error);
379
- document.getElementById('{container_id}').innerHTML =
380
- '<div style="color:#fff;padding:20px;text-align:center;">' +
381
- 'Error loading viewer: ' + error.message + '</div>';
382
- }}
383
- }})();
384
- </script>
385
- """
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
281
  height: int = 400,
282
  ) -> str:
283
  """
284
+ Create HTML for NiiVue viewer (static content only).
285
 
286
+ This function generates an HTML snippet with data attributes containing
287
+ volume URLs. The actual NiiVue initialization is handled by js_on_load
288
+ in the gr.HTML component (see NIIVUE_JS_ON_LOAD).
289
+
290
+ IMPORTANT: Gradio's gr.HTML strips <script> tags for security.
291
+ JavaScript must be passed via the js_on_load parameter instead.
292
 
293
  Args:
294
  volume_url: Data URL or URL to volume NIfTI file
 
296
  height: Viewer height in pixels
297
 
298
  Returns:
299
+ HTML string with data attributes for NiiVue viewer
300
 
301
  Note:
302
+ The volume URLs are stored in data-* attributes and read by
303
+ the js_on_load JavaScript code. This pattern works because
304
+ js_on_load has access to the 'element' variable.
305
  """
306
  # Generate unique ID for this viewer instance
307
  viewer_id = uuid.uuid4().hex[:8]
308
+
309
+ # Safely encode URLs for HTML data attributes
310
+ # Using json.dumps ensures proper escaping
311
+ volume_attr = f"data-volume-url={json.dumps(volume_url)}"
312
+ mask_attr = f"data-mask-url={json.dumps(mask_url)}" if mask_url else 'data-mask-url=""'
313
+
314
+ return f"""<div
315
+ id="niivue-container-{viewer_id}"
316
+ class="niivue-viewer"
317
+ {volume_attr}
318
+ {mask_attr}
319
+ style="width:100%; height:{height}px; background:#000; border-radius:8px; position:relative;"
320
+ >
321
+ <canvas style="width:100%; height:100%;"></canvas>
322
+ <div class="niivue-status" style="position:absolute;top:50%;left:50%;transform:translate(-50%,-50%);color:#666;">
323
+ Loading viewer...
 
 
 
 
 
 
324
  </div>
325
+ </div>"""
326
+
327
+
328
+ # JavaScript code for js_on_load parameter
329
+ # This runs when the gr.HTML component loads/updates
330
+ # Variables available: element, props, trigger
331
+ NIIVUE_JS_ON_LOAD = f"""
332
+ (async () => {{
333
+ const container = element.querySelector('.niivue-viewer') || element;
334
+ const canvas = element.querySelector('canvas');
335
+ const status = element.querySelector('.niivue-status');
336
+
337
+ // Get URLs from data attributes
338
+ const volumeUrl = container.dataset.volumeUrl;
339
+ const maskUrl = container.dataset.maskUrl;
340
+
341
+ // Skip if no volume URL (initial empty state)
342
+ if (!volumeUrl) {{
343
+ if (status) status.innerText = 'Waiting for segmentation...';
344
+ return;
345
+ }}
346
+
347
+ try {{
348
+ if (status) status.innerText = 'Checking WebGL2...';
349
+
350
+ // Check WebGL2 support
351
+ const gl = canvas.getContext('webgl2');
352
+ if (!gl) {{
353
+ container.innerHTML = '<div style="color:#fff;padding:20px;text-align:center;">WebGL2 not supported. Please use a modern browser.</div>';
354
+ return;
355
+ }}
356
+
357
+ if (status) status.innerText = 'Loading NiiVue...';
358
+
359
+ // Dynamically import NiiVue from CDN
360
+ const {{ Niivue }} = await import('{NIIVUE_CDN_URL}');
361
+
362
+ // Initialize NiiVue
363
+ const nv = new Niivue({{
364
+ logging: false,
365
+ show3Dcrosshair: true,
366
+ textHeight: 0.04,
367
+ backColor: [0, 0, 0, 1],
368
+ crosshairColor: [0.2, 0.8, 0.2, 1]
369
+ }});
370
+
371
+ // Attach to canvas
372
+ await nv.attachToCanvas(canvas);
373
+
374
+ // Hide status message
375
+ if (status) status.style.display = 'none';
376
+
377
+ // Prepare volumes
378
+ const volumes = [{{ url: volumeUrl, name: 'input.nii.gz' }}];
379
+
380
+ if (maskUrl) {{
381
+ volumes.push({{
382
+ url: maskUrl,
383
+ colorMap: 'red',
384
+ opacity: 0.5
385
+ }});
386
+ }}
387
+
388
+ // Load volumes
389
+ await nv.loadVolumes(volumes);
390
+
391
+ // Configure view: multiplanar + 3D
392
+ nv.setSliceType(nv.sliceTypeMultiplanar);
393
+ if (typeof nv.setMultiplanarLayout === 'function') {{
394
+ nv.setMultiplanarLayout(2);
395
+ }}
396
+ nv.opts.show3Dcrosshair = true;
397
+ nv.setRenderAzimuthElevation(120, 10);
398
+ nv.drawScene();
399
+
400
+ console.log('NiiVue viewer initialized successfully');
401
+
402
+ }} catch (error) {{
403
+ console.error('NiiVue initialization error:', error);
404
+ // Use textContent instead of innerHTML to prevent XSS
405
+ const errorDiv = document.createElement('div');
406
+ errorDiv.style.cssText = 'color:#f66;padding:20px;text-align:center;';
407
+ errorDiv.textContent = 'Error loading viewer: ' + error.message;
408
+ container.innerHTML = '';
409
+ container.appendChild(errorDiv);
410
+ }}
411
+ }})();
412
+ """