File size: 31,619 Bytes
c816150
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
<!doctype html>
<html lang="ar" dir="rtl">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>OmniDiag Architecture</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=IBM+Plex+Sans+Arabic:wght@400;500;700&family=IBM+Plex+Mono:wght@400;500&display=swap">
<style>
/* Layout: right-to-left reading column, sticky section index, one diagram per section */
:root{--bg:#f6f7f9;--surface:#ffffff;--fg:#14202b;--muted:#566574;--line:#d9dee4;--accent:#0b6e6e;--accent-soft:#e0f0ef;--warn:#9a5b00;--warn-soft:#fbf0dc;
--font-body:'IBM Plex Sans Arabic',system-ui,sans-serif;--font-mono:'IBM Plex Mono',ui-monospace,monospace}
@media (prefers-color-scheme:dark){:root:not([data-theme="light"]){--bg:#0f161c;--surface:#16202a;--fg:#e6edf3;--muted:#9bacba;--line:#26333f;--accent:#4fc3c0;--accent-soft:#12302f;--warn:#e0a84a;--warn-soft:#2c2412;color-scheme:dark}}
:root[data-theme="dark"]{--bg:#0f161c;--surface:#16202a;--fg:#e6edf3;--muted:#9bacba;--line:#26333f;--accent:#4fc3c0;--accent-soft:#12302f;--warn:#e0a84a;--warn-soft:#2c2412;color-scheme:dark}
*{box-sizing:border-box}
body{background:var(--bg);color:var(--fg);font-family:var(--font-body);font-size:16px;line-height:1.7;padding-inline:16px;padding-block:0}
.wrap{max-width:1180px;margin-inline:auto}
header{padding-block:32px 20px}
h1{font-size:clamp(1.6rem,4vw,2.3rem);line-height:1.25;margin:0 0 8px;text-wrap:balance}
.sub{color:var(--muted);margin:0;max-width:70ch}
.facts{display:grid;grid-template-columns:repeat(auto-fit,minmax(210px,1fr));gap:12px;margin-block:20px}
.fact{background:var(--surface);border:1px solid var(--line);border-radius:8px;padding:12px 14px;min-width:0}
.fact b{display:block;font-size:.8rem;color:var(--muted);font-weight:500;letter-spacing:.02em}
.fact span{font-family:var(--font-mono);font-size:.9rem;overflow-wrap:anywhere}
nav{position:sticky;top:env(safe-area-inset-top,0px);z-index:5;background:var(--bg);border-block:1px solid var(--line);display:flex;gap:6px;overflow-x:auto;padding-block:8px;margin-block:8px 24px}
nav a{white-space:nowrap;font-size:.85rem;color:var(--fg);text-decoration:none;padding:4px 10px;border-radius:999px;border:1px solid var(--line);background:var(--surface)}
nav a:hover,nav a:focus-visible{border-color:var(--accent);color:var(--accent);outline:none}
section{margin-block:0 40px;scroll-margin-top:64px}
h2{font-size:1.3rem;margin:0 0 6px;text-wrap:balance}
.cap{color:var(--muted);margin:0 0 12px;max-width:80ch}
.fig{background:var(--surface);border:1px solid var(--line);border-radius:10px;padding:16px;overflow-x:auto;min-width:0}
pre.mermaid{margin:0;text-align:center;background:transparent}
.note{background:var(--warn-soft);border:1px solid var(--line);border-inline-start:4px solid var(--warn);border-radius:6px;padding:10px 14px;margin-block:12px;color:var(--fg);font-size:.95rem}
footer{color:var(--muted);font-size:.85rem;padding-block:8px 40px}
</style>

</head>
<body>
<div class="wrap">
<header>
<h1>هيكلية OmniDiag: منصة تشخيص سريري متعددة الأمراض</h1>
<p class="sub">Modular Monolith بعملية FastAPI واحدة وواجهة React. ١٣ مخططاً تغطي المعمارية، تدفق الطلب، النماذج، الإنسان في الحلقة، NLP والـ LLM، الأمن، المراقبة، قاعدة البيانات، الواجهة، النشر، والتعلّم الاتحادي.</p>
<div class="facts">
<div class="fact"><b>Backend</b><span>FastAPI, SQLAlchemy async, Alembic</span></div>
<div class="fact"><b>Frontend</b><span>React, Vite, Tailwind, PWA</span></div>
<div class="fact"><b>Diseases</b><span>heart_disease, diabetes_nhanes (BRFSS diabetes retired: 410)</span></div>
<div class="fact"><b>State</b><span>SQLite dev / PostgreSQL, Redis, MLflow</span></div>
<div class="fact"><b>Ops</b><span>Docker, HF Space, Vercel, k8s, Helm, Prometheus, Grafana</span></div>
<div class="fact"><b>Tests</b><span>about 60 pytest files</span></div>
</div>
<div class="note">الحالة الصادقة: التعلّم الاتحادي ليس جزءاً من المنتج المشغَّل (القسم ١٣)، وترقية النموذج المرشَّح بعد إعادة التدريب قرار بشري لا يوجد كود يفعله تلقائياً (القسم ٦).</div>
</header>
<nav aria-label="فهرس الأقسام"><a href="#overview">الصورة الكبرى: Modular Monolith</a><a href="#modules">خريطة الوحدات (Package Map)</a><a href="#predict">رحلة طلب التنبؤ (Predict Request)</a><a href="#registry">الإضافة بالإعدادات: Config-driven Registry</a><a href="#heart">نموذج القلب: Spline-GLM + Venn-Abers + Conformal</a><a href="#hitl">الإنسان في الحلقة والتعلّم النشط (HITL + Active Learning)</a><a href="#nlpllm">معالجة اللغة (NLP) والتقرير بالـ LLM</a><a href="#security">الأمن والصلاحيات والتدقيق</a><a href="#monitor">المراقبة والـ Drift و MLflow</a><a href="#er">نموذج البيانات (Database)</a><a href="#frontend">الواجهة الأمامية (React SPA)</a><a href="#deploy">النشر والبنية التحتية (Deployment / CI)</a><a href="#fl">التعلّم الاتحادي (Federated Learning): الحالة الفعلية</a></nav>
<section id="overview"><h2>١. الصورة الكبرى: Modular Monolith</h2><p class="cap">عملية FastAPI واحدة (deployable واحد) مقسّمة داخلياً إلى وحدات (modules) بحدود واضحة؛ الأمراض تُضاف بملف YAML دون لمس باقي الكود.</p>
<div class="fig" dir="ltr"><pre class="mermaid">flowchart TB
  subgraph CL[&quot;Clients&quot;]
    SPA[&quot;React SPA - Vite, Tailwind, PWA&lt;br/&gt;(Vercel)&quot;]
    EMR[&quot;EMR / API clients&lt;br/&gt;JWT cookie or X-API-Key&quot;]
  end
  subgraph MW[&quot;Cross-cutting middleware (order of wrapping)&quot;]
    direction LR
    CORS[&quot;CORS&quot;] --&gt; SEC[&quot;SecurityHeaders&lt;br/&gt;HTTPS enforce&quot;] --&gt; AUD[&quot;AuditMiddleware&lt;br/&gt;writes audit_logs&quot;] --&gt; MET[&quot;MetricsMiddleware&lt;br/&gt;Prometheus&quot;] --&gt; RL[&quot;slowapi rate limit&lt;br/&gt;per-endpoint&quot;]
  end
  subgraph APP[&quot;FastAPI process = the monolith (backend/main.py)&quot;]
    subgraph MODS[&quot;Feature modules (APIRouter each)&quot;]
      AUTH[&quot;auth&lt;br/&gt;/auth&quot;]
      ADM[&quot;admin&lt;br/&gt;/admin&quot;]
      PAT[&quot;patients&lt;br/&gt;/api/v4/patients&quot;]
      CLIN[&quot;clinical diagnosis&lt;br/&gt;/api/v4/{disease}/...&quot;]
      AL[&quot;active_learning (HITL)&lt;br/&gt;/api/v4/review&quot;]
      MON[&quot;monitoring&lt;br/&gt;/api/v4/admin/drift, mlflow&quot;]
      NLP[&quot;nlp&lt;br/&gt;notes_parser&quot;]
      LLM[&quot;llm&lt;br/&gt;report_generator&quot;]
      FED[&quot;federated&lt;br/&gt;SUPERSEDED prototype&quot;]
    end
    subgraph CORE[&quot;Shared core services&quot;]
      ROUTER[&quot;OmniDiagRouter&lt;br/&gt;scans configs/*.yaml&quot;]
      PS[&quot;probability_scale +&lt;br/&gt;prevalence_correction&quot;]
      SHAP[&quot;shap_service&quot;]
      CF[&quot;counterfactual_generator +&lt;br/&gt;diabetes_what_if_levers&quot;]
      CACHE[&quot;cache.py&lt;br/&gt;Redis or in-memory&quot;]
    end
    subgraph REG[&quot;Model-family registry (backend/model_backends)&quot;]
      B1[&quot;glm_ivap_conformal&lt;br/&gt;HEART&quot;]
      B3[&quot;ebm_platt_conformal&lt;br/&gt;DIABETES NHANES&quot;]
      B4[&quot;sklearn_pipeline /&lt;br/&gt;sklearn_generic&quot;]
    end
  end
  subgraph DATA[&quot;State and artifacts&quot;]
    DB[(&quot;SQL DB&lt;br/&gt;SQLite dev / PostgreSQL prod&lt;br/&gt;SQLAlchemy async + Alembic&quot;)]
    RED[(&quot;Redis cache&quot;)]
    ART[(&quot;models/ bundles&lt;br/&gt;built at image build&quot;)]
    MLF[(&quot;MLflow&lt;br/&gt;runs and registry&quot;)]
    CFG[&quot;configs/*.yaml&lt;br/&gt;single source of truth per disease&quot;]
  end
  subgraph OBS[&quot;Observability&quot;]
    PROM[&quot;Prometheus&quot;] --&gt; GRAF[&quot;Grafana&quot;]
  end
  subgraph EXT[&quot;External&quot;]
    DS[&quot;DeepSeek API&lt;br/&gt;(OpenAI-compatible)&quot;]
    BERT[&quot;BioBERT NER&lt;br/&gt;HuggingFace, optional&quot;]
  end
  CL --&gt; MW --&gt; MODS
  CLIN --&gt; ROUTER --&gt; REG
  CLIN --&gt; PS
  CLIN --&gt; SHAP
  CLIN --&gt; CF
  CLIN --&gt; CACHE
  CACHE -.-&gt; RED
  CFG --&gt; ROUTER
  REG --&gt; ART
  AUTH --&gt; DB
  PAT --&gt; DB
  ADM --&gt; DB
  AL --&gt; DB
  AUD -.-&gt; DB
  MON --&gt; MLF
  AL --&gt; MLF
  MET --&gt; PROM
  LLM -.-&gt; DS
  NLP -.-&gt; BERT</pre></div></section>
<section id="modules"><h2>٢. خريطة الوحدات (Package Map)</h2><p class="cap">كل مجلد = وحدة بمسؤولية واحدة. الوحدات تتحدث عبر استدعاءات داخلية (in-process) لا عبر شبكة، وهذا ما يجعله Monolith، والحدود النظيفة بينها هي ما يجعله Modular.</p>
<div class="fig" dir="ltr"><pre class="mermaid">flowchart LR
  ROOT([&quot;OmniDiag repo&quot;])
  ROOT --&gt; BE[&quot;backend/&quot;]
  ROOT --&gt; FE[&quot;frontend/src&quot;]
  ROOT --&gt; CF[&quot;configs/ + features/&quot;]
  ROOT --&gt; SC[&quot;scripts/ + models/ + experiment_files/&quot;]
  ROOT --&gt; OPS[&quot;Dockerfile, docker-compose, k8s, helm, deploy, .github&quot;]
  ROOT --&gt; TST[&quot;tests/ (about 60 files)&quot;]
  ROOT --&gt; DOC[&quot;docs/, evaluation_evidence/, alembic/&quot;]
  BE --&gt; m1[&quot;main.py - app wiring, lifespan,&lt;br/&gt;predict/explain/batch/report/parse-notes&quot;]
  BE --&gt; m2[&quot;router.py - disease dispatcher&quot;]
  BE --&gt; m3[&quot;model_backends/ - 4 families,&lt;br/&gt;auto-registered by file&quot;]
  BE --&gt; m4[&quot;heart_glm/ - stack.py, candidate.py,&lt;br/&gt;reference_scores.json&quot;]
  BE --&gt; m5[&quot;auth/ - jwt, hashing, rbac, api_key&quot;]
  BE --&gt; m6[&quot;admin/ - users, audit, cache flush,&lt;br/&gt;api-keys, stats, retrain&quot;]
  BE --&gt; m7[&quot;patients/ - EMR: patients, visits,&lt;br/&gt;predictions, export, notes&quot;]
  BE --&gt; m8[&quot;active_learning/ - sampler, routes,&lt;br/&gt;retrain, nhanes candidate&quot;]
  BE --&gt; m9[&quot;monitoring/ - drift, drift_stats,&lt;br/&gt;metrics, mlflow_tracker&quot;]
  BE --&gt; m10[&quot;nlp/ + llm/&quot;]
  BE --&gt; m11[&quot;middleware/ - audit, security&quot;]
  BE --&gt; m12[&quot;db_models/ - 8 ORM tables&quot;]
  BE --&gt; m13[&quot;federated/ - superseded&quot;]
  CF --&gt; c1[&quot;heart_disease.yaml&lt;br/&gt;diabetes_nhanes.yaml&quot;]
  CF --&gt; c2[&quot;base_features.py&lt;br/&gt;(extension point; no live module uses it)&quot;]
  SC --&gt; s1[&quot;train_heart_glm.py&lt;br/&gt;build_drift_reference.py&lt;br/&gt;retrain.py, seed_db.py&quot;]</pre></div></section>
<section id="predict"><h2>٣. رحلة طلب التنبؤ (Predict Request)</h2><p class="cap">ما يحدث من الضغط على Predict حتى ظهور النتيجة: المصادقة، الكاش، النموذج، التسجيل، وإضافة الحالات غير المؤكدة لطابور المراجعة البشرية.</p>
<div class="fig" dir="ltr"><pre class="mermaid">sequenceDiagram
  autonumber
  participant U as Clinician (React)
  participant MW as Middleware
  participant API as POST /api/v4/{disease}/predict
  participant AU as Auth + RBAC
  participant C as Cache (Redis or memory)
  participant R as OmniDiagRouter
  participant B as ModelBackend (per family)
  participant DB as SQL DB
  participant AL as Active-learning sampler
  U-&gt;&gt;MW: JSON features
  MW-&gt;&gt;API: rate limit, security headers, start timer
  API-&gt;&gt;AU: JWT cookie or API key
  AU--&gt;&gt;API: user, roles must be doctor, nurse or super_admin
  API-&gt;&gt;API: validate against schema from YAML
  API-&gt;&gt;C: get(hash of disease + features)
  alt cache hit
    C--&gt;&gt;API: stored result (scale checked)
  else cache miss
    API-&gt;&gt;R: predict in threadpool
    R-&gt;&gt;B: engineer features, model, calibrate
    B--&gt;&gt;R: probability, decision, bounds
    R--&gt;&gt;API: result
    API-&gt;&gt;C: set(result, ttl)
  end
  API-&gt;&gt;API: record Prometheus counters (hit or miss)
  API-&gt;&gt;DB: INSERT predictions (scale and threshold stored)
  API-&gt;&gt;AL: uncertain?
  alt conformal decision is uncertain, or entropy near threshold
    AL-&gt;&gt;DB: INSERT review_queue (pending)
  end
  API--&gt;&gt;MW: PredictResponse + Cache-Hit header
  MW-&gt;&gt;DB: INSERT audit_logs (user, endpoint, status, ms)
  MW--&gt;&gt;U: result, SHAP available via /explain</pre></div></section>
<section id="registry"><h2>٤. الإضافة بالإعدادات: Config-driven Registry</h2><p class="cap">إضافة مرض = ملف YAML (وملف features فقط إن احتاجته العائلة). الـ Router لا يعرف أسماء الأمراض ولا العائلات. كل عائلة نموذج تسجّل نفسها بمجرد وجود ملفها في model_backends.</p>
<div class="fig" dir="ltr"><pre class="mermaid">flowchart LR
  subgraph FS[&quot;Filesystem&quot;]
    Y1[&quot;configs/heart_disease.yaml&lt;br/&gt;family: glm_ivap_conformal&quot;]
    Y3[&quot;configs/diabetes_nhanes.yaml&lt;br/&gt;family: ebm_platt_conformal&quot;]
  end
  subgraph STARTUP[&quot;At startup&quot;]
    SCAN[&quot;OmniDiagRouter._load_all_configs()&lt;br/&gt;one entry per YAML&quot;]
    IMP[&quot;model_backends/__init__.py&lt;br/&gt;pkgutil imports every module&quot;]
    REGI[(&quot;registry&lt;br/&gt;@register_backend(&#x27;name&#x27;)&quot;)]
    IMP --&gt; REGI
  end
  Y1 --&gt; SCAN
  Y3 --&gt; SCAN
  SCAN --&gt;|&quot;family key&quot;| GET[&quot;get_backend(family)&quot;]
  REGI --&gt; GET
  GET --&gt;|&quot;unknown family fails at startup,&lt;br/&gt;names the registered ones&quot;| ERR[&quot;UnknownModelFamilyError&quot;]
  GET --&gt; L1[&quot;HeartGlmConformal&quot;]
  GET --&gt; L3[&quot;EbmPlattConformal&quot;]
  GET -.-&gt; L4[&quot;sklearn_pipeline / sklearn_generic&quot;]
  L1 -.-&gt;|&quot;lazy load on first request&quot;| W1[(&quot;heart_l3_glm_stack.pkl&quot;)]
  L3 -.-&gt; W3[(&quot;diabetes_nhanes_ebm.joblib&quot;)]
  subgraph IFACE[&quot;ModelBackend interface&quot;]
    I1[&quot;predict_proba, feature_names, shap_values&quot;]
    I2[&quot;predict, predict_batch, explain&quot;]
    I3[&quot;BackendCapabilities:&lt;br/&gt;tree_shap, vectorized_batch,&lt;br/&gt;counterfactuals, explainer&quot;]
  end
  L1 --- IFACE</pre></div></section>
<section id="heart"><h2>٥. نموذج القلب: Spline-GLM + Venn-Abers + Conformal</h2><p class="cap">القرار ليس احتمالاً مقابل عتبة واحدة، بل مجموعة conformal تعطي referral / no_referral / uncertain، مع فاصل [p_lower, p_upper]. 7 مدخلات فقط، مدرَّب على 920 مريضاً من 4 مستشفيات.</p>
<div class="fig" dir="ltr"><pre class="mermaid">flowchart LR
  IN[&quot;7 inputs (L3)&lt;br/&gt;Age, Sex, ChestPainType, RestingBP,&lt;br/&gt;Cholesterol, FastingBS, RestingECG&quot;]
  PRE[&quot;Frozen preprocessing&lt;br/&gt;IterativeImputer, scaler,&lt;br/&gt;SplineTransformer, one-hot&quot;]
  GLM[&quot;Spline-GLM&lt;br/&gt;logistic, ridge&quot;]
  S[&quot;raw score&lt;br/&gt;(log-odds, exact additive SHAP)&quot;]
  IVAP[&quot;IVAP&lt;br/&gt;inductive Venn-Abers&quot;]
  P[&quot;calibrated p&lt;br/&gt;+ interval [p_lower, p_upper]&quot;]
  MC[&quot;Mondrian conformal&lt;br/&gt;groups: Sex x class&lt;br/&gt;alpha = 0.10&quot;]
  DEC{&quot;decision&quot;}
  R1[&quot;referral&quot;]
  R2[&quot;no_referral&quot;]
  R3[&quot;uncertain&quot;]
  Q[&quot;review_queue&lt;br/&gt;(human in the loop)&quot;]
  IN --&gt; PRE --&gt; GLM --&gt; S
  S --&gt; IVAP --&gt; P
  S --&gt; MC --&gt; DEC
  DEC --&gt; R1
  DEC --&gt; R2
  DEC --&gt; R3 --&gt; Q
  subgraph LIM[&quot;Declared limits shown in config&quot;]
    L1[&quot;Scope: patients already referred for catheterisation&quot;]
    L2[&quot;No HIGH/MODERATE/LOW bands (risk_bands: null)&quot;]
    L3[&quot;LOHO AUC 0.802, calibration does not transport between hospitals&quot;]
  end
  DEC -.-&gt; LIM</pre></div></section>
<section id="hitl"><h2>٦. الإنسان في الحلقة والتعلّم النشط (HITL + Active Learning)</h2><p class="cap">النظام لا يتعلّم من نفسه تلقائياً: الحالات غير المؤكدة تذهب لطبيب، تسميته تُخزَّن، والإعادة التدريبية تنتج مرشَّحاً (candidate) يقرّر إنسان ترقيته.</p>
<div class="fig" dir="ltr"><pre class="mermaid">flowchart TB
  P[&quot;Prediction&quot;] --&gt; Q1{&quot;module type&quot;}
  Q1 --&gt;|&quot;conformal (heart, NHANES)&quot;| C1[&quot;decision == uncertain&quot;]
  Q1 --&gt;|&quot;threshold-only module&lt;br/&gt;(none live; the retired BRFSS was one)&quot;| C2[&quot;threshold-centred entropy &gt;= 0.88&lt;br/&gt;(prior-shift map sends threshold to 0.5)&quot;]
  C1 --&gt; QUEUE
  C2 --&gt; QUEUE
  QUEUE[(&quot;review_queue&lt;br/&gt;status = pending&lt;br/&gt;uncertainty_score, scale, threshold&quot;)]
  QUEUE --&gt; UI[&quot;ReviewQueuePanel (doctor)&lt;br/&gt;GET /api/v4/review/queue&quot;]
  UI --&gt; ACT{&quot;doctor action&quot;}
  ACT --&gt;|&quot;annotate: label 0/1 + notes&quot;| REV[&quot;status = reviewed&quot;]
  ACT --&gt;|&quot;skip&quot;| SKP[&quot;status = skipped&quot;]
  REV --&gt; RT[&quot;super_admin: POST /admin/retrain&lt;br/&gt;min-samples gate&quot;]
  RT --&gt; FAM{&quot;model.family&quot;}
  FAM --&gt;|&quot;heart: glm_ivap_conformal&quot;| H1[&quot;build_bundle() on 920 UCI rows&lt;br/&gt;+ reviewed rows&lt;br/&gt;(same code as shipped build)&quot;]
  H1 --&gt; H2[&quot;models/heart_disease/candidates/RUN_ID&quot;]
  H2 --&gt; H3[&quot;compare to SHIPPED model:&lt;br/&gt;conformal coverage +&lt;br/&gt;decision transition matrix&quot;]
  H3 --&gt; H4[&quot;log to MLflow&quot;]
  H4 --&gt; H5{{&quot;HUMAN decides promotion&lt;br/&gt;(no code path promotes)&quot;}}
  FAM --&gt;|&quot;NHANES: ebm_platt_conformal&quot;| N1[&quot;status: unsupported (not wired yet);&lt;br/&gt;diabetes_nhanes_candidate.py exists:&lt;br/&gt;candidate only, HbA1c provenance required&quot;]
  FAM --&gt;|&quot;any other family&quot;| D1[&quot;status: unsupported&lt;br/&gt;(nothing read or written)&quot;]
  FAM --&gt;|&quot;retired disease (diabetes)&quot;| D2[&quot;410 DISEASE_RETIRED&quot;]
  SKP -.-&gt; STATS[&quot;GET /review/stats&quot;]
  REV -.-&gt; STATS</pre></div></section>
<section id="nlpllm"><h2>٧. معالجة اللغة (NLP) والتقرير بالـ LLM</h2><p class="cap">ملاحظة سريرية نصية تتحول إلى حقول جاهزة للنموذج؛ وبعد التنبؤ يُولَّد تقرير سريري بالـ LLM مع حارس يمنع المحتوى الممنوع وبديل قاعدي عند الفشل.</p>
<div class="fig" dir="ltr"><pre class="mermaid">flowchart TB
  subgraph NLPX[&quot;NLP: POST /api/v4/parse-notes (20/min, no auth) or /patients/ID/notes&quot;]
    N0[&quot;free-text note&quot;] --&gt; N1{&quot;detect_script&quot;}
    N1 --&gt;|&quot;Arabic&quot;| N1a[&quot;language_support warning:&lt;br/&gt;patterns are English only&quot;]
    N1 --&gt;|&quot;Latin&quot;| N2[&quot;1. Regex baseline&lt;br/&gt;spelled numbers to digits, negation&quot;]
    N2 --&gt; N3[&quot;2. spaCy dependency parse&lt;br/&gt;numeric mentions, linking words,&lt;br/&gt;overrides regex (optional dep)&quot;]
    N3 --&gt; N4[&quot;3. BioBERT NER via HuggingFace&lt;br/&gt;lazy load, overrides on overlap&quot;]
    N4 --&gt; N5[&quot;map_to_disease_schema&lt;br/&gt;(heart only; NHANES has no map)&quot;]
    N5 --&gt; N6[&quot;pre-filled form&lt;br/&gt;missing fields omitted, user confirms&quot;]
  end
  N6 --&gt; PRED[&quot;/predict&quot;]
  subgraph LLMX[&quot;LLM: POST /api/v4/generate-report (10/min)&quot;]
    PRED --&gt; L0[&quot;result: probability, decision, bounds, SHAP top 5&quot;]
    L0 --&gt; L1[&quot;band_for_report&lt;br/&gt;one band value used everywhere&quot;]
    L1 --&gt; L2{&quot;DEEPSEEK_API_KEY set?&quot;}
    L2 --&gt;|&quot;yes&quot;| L3[&quot;DeepSeek chat via OpenAI SDK&quot;]
    L3 --&gt; L4{&quot;forbidden_content(text)?&quot;}
    L4 --&gt;|&quot;clean&quot;| L5[&quot;source = llm&quot;]
    L4 --&gt;|&quot;violations&quot;| L6[&quot;rule-based template&lt;br/&gt;fallback_reason recorded&quot;]
    L2 --&gt;|&quot;no&quot;| L6
    L6 --&gt; L7[&quot;source = rule_based&quot;]
  end
  L5 --&gt; OUT[&quot;ClinicalReportModal / PDFReport&quot;]
  L7 --&gt; OUT</pre></div></section>
<section id="security"><h2>٨. الأمن والصلاحيات والتدقيق</h2><p class="cap">JWT في كوكي + Refresh، مفتاح API بديل للأنظمة الخارجية، صلاحيات RBAC، وسجل تدقيق لكل طلب.</p>
<div class="fig" dir="ltr"><pre class="mermaid">flowchart TB
  subgraph AUTHF[&quot;Authentication&quot;]
    REG[&quot;POST /auth/register&quot;] --&gt; HASH[&quot;password hashing&quot;]
    LOG[&quot;POST /auth/login&quot;] --&gt; JWT[&quot;access JWT + refresh cookie&quot;]
    REF[&quot;POST /auth/refresh&quot;] --&gt; JWT
    OUT[&quot;POST /auth/logout&quot;]
    KEY[&quot;X-API-Key (hash stored)&lt;br/&gt;POST /admin/api-keys&quot;]
  end
  subgraph AUTHZ[&quot;Authorisation (RBAC)&quot;]
    DEP[&quot;get_current_active_user&quot;]
    RR[&quot;require_role(...)&quot;]
    ROLES[&quot;roles: doctor, nurse, super_admin&lt;br/&gt;CLINICAL_ROLES = doctor, nurse, super_admin&lt;br/&gt;ADMIN_ROLES = super_admin&quot;]
    DEP --&gt; RR --&gt; ROLES
  end
  JWT --&gt; DEP
  KEY --&gt; DEP
  subgraph PROT[&quot;What each role reaches&quot;]
    P1[&quot;predict, explain, patients, review: clinical roles&quot;]
    P2[&quot;users, audit-logs, cache flush, retrain, drift run, mlflow: super_admin&quot;]
  end
  RR --&gt; P1
  RR --&gt; P2
  subgraph DEF[&quot;Defence in depth&quot;]
    D1[&quot;SecurityHeadersMiddleware + HTTPS enforcement&quot;]
    D2[&quot;slowapi limits: LIMIT_CLINICAL, LIMIT_ADMIN, 10/min report, 20/min notes&quot;]
    D3[&quot;AuditMiddleware: user, endpoint, method, status, IP, duration_ms&quot;]
    D4[&quot;patient soft-delete (deleted_at)&quot;]
  end
  D3 --&gt; AL[(&quot;audit_logs&quot;)]</pre></div></section>
<section id="monitor"><h2>٩. المراقبة والـ Drift و MLflow</h2><p class="cap">مراقبة تشغيلية (Prometheus/Grafana) ومراقبة انحراف المدخلات بإحصاءات KS و chi-square و PSI مقابل مرجع مجمَّد. الانحراف في المدخلات تنبيه للفحص وليس دليلاً على تدهور النموذج.</p>
<div class="fig" dir="ltr"><pre class="mermaid">flowchart LR
  subgraph RUNTIME[&quot;Runtime metrics&quot;]
    M1[&quot;MetricsMiddleware&lt;br/&gt;request duration, active requests&quot;]
    M2[&quot;record_prediction&lt;br/&gt;predictions_total, confidence histogram&quot;]
    M3[&quot;cache hits / misses&quot;]
    M4[&quot;conformal decisions, batch rows&quot;]
    M5[&quot;drift_share gauge&quot;]
  end
  M1 --&gt; ME[&quot;GET /metrics&quot;]
  M2 --&gt; ME
  M3 --&gt; ME
  M4 --&gt; ME
  M5 --&gt; ME
  ME --&gt; PR[&quot;Prometheus&lt;br/&gt;deploy/prometheus.yml&quot;] --&gt; GR[&quot;Grafana dashboards&quot;]
  subgraph DRIFT[&quot;Input drift (Gate 8.7c)&quot;]
    REFP[(&quot;drift_reference.json&lt;br/&gt;frozen profile&lt;br/&gt;built by build_drift_reference.py&quot;)]
    RECENT[&quot;recent predictions from DB&quot;]
    PM[&quot;ProfileDriftMonitor&lt;br/&gt;KS, chi-square, PSI on scipy&quot;]
    REFP --&gt; PM
    RECENT --&gt; PM
    PM --&gt; ST[&quot;GET /drift/DISEASE/status&quot;]
    PM --&gt; RUN[&quot;POST /drift/DISEASE/run&quot;]
    PM --&gt; REP[&quot;GET /drift/DISEASE/report&quot;]
    RUN --&gt; M5
  end
  subgraph ML[&quot;MLflow&quot;]
    TR[&quot;mlflow_tracker&lt;br/&gt;experiment OmniDiag&quot;]
    TR --&gt; RUNS[&quot;GET /mlflow/runs&quot;]
    TR --&gt; RG[&quot;POST /mlflow/register&quot;]
  end
  RUN --&gt; TR
  RET[&quot;retrain candidates&quot;] --&gt; TR</pre></div></section>
<section id="er"><h2>١٠. نموذج البيانات (Database)</h2><p class="cap">ثمانية جداول؛ كل تنبؤ يُسجَّل مع مقياسه وعتبته، وطابور المراجعة مرتبط بالتنبؤ بعلاقة واحد لواحد.</p>
<div class="fig" dir="ltr"><pre class="mermaid">erDiagram
  USERS ||--o{ USER_ROLES : has
  ROLES ||--o{ USER_ROLES : grants
  USERS ||--o{ PATIENTS : created_by
  PATIENTS ||--o{ PATIENT_VISITS : has
  PATIENTS ||--o{ PREDICTIONS : linked
  USERS ||--o{ PREDICTIONS : created_by
  PREDICTIONS ||--o| REVIEW_QUEUE : flagged
  USERS ||--o{ REVIEW_QUEUE : reviewer
  USERS ||--o{ AUDIT_LOGS : actor
  USERS {
    string id PK
    string email
    string hashed_password
    string api_key_hash
    bool is_active
  }
  ROLES {
    int id PK
    string name
  }
  PATIENTS {
    string id PK
    string mrn UK
    string full_name
    date date_of_birth
    datetime deleted_at
  }
  PATIENT_VISITS {
    string id PK
    string patient_id FK
  }
  PREDICTIONS {
    string id PK
    string disease
    float confidence
    string probability_scale
    string decision
    float probability_lower
    float probability_upper
  }
  REVIEW_QUEUE {
    string id PK
    string prediction_id UK
    float uncertainty_score
    float decision_threshold
    string status
    int label
    string notes
  }
  AUDIT_LOGS {
    string id PK
    string endpoint
    string method
    int status_code
    string ip_address
    int duration_ms
  }</pre></div></section>
<section id="frontend"><h2>١١. الواجهة الأمامية (React SPA)</h2><p class="cap">الفورم يُبنى ديناميكياً من الـ schema القادم من الـ API (لا حقول مكتوبة يدوياً لكل مرض)، ولذلك تظهر الأمراض الجديدة بدون تعديل الواجهة.</p>
<div class="fig" dir="ltr"><pre class="mermaid">flowchart TB
  MAIN[&quot;main.jsx&quot;] --&gt; PROV[&quot;Providers: Auth, Disease, Theme&quot;]
  PROV --&gt; APP[&quot;App.jsx - DiseaseSelector, theme toggle, view tabs&quot;]
  APP --&gt; V1[&quot;Engineering Mode&quot;]
  APP --&gt; V2[&quot;Clinical EMR Mode&quot;]
  APP --&gt; V3[&quot;Batch Prediction (CSV, max 500 rows)&quot;]
  APP --&gt; V4[&quot;Patient Comparison&quot;]
  APP --&gt; V5[&quot;Review Queue (clinical roles)&quot;]
  APP --&gt; V6[&quot;AdminDashboard&quot;]
  subgraph SCH[&quot;Schema-driven form&quot;]
    H1[&quot;useDiseaseSchema&lt;br/&gt;GET /api/v4/DISEASE/schema&quot;]
    H2[&quot;schemaFieldParser + schemaToZod&quot;]
    H3[&quot;DynamicClinicalForm + SchemaFieldFactory&quot;]
    H4[&quot;ClinicalNotesInput (NLP prefill)&quot;]
    H1 --&gt; H2 --&gt; H3
    H4 --&gt; H3
  end
  V1 --&gt; SCH
  V2 --&gt; SCH
  subgraph RES[&quot;Result views&quot;]
    R1[&quot;ThresholdBar + decision&quot;]
    R2[&quot;ShapBarChart (useShapSnapshot)&quot;]
    R3[&quot;WhatIfScenarioCard&lt;br/&gt;counterfactuals.js + /counterfactuals&quot;]
    R4[&quot;ClinicalActionPlan&quot;]
    R5[&quot;ClinicalReportModal + PDFReport&quot;]
    R6[&quot;PatientRiskTimeline / PatientTimeline&quot;]
  end
  SCH --&gt; RES
  V5 --&gt; RL[&quot;ReviewLabelBox&lt;br/&gt;annotate or skip&quot;]
  APP --&gt; PWA[&quot;PWA: InstallPrompt, OfflineBanner&quot;]
  API[&quot;api.js - fetch wrapper, cookies&quot;] --- SCH
  API --- RES
  API --- V5
  API --- V6</pre></div></section>
<section id="deploy"><h2>١٢. النشر والبنية التحتية (Deployment / CI)</h2><p class="cap">نفس الصورة (image) تعمل محلياً وعلى Hugging Face Spaces وKubernetes. النماذج تُبنى أو تُحمَّل وقت بناء الصورة، فلا ملفات ثنائية في الريبو.</p>
<div class="fig" dir="ltr"><pre class="mermaid">flowchart LR
  DEV[&quot;Developer&quot;] --&gt; GH[&quot;GitHub repo&quot;]
  GH --&gt; CI[&quot;CI: .github/workflows/ci.yml&lt;br/&gt;1. pip install, import backend&lt;br/&gt;2. start API check&lt;br/&gt;3. npm ci, npm run build&quot;]
  GH --&gt; DF[&quot;Dockerfile&lt;br/&gt;python 3.13-slim&quot;]
  DF --&gt; BUILD[&quot;Build time:&lt;br/&gt;train_heart_glm.py (CSV sha256 verified)&lt;br/&gt;download NHANES bundle from HF Hub&lt;br/&gt;(sha256-checked)&quot;]
  BUILD --&gt; IMG[&quot;Image: model baked in&lt;br/&gt;port 7860, instant start&quot;]
  IMG --&gt; HF[&quot;Hugging Face Space&lt;br/&gt;yahyoha/omnidiag&quot;]
  IMG --&gt; K8[&quot;Kubernetes&lt;br/&gt;k8s/*.yaml, helm chart,&lt;br/&gt;HPA, ingress, secrets&quot;]
  IMG --&gt; DC[&quot;docker-compose&quot;]
  subgraph COMPOSE[&quot;docker-compose services&quot;]
    PG[(&quot;postgres 15&quot;)]
    RD[(&quot;redis 7&quot;)]
    BE[&quot;backend&quot;]
    MF[&quot;mlflow&quot;]
    PRM[&quot;prometheus&quot;]
    GRF[&quot;grafana&quot;]
    RTN[&quot;retrain (profile: retrain)&lt;br/&gt;scripts/retrain.py&quot;]
  end
  DC --&gt; COMPOSE
  BE --&gt; PG
  BE --&gt; RD
  BE --&gt; MF
  PRM --&gt; BE
  GRF --&gt; PRM
  RTN --&gt; PG
  RTN --&gt; MF
  FE[&quot;frontend build (vite dist)&quot;] --&gt; VC[&quot;Vercel&lt;br/&gt;SPA rewrite to index.html&quot;]
  VC --&gt;|&quot;HTTPS API calls&quot;| HF</pre></div></section>
<section id="fl"><h2>١٣. التعلّم الاتحادي (Federated Learning): الحالة الفعلية</h2><p class="cap">غير مُشغَّل في المنتج. النموذج الأولي (Flower + FedAvg + XGBoost) لا يستطيع التجميع، والبديل GLORE مُنفَّذ ومقاس في مستودع التجارب فقط.</p>
<div class="fig" dir="ltr"><pre class="mermaid">flowchart LR
  subgraph OLD[&quot;backend/federated/ (SUPERSEDED, not used)&quot;]
    O1[&quot;client.py&lt;br/&gt;pickle.dumps(XGBoost)&quot;]
    O2[&quot;aggregator.py&lt;br/&gt;Flower FedAvg&quot;]
    O3[&quot;add_dp_noise()&lt;br/&gt;never called, no epsilon&quot;]
    O1 --&gt;|&quot;opaque bytes, cannot be averaged&quot;| O2
    O2 -.-x O3
  end
  subgraph NEW[&quot;omnidiag_experiments (research only, not merged)&quot;]
    G0[&quot;4 sites: Cleveland, Hungarian,&lt;br/&gt;Switzerland, Long Beach VA&quot;]
    G1[&quot;each site computes locally&lt;br/&gt;gradient g and Hessian H&quot;]
    G2[&quot;coordinator sums g, H&lt;br/&gt;arrow-block Newton step&lt;br/&gt;per-site intercept, shared slopes&quot;]
    G3[&quot;7 iterations =&lt;br/&gt;centralised fit (error 2e-14)&quot;]
    G0 --&gt; G1 --&gt; G2 --&gt; G3
    G2 --&gt;|&quot;updated beta&quot;| G1
  end
  NEW --&gt; RES[&quot;Measured: LOSO AUC 0.779 vs 0.802 shipped (-0.023)&lt;br/&gt;no differential privacy, simulation on one machine&quot;]</pre></div></section>

<footer>مستخرَج من كود المستودع الفعلي: backend/main.py، router.py، model_backends، active_learning، nlp، llm، monitoring، db_models، configs، Dockerfile، docker-compose، frontend/src.</footer>
</div>

<script type="module">
import mermaid from "https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.esm.min.mjs";
const dark = window.matchMedia("(prefers-color-scheme: dark)").matches;
mermaid.initialize({startOnLoad:false, theme: dark ? "dark" : "default", securityLevel:"loose", flowchart:{htmlLabels:true, useMaxWidth:false}});
await mermaid.run({querySelector:"pre.mermaid"});
</script>
</body>
</html>