File size: 53,341 Bytes
2e0b5ad b44d190 2e0b5ad b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 21271dd b44d190 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 | ---
title: BP Monitoring Pipeline
emoji: 🩺
colorFrom: red
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
---
<div align="center">
# BP Monitoring Pipeline
### Clean Architecture & System Design Documentation
*Dokumen ini menjelaskan arsitektur, design patterns, dan keputusan teknis yang digunakan dalam pipeline data monitoring tekanan darah berbasis sinyal PPG (Photoplethysmography) dari perangkat IoT.*
---
**Status**: `Production-Ready Design` · **Versi**: `1.0.0` · **Terakhir Diperbarui**: `30 Mei 2026`
</div>
---
## 📑 Daftar Isi
- [1. Executive Summary](#1-executive-summary)
- [2. Architecture Overview](#2-architecture-overview)
- [2.1. High-Level System Architecture](#21-high-level-system-architecture)
- [2.2. Clean Architecture Layers](#22-clean-architecture-layers)
- [2.3. Dependency Rule](#23-dependency-rule)
- [3. Layer-by-Layer Breakdown](#3-layer-by-layer-breakdown)
- [3.1. Domain Layer](#31-domain-layer)
- [3.2. Application Layer](#32-application-layer)
- [3.3. Infrastructure Layer](#33-infrastructure-layer)
- [3.4. Interface Layer](#34-interface-layer)
- [3.5. Shared Module](#35-shared-module)
- [4. Design Principles](#4-design-principles)
- [4.1. SOLID Principles](#41-solid-principles)
- [4.2. DRY (Don't Repeat Yourself)](#42-dry-dont-repeat-yourself)
- [5. Design Patterns](#5-design-patterns)
- [6. Data Flow & ETL Pipeline](#6-data-flow--etl-pipeline)
- [6.1. ETL #1 — Data Ingestion](#61-etl-1--data-ingestion)
- [6.2. ETL #2 — Processing & Inference](#62-etl-2--processing--inference)
- [6.3. End-to-End Data Flow](#63-end-to-end-data-flow)
- [7. Project Structure](#7-project-structure)
- [8. Deployment Architecture](#8-deployment-architecture)
- [8.1. Deployment Topology](#81-deployment-topology)
- [8.2. Technology Stack](#82-technology-stack)
- [8.3. Entry Points](#83-entry-points)
- [9. Configuration Management](#9-configuration-management)
- [10. Testing Strategy](#10-testing-strategy)
- [11. Glossary](#11-glossary)
- [12. References](#12-references)
---
## 1. Executive Summary
**BP Monitoring Pipeline** adalah sistem data pipeline yang dirancang untuk memproses sinyal **PPG (Photoplethysmography)** dari perangkat IoT melalui smartphone, kemudian melakukan inferensi menggunakan model deep learning (**GAN** dan **VGTL-Net**) untuk memprediksi **ABP (Arterial Blood Pressure)** secara non-invasif.
### Tujuan Utama
| # | Tujuan | Deskripsi |
|---|--------|-----------|
| 1 | **Reliability** | Menjamin data PPG dari sensor IoT tidak hilang selama proses transmisi dan penyimpanan |
| 2 | **Scalability** | Arsitektur yang dapat berkembang seiring bertambahnya jumlah sensor dan pengguna |
| 3 | **Maintainability** | Codebase yang mudah dipahami, dimodifikasi, dan di-debug oleh seluruh anggota tim |
| 4 | **Replaceability** | Setiap komponen eksternal (database, message broker, model AI) dapat diganti tanpa mengubah business logic |
| 5 | **Testability** | Setiap layer dapat di-test secara independen melalui unit test dan integration test |
### Keputusan Arsitektural Utama
| Keputusan | Pilihan | Alasan |
|-----------|---------|--------|
| Architecture Style | Clean Architecture | Separation of concerns, dependency inversion, framework independence |
| Communication Pattern | Asynchronous Messaging (Message Queue) | Decoupling antara Data Node dan Model Node |
| Data Pipeline Pattern | ETL (Extract-Transform-Load) | Data harus di-transform sebelum masuk ke downstream (model dan database) |
| Deployment Strategy | Multi-platform (HF Spaces + Kaggle) | Cost optimization ($0) dengan memanfaatkan free tier |
---
## 2. Architecture Overview
### 2.1. High-Level System Architecture
Sistem terdiri dari **tiga node** utama yang berkomunikasi secara asinkron:
```
┌─────────────────────────────────────────────────────────────────────┐
│ SYSTEM ARCHITECTURE │
│ │
│ ┌──────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ IoT Node │───────→│ Data Node │───────→│ Model Node │ │
│ │ (Sensor │ PPG │ (ETL #1) │ Queue │ (ETL #2) │ │
│ │ + HP) │ Signal │ │ │ │ │
│ └──────────┘ │ ┌────────┐ │ │ ┌────────────┐ │ │
│ │ │FastAPI │ │ │ │ GAN │ │ │
│ │ │+ ETL │ │ │ │ + VGTL-Net │ │ │
│ │ └────────┘ │ │ └────────────┘ │ │
│ └──────┬───────┘ └────────┬─────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────────────────────┐ │
│ │ Database │ │
│ │ (Raw PPG + Predictions) │ │
│ └──────────────────────────────────┘ │
│ ▲ │
│ │ │
│ ┌──────┴───────┐ │
│ │ Application │ │
│ │ Node │ │
│ │ (Frontend + │ │
│ │ Backend) │ │
│ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
### 2.2. Clean Architecture Layers
Seluruh codebase diorganisasi ke dalam **empat layer konsentris** sesuai prinsip Clean Architecture oleh Robert C. Martin:
```
╔═══════════════════════════════════════════════════════╗
║ ║
║ ┌───────────────────────────────────────────┐ ║
║ │ INTERFACE LAYER │ ║
║ │ (FastAPI Routes, CLI, Consumer) │ ║
║ │ │ ║
║ │ ┌───────────────────────────────────┐ │ ║
║ │ │ INFRASTRUCTURE LAYER │ │ ║
║ │ │ (PostgreSQL, RabbitMQ, SciPy, │ │ ║
║ │ │ Model Service Implementations) │ │ ║
║ │ │ │ │ ║
║ │ │ ┌───────────────────────────┐ │ │ ║
║ │ │ │ APPLICATION LAYER │ │ │ ║
║ │ │ │ (Use Cases, DTOs) │ │ │ ║
║ │ │ │ │ │ │ ║
║ │ │ │ ┌─────────────────┐ │ │ │ ║
║ │ │ │ │ DOMAIN LAYER │ │ │ │ ║
║ │ │ │ │ (Entities, │ │ │ │ ║
║ │ │ │ │ Interfaces, │ │ │ │ ║
║ │ │ │ │ Value Objects)│ │ │ │ ║
║ │ │ │ └─────────────────┘ │ │ │ ║
║ │ │ └───────────────────────────┘ │ │ ║
║ │ └───────────────────────────────────┘ │ ║
║ └───────────────────────────────────────────┘ ║
║ ║
╚═══════════════════════════════════════════════════════╝
```
### 2.3. Dependency Rule
> **"Source code dependencies harus selalu mengarah KE DALAM (ke arah Domain Layer)."**
> — Robert C. Martin, *Clean Architecture* (2017)
```
Interface ──depends on──→ Application ──depends on──→ Domain
▲
Infrastructure ──────────────implements────────────────┘
```
**Implikasi penting:**
- Domain Layer **tidak mengetahui** keberadaan PostgreSQL, RabbitMQ, FastAPI, atau framework apapun.
- Application Layer (Use Cases) bergantung pada **interfaces** yang didefinisikan di Domain, bukan pada implementasi konkret.
- Infrastructure Layer **mengimplementasikan** interfaces Domain, sehingga bisa di-swap tanpa mengubah business logic.
- Interface Layer bertindak sebagai **entry point** — hanya melakukan wiring/composition.
---
## 3. Layer-by-Layer Breakdown
### 3.1. Domain Layer
**Lokasi**: `src/domain/`
Domain Layer adalah **inti dari sistem** — berisi business rules murni yang tidak bergantung pada teknologi apapun. Layer ini harus bisa berjalan **tanpa framework, tanpa database, tanpa network**.
#### 3.1.1. Entities
Entity merepresentasikan objek bisnis utama dengan identitas dan lifecycle.
| Entity | File | Deskripsi | Key Properties |
|--------|------|-----------|----------------|
| `PPGSignal` | `entities/ppg_signal.py` | Representasi sinyal PPG yang masuk dari sensor | `device_id`, `user_id`, `timestamp`, `sampling_rate`, `ppg_values`, `duration_seconds` |
| `BPPrediction` | `entities/prediction.py` | Hasil prediksi tekanan darah dari model AI | `predicted_sbp`, `predicted_dbp`, `model_version`, `inference_time_ms` |
**Karakteristik Entity:**
- Menggunakan `@dataclass` murni Python — **bukan** SQLAlchemy model, **bukan** Pydantic model.
- Berisi **domain validation** (`validate()`) yang merepresentasikan business rules.
- Berisi **computed properties** (e.g., `num_samples`, `mean_arterial_pressure`).
#### 3.1.2. Value Objects
Value Object adalah objek immutable yang didefinisikan oleh atributnya, bukan identitasnya.
| Value Object | File | Deskripsi |
|-------------|------|-----------|
| `DeviceInfo` | `value_objects/device_info.py` | Informasi perangkat sensor IoT |
| `SignalMetadata` | `value_objects/signal_metadata.py` | Metadata sinyal (sampling rate, durasi, jumlah sampel) |
#### 3.1.3. Interfaces (Contracts)
Interfaces mendefinisikan **kontrak** yang harus dipenuhi oleh Infrastructure Layer. Menggunakan Abstract Base Class (ABC).
| Interface | File | Kontrak |
|-----------|------|---------|
| `BaseRepository[T]` | `interfaces/repositories/base_repository.py` | Generic CRUD: `add()`, `get_by_id()`, `get_all()`, `delete()` |
| `PPGRepository` | `interfaces/repositories/ppg_repository.py` | Extends `BaseRepository[PPGSignal]` + `get_by_user()`, `get_by_device()` |
| `PredictionRepository` | `interfaces/repositories/prediction_repository.py` | Extends `BaseRepository[BPPrediction]` + query spesifik prediksi |
| `MessageBroker` | `interfaces/services/message_broker.py` | `publish()`, `consume()`, `connect()`, `disconnect()` |
| `SignalProcessor` | `interfaces/services/signal_processor.py` | `filter_signal()`, `normalize()`, `segment()`, `process()` |
| `ModelService` | `interfaces/services/model_service.py` | `load_model()`, `predict()` |
> **📌 Catatan Penting**: Interface `MessageBroker` tidak menyebut "RabbitMQ" dimanapun. Jika di kemudian hari ingin berpindah ke **Apache Kafka** atau **Redis Streams**, cukup buat implementasi baru tanpa mengubah satu baris pun di Domain atau Application Layer.
#### 3.1.4. Domain Exceptions
```
DomainException (base)
├── InvalidSignalError
├── PredictionOutOfRangeError
└── EntityNotFoundError
```
---
### 3.2. Application Layer
**Lokasi**: `src/application/`
Application Layer berisi **use cases** — orkestrasi langkah-langkah business logic untuk memenuhi satu kebutuhan pengguna yang spesifik.
#### 3.2.1. Use Cases
| Use Case | File | Deskripsi | Input → Output |
|----------|------|-----------|----------------|
| `IngestPPGUseCase` | `use_cases/ingest_ppg.py` | ETL #1: Terima PPG → Store → Publish | `PPGIngestRequest` → `PPGIngestResponse` |
| `ProcessAndPredictUseCase` | `use_cases/process_and_predict.py` | ETL #2: Consume → Preprocess → Predict → Store | `dict (message)` → `BPPrediction` |
| `GetPredictionHistoryUseCase` | `use_cases/get_prediction_history.py` | Query riwayat prediksi per user | `user_id, date_range` → `list[BPPrediction]` |
**Prinsip Use Case:**
- Setiap use case memiliki **satu method publik**: `execute()`.
- Dependencies di-inject melalui **constructor** (Dependency Inversion).
- Use case **tidak mengetahui** dari mana dia dipanggil (API? CLI? Consumer?).
```
┌──────────────────────────────────────────────────────────────┐
│ IngestPPGUseCase │
│ │
│ __init__(ppg_repo: PPGRepository, broker: MessageBroker) │
│ ▲ ▲ │
│ │ interface │ interface │
│ │ │ │
│ execute(request: PPGIngestRequest) → PPGIngestResponse │
│ 1. Create PPGSignal entity │
│ 2. entity.validate() │
│ 3. ppg_repo.add(entity) │
│ 4. broker.publish("ppg_queue", message) │
│ 5. Return response DTO │
└──────────────────────────────────────────────────────────────┘
```
#### 3.2.2. Data Transfer Objects (DTOs)
DTO digunakan untuk transfer data antar layer boundary. Menggunakan **Pydantic** untuk validasi otomatis.
| DTO | Tipe | Deskripsi |
|-----|------|-----------|
| `PPGIngestRequest` | Input | Data yang diterima dari HP via API |
| `PPGIngestResponse` | Output | Response setelah ingest berhasil |
| `PredictionResponse` | Output | Hasil prediksi untuk ditampilkan ke frontend |
> **Kenapa bukan Entity langsung?**
> Entity adalah representasi domain murni. DTO adalah representasi "bentuk data" yang masuk/keluar dari sistem. Memisahkan keduanya mencegah domain entity terkontaminasi oleh kebutuhan serialization/validation framework.
---
### 3.3. Infrastructure Layer
**Lokasi**: `src/infrastructure/`
Infrastructure Layer berisi **implementasi konkret** dari interfaces yang didefinisikan di Domain Layer. Layer ini berhubungan langsung dengan teknologi pihak ketiga.
#### 3.3.1. Database
| Komponen | File | Deskripsi |
|----------|------|-----------|
| `connection.py` | `database/connection.py` | SQLAlchemy async engine + session factory |
| `Base` | `database/models/base.py` | Declarative base dengan `id` dan `created_at` otomatis |
| `PPGModel` | `database/models/ppg_model.py` | ORM model untuk tabel `raw_ppg` |
| `PredictionModel` | `database/models/prediction_model.py` | ORM model untuk tabel `predictions` |
| `SQLAlchemyBaseRepository` | `database/repositories/sqlalchemy_base.py` | Generic CRUD implementation (DRY) |
| `SQLAlchemyPPGRepository` | `database/repositories/ppg_repository.py` | Implementasi `PPGRepository` |
| `SQLAlchemyPredictionRepository` | `database/repositories/prediction_repository.py` | Implementasi `PredictionRepository` |
**Mapping Pattern:**
```
Domain Entity (PPGSignal) ←─── _to_entity() ─── ORM Model (PPGModel)
─── _to_model() ───→
```
Setiap repository konkret wajib mengimplementasikan dua method mapping:
- `_to_entity(model) → entity` — Mengkonversi ORM model ke domain entity.
- `_to_model(entity) → model` — Mengkonversi domain entity ke ORM model.
#### 3.3.2. Messaging
| Komponen | File | Deskripsi |
|----------|------|-----------|
| `RabbitMQBroker` | `messaging/rabbitmq_broker.py` | Implementasi `MessageBroker` menggunakan `aio-pika` |
Mendukung koneksi **AMQP** (lokal) dan **AMQPS** (CloudAMQP — SSL) melalui konfigurasi URL.
#### 3.3.3. Signal Processing
| Komponen | File | Deskripsi |
|----------|------|-----------|
| `ScipySignalProcessor` | `processing/scipy_signal_processor.py` | Implementasi `SignalProcessor` menggunakan SciPy |
Pipeline pemrosesan sinyal:
```
Raw PPG Signal
│
▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Bandpass Filter │────→│ Normalization │────→│ Segmentation │
│ (0.5 – 8.0 Hz) │ │ (Z-score) │ │ (8s windows) │
│ Butterworth 4th │ │ │ │ │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│
▼
Processed Segments
shape: (N, window_size)
```
#### 3.3.4. Model Service
| Komponen | File | Deskripsi |
|----------|------|-----------|
| `MockModelService` | `model/mock_model_service.py` | Mock implementation untuk testing tanpa model asli |
| `GANVGTLNetService` | `model/gan_vgtlnet_service.py` | Implementasi production: GAN + VGTL-Net inference |
---
### 3.4. Interface Layer
**Lokasi**: `src/interface/`
Interface Layer adalah **entry point** ke sistem. Layer ini bertanggung jawab atas:
- Menerima input dari dunia luar (HTTP request, message queue, CLI command).
- Melakukan **dependency wiring** (composition root).
- Memanggil use case yang sesuai.
- Mengembalikan output dalam format yang diharapkan.
#### 3.4.1. API (Entry Point 1 — HF Spaces)
| Komponen | File | Deskripsi |
|----------|------|-----------|
| `create_app()` | `api/app.py` | FastAPI application factory |
| `dependencies.py` | `api/dependencies.py` | Dependency injection wiring menggunakan `Depends()` |
| `ppg_routes.py` | `api/routes/ppg_routes.py` | REST endpoint untuk ingest PPG |
| `prediction_routes.py` | `api/routes/prediction_routes.py` | REST endpoint untuk query predictions |
#### 3.4.2. Consumer (Entry Point 2 — Kaggle)
| Komponen | File | Deskripsi |
|----------|------|-----------|
| `run_consumer.py` | `consumer/run_consumer.py` | Standalone consumer service |
| `colab_notebook.ipynb` | `consumer/colab_notebook.ipynb` | Jupyter notebook untuk menjalankan consumer di Colab |
> **📌 Kedua entry point menggunakan use case, repository, dan interface yang PERSIS SAMA.** Perbedaannya hanya pada cara bootstrapping dan dependency wiring.
---
### 3.5. Shared Module
**Lokasi**: `src/shared/`
Cross-cutting concerns yang digunakan oleh seluruh layer.
| Komponen | File | Deskripsi |
|----------|------|-----------|
| `Settings` | `config.py` | Konfigurasi aplikasi dari environment variables (Pydantic Settings) |
| `get_logger()` | `logger.py` | Logging factory dengan format konsisten |
| `constants.py` | `constants.py` | Named constants (menghilangkan magic numbers) |
---
## 4. Design Principles
### 4.1. SOLID Principles
#### **S — Single Responsibility Principle**
> *"Sebuah class harus memiliki satu, dan hanya satu, alasan untuk berubah."*
| Komponen | Responsibility Tunggal |
|----------|----------------------|
| `PPGSignal` | Merepresentasikan sinyal PPG dan validasi domain-nya |
| `IngestPPGUseCase` | Mengorkestrasikan alur ingest data PPG |
| `ScipySignalProcessor` | Memproses sinyal menggunakan SciPy |
| `SQLAlchemyPPGRepository` | Mempersist PPGSignal ke PostgreSQL |
| `RabbitMQBroker` | Mengirim/menerima pesan via RabbitMQ |
| `ppg_routes.py` | Menangani HTTP routing untuk endpoint PPG |
#### **O — Open/Closed Principle**
> *"Entitas software harus terbuka untuk ekstensi, tertutup untuk modifikasi."*
| Skenario | Solusi (Tanpa Modifikasi) |
|----------|--------------------------|
| Ganti PostgreSQL → MongoDB | Buat `MongoDBPPGRepository` yang implement `PPGRepository`. Use case **tidak berubah**. |
| Ganti RabbitMQ → Kafka | Buat `KafkaBroker` yang implement `MessageBroker`. Use case **tidak berubah**. |
| Ganti SciPy → PyTorch preprocessing | Buat `TorchSignalProcessor` yang implement `SignalProcessor`. Use case **tidak berubah**. |
| Tambah model baru selain GAN+VGTL-Net | Buat `NewModelService` yang implement `ModelService`. Use case **tidak berubah**. |
#### **L — Liskov Substitution Principle**
> *"Objek dari superclass harus bisa digantikan oleh objek subclass tanpa merusak program."*
```python
# Keduanya bisa digunakan di mana pun PPGRepository dibutuhkan
repo: PPGRepository = SQLAlchemyPPGRepository(session) # ✅ Production
repo: PPGRepository = InMemoryPPGRepository() # ✅ Testing
# Keduanya bisa digunakan di mana pun MessageBroker dibutuhkan
broker: MessageBroker = RabbitMQBroker() # ✅ Production
broker: MessageBroker = InMemoryBroker() # ✅ Testing
```
#### **I — Interface Segregation Principle**
> *"Client tidak boleh dipaksa bergantung pada interface yang tidak mereka gunakan."*
```
┌──── PPGRepository ────┐
│ get_by_user() │
BaseRepository[T] │ get_by_device() │
add() └──────────────────────┘
get_by_id()
get_all() ┌── PredictionRepository ──┐
delete() │ get_by_user_latest() │
│ get_by_date_range() │
└──────────────────────────┘
```
- `IngestPPGUseCase` hanya bergantung pada `PPGRepository` dan `MessageBroker` — **bukan** pada `PredictionRepository` atau `SignalProcessor`.
- `ProcessAndPredictUseCase` bergantung pada `PPGRepository`, `PredictionRepository`, `SignalProcessor`, dan `ModelService` — masing-masing interface kecil dan fokus.
#### **D — Dependency Inversion Principle**
> *"Module high-level tidak boleh bergantung pada module low-level. Keduanya harus bergantung pada abstraksi."*
```
┌──────────────────┐ ┌──────────────────────┐
│ Use Case │ │ Interface (ABC) │
│ (High-Level) │──────→│ PPGRepository │
│ │ │ MessageBroker │
└──────────────────┘ └──────────┬───────────┘
│
│ implements
│
┌──────────▼───────────┐
│ Concrete Class │
│ (Low-Level) │
│ SQLAlchemyPPGRepo │
│ RabbitMQBroker │
└──────────────────────┘
```
Use case menerima dependency melalui **constructor injection**:
```python
class IngestPPGUseCase:
def __init__(
self,
ppg_repo: PPGRepository, # ← interface, bukan SQLAlchemyPPGRepository
broker: MessageBroker, # ← interface, bukan RabbitMQBroker
):
self._ppg_repo = ppg_repo
self._broker = broker
```
---
### 4.2. DRY (Don't Repeat Yourself)
| Penerapan | Lokasi | Sebelum (Redundant) | Sesudah (DRY) |
|-----------|--------|---------------------|---------------|
| Generic CRUD | `SQLAlchemyBaseRepository` | Setiap repository menulis `add()`, `get_by_id()`, `delete()` sendiri-sendiri | Tulis sekali di base class, semua repo inherit |
| ORM Base Model | `database/models/base.py` | Setiap ORM model mendefinisikan `id` dan `created_at` | Definisi di `Base`, semua model inherit |
| Magic Numbers | `shared/constants.py` | `0.5` dan `8.0` tersebar di banyak file | `PPG_BANDPASS_LOW = 0.5` didefinisikan sekali |
| Signal Processing Pipeline | `SignalProcessor.process()` | Urutan `filter → normalize → segment` ditulis ulang di setiap pemanggil | Template Method di base class: `process()` memanggil tiga langkah secara berurutan |
| Logger Creation | `shared/logger.py` | `logging.getLogger()` + handler setup di setiap file | `get_logger(__name__)` — factory function |
---
## 5. Design Patterns
| Pattern | Lokasi | Penggunaan |
|---------|--------|------------|
| **Repository Pattern** | `domain/interfaces/repositories/` | Abstraksi akses data, memisahkan domain dari persistence layer |
| **Factory Pattern** | `interface/api/app.py` → `create_app()` | Membuat instance FastAPI application dengan konfigurasi yang konsisten |
| **Dependency Injection** | `interface/api/dependencies.py` | Wiring dependencies menggunakan FastAPI `Depends()` |
| **Template Method** | `SignalProcessor.process()` | Mendefinisikan skeleton algoritma; subclass override langkah-langkah spesifik |
| **Data Transfer Object (DTO)** | `application/dto/` | Transfer data antar layer boundary tanpa mengekspos domain entity |
| **Data Mapper** | `_to_entity()` / `_to_model()` | Mapping antara domain entity dan ORM model |
| **Observer (Pub/Sub)** | `MessageBroker` + RabbitMQ | Publisher (ETL #1) dan Consumer (ETL #2) loosely coupled melalui message queue |
| **Strategy Pattern** | `SignalProcessor`, `ModelService` | Algoritma preprocessing dan model inference dapat di-swap melalui interface |
| **Singleton** | `_broker` di `dependencies.py` | Satu instance broker di-share ke seluruh request |
---
## 6. Data Flow & ETL Pipeline
### 6.1. ETL #1 — Data Ingestion
**Dijalankan di**: 🤗 Hugging Face Spaces
**Trigger**: HTTP POST request dari mobile app
```
┌─────────┐ ┌────────────────────────────────────────────────┐
│ Mobile │ │ ETL #1 (HF Spaces) │
│ App │ │ │
│ (HP) │─────→│ ┌───────────┐ ┌──────────────┐ ┌────────┐ │
│ │ POST │ │ EXTRACT │→ │ TRANSFORM │→ │ LOAD │ │
└─────────┘ │ │ │ │ │ │ │ │
│ │ Parse │ │ Create │ │ ┌────┐ │ │
│ │ JSON body │ │ PPGSignal │ │ │ DB │ │ │
│ │ Validate │ │ entity │ │ └────┘ │ │
│ │ DTO │ │ domain │ │ ┌────┐ │ │
│ │ │ │ validation │ │ │ MQ │ │ │
│ └───────────┘ └──────────────┘ │ └────┘ │ │
│ └────────┘ │
└────────────────────────────────────────────────┘
```
| Tahap | Aksi | Output |
|-------|------|--------|
| **Extract** | Parse JSON request body, validasi melalui Pydantic DTO | `PPGIngestRequest` object |
| **Transform** | Buat `PPGSignal` entity, jalankan `entity.validate()` | Validated `PPGSignal` entity |
| **Load** | `ppg_repo.add()` → Supabase; `broker.publish()` → CloudAMQP | Stored record + queued message |
### 6.2. ETL #2 — Processing & Inference
**Dijalankan di**: 🧪 Kaggle (GPU)
**Trigger**: Message baru di RabbitMQ queue
```
┌───────────┐ ┌─────────────────────────────────────────────────────┐
│ CloudAMQP │ │ ETL #2 (Kaggle) │
│ (Queue) │ │ │
│ │─────→│ ┌───────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ msg │ │ EXTRACT │→ │ TRANSFORM │→ │ LOAD │ │
└───────────┘ │ │ │ │ │ │ │ │
│ │ Consume │ │ Bandpass │ │ GAN predict │ │
│ │ message │ │ filter │ │ VGTL-Net │ │
│ │ Fetch PPG │ │ Normalize │ │ predict │ │
│ │ from DB │ │ Segment │ │ Store to DB │ │
│ └───────────┘ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────┘
```
| Tahap | Aksi | Output |
|-------|------|--------|
| **Extract** | Consume message dari queue, fetch full PPG signal dari database | `PPGSignal` entity |
| **Transform** | Bandpass filter → Z-score normalization → 8-second segmentation | `np.ndarray` shape `(N, window_size)` |
| **Load** | GAN inference → VGTL-Net inference → `prediction_repo.add()` | `BPPrediction` stored in database |
### 6.3. End-to-End Data Flow
```
📱 HP 🤗 HF Spaces 🐰 CloudAMQP 🧪 Kaggle ☁️ Supabase
│ │ │ │ │
│ POST /ppg/ingest │ │ │ │
├───────────────────────→│ │ │ │
│ │ │ │ │
│ │── Store raw PPG ──────────────────────────────────────────────→│
│ │ │ │ │
│ │── Publish message ───→│ │ │
│ │ │ │ │
│ ← 200 OK (response) │ │ │ │
│←───────────────────────│ │ │ │
│ │ │ │ │
│ │ │── Consume ─────────→│ │
│ │ │ │ │
│ │ │ │── Fetch raw PPG ──→│
│ │ │ │←── PPG data ───────│
│ │ │ │ │
│ │ │ │ [ETL #2] │
│ │ │ │ Filter → Norm │
│ │ │ │ → Segment │
│ │ │ │ │
│ │ │ │ [Model] │
│ │ │ │ GAN → VGTL-Net │
│ │ │ │ │
│ │ │ │── Store prediction→│
│ │ │ │ │
```
---
## 7. Project Structure
```
bp-monitoring-pipeline/
│
├── README.md # Project overview
├── ARCHITECTURE.md # ⬅ Dokumen ini
├── docker-compose.yml # Local development stack
├── docker-compose.prod.yml # Production stack
├── Dockerfile # HF Spaces deployment
├── pyproject.toml # Project metadata & dependencies
├── alembic.ini # Database migration config
├── .env.example # Environment variables template
│
├── src/
│ ├── __init__.py
│ │
│ ├── domain/ # 🟢 LAYER 1: Domain (Innermost)
│ │ ├── __init__.py #
│ │ ├── entities/ # Core business objects
│ │ │ ├── __init__.py #
│ │ │ ├── ppg_signal.py # PPG signal entity
│ │ │ └── prediction.py # Blood pressure prediction entity
│ │ ├── value_objects/ # Immutable value types
│ │ │ ├── __init__.py #
│ │ │ ├── device_info.py # IoT device information
│ │ │ └── signal_metadata.py # Signal metadata (rate, duration)
│ │ ├── interfaces/ # Abstract contracts (ABC)
│ │ │ ├── __init__.py #
│ │ │ ├── repositories/ # Data persistence contracts
│ │ │ │ ├── __init__.py #
│ │ │ │ ├── base_repository.py # Generic CRUD interface
│ │ │ │ ├── ppg_repository.py # PPG-specific queries
│ │ │ │ └── prediction_repository.py # Prediction-specific queries
│ │ │ └── services/ # External service contracts
│ │ │ ├── __init__.py #
│ │ │ ├── message_broker.py # Pub/Sub messaging
│ │ │ ├── signal_processor.py # Signal preprocessing
│ │ │ └── model_service.py # AI model inference
│ │ └── exceptions/ # Domain-specific errors
│ │ ├── __init__.py #
│ │ └── domain_exceptions.py # InvalidSignalError, etc.
│ │
│ ├── application/ # 🔵 LAYER 2: Application
│ │ ├── __init__.py #
│ │ ├── use_cases/ # Business use case orchestrators
│ │ │ ├── __init__.py #
│ │ │ ├── ingest_ppg.py # ETL #1: Ingest from mobile
│ │ │ ├── process_and_predict.py # ETL #2: Process + AI predict
│ │ │ └── get_prediction_history.py # Query historical predictions
│ │ └── dto/ # Data Transfer Objects
│ │ ├── __init__.py #
│ │ ├── ppg_dto.py # Request/Response for PPG
│ │ └── prediction_dto.py # Request/Response for predictions
│ │
│ ├── infrastructure/ # 🟠 LAYER 3: Infrastructure
│ │ ├── __init__.py #
│ │ ├── database/ # PostgreSQL implementation
│ │ │ ├── __init__.py #
│ │ │ ├── connection.py # Engine & session factory
│ │ │ ├── models/ # SQLAlchemy ORM models
│ │ │ │ ├── __init__.py #
│ │ │ │ ├── base.py # Shared base (id, created_at)
│ │ │ │ ├── ppg_model.py # raw_ppg table
│ │ │ │ └── prediction_model.py # predictions table
│ │ │ └── repositories/ # Concrete repository implementations
│ │ │ ├── __init__.py #
│ │ │ ├── sqlalchemy_base.py # Generic CRUD impl (DRY)
│ │ │ ├── ppg_repository.py # PPGRepository impl
│ │ │ └── prediction_repository.py # PredictionRepository impl
│ │ ├── messaging/ # RabbitMQ implementation
│ │ │ ├── __init__.py #
│ │ │ └── rabbitmq_broker.py # MessageBroker impl
│ │ ├── processing/ # Signal processing implementation
│ │ │ ├── __init__.py #
│ │ │ └── scipy_signal_processor.py # SignalProcessor impl (SciPy)
│ │ └── model/ # AI model implementation
│ │ ├── __init__.py #
│ │ ├── mock_model_service.py # Mock for testing
│ │ └── gan_vgtlnet_service.py # Production GAN + VGTL-Net
│ │
│ ├── interface/ # 🔴 LAYER 4: Interface (Outermost)
│ │ ├── __init__.py #
│ │ ├── api/ # Entry Point 1: REST API (HF Spaces)
│ │ │ ├── __init__.py #
│ │ │ ├── app.py # FastAPI application factory
│ │ │ ├── dependencies.py # Dependency injection wiring
│ │ │ └── routes/ # HTTP route handlers
│ │ │ ├── __init__.py #
│ │ │ ├── ppg_routes.py # POST /api/v1/ppg/ingest
│ │ │ └── prediction_routes.py # GET /api/v1/predictions
│ │ └── consumer/ # Entry Point 2: MQ Consumer (Colab)
│ │ ├── __init__.py #
│ │ ├── run_consumer.py # Standalone async consumer
│ │ └── colab_notebook.ipynb # Jupyter notebook for Colab
│ │
│ └── shared/ # 🟣 Cross-cutting Concerns
│ ├── __init__.py #
│ ├── config.py # Settings from .env (Pydantic)
│ ├── logger.py # Logging factory
│ └── constants.py # Named constants (no magic numbers)
│
├── tests/ # Test suite
│ ├── __init__.py #
│ ├── unit/ # Unit tests (no external deps)
│ │ ├── test_entities.py # Domain entity tests
│ │ ├── test_use_cases.py # Use case tests (mocked deps)
│ │ └── test_signal_processor.py # Signal processing tests
│ └── integration/ # Integration tests
│ ├── test_api.py # API endpoint tests
│ └── test_rabbitmq.py # Message broker tests
│
├── migrations/ # Alembic database migrations
│ ├── env.py #
│ └── versions/ #
│
└── scripts/ # Utility scripts
├── mock_iot_sender.py # Simulate IoT data
└── seed_database.py # Seed initial data
```
---
## 8. Deployment Architecture
### 8.1. Deployment Topology
```
┌──────────────────────────────────┐
│ GitHub Repository │
│ bp-monitoring-pipeline │
└───────────┬──────────┬───────────┘
│ │
git push │ │ git clone
(subset) │ │ (full repo)
│ │
┌──────────────────▼──┐ ┌────▼──────────────────┐
│ 🤗 Hugging Face │ │ 🧪 Kaggle │
│ Spaces │ │ (GPU: T4) │
│ │ │ │
│ ┌────────────────┐ │ │ ┌──────────────────┐ │
│ │ Docker │ │ │ │ run_consumer.py │ │
│ │ ┌──────────┐ │ │ │ │ │ │
│ │ │ FastAPI │ │ │ │ │ ETL #2 │ │
│ │ │ ETL #1 │ │ │ │ │ + GAN │ │
│ │ │ port 7860│ │ │ │ │ + VGTL-Net │ │
│ │ └──────────┘ │ │ │ └──────────────────┘ │
│ └────────────────┘ │ └────────────────────────┘
└───────┬────┬────────┘ │ │
│ │ │ │
┌──────────▼┐ ┌▼──────────┐ ┌─────▼──┐ ┌──▼───────┐
│ ☁️Supabase │ │🐰CloudAMQP│ │☁️Supa- │ │🐰Cloud- │
│ PostgreSQL│ │ RabbitMQ │ │ base │ │ AMQP │
│ (Store) │ │ (Publish) │ │ (R/W) │ │(Consume)│
└───────────┘ └────────────┘ └────────┘ └─────────┘
```
### 8.2. Technology Stack
| Layer | Teknologi | Versi | Justifikasi |
|-------|-----------|-------|-------------|
| **Runtime** | Python | 3.11+ | Async support, type hints, ecosystem ML matang |
| **Web Framework** | FastAPI | ≥ 0.110 | Async-native, auto-docs (Swagger), Pydantic integration |
| **ORM** | SQLAlchemy | ≥ 2.0 | Async support, mature, database-agnostic |
| **Database Driver** | asyncpg | ≥ 0.29 | Fastest Python PostgreSQL async driver |
| **Database** | PostgreSQL (Supabase) | 15 | JSONB support, reliability, free tier 500MB |
| **Message Broker** | RabbitMQ (CloudAMQP) | 3.x | AMQP protocol, persistent messages, free tier 1M msg/month |
| **Message Client** | aio-pika | ≥ 9.4 | Async RabbitMQ client, SSL support |
| **Signal Processing** | SciPy | ≥ 1.12 | Industry-standard scientific computing |
| **Deep Learning** | PyTorch | ≥ 2.0 | GPU inference, GAN + VGTL-Net compatibility |
| **Validation** | Pydantic | ≥ 2.0 | DTO validation, settings management |
| **Containerization** | Docker | — | Reproducible deployments |
| **Hosting (API)** | Hugging Face Spaces | Free | Docker SDK, 2 vCPU, 16GB RAM |
| **Hosting (GPU)** | Kaggle | Free | T4 GPU, 15GB VRAM |
### 8.3. Entry Points
| # | Entry Point | Platform | Command | Fungsi |
|---|-------------|----------|---------|--------|
| 1 | FastAPI Server | HF Spaces | `uvicorn src.interface.api.app:create_app` | Terima PPG dari HP, store, publish |
| 2 | MQ Consumer | Kaggle | `python -m src.interface.consumer.run_consumer` | Consume, preprocess, predict, store |
> **📌 Kedua entry point berbagi ~85% codebase yang sama.** Yang berbeda hanyalah bootstrapping dan dependency wiring.
---
## 9. Configuration Management
Seluruh konfigurasi dikelola melalui **environment variables** menggunakan Pydantic Settings. Tidak ada credentials yang di-hardcode.
### Environment Variables
| Variable | Deskripsi | Contoh | Wajib |
|----------|-----------|--------|:-----:|
| `DATABASE_URL` | PostgreSQL connection string (async) | `postgresql+asyncpg://user:pass@host:6543/db` | ✅ |
| `RABBITMQ_URL` | RabbitMQ/CloudAMQP connection string | `amqps://user:pass@host/vhost` | ✅ |
| `APP_PORT` | Port untuk FastAPI (default: 7860 untuk HF) | `7860` | ❌ |
| `APP_HOST` | Host binding (default: 0.0.0.0) | `0.0.0.0` | ❌ |
| `DEBUG` | Enable debug mode | `false` | ❌ |
| `LOG_LEVEL` | Logging level | `INFO` | ❌ |
### Secrets Management per Platform
| Platform | Metode | Lokasi |
|----------|--------|--------|
| HF Spaces | Space Secrets | Settings → Variables and Secrets |
| Kaggle | Colab Secrets | 🔑 Sidebar → Secrets |
| Local Development | `.env` file | Root project directory |
---
## 10. Testing Strategy
### Testing Pyramid
```
╱╲
╱ ╲ E2E Tests
╱ ╲ (curl → HF Spaces → Supabase → Colab)
╱──────╲
╱ ╲ Integration Tests
╱ ╲ (API + real DB, Consumer + real Queue)
╱────────────╲
╱ ╲ Unit Tests
╱ ╲ (Entities, Use Cases, Processors — mocked deps)
╱══════════════════╲
```
### Test Categories
| Kategori | Scope | Dependencies | Tools |
|----------|-------|-------------|-------|
| **Unit** | Domain entities, value objects | Tidak ada | `pytest` |
| **Unit** | Use cases | Mocked repositories & services | `pytest`, `unittest.mock` |
| **Unit** | Signal processor | Tidak ada (pure NumPy/SciPy) | `pytest`, `numpy.testing` |
| **Integration** | API endpoints | Test database (SQLite/PostgreSQL) | `pytest`, `httpx.AsyncClient` |
| **Integration** | Message broker | Test RabbitMQ (Docker) | `pytest`, `aio-pika` |
| **E2E** | Full pipeline | All services running | `curl`, `pytest` |
### Testability by Layer
| Layer | Testable Without External Deps? | Bagaimana |
|-------|:-------------------------------:|-----------|
| Domain | ✅ Ya | Pure Python — tidak ada dependency |
| Application | ✅ Ya | Inject mock repositories & services |
| Infrastructure | ⚠️ Perlu test DB/Queue | Gunakan testcontainers atau in-memory substitutes |
| Interface | ⚠️ Perlu running app | Gunakan `TestClient` (FastAPI) |
---
## 11. Glossary
| Istilah | Definisi |
|---------|----------|
| **PPG** | Photoplethysmography — teknik optik non-invasif untuk mengukur perubahan volume darah dalam jaringan mikrovaskular |
| **ABP** | Arterial Blood Pressure — tekanan darah arteri yang diprediksi oleh model |
| **SBP** | Systolic Blood Pressure — tekanan darah saat jantung berkontraksi (angka atas) |
| **DBP** | Diastolic Blood Pressure — tekanan darah saat jantung berelaksasi (angka bawah) |
| **GAN** | Generative Adversarial Network — arsitektur deep learning untuk mentranslasikan sinyal PPG ke ECG |
| **VGTL-Net** | Model deep learning yang menerima PPG + ECG untuk memprediksi ABP |
| **ETL** | Extract-Transform-Load — pola pipeline data |
| **Entity** | Objek domain dengan identitas unik dan lifecycle |
| **Value Object** | Objek domain immutable yang didefinisikan oleh atributnya |
| **DTO** | Data Transfer Object — objek untuk transfer data antar layer boundary |
| **Repository** | Abstraksi akses data yang menyembunyikan detail persistence |
| **Use Case** | Orkestrasi satu unit business logic yang spesifik |
| **Message Broker** | Middleware yang mengelola antrian pesan antara producer dan consumer |
| **Dependency Injection** | Teknik menyediakan dependency dari luar, bukan membuat di dalam class |
---
## 12. References
| # | Referensi | Penulis |
|---|-----------|---------|
| 1 | *Clean Architecture: A Craftsman's Guide to Software Structure and Design* | Robert C. Martin (2017) |
| 2 | *Design Patterns: Elements of Reusable Object-Oriented Software* | Gamma, Helm, Johnson, Vlissides (1994) |
| 3 | *Patterns of Enterprise Application Architecture* | Martin Fowler (2002) |
| 4 | *Domain-Driven Design: Tackling Complexity in the Heart of Software* | Eric Evans (2003) |
| 5 | FastAPI Documentation | https://fastapi.tiangolo.com |
| 6 | SQLAlchemy 2.0 Documentation | https://docs.sqlalchemy.org |
| 7 | RabbitMQ Tutorials | https://www.rabbitmq.com/tutorials |
| 8 | Hugging Face Spaces Docker SDK | https://huggingface.co/docs/hub/spaces-sdks-docker |
---
<div align="center">
---
*Dokumen ini di-maintain oleh Data Engineering Team.*
*Untuk pertanyaan, silakan buat issue di repository GitHub.*
**© 2026 BP Monitoring Pipeline — All rights reserved.**
</div>
|