docs/architecture/05-dataflow-diagram.md

05 — Data Flow Diagram (DFD)

Overview

This diagram traces how data flows through the BAYAN system — from user input to final storage, through all NLP processing stages.

Level 0 — Context Diagram

graph LR
    USER["👤 User"] -->|"Arabic Text"| BAYAN["🔵 BAYAN System"]
    BAYAN -->|"Corrections · Summaries · Documents"| USER
    BAYAN <-->|"Auth · Data"| SUPABASE["🗄️ Supabase"]
    BAYAN <-->|"Models"| HUGGINGFACE["🤗 HuggingFace"]

Level 1 — System DFD

graph TD
    subgraph "Data Sources"
        INPUT["📝 User Input Text"]
        IMPORT["📥 Imported File<br/>(TXT / DOCX / PDF)"]
    end

    subgraph "1.0 Text Correction Pipeline"
        SPELL["1.1 AraSpell<br/>Spelling Correction"]
        GRAMMAR["1.2 Grammar Check<br/>Bayan Grammar"]
        PUNCT["1.3 Punctuation<br/>PuncAra-v1"]
        MERGE["1.4 Merge Results<br/>Offset Mapping"]
    end

    subgraph "2.0 Display Pipeline"
        HIGHLIGHT["2.1 Error Highlighting<br/>renderer.js"]
        SIDEBAR["2.2 Suggestions Panel<br/>ui.js"]
        SCORE["2.3 Writing Score<br/>Calculation"]
        STATS["2.4 Editor Stats<br/>Words · Chars · Sentences"]
    end

    subgraph "3.0 Summarization Pipeline"
        SUMM_IN["3.1 Extract Clean Text"]
        SUMM_MODEL["3.2 MBart Summarization"]
        SUMM_OUT["3.3 Summary Display"]
    end

    subgraph "4.0 AutoComplete Pipeline"
        AC_IN["4.1 Cursor Context"]
        AC_MODEL["4.2 AutoComplete Model"]
        AC_OUT["4.3 Dropdown Suggestions"]
    end

    subgraph "5.0 Storage"
        LOCAL["5.1 localStorage<br/>Draft · Dismissed Words"]
        DOCUMENTS_DB["5.2 Supabase: documents"]
        SUMMARIES_DB["5.3 Supabase: summaries"]
        SETTINGS_DB["5.4 Supabase: settings"]
        PROFILES_DB["5.5 Supabase: profiles"]
    end

    subgraph "6.0 Export"
        EXPORT_TXT["6.1 Export TXT"]
        EXPORT_DOCX["6.2 Export DOCX"]
        EXPORT_PDF["6.3 Export PDF"]
    end

    INPUT --> SPELL
    IMPORT --> SPELL
    SPELL -->|"Corrected tokens"| GRAMMAR
    GRAMMAR -->|"Grammar-checked text"| PUNCT
    PUNCT -->|"Punctuated text"| MERGE
    MERGE -->|"All suggestions + offsets"| HIGHLIGHT
    MERGE --> SIDEBAR
    MERGE --> SCORE
    INPUT --> STATS

    INPUT --> SUMM_IN
    SUMM_IN --> SUMM_MODEL
    SUMM_MODEL --> SUMM_OUT
    SUMM_OUT --> SUMMARIES_DB

    INPUT --> AC_IN
    AC_IN --> AC_MODEL
    AC_MODEL --> AC_OUT

    INPUT --> LOCAL
    LOCAL -->|"Sync"| DOCUMENTS_DB

    INPUT --> EXPORT_TXT
    INPUT --> EXPORT_DOCX
    INPUT --> EXPORT_PDF

    style SPELL fill:#EF4444,color:#fff
    style GRAMMAR fill:#F59E0B,color:#000
    style PUNCT fill:#3B82F6,color:#fff
    style SUMM_MODEL fill:#8B5CF6,color:#fff
    style AC_MODEL fill:#10B981,color:#fff

Data Stores Summary

Store	Type	Data	Access Pattern
localStorage	Client-side	Draft HTML, dismissed words, word goal, theme	Read/write on every input
documents	Supabase	id, user_id, title, content, timestamps	CRUD per user (RLS)
summaries	Supabase	id, document_id, user_id, original, summary	Append-mostly
settings	Supabase	id, user_id, preferences JSON	Read on login, write on change
profiles	Supabase	id, display_name, avatar_url, auth_provider	Auto-created on signup

Data Transformation Chain

Raw Arabic Text
  │
  ├──→ AraSpell ──→ Corrected words (with alternatives)
  │         │
  │         ▼
  │    Grammar Engine ──→ Grammar errors (with suggestions)
  │         │
  │         ▼
  │    Punctuation ──→ Punctuation insertions
  │         │
  │         ▼
  │    Offset Mapper ──→ Maps corrected offsets → original offsets
  │         │
  │         ▼
  │    Unified Suggestions Array
  │         │
  │         ├──→ Error Spans (renderer.js)
  │         ├──→ Sidebar Cards (ui.js)
  │         └──→ Score Calculation
  │
  ├──→ AutoComplete Model ──→ Top-K word suggestions
  │
  └──→ MBart Summarizer ──→ Compressed summary text