refactored project structure. renamed scheduler dir to src

docs/README.md

@@ -0,0 +1,29 @@
+### Scheduler Documentation (Core Algorithm and Surrounding Modules)
+
+This documentation explains the end‑to‑end flow for the scheduler and its supporting modules, and provides detailed per‑directory guides that document individual files. Start with the high‑level flowchart, then dive into the directory guides.
+
+- High‑level flowchart: see `docs/scheduler_flowchart.md`
+- Directory guides (each includes per‑file details):
+  - `docs/core.md` — core scheduler domain and algorithm
+  - `docs/simulation.md` — discrete‑event simulation and policies
+  - `docs/control.md` — manual overrides and explainability surfaces
+  - `docs/dashboard.md` — Streamlit dashboard and pages
+  - `docs/data.md` — data loaders, defaults and configuration
+  - `docs/metrics.md` — basic metrics utilities
+  - `docs/monitoring.md` — ripeness monitoring (not yet integrated)
+  - `docs/utils.md` — utilities (calendar)
+
+Related (outside current scope but referenced):
+- Outputs (cause lists, reports) live under `scheduler/output/` and `outputs/`.
+
+#### Quick Start Reading Order
+1) `scheduler_flowchart.md` (overview)  
+2) `core.md` (domain + algorithm)  
+3) `simulation.md` (how scenarios are evaluated at scale)  
+4) `control.md` (how human inputs override the algorithm)  
+5) `dashboard.md` (how the UI wires it together)
+
+#### Definitions at a Glance
+- Case ripeness: classification indicating whether a case is ready to be scheduled and with what priority. Implemented in `scheduler/core/ripeness.py`.
+- Scheduling algorithm: orchestrates filtering, prioritization, overrides, and courtroom allocation. Implemented in `scheduler/core/algorithm.py`.
+- Simulation: forward model to evaluate policy performance over time. Implemented in `scheduler/simulation/engine.py` and related modules.

docs/control.md

@@ -0,0 +1,23 @@
+### Control Module — Overrides and Explainability
+
+Directory: `scheduler/control/`
+
+This package houses the manual control plane that allows judges or operators to override algorithmic decisions and to inspect the reasoning behind suggestions.
+
+#### Files overview
+
+- `overrides.py`
+  - Purpose: Define override types, manage override drafts per cause list, validate changes, and collect judge preferences and audit trails.
+  - Key elements:
+    - `OverrideType` (Enum): categories like ripeness override, capacity change, add/remove case.
+    - `Override`: a single change request with metadata; `to_dict()`, `to_readable_text()` helpers.
+    - `JudgePreferences`: persistent preferences per judge; `to_dict()`.
+    - `CauseListDraft`: a draft editable list for a given date/courtroom/judge; exposes `get_acceptance_rate()` and `get_modifications_summary()`.
+    - `OverrideValidator`: validates requested changes; surface `get_errors()`; targeted validate methods for each override kind (ripeness, capacity, add/remove case).
+    - `OverrideManager`: creates drafts, applies overrides, finalizes drafts, fetches preferences, produces statistics, and exports an audit trail.
+  - Interactions: Consumed by `core/algorithm.py` to apply judge preferences and manual modifications before final allocation.
+
+- `explainability.py`
+  - Purpose: Human‑readable explanations for why cases were selected or not.
+  - Typical content: utilities that convert ripeness/eligibility checks and policy scores into readable reasons for UI display and audit logs.
+  - Interactions: Used by dashboard pages and potentially during `SchedulingAlgorithm.schedule_day` explanation generation.

docs/core.md

@@ -0,0 +1,71 @@
+### Core Module — Domain and Scheduling Algorithm
+
+Directory: `scheduler/core/`
+
+This package defines the domain objects (cases, hearings, courtrooms, judges), the ripeness classifier, policy interfaces, and the main scheduling algorithm that orchestrates daily scheduling.
+
+#### Files overview
+
+- `algorithm.py`
+  - Purpose: Implements the daily scheduling pipeline.
+  - Key classes:
+    - `SchedulingResult`: Aggregates scheduled cases, unscheduled reasons, applied overrides, and explanations.
+    - `SchedulingAlgorithm`: Orchestrates the day’s scheduling.
+      - `schedule_day(cases, courtrooms, current_date, overrides, preferences, max_explanations_unscheduled)`
+        - Flow:
+          1) Compute judge preference overrides (if provided).
+          2) Filter by ripeness (via `RipenessClassifier`), tracking unscheduled reasons for unripe cases.
+          3) Filter eligibility (age gaps, disposed, same‑day constraints) and compute priority.
+          4) Apply manual overrides (add/remove cases; capacity changes).
+          5) Allocate cases to courtrooms (via allocator interface used in simulation layer).
+          6) Generate explanations for unscheduled cases (up to cap) and successful allocations.
+        - Inputs: list of `Case`, list of `Courtroom`, `date`, optional `Override` list, optional `JudgePreferences`.
+        - Outputs: `SchedulingResult` with scheduled list and explanations.
+      - Internal helpers: `_filter_by_ripeness`, `_filter_eligible`, `_get_preference_overrides`, `_apply_manual_overrides`, `_allocate_cases`, `_clear_temporary_case_flags`.
+  - Interactions: `core.ripeness`, `control.overrides`, `simulation.allocator`.
+
+- `case.py`
+  - Purpose: Core case model and priority logic.
+  - Key classes:
+    - `CaseStatus`: Enum of lifecycle statuses.
+    - `Case`: Fields describe type, stage, history, last hearing, etc.
+      - Selected methods: `progress_to_stage`, `record_hearing`, `update_age`, `compute_readiness_score`, `is_ready_for_scheduling`, `needs_alert`, `get_priority_score`, `mark_unripe`, `mark_ripe`, `mark_scheduled`, `is_disposed`, `to_dict`.
+  - Interactions: Used by ripeness classifier, scheduler, and simulation.
+
+- `courtroom.py`
+  - Purpose: Represents courtroom capacity and scheduling slots.
+  - Typical members: courtroom identifier, capacity/availability for a given `date`.
+  - Interactions: Allocator consumes courtroom capacity when placing cases.
+
+- `hearing.py`
+  - Purpose: Hearing record representation (date, outcome, adjournment, etc.).
+  - Interactions: Case history and scheduler eligibility checks (min gap, outcomes).
+
+- `judge.py`
+  - Purpose: Judge metadata and optional preference hooks.
+  - Interactions: `control.overrides.JudgePreferences` integrates with algorithm preference overrides.
+
+- `policy.py`
+  - Purpose: Defines the scheduling policy interface and default implementations for prioritization heuristics.
+  - Interactions: Injected into `SchedulingAlgorithm` to determine case ordering.
+
+- `ripeness.py`
+  - Purpose: Ripeness classifier and thresholds.
+  - Key elements:
+    - `RipenessStatus` enum with helpers `is_ripe()` and `is_unripe()`.
+    - `RipenessClassifier` with methods:
+      - `classify(case, current_date)`: returns `RipenessStatus` and sets reasons.
+      - `get_ripeness_priority(case, current_date)`: numeric priority for scheduling.
+      - `is_schedulable(case, current_date)`: boolean gate for eligibility.
+      - `get_ripeness_reason(status)`: human‑readable reason strings.
+      - `estimate_ripening_time(case, current_date)`: expected time until ready.
+      - `set_thresholds(new_thresholds)`, `get_current_thresholds()`.
+  - Interactions: Used both by `algorithm.py` and the simulation engine.
+
+#### Core flow (within `SchedulingAlgorithm.schedule_day`)
+1) Collect inputs (cases, courtrooms, date, optional overrides/preferences).  
+2) Ripeness filter: mark unripe with reasons; compute ripeness‑based priority.  
+3) Eligibility filter: ensure min gap since last hearing, exclude disposed, enforce capacity constraints.  
+4) Apply overrides (manual changes and judge preferences).  
+5) Allocate to courtrooms; finalize schedule.  
+6) Produce explanations and a `SchedulingResult`.

docs/dashboard.md

@@ -0,0 +1,47 @@
+### Dashboard Module — Streamlit App and Pages
+
+Directory: `scheduler/dashboard/`
+
+The dashboard provides a Streamlit UI to explore data, inspect the ripeness classifier, run simulations, configure overrides, and view analytics/reports.
+
+#### Files overview
+
+- `app.py`
+  - Purpose: Streamlit entrypoint that initializes navigation, shared state, and page routing.
+  - Interactions: Uses helpers under `dashboard/utils/` to load data and run simulations; mounts pages under `dashboard/pages/`.
+
+- `__init__.py`
+  - Purpose: Package initialization for the dashboard module.
+
+- `pages/1_Data_And_Insights.py`
+  - Purpose: Data loading, cleaning summaries, EDA‑style insights and figures.
+  - Inputs: parquet/CSV datasets and computed reports in `reports/figures/*`.
+
+- `pages/2_Ripeness_Classifier.py`
+  - Purpose: Visualize and test `core.ripeness.RipenessClassifier` on sample cases; display reasons and thresholds.
+  - Interactions: Can call into `RipenessClassifier.get_current_thresholds()` and `classify()`.
+
+- `pages/3_Simulation_Workflow.py`
+  - Purpose: Configure and run simulation scenarios; display outputs (events, metrics, summaries).
+  - Interactions: `dashboard/utils/simulation_runner.py`; backend `scheduler/simulation/engine.py`.
+
+- `pages/4_Cause_Lists_And_Overrides.py`
+  - Purpose: Generate draft cause lists for a date; review/apply overrides; export finalized lists.
+  - Interactions: `control/overrides.py` for `OverrideManager` and `CauseListDraft`.
+
+- `pages/6_Analytics_And_Reports.py`
+  - Purpose: Aggregate analytics, charts, and report downloads; links to `reports/figures/*` and `outputs/simulation_runs/*`.
+
+- `utils/__init__.py`
+  - Purpose: Package initialization for dashboard utilities.
+
+- `utils/data_loader.py`
+  - Purpose: Load datasets and parameter files for use in the UI; cache artifacts for responsiveness.
+  - Inputs: `Data/`, `reports/figures/*/params/*.csv`, and defaults in `scheduler/data/defaults/*`.
+
+- `utils/simulation_runner.py`
+  - Purpose: Convenience layer to assemble `CourtSimConfig`, run the simulation, and collect outputs for display/download.
+  - Interactions: `scheduler/simulation/engine.py` and policy modules.
+
+- `utils/ui_input_parser.py`
+  - Purpose: Parse and validate user inputs from Streamlit widgets into typed configs/overrides.

docs/data.md

@@ -0,0 +1,34 @@
+### Data Module — Parameters, Defaults, and Case Generation
+
+Directory: `scheduler/data/`
+
+This package is responsible for loading configuration/parameters, exposing default parameter tables, and (optionally) generating synthetic cases. These inputs are used by both the core scheduler and the simulation engine.
+
+#### Files overview
+
+- `config.py`
+  - Purpose: Data/config structures and helpers to read configuration (e.g., TOML under `configs/`).
+  - Typical contents: typed config models, IO utilities to load/validate values, and a central place to map file paths to parameter tables.
+  - Interactions: Consumed by dashboard utilities, simulation runner, and EDA.
+
+- `param_loader.py`
+  - Purpose: Load parameter tables required by the scheduler and simulation from CSV/JSON defaults or project artifacts.
+  - Inputs (defaults): files under `scheduler/data/defaults/`, including:
+    - `adjournment_proxies.csv` — signals/statistics used to approximate adjournment likelihood.
+    - `case_type_summary.csv` — frequencies and basic properties by case type.
+    - `court_capacity_global.json` — nominal/maximum capacity settings.
+    - `stage_duration.csv` — typical durations per stage.
+    - `stage_transition_entropy.csv` — transition uncertainty by stage.
+    - `stage_transition_probs.csv` — Markov transition probabilities between stages.
+  - Outputs: in‑memory DataFrames/objects consumed by `core` and `simulation`.
+
+- `case_generator.py`
+  - Purpose: Produce synthetic cases consistent with the parameter distributions for use in simulations or demos.
+  - Interactions: Reads parameters via `param_loader.py`; emits `core/Case` instances ready for ripeness evaluation and scheduling.
+
+- `__init__.py`
+  - Purpose: Package initialization; may expose convenience imports for loader/generator utilities.
+
+#### Data sources
+- Project data: `Data/*.parquet` and derived artifacts in `reports/figures/*` (e.g., cleaned parquet, params CSVs).
+- Defaults: `scheduler/data/defaults/*` bundled with the package as fallback/configuration baselines.

docs/metrics.md

@@ -0,0 +1,14 @@
+### Metrics Module — Basic Metrics Utilities
+
+Directory: `scheduler/metrics/`
+
+This package provides lightweight metrics helpers used by simulation runs and analyses.
+
+#### Files overview
+
+- `basic.py`
+  - Purpose: Compute aggregate metrics from simulation or scheduling outputs (e.g., throughput, utilization, wait times, adjournment rates).
+  - Typical usage: Imported by simulation/reporting code to summarize daily/event logs into CSVs found under `outputs/simulation_runs/*/metrics.csv`.
+
+- `__init__.py`
+  - Purpose: Package initialization; may re‑export common helpers.

docs/monitoring.md

@@ -0,0 +1,24 @@
+### Monitoring Module — Ripeness Tracking and Calibration
+
+Directory: `scheduler/monitoring/`
+
+This package provides components to track ripeness prediction outcomes and calibrate thresholds based on observed accuracy. It is currently not wired into the main scheduling flow but is designed for future integration.
+
+#### Files overview
+
+- `ripeness_metrics.py`
+  - Purpose: Record per‑case ripeness predictions and later outcomes to compute accuracy statistics and error types.
+  - Key elements:
+    - `RipenessPrediction` (dataclass): captures `case_id`, `predicted_status`, `prediction_date`, and later the `actual_outcome`, `was_adjourned`, and `outcome_date`.
+    - `RipenessMetrics`:
+      - `record_prediction(case_id, predicted_status, prediction_date)` — log a prediction when the classifier is invoked.
+      - `record_outcome(case_id, actual_outcome, was_adjourned, outcome_date)` — attach ground truth when available.
+      - Export utilities to summarize false positives/negatives and overall accuracy; convenience frame exports via pandas.
+  - Interactions: Expects callers (e.g., the scheduler) to log predictions and later outcomes; can write CSV summaries for analysis.
+
+- `ripeness_calibrator.py`
+  - Purpose: Use aggregated metrics to suggest or set revised ripeness thresholds that improve real‑world performance.
+  - Typical responsibilities: fit threshold adjustments to minimize misclassification costs; produce human‑readable calibration reports and recommended parameter deltas consumable by `core/ripeness.py` via `RipenessClassifier.set_thresholds()`.
+
+- `__init__.py`
+  - Purpose: Package initialization; may expose convenience imports for metrics/calibration.

docs/scheduler_flowchart.md

@@ -0,0 +1,114 @@
+### Scheduler End-to-End Flowchart
+
+This document shows the high-level flow across modules using Mermaid. It covers input data preparation, ripeness classification, scheduling, overrides, allocation, outputs, and optional simulation and dashboard flows.
+
+```mermaid
+flowchart TD
+
+%% Inputs
+subgraph Data_Layer
+  DCFG[configs/*.toml]
+  DLOAD[scheduler/data/param_loader.py<br/>scheduler/data/config.py]
+  DDEFS[scheduler/data/defaults/*]
+  DCASES[Data/cases.parquet<br/>reports/figures/*/cases_clean.parquet]
+  DHEAR[Data/hearings.parquet<br/>reports/figures/*/hearings_clean.parquet]
+end
+
+%% Core Domain
+subgraph Core_Domain
+  CASE[scheduler/core/case.py]
+  RIPEN[scheduler/core/ripeness.py]
+  HEAR[scheduler/core/hearing.py]
+  ROOM[scheduler/core/courtroom.py]
+  JUDGE[scheduler/core/judge.py]
+  POLICY[scheduler/core/policy.py]
+  ALGO[scheduler/core/algorithm.py]
+end
+
+%% Control
+subgraph Control_and_Overrides
+  OV[scheduler/control/overrides.py]
+  EXP[scheduler/control/explainability.py]
+end
+
+%% Allocation/Simulation
+subgraph Simulation_Engine
+  SIM[scheduler/simulation/engine.py]
+  ALLOC[scheduler/simulation/allocator.py]
+  POLS[scheduler/simulation/policies/*]
+end
+
+%% UI
+subgraph Dashboard_UI
+  APP[scheduler/dashboard/app.py]
+  PAGES[scheduler/dashboard/pages/*]
+  DUTIL[scheduler/dashboard/utils/*]
+end
+
+%% Metrics and Monitoring
+subgraph Metrics_and_Monitoring
+  MET[scheduler/metrics/basic.py]
+  MONCAL[scheduler/monitoring/ripeness_calibrator.py]
+  MONMET[scheduler/monitoring/ripeness_metrics.py]
+end
+
+%% Outputs
+subgraph Outputs
+  OUTCL[scheduler/output/cause_list.py]
+  OUTFILES[outputs/simulation_runs/*<br/>reports/*]
+end
+
+%% Data flow
+DCFG --> DLOAD
+DDEFS --> DLOAD
+DCASES --> DLOAD
+DHEAR --> DLOAD
+
+DLOAD --> CASE
+CASE --> RIPEN
+RIPEN --> ALGO
+HEAR --> ALGO
+ROOM --> ALGO
+JUDGE --> ALGO
+POLICY --> ALGO
+
+%% Scheduling path for a real day
+ALGO -->|filters by ripeness| RIPEN
+ALGO -->|eligibility and priority| CASE
+ALGO -->|manual overrides| OV
+ALGO -->|explanations| EXP
+ALGO -->|allocation| ALLOC
+ALLOC --> OUTCL
+OUTCL --> OUTFILES
+
+%% Simulation path (optional)
+DLOAD --> SIM
+SIM --> RIPEN
+SIM --> POLS
+SIM --> ALLOC
+SIM --> MET
+SIM --> OUTFILES
+
+%% Dashboard path
+DUTIL --> APP
+APP --> PAGES
+PAGES --> ALGO
+PAGES --> SIM
+PAGES --> OUTFILES
+PAGES --> EXP
+PAGES --> OV
+
+%% Monitoring (future integration)
+ALGO -. record predictions .-> MONMET
+MONMET -. calibrate thresholds .-> MONCAL
+MONCAL -. update ripeness thresholds .-> RIPEN
+```
+
+#### Narrative
+1) Data layer assembles parameters and input datasets via `param_loader.py` and `config.py`, pulling defaults from `scheduler/data/defaults/*` and optionally real case/hearing data.
+2) Core domain models (`case.py`, `hearing.py`, `courtroom.py`, `judge.py`) define the state. Ripeness (`ripeness.py`) classifies cases as ripe/unripe and provides reasons and thresholds.
+3) The scheduling algorithm (`algorithm.py`) orchestrates: ripeness filtering, priority computation, manual overrides, and courtroom allocation. Explanations are available through `control/explainability.py`.
+4) Allocations produce cause lists (`output/cause_list.py`) and artifacts.
+5) The simulation engine (`simulation/engine.py`) uses the same domain and policies to stress‑test scheduling strategies at scale, producing metrics and reports.
+6) The dashboard (`scheduler/dashboard/*`) provides a UI to run data explorations, inspect ripeness, run simulations, and export cause lists.
+7) Monitoring components track classification accuracy and enable future threshold calibration; currently not wired into the live flow.

docs/simulation.md

@@ -0,0 +1,40 @@
+### Simulation Module — Engine, Allocator, and Policies
+
+Directory: `scheduler/simulation/`
+
+This package provides a discrete‑event simulation of the court scheduling system. It generates synthetic case flows (or consumes real‑like parameters), evaluates scheduling/ripeness policies, and produces quantitative outputs and reports.
+
+#### Files overview
+
+- `engine.py`
+  - Purpose: Main simulation loop and state transition engine.
+  - Key classes:
+    - `CourtSimConfig`: Parameters (capacity, stage transition matrices, adjournment params, filings, horizon, etc.).
+    - `CourtSimResult`: Aggregated outputs produced by a run.
+    - `CourtSim`: The engine orchestrating daily cycles:
+      - `_init_stage_ready()`: initializes stage readiness from configuration.
+      - `_evaluate_ripeness(current)`: classify cases using `core.ripeness.RipenessClassifier`.
+      - `_choose_cases_for_day(current)`: select candidates (potentially via policies/priority).
+      - `_file_new_cases(current, n)`: generate new cases based on expected filings.
+      - `_day_process(current)`: core daily loop combining filings, ripeness, scheduling, hearings, adjournments, and disposals.
+      - `run()`: iterate over the configured horizon; write metrics and event logs.
+  - Interactions: Uses domain models in `scheduler/core/*`, policies in `scheduler/simulation/policies/*`, and allocator.
+
+- `allocator.py`
+  - Purpose: Allocate prioritized cases to available courtroom capacity.
+  - Typical responsibilities: respect per‑courtroom limits, avoid duplicate bookings, ensure eligibility constraints.
+  - Interactions: Invoked by both the simulation engine and the core scheduling algorithm.
+
+- `policies/`
+  - Purpose: Pluggable case selection strategies.
+  - Files:
+    - `age.py`: age‑based prioritization (older cases first, possibly weighted).
+    - `fifo.py`: first‑in first‑out ordering respecting readiness.
+    - `readiness.py`: prioritization by computed readiness or ripeness score.
+  - Interface: Each policy exposes a common callable/signature expected by `engine.py` and the core algorithm’s policy abstraction (`core/policy.py`).
+
+#### Simulation flow (high level)
+1) Initialize `CourtSimConfig` from data defaults and parameters.
+2) Seed a case population and stage readiness.
+3) For each simulated day: file new cases, evaluate ripeness, select cases, allocate to courtrooms, conduct hearings, sample adjournments/next stages/disposals.
+4) Persist daily summaries, metrics, and event logs to `outputs/simulation_runs/<version_timestamp>/`.

docs/utils.md

@@ -0,0 +1,15 @@
+### Utils Module — Calendar Utilities
+
+Directory: `scheduler/utils/`
+
+Utility functions that support scheduling and simulation. Currently focused on court calendar handling.
+
+#### Files overview
+
+- `calendar.py`
+  - Purpose: Calendar/date helpers used by the scheduler and simulation.
+  - Typical responsibilities: business day calculations, holiday/weekend handling, date ranges for scheduling windows, and formatting for outputs.
+  - Interactions: Imported where date arithmetic and calendar rules are required (e.g., selecting valid hearing days).
+
+- `__init__.py`
+  - Purpose: Package initialization.

pyproject.toml

@@ -54,7 +54,7 @@ court-scheduler = "cli.main:app"
 include = [
     "cli",
     "eda",
-    "scheduler",
+    "src",
 ]
 exclude = [
     "Data",