Spaces:
Sleeping
Sleeping
Initial deploy: DELPHI Pump Analytics (renamed from MURPHY)
Browse files- .gitignore +6 -0
- .streamlit/config.toml +9 -0
- Dockerfile +22 -0
- README.md +46 -5
- analysis/__init__.py +0 -0
- analysis/advanced_analysis.py +159 -0
- analysis/basic_analysis.py +184 -0
- analysis/execution_engine.py +203 -0
- analysis/knowledge/failure_modes.yaml +361 -0
- analysis/knowledge/operational_context.yaml +154 -0
- analysis/knowledge/reasoning_chains.yaml +302 -0
- analysis/knowledge_retriever.py +432 -0
- analysis/llm_analyzer.py +418 -0
- analysis/nl2sql.py +217 -0
- analysis/operations.py +657 -0
- analysis/pump_cycle_detector.py +319 -0
- analysis/query_planner.py +459 -0
- analysis/result_interpreter.py +205 -0
- app.py +87 -0
- assets/csh2_logo.jpeg +0 -0
- core/__init__.py +0 -0
- core/cached_queries.py +50 -0
- core/config.py +174 -0
- core/date_parser.py +341 -0
- core/db_connector.py +492 -0
- core/export_utils.py +73 -0
- core/timezone.py +100 -0
- pages/1_data_explorer.py +236 -0
- pages/2_pump_cycles.py +117 -0
- pages/3_cycle_detail.py +179 -0
- pages/4_variation_analysis.py +113 -0
- pages/5_cycle_comparison.py +245 -0
- pages/6_advanced_explorer.py +437 -0
- pages/7_search.py +187 -0
- pages/__init__.py +0 -0
- requirements.txt +9 -0
- ui/__init__.py +0 -0
- ui/components.py +260 -0
- ui/plotly_charts.py +383 -0
- ui/styles.py +442 -0
.gitignore
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
.env
|
| 2 |
+
.env.local
|
| 3 |
+
secrets.toml
|
| 4 |
+
__pycache__/
|
| 5 |
+
*.pyc
|
| 6 |
+
.DS_Store
|
.streamlit/config.toml
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[theme]
|
| 2 |
+
primaryColor = "#D4A04A"
|
| 3 |
+
backgroundColor = "#0C0A08"
|
| 4 |
+
secondaryBackgroundColor = "#12100E"
|
| 5 |
+
textColor = "#E8E0D4"
|
| 6 |
+
font = "monospace"
|
| 7 |
+
|
| 8 |
+
[server]
|
| 9 |
+
headless = true
|
Dockerfile
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
FROM python:3.11-slim
|
| 2 |
+
|
| 3 |
+
WORKDIR /app
|
| 4 |
+
|
| 5 |
+
# System deps for psycopg2-binary
|
| 6 |
+
RUN apt-get update && apt-get install -y --no-install-recommends \
|
| 7 |
+
libpq-dev \
|
| 8 |
+
&& rm -rf /var/lib/apt/lists/*
|
| 9 |
+
|
| 10 |
+
COPY requirements.txt .
|
| 11 |
+
RUN pip install --no-cache-dir -r requirements.txt
|
| 12 |
+
|
| 13 |
+
COPY . .
|
| 14 |
+
|
| 15 |
+
# HF Spaces expects port 7860
|
| 16 |
+
EXPOSE 7860
|
| 17 |
+
|
| 18 |
+
CMD ["streamlit", "run", "app.py", \
|
| 19 |
+
"--server.port=7860", \
|
| 20 |
+
"--server.address=0.0.0.0", \
|
| 21 |
+
"--server.headless=true", \
|
| 22 |
+
"--browser.gatherUsageStats=false"]
|
README.md
CHANGED
|
@@ -1,10 +1,51 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: docker
|
|
|
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: CSH2 DELPHI — Pump Analytics
|
| 3 |
+
emoji: ⚙️
|
| 4 |
+
colorFrom: yellow
|
| 5 |
+
colorTo: gray
|
| 6 |
sdk: docker
|
| 7 |
+
app_file: app.py
|
| 8 |
pinned: false
|
| 9 |
---
|
| 10 |
|
| 11 |
+
# CSH2 DELPHI -- Pump Analytics Dashboard
|
| 12 |
+
|
| 13 |
+
Interactive Streamlit dashboard for the CSH2 cryogenic hydrogen pump test platform.
|
| 14 |
+
Connects to TimescaleDB for real-time and historical sensor analytics across 74 sensors,
|
| 15 |
+
with LLM-powered diagnostic analysis via Claude API.
|
| 16 |
+
|
| 17 |
+
## Pages
|
| 18 |
+
|
| 19 |
+
1. **Data Explorer** -- Browse sensor time-series with NL query support (ask in plain English)
|
| 20 |
+
2. **Testing Cycles** -- Browse pre-computed pump test cycles with summary metrics
|
| 21 |
+
3. **Cycle Detail** -- Deep-dive into a single cycle with multi-sensor overlay plots
|
| 22 |
+
4. **Variation Analysis** -- LLM-powered analysis of within-cycle stabilizations and variations
|
| 23 |
+
5. **Cycle Comparison** -- Side-by-side comparison of two cycles with LLM diagnostic commentary
|
| 24 |
+
6. **Advanced Data Explorer** -- NL-to-SQL analytical pipeline with structured operation execution
|
| 25 |
+
|
| 26 |
+
## Secrets (Required)
|
| 27 |
+
|
| 28 |
+
Set these as Space Secrets in the HuggingFace Space settings:
|
| 29 |
+
|
| 30 |
+
| Secret | Description |
|
| 31 |
+
|--------|-------------|
|
| 32 |
+
| `DB_HOST` | TimescaleDB hostname |
|
| 33 |
+
| `DB_PORT` | Connection port |
|
| 34 |
+
| `DB_NAME` | Database name |
|
| 35 |
+
| `DB_USER` | Read-only database user |
|
| 36 |
+
| `DB_PASSWORD` | Database password |
|
| 37 |
+
| `ANTHROPIC_API_KEY` | Claude API key (optional -- pages 1-3 work without it) |
|
| 38 |
+
|
| 39 |
+
Without `ANTHROPIC_API_KEY`, the NL query parser (page 1), variation analysis (page 4),
|
| 40 |
+
cycle comparison (page 5), and advanced explorer interpretation (page 6) will show a
|
| 41 |
+
graceful fallback message. All data browsing and charting works without it.
|
| 42 |
+
|
| 43 |
+
## Architecture
|
| 44 |
+
|
| 45 |
+
- `core/config.py` -- Reads secrets from `st.secrets` (Streamlit Cloud / HF Spaces) first, falls back to `os.environ` / `.env`
|
| 46 |
+
- `core/db_connector.py` -- Singleton PostgreSQL connector with smart table routing (raw / 1sec / 15sec aggregates)
|
| 47 |
+
- `analysis/` -- LLM analyzer, NL-to-SQL parser, domain knowledge YAML retriever, cycle detection
|
| 48 |
+
- `ui/` -- Custom HUD-themed components, Plotly chart builders, CSS injection
|
| 49 |
+
- `pages/` -- Streamlit multi-page app (6 pages)
|
| 50 |
+
|
| 51 |
+
Built for Clear Skies Hydrogen (CSH2).
|
analysis/__init__.py
ADDED
|
File without changes
|
analysis/advanced_analysis.py
ADDED
|
@@ -0,0 +1,159 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Advanced Analysis Module - adapted from clearskies_agent_v1.1
|
| 3 |
+
Peak pressure detection, plateau detection, downsampling
|
| 4 |
+
"""
|
| 5 |
+
import pandas as pd
|
| 6 |
+
import numpy as np
|
| 7 |
+
from typing import Dict, List, Optional
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
class AdvancedAnalyzer:
|
| 11 |
+
"""Advanced analysis functions for cycle detail views"""
|
| 12 |
+
|
| 13 |
+
def find_peak_pressure_blocks(
|
| 14 |
+
self,
|
| 15 |
+
df: pd.DataFrame,
|
| 16 |
+
pressure_tag: str = 'PT130',
|
| 17 |
+
threshold_bar: float = 900.0,
|
| 18 |
+
) -> Dict:
|
| 19 |
+
"""Find time blocks where pressure exceeded a threshold"""
|
| 20 |
+
if pressure_tag not in df.columns:
|
| 21 |
+
# Try long format
|
| 22 |
+
pressure_data = df[df.get('tag_name', pd.Series()) == pressure_tag].copy()
|
| 23 |
+
if pressure_data.empty:
|
| 24 |
+
return {'error': f'No data for {pressure_tag}', 'num_blocks': 0}
|
| 25 |
+
pressure_data = pressure_data.sort_values('timestamp')
|
| 26 |
+
values = pressure_data['value']
|
| 27 |
+
timestamps = pressure_data['timestamp']
|
| 28 |
+
else:
|
| 29 |
+
# Wide format
|
| 30 |
+
pressure_data = df[['timestamp', pressure_tag]].dropna(subset=[pressure_tag]).copy()
|
| 31 |
+
pressure_data = pressure_data.sort_values('timestamp')
|
| 32 |
+
values = pressure_data[pressure_tag]
|
| 33 |
+
timestamps = pressure_data['timestamp']
|
| 34 |
+
|
| 35 |
+
above = values > threshold_bar
|
| 36 |
+
if not above.any():
|
| 37 |
+
return {
|
| 38 |
+
'threshold': threshold_bar,
|
| 39 |
+
'sensor': pressure_tag,
|
| 40 |
+
'num_blocks': 0,
|
| 41 |
+
'total_points_above': 0,
|
| 42 |
+
'message': f'No data points exceeded {threshold_bar} bar',
|
| 43 |
+
}
|
| 44 |
+
|
| 45 |
+
# Group consecutive above-threshold points
|
| 46 |
+
above_df = pd.DataFrame({'timestamp': timestamps[above].values, 'value': values[above].values})
|
| 47 |
+
blocks = self._group_consecutive_timestamps(above_df)
|
| 48 |
+
|
| 49 |
+
block_stats = []
|
| 50 |
+
for block in blocks:
|
| 51 |
+
mask = (above_df['timestamp'] >= block['start']) & (above_df['timestamp'] <= block['end'])
|
| 52 |
+
block_data = above_df[mask]
|
| 53 |
+
block_stats.append({
|
| 54 |
+
'start_time': block['start'],
|
| 55 |
+
'end_time': block['end'],
|
| 56 |
+
'duration_minutes': (block['end'] - block['start']).total_seconds() / 60,
|
| 57 |
+
'num_points': len(block_data),
|
| 58 |
+
'peak_pressure': block_data['value'].max(),
|
| 59 |
+
'avg_pressure': block_data['value'].mean(),
|
| 60 |
+
})
|
| 61 |
+
|
| 62 |
+
return {
|
| 63 |
+
'threshold': threshold_bar,
|
| 64 |
+
'sensor': pressure_tag,
|
| 65 |
+
'num_blocks': len(block_stats),
|
| 66 |
+
'total_points_above': int(above.sum()),
|
| 67 |
+
'percentage_above': (above.sum() / len(values)) * 100,
|
| 68 |
+
'blocks': block_stats,
|
| 69 |
+
'overall_peak': values.max(),
|
| 70 |
+
}
|
| 71 |
+
|
| 72 |
+
def detect_constant_periods(
|
| 73 |
+
self,
|
| 74 |
+
df: pd.DataFrame,
|
| 75 |
+
tag: str,
|
| 76 |
+
tolerance: float = 0.01,
|
| 77 |
+
min_duration_minutes: float = 2.0,
|
| 78 |
+
) -> List[Dict]:
|
| 79 |
+
"""
|
| 80 |
+
Detect periods where a sensor value is constant within tolerance.
|
| 81 |
+
|
| 82 |
+
Works with both wide-format (tag as column) and long-format data.
|
| 83 |
+
"""
|
| 84 |
+
if tag in df.columns:
|
| 85 |
+
# Wide format
|
| 86 |
+
sensor_data = df[['timestamp', tag]].dropna(subset=[tag]).sort_values('timestamp')
|
| 87 |
+
timestamps = sensor_data['timestamp'].values
|
| 88 |
+
values = sensor_data[tag].values
|
| 89 |
+
else:
|
| 90 |
+
# Long format
|
| 91 |
+
sensor_data = df[df.get('tag_name', pd.Series()) == tag].sort_values('timestamp')
|
| 92 |
+
if sensor_data.empty:
|
| 93 |
+
return []
|
| 94 |
+
timestamps = sensor_data['timestamp'].values
|
| 95 |
+
values = sensor_data['value'].values
|
| 96 |
+
|
| 97 |
+
if len(values) < 2:
|
| 98 |
+
return []
|
| 99 |
+
|
| 100 |
+
constant_periods = []
|
| 101 |
+
period_start = 0
|
| 102 |
+
period_value = values[0]
|
| 103 |
+
|
| 104 |
+
for i in range(1, len(values)):
|
| 105 |
+
# Check if value is constant within tolerance
|
| 106 |
+
denom = abs(period_value) + 1e-10
|
| 107 |
+
if abs(values[i] - period_value) / denom <= tolerance:
|
| 108 |
+
continue
|
| 109 |
+
|
| 110 |
+
# Period ended — check duration
|
| 111 |
+
duration_sec = (pd.Timestamp(timestamps[i - 1]) - pd.Timestamp(timestamps[period_start])).total_seconds()
|
| 112 |
+
duration_min = duration_sec / 60
|
| 113 |
+
|
| 114 |
+
if duration_min >= min_duration_minutes:
|
| 115 |
+
constant_periods.append({
|
| 116 |
+
'start': pd.Timestamp(timestamps[period_start]),
|
| 117 |
+
'end': pd.Timestamp(timestamps[i - 1]),
|
| 118 |
+
'duration_minutes': duration_min,
|
| 119 |
+
'value': float(period_value),
|
| 120 |
+
})
|
| 121 |
+
|
| 122 |
+
period_start = i
|
| 123 |
+
period_value = values[i]
|
| 124 |
+
|
| 125 |
+
# Check last period
|
| 126 |
+
duration_sec = (pd.Timestamp(timestamps[-1]) - pd.Timestamp(timestamps[period_start])).total_seconds()
|
| 127 |
+
duration_min = duration_sec / 60
|
| 128 |
+
if duration_min >= min_duration_minutes:
|
| 129 |
+
constant_periods.append({
|
| 130 |
+
'start': pd.Timestamp(timestamps[period_start]),
|
| 131 |
+
'end': pd.Timestamp(timestamps[-1]),
|
| 132 |
+
'duration_minutes': duration_min,
|
| 133 |
+
'value': float(period_value),
|
| 134 |
+
})
|
| 135 |
+
|
| 136 |
+
return constant_periods
|
| 137 |
+
|
| 138 |
+
def _group_consecutive_timestamps(self, df: pd.DataFrame, max_gap_seconds: float = 2.0) -> List[Dict]:
|
| 139 |
+
"""Group consecutive timestamps into blocks"""
|
| 140 |
+
if df.empty:
|
| 141 |
+
return []
|
| 142 |
+
|
| 143 |
+
blocks = []
|
| 144 |
+
current_start = df.iloc[0]['timestamp']
|
| 145 |
+
current_end = df.iloc[0]['timestamp']
|
| 146 |
+
|
| 147 |
+
for i in range(1, len(df)):
|
| 148 |
+
t = df.iloc[i]['timestamp']
|
| 149 |
+
gap = (t - current_end).total_seconds() if hasattr(t - current_end, 'total_seconds') else 0
|
| 150 |
+
|
| 151 |
+
if gap <= max_gap_seconds:
|
| 152 |
+
current_end = t
|
| 153 |
+
else:
|
| 154 |
+
blocks.append({'start': current_start, 'end': current_end})
|
| 155 |
+
current_start = t
|
| 156 |
+
current_end = t
|
| 157 |
+
|
| 158 |
+
blocks.append({'start': current_start, 'end': current_end})
|
| 159 |
+
return blocks
|
analysis/basic_analysis.py
ADDED
|
@@ -0,0 +1,184 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Basic Analysis Module - adapted from clearskies_agent_v1.1
|
| 3 |
+
Compression ratio, ramp rate, flow stats, fill event analysis
|
| 4 |
+
"""
|
| 5 |
+
import pandas as pd
|
| 6 |
+
import numpy as np
|
| 7 |
+
from typing import Dict, Optional
|
| 8 |
+
from core import config
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
class DataAnalyzer:
|
| 12 |
+
"""Performs analysis on testing cycle / fill event data"""
|
| 13 |
+
|
| 14 |
+
def __init__(self):
|
| 15 |
+
self.performance_targets = config.PERFORMANCE_TARGETS
|
| 16 |
+
|
| 17 |
+
def analyze_cycle(
|
| 18 |
+
self,
|
| 19 |
+
df_pivot: pd.DataFrame,
|
| 20 |
+
discharge_tag: str = 'PT130',
|
| 21 |
+
supply_tag: str = 'PT01T',
|
| 22 |
+
flow_tag: str = 'FT140',
|
| 23 |
+
temp_tags: list = None,
|
| 24 |
+
) -> Dict:
|
| 25 |
+
"""
|
| 26 |
+
Comprehensive analysis of a testing cycle from pivoted data.
|
| 27 |
+
|
| 28 |
+
Args:
|
| 29 |
+
df_pivot: Wide-format DataFrame with timestamp and tag columns
|
| 30 |
+
discharge_tag: Discharge pressure sensor
|
| 31 |
+
supply_tag: Supply pressure sensor
|
| 32 |
+
flow_tag: Flow meter sensor
|
| 33 |
+
temp_tags: Temperature sensor tags
|
| 34 |
+
|
| 35 |
+
Returns:
|
| 36 |
+
Dict with comprehensive cycle analysis
|
| 37 |
+
"""
|
| 38 |
+
if temp_tags is None:
|
| 39 |
+
temp_tags = ['TT110', 'TT130']
|
| 40 |
+
|
| 41 |
+
analysis = {
|
| 42 |
+
'time_range': {
|
| 43 |
+
'start': df_pivot['timestamp'].min(),
|
| 44 |
+
'end': df_pivot['timestamp'].max(),
|
| 45 |
+
'duration_minutes': (df_pivot['timestamp'].max() - df_pivot['timestamp'].min()).total_seconds() / 60,
|
| 46 |
+
}
|
| 47 |
+
}
|
| 48 |
+
|
| 49 |
+
# Discharge pressure
|
| 50 |
+
if discharge_tag in df_pivot.columns:
|
| 51 |
+
col = df_pivot[discharge_tag].dropna()
|
| 52 |
+
if len(col) > 0:
|
| 53 |
+
analysis['discharge_pressure'] = {
|
| 54 |
+
'peak': float(col.max()),
|
| 55 |
+
'avg': float(col.mean()),
|
| 56 |
+
'std': float(col.std()) if len(col) > 1 else 0.0,
|
| 57 |
+
'initial': float(col.iloc[0]),
|
| 58 |
+
'final': float(col.iloc[-1]),
|
| 59 |
+
'unit': 'bar',
|
| 60 |
+
}
|
| 61 |
+
|
| 62 |
+
# Supply pressure
|
| 63 |
+
if supply_tag in df_pivot.columns:
|
| 64 |
+
col = df_pivot[supply_tag].dropna()
|
| 65 |
+
if len(col) > 0:
|
| 66 |
+
analysis['supply_pressure'] = {
|
| 67 |
+
'avg': float(col.mean()),
|
| 68 |
+
'min': float(col.min()),
|
| 69 |
+
'max': float(col.max()),
|
| 70 |
+
'unit': 'bar',
|
| 71 |
+
}
|
| 72 |
+
|
| 73 |
+
# Compression ratio
|
| 74 |
+
if discharge_tag in df_pivot.columns and supply_tag in df_pivot.columns:
|
| 75 |
+
valid = df_pivot[[discharge_tag, supply_tag]].dropna()
|
| 76 |
+
if len(valid) > 0 and (valid[supply_tag] != 0).any():
|
| 77 |
+
cr = valid[discharge_tag] / valid[supply_tag].replace(0, np.nan)
|
| 78 |
+
cr = cr.dropna()
|
| 79 |
+
if len(cr) > 0:
|
| 80 |
+
analysis['compression_ratio'] = {
|
| 81 |
+
'avg': float(cr.mean()),
|
| 82 |
+
'max': float(cr.max()),
|
| 83 |
+
'min': float(cr.min()),
|
| 84 |
+
}
|
| 85 |
+
|
| 86 |
+
# Temperature analysis
|
| 87 |
+
for tag in temp_tags:
|
| 88 |
+
if tag in df_pivot.columns:
|
| 89 |
+
col = df_pivot[tag].dropna()
|
| 90 |
+
if len(col) > 0:
|
| 91 |
+
analysis[f'temperature_{tag}'] = {
|
| 92 |
+
'peak': float(col.max()),
|
| 93 |
+
'min': float(col.min()),
|
| 94 |
+
'avg': float(col.mean()),
|
| 95 |
+
'unit': 'K',
|
| 96 |
+
}
|
| 97 |
+
|
| 98 |
+
# Flow analysis
|
| 99 |
+
if flow_tag in df_pivot.columns:
|
| 100 |
+
col = df_pivot[flow_tag].dropna()
|
| 101 |
+
if len(col) > 0:
|
| 102 |
+
analysis['flow'] = {
|
| 103 |
+
'avg_flow': float(col.mean()),
|
| 104 |
+
'max_flow': float(col.max()),
|
| 105 |
+
'unit': 'kg/min',
|
| 106 |
+
}
|
| 107 |
+
# Estimate total mass (trapezoidal integration)
|
| 108 |
+
flow_sorted = df_pivot[['timestamp', flow_tag]].dropna().sort_values('timestamp')
|
| 109 |
+
if len(flow_sorted) > 1:
|
| 110 |
+
time_diff_min = flow_sorted['timestamp'].diff().dt.total_seconds().fillna(0) / 60
|
| 111 |
+
mass_increment = flow_sorted[flow_tag] * time_diff_min
|
| 112 |
+
analysis['flow']['total_mass_kg'] = float(mass_increment.sum())
|
| 113 |
+
|
| 114 |
+
# Pressure ramp rate
|
| 115 |
+
if discharge_tag in df_pivot.columns:
|
| 116 |
+
ramp = self._calculate_ramp_rate(df_pivot, discharge_tag)
|
| 117 |
+
if ramp:
|
| 118 |
+
analysis['ramp_rate'] = ramp
|
| 119 |
+
|
| 120 |
+
# Motor speed — apply display multiplier (5/3) to convert raw signal to RPM
|
| 121 |
+
speed_mult = getattr(config, 'MOTOR_SPEED_DISPLAY_MULTIPLIER', 1.0)
|
| 122 |
+
for speed_tag in ['M130_Speed', 'MC130_VFD_Speed', 'M130_RPM']:
|
| 123 |
+
if speed_tag in df_pivot.columns:
|
| 124 |
+
col = df_pivot[speed_tag].dropna()
|
| 125 |
+
if len(col) > 0:
|
| 126 |
+
analysis['motor'] = {
|
| 127 |
+
'peak_speed': float(col.max()) * speed_mult,
|
| 128 |
+
'avg_speed': float(col.mean()) * speed_mult,
|
| 129 |
+
'tag': speed_tag,
|
| 130 |
+
'unit': 'RPM',
|
| 131 |
+
}
|
| 132 |
+
break
|
| 133 |
+
|
| 134 |
+
# Performance vs targets
|
| 135 |
+
analysis['performance_vs_targets'] = self._compare_to_targets(analysis)
|
| 136 |
+
|
| 137 |
+
return analysis
|
| 138 |
+
|
| 139 |
+
def _calculate_ramp_rate(self, df: pd.DataFrame, pressure_tag: str) -> Optional[Dict]:
|
| 140 |
+
"""Calculate pressure ramp rate in bar/min"""
|
| 141 |
+
data = df[['timestamp', pressure_tag]].dropna().sort_values('timestamp')
|
| 142 |
+
if len(data) < 2:
|
| 143 |
+
return None
|
| 144 |
+
|
| 145 |
+
time_diff_min = data['timestamp'].diff().dt.total_seconds() / 60
|
| 146 |
+
pressure_diff = data[pressure_tag].diff()
|
| 147 |
+
ramp_rate = pressure_diff / time_diff_min
|
| 148 |
+
|
| 149 |
+
valid_rates = ramp_rate.replace([np.inf, -np.inf], np.nan).dropna()
|
| 150 |
+
if len(valid_rates) == 0:
|
| 151 |
+
return None
|
| 152 |
+
|
| 153 |
+
return {
|
| 154 |
+
'avg': float(valid_rates.mean()),
|
| 155 |
+
'max': float(valid_rates.max()),
|
| 156 |
+
'min': float(valid_rates.min()),
|
| 157 |
+
'unit': 'bar/min',
|
| 158 |
+
}
|
| 159 |
+
|
| 160 |
+
def _compare_to_targets(self, analysis: Dict) -> Dict:
|
| 161 |
+
"""Compare analysis results against performance targets"""
|
| 162 |
+
comparison = {}
|
| 163 |
+
|
| 164 |
+
if 'discharge_pressure' in analysis:
|
| 165 |
+
peak = analysis['discharge_pressure']['peak']
|
| 166 |
+
targets = self.performance_targets['pressure']
|
| 167 |
+
comparison['pressure'] = {
|
| 168 |
+
'achieved': peak,
|
| 169 |
+
'phase1_target': targets['phase1'],
|
| 170 |
+
'h70_target': targets['h70'],
|
| 171 |
+
'meets_phase1': peak >= targets['phase1'],
|
| 172 |
+
'meets_h70': peak >= targets['h70'],
|
| 173 |
+
}
|
| 174 |
+
|
| 175 |
+
if 'flow' in analysis:
|
| 176 |
+
avg = analysis['flow']['avg_flow']
|
| 177 |
+
targets = self.performance_targets['flow']
|
| 178 |
+
comparison['flow'] = {
|
| 179 |
+
'achieved': avg,
|
| 180 |
+
'simplex_target': targets['simplex'],
|
| 181 |
+
'pct_of_target': (avg / targets['simplex']) * 100 if targets['simplex'] else 0,
|
| 182 |
+
}
|
| 183 |
+
|
| 184 |
+
return comparison
|
analysis/execution_engine.py
ADDED
|
@@ -0,0 +1,203 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Execution Engine for Advanced Data Explorer.
|
| 3 |
+
|
| 4 |
+
Validates an ExecutionPlan, fetches the required data, dispatches
|
| 5 |
+
operations sequentially via OperationDispatcher, and packages results.
|
| 6 |
+
"""
|
| 7 |
+
import time
|
| 8 |
+
import pandas as pd
|
| 9 |
+
from typing import Any, Dict, List, Optional
|
| 10 |
+
from dataclasses import dataclass, field
|
| 11 |
+
from datetime import datetime
|
| 12 |
+
|
| 13 |
+
from core import config
|
| 14 |
+
from core.db_connector import get_db_connector
|
| 15 |
+
from core.cached_queries import cached_get_pivot_data
|
| 16 |
+
from analysis.operations import OperationDispatcher, OperationOutput
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
# ── Result Dataclasses ───────────────────────────────────────────
|
| 20 |
+
|
| 21 |
+
@dataclass
|
| 22 |
+
class ExecutionResult:
|
| 23 |
+
"""Complete result of executing a plan."""
|
| 24 |
+
success: bool
|
| 25 |
+
query_type: str
|
| 26 |
+
explanation: str
|
| 27 |
+
source_data: Optional[pd.DataFrame] = None
|
| 28 |
+
operation_results: List[OperationOutput] = field(default_factory=list)
|
| 29 |
+
errors: List[str] = field(default_factory=list)
|
| 30 |
+
warnings: List[str] = field(default_factory=list)
|
| 31 |
+
execution_time_seconds: float = 0.0
|
| 32 |
+
table_used: str = ""
|
| 33 |
+
row_count: int = 0
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
# ── Table label mapping (for display) ───────────────────────────
|
| 37 |
+
|
| 38 |
+
TABLE_LABELS = {
|
| 39 |
+
config.TABLES['raw']: 'Raw (10Hz)',
|
| 40 |
+
config.TABLES['agg_1sec']: '1-second aggregates',
|
| 41 |
+
config.TABLES['agg_15sec']: '15-second aggregates',
|
| 42 |
+
}
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
# ── Execution Engine ─────────────────────────────────────────────
|
| 46 |
+
|
| 47 |
+
class ExecutionEngine:
|
| 48 |
+
"""
|
| 49 |
+
Validate an ExecutionPlan, fetch data, run operations, package results.
|
| 50 |
+
|
| 51 |
+
Usage:
|
| 52 |
+
engine = ExecutionEngine()
|
| 53 |
+
result = engine.execute(plan)
|
| 54 |
+
"""
|
| 55 |
+
|
| 56 |
+
def __init__(self):
|
| 57 |
+
self.dispatcher = OperationDispatcher()
|
| 58 |
+
|
| 59 |
+
def execute(self, plan: Dict) -> ExecutionResult:
|
| 60 |
+
"""Execute a validated plan end-to-end."""
|
| 61 |
+
t0 = time.time()
|
| 62 |
+
errors = []
|
| 63 |
+
warnings = []
|
| 64 |
+
|
| 65 |
+
# ── 1. Validate plan structure ──
|
| 66 |
+
qt = plan.get("query_type", "FETCH")
|
| 67 |
+
explanation = plan.get("explanation", "")
|
| 68 |
+
dr = plan.get("data_requirements", {})
|
| 69 |
+
operations = plan.get("operations", [])
|
| 70 |
+
|
| 71 |
+
sensors = dr.get("sensors", [])
|
| 72 |
+
start_time = dr.get("start_time")
|
| 73 |
+
end_time = dr.get("end_time")
|
| 74 |
+
resolution = dr.get("resolution", "auto")
|
| 75 |
+
|
| 76 |
+
if not sensors:
|
| 77 |
+
return self._fail("No sensors specified in plan", qt, explanation, t0)
|
| 78 |
+
|
| 79 |
+
if not start_time or not end_time:
|
| 80 |
+
return self._fail("Missing start_time or end_time in plan", qt, explanation, t0)
|
| 81 |
+
|
| 82 |
+
# Ensure datetimes are datetime objects
|
| 83 |
+
if isinstance(start_time, str):
|
| 84 |
+
start_time = datetime.fromisoformat(start_time)
|
| 85 |
+
if isinstance(end_time, str):
|
| 86 |
+
end_time = datetime.fromisoformat(end_time)
|
| 87 |
+
|
| 88 |
+
# ── 2. Resolve table from resolution ──
|
| 89 |
+
table_override = None
|
| 90 |
+
if resolution in config.RESOLUTION_MAP:
|
| 91 |
+
table_override = config.RESOLUTION_MAP[resolution]
|
| 92 |
+
elif resolution == "auto":
|
| 93 |
+
# Let the DB connector decide via smart routing
|
| 94 |
+
table_override = None
|
| 95 |
+
else:
|
| 96 |
+
warnings.append(f"Unknown resolution '{resolution}', using auto routing.")
|
| 97 |
+
|
| 98 |
+
# ── 3. Safety: raw table time-range cap ──
|
| 99 |
+
if table_override == config.TABLES['raw']:
|
| 100 |
+
duration_h = (end_time - start_time).total_seconds() / 3600
|
| 101 |
+
if duration_h > 1.0:
|
| 102 |
+
return self._fail(
|
| 103 |
+
f"Raw resolution limited to 1 hour. Requested {duration_h:.1f} hours. "
|
| 104 |
+
f"Use '1sec' or '15sec' for longer ranges.",
|
| 105 |
+
qt, explanation, t0,
|
| 106 |
+
)
|
| 107 |
+
|
| 108 |
+
# ── 4. Fetch data ──
|
| 109 |
+
try:
|
| 110 |
+
db = get_db_connector()
|
| 111 |
+
df = cached_get_pivot_data(
|
| 112 |
+
tag_names=tuple(sensors),
|
| 113 |
+
start_time=start_time,
|
| 114 |
+
end_time=end_time,
|
| 115 |
+
table_override=table_override,
|
| 116 |
+
)
|
| 117 |
+
except Exception as e:
|
| 118 |
+
return self._fail(f"Data fetch failed: {e}", qt, explanation, t0)
|
| 119 |
+
|
| 120 |
+
if df is None or df.empty:
|
| 121 |
+
return self._fail(
|
| 122 |
+
"No data returned for the specified sensors and time range. "
|
| 123 |
+
"Check that the sensors exist and the time range is within March 14 – September 25, 2025.",
|
| 124 |
+
qt, explanation, t0,
|
| 125 |
+
)
|
| 126 |
+
|
| 127 |
+
# Determine which table was actually used (for display)
|
| 128 |
+
if table_override:
|
| 129 |
+
table_used = table_override
|
| 130 |
+
else:
|
| 131 |
+
table_used = db._select_table(start_time, end_time)
|
| 132 |
+
|
| 133 |
+
table_label = TABLE_LABELS.get(table_used, table_used)
|
| 134 |
+
|
| 135 |
+
# Check which sensors actually came back
|
| 136 |
+
returned_sensors = [s for s in sensors if s in df.columns]
|
| 137 |
+
missing = [s for s in sensors if s not in df.columns and s != "timestamp"]
|
| 138 |
+
if missing:
|
| 139 |
+
warnings.append(f"Sensors not found in data: {', '.join(missing)}")
|
| 140 |
+
|
| 141 |
+
# ── 5. Run operations sequentially ──
|
| 142 |
+
op_results = []
|
| 143 |
+
prev_result = None
|
| 144 |
+
|
| 145 |
+
for i, op_spec in enumerate(operations):
|
| 146 |
+
op_name = op_spec.get("op", "")
|
| 147 |
+
op_params = op_spec.get("params", {})
|
| 148 |
+
op_label = op_spec.get("label", op_name)
|
| 149 |
+
|
| 150 |
+
try:
|
| 151 |
+
output = self.dispatcher.run(
|
| 152 |
+
op_name=op_name,
|
| 153 |
+
df=df,
|
| 154 |
+
params=op_params,
|
| 155 |
+
prev_result=prev_result,
|
| 156 |
+
db_connector=get_db_connector(),
|
| 157 |
+
)
|
| 158 |
+
op_results.append(output)
|
| 159 |
+
prev_result = output
|
| 160 |
+
|
| 161 |
+
# Check for operation-level errors
|
| 162 |
+
if output.metadata.get("error"):
|
| 163 |
+
warnings.append(f"Operation '{op_label}': {output.metadata['error']}")
|
| 164 |
+
|
| 165 |
+
except Exception as e:
|
| 166 |
+
error_msg = f"Operation '{op_label}' failed: {e}"
|
| 167 |
+
errors.append(error_msg)
|
| 168 |
+
# Create a placeholder output so downstream chaining can detect the failure
|
| 169 |
+
op_results.append(OperationOutput(
|
| 170 |
+
op_name=op_name,
|
| 171 |
+
label=f"{op_label} (FAILED)",
|
| 172 |
+
data=None,
|
| 173 |
+
metadata={"error": str(e)},
|
| 174 |
+
))
|
| 175 |
+
prev_result = op_results[-1]
|
| 176 |
+
|
| 177 |
+
# ── 6. Package result ──
|
| 178 |
+
elapsed = time.time() - t0
|
| 179 |
+
success = len(errors) == 0
|
| 180 |
+
|
| 181 |
+
return ExecutionResult(
|
| 182 |
+
success=success,
|
| 183 |
+
query_type=qt,
|
| 184 |
+
explanation=explanation,
|
| 185 |
+
source_data=df,
|
| 186 |
+
operation_results=op_results,
|
| 187 |
+
errors=errors,
|
| 188 |
+
warnings=warnings,
|
| 189 |
+
execution_time_seconds=round(elapsed, 2),
|
| 190 |
+
table_used=table_label,
|
| 191 |
+
row_count=len(df),
|
| 192 |
+
)
|
| 193 |
+
|
| 194 |
+
@staticmethod
|
| 195 |
+
def _fail(msg: str, qt: str, explanation: str, t0: float) -> ExecutionResult:
|
| 196 |
+
"""Return a failed ExecutionResult."""
|
| 197 |
+
return ExecutionResult(
|
| 198 |
+
success=False,
|
| 199 |
+
query_type=qt,
|
| 200 |
+
explanation=explanation,
|
| 201 |
+
errors=[msg],
|
| 202 |
+
execution_time_seconds=round(time.time() - t0, 2),
|
| 203 |
+
)
|
analysis/knowledge/failure_modes.yaml
ADDED
|
@@ -0,0 +1,361 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CSH2 Cryogenic Pump — Failure Mode Knowledge Base
|
| 2 |
+
# Derived from maintenance_log.xlsx (Jul 2025) and Technical PDF (Phase 1B results)
|
| 3 |
+
# 8 failure modes ordered by frequency and system priority
|
| 4 |
+
|
| 5 |
+
failure_modes:
|
| 6 |
+
- id: FM-001
|
| 7 |
+
name: "ICV Reverse Flow"
|
| 8 |
+
category: "check_valve"
|
| 9 |
+
symptoms:
|
| 10 |
+
primary: "Low pressure build — PT130 plateaus well below target"
|
| 11 |
+
secondary:
|
| 12 |
+
- "Degrades more at higher motor speeds"
|
| 13 |
+
- "DCV leak test normal (rules out DCV)"
|
| 14 |
+
- "Volumetric efficiency significantly below expected"
|
| 15 |
+
sensor_signatures:
|
| 16 |
+
- tag: PT130
|
| 17 |
+
tagindex: 18
|
| 18 |
+
behavior: "Plateaus at 40-60% of expected peak pressure"
|
| 19 |
+
- tag: TT110
|
| 20 |
+
tagindex: 24
|
| 21 |
+
behavior: "Gradual warming from high-pressure leakback through ICV"
|
| 22 |
+
- tag: FT140
|
| 23 |
+
tagindex: 7
|
| 24 |
+
behavior: "Flow rate reduced proportionally to leak severity"
|
| 25 |
+
physical_mechanism: >
|
| 26 |
+
ICV does not fully seat against head block during compression stroke.
|
| 27 |
+
Creates reverse flow path from high-pressure chamber back to inlet.
|
| 28 |
+
Volumetric efficiency drops proportionally with gap area and pressure
|
| 29 |
+
differential. Higher motor speeds worsen the leak because the pressure
|
| 30 |
+
differential per stroke increases while the seating time decreases.
|
| 31 |
+
candidate_causes:
|
| 32 |
+
- cause: "ICV installation orientation error"
|
| 33 |
+
prior: 0.45
|
| 34 |
+
evidence: "Jul 22 — fixed orientation restored 700 bar"
|
| 35 |
+
refs: ["2025-07-22"]
|
| 36 |
+
- cause: "ICV/head block tolerance mismatch"
|
| 37 |
+
prior: 0.30
|
| 38 |
+
evidence: "Jul 14 — manually polished ICV not flat"
|
| 39 |
+
refs: ["2025-07-14"]
|
| 40 |
+
- cause: "ICV sealing surface wear"
|
| 41 |
+
prior: 0.15
|
| 42 |
+
evidence: "Jul 1 — new valve helped but some leakage remained"
|
| 43 |
+
refs: ["2025-07-01"]
|
| 44 |
+
- cause: "Thermal distortion from cryo cycling"
|
| 45 |
+
prior: 0.10
|
| 46 |
+
evidence: "Theoretical — differential CTE at ICV/head block interface"
|
| 47 |
+
discriminating_tests:
|
| 48 |
+
- test: "DCV leak test (PT130 decay, pump stopped)"
|
| 49 |
+
rules_out: "DCV leak if decay < 0.5 bar/min"
|
| 50 |
+
- test: "Compare pressure build at 10% vs 50% motor speed"
|
| 51 |
+
interpretation: "ICV leak worsens disproportionately at higher speeds"
|
| 52 |
+
- test: "Visual inspect ICV seating — flashlight gap check"
|
| 53 |
+
confirms: "Installation tolerance issue"
|
| 54 |
+
recommended_actions:
|
| 55 |
+
immediate: "Check ICV seating orientation and gap against head block"
|
| 56 |
+
long_term: "Implement ICV installation jig or go/no-go gauge"
|
| 57 |
+
system_priority: "HIGH"
|
| 58 |
+
occurrence_count: 3
|
| 59 |
+
notes: "Most frequently recurring failure mode (3x July 2025)"
|
| 60 |
+
|
| 61 |
+
- id: FM-002
|
| 62 |
+
name: "DCV Leakage"
|
| 63 |
+
category: "check_valve"
|
| 64 |
+
symptoms:
|
| 65 |
+
primary: "PT130 decays during static hold test"
|
| 66 |
+
secondary:
|
| 67 |
+
- "Pressure build OK but cannot maintain pressure after pump stops"
|
| 68 |
+
- "Rate of PT130 decay proportional to leak severity"
|
| 69 |
+
sensor_signatures:
|
| 70 |
+
- tag: PT130
|
| 71 |
+
tagindex: 18
|
| 72 |
+
behavior: "Exponential decay during hold period (pump stopped)"
|
| 73 |
+
- tag: TT130
|
| 74 |
+
tagindex: 27
|
| 75 |
+
behavior: "May show slight cooling as high-pressure fluid expands backward"
|
| 76 |
+
physical_mechanism: >
|
| 77 |
+
DCV sealing surface does not fully seat, allowing high-pressure fluid
|
| 78 |
+
to leak backward through the discharge check valve. The leak rate is
|
| 79 |
+
proportional to the pressure differential and the effective orifice area
|
| 80 |
+
of the gap. This is a static leak — it occurs even when the pump is stopped.
|
| 81 |
+
candidate_causes:
|
| 82 |
+
- cause: "DCV sealing surface damage/scoring"
|
| 83 |
+
prior: 0.50
|
| 84 |
+
evidence: "Jul 1 — DCV sealing surface polished to restore seal"
|
| 85 |
+
refs: ["2025-07-01"]
|
| 86 |
+
- cause: "DCV spring fatigue"
|
| 87 |
+
prior: 0.25
|
| 88 |
+
evidence: "Theoretical — spring force decreases with cycling"
|
| 89 |
+
- cause: "Particulate contamination on seat"
|
| 90 |
+
prior: 0.25
|
| 91 |
+
evidence: "Theoretical — cryo system should be clean but possible"
|
| 92 |
+
discriminating_tests:
|
| 93 |
+
- test: "Pressurize to 500 bar, stop pump, log PT130 for 5 minutes"
|
| 94 |
+
interpretation: "Decay rate > 5 bar/min = actionable leak"
|
| 95 |
+
- test: "Compare decay rates before and after DCV polishing"
|
| 96 |
+
interpretation: "Improvement confirms surface damage as cause"
|
| 97 |
+
recommended_actions:
|
| 98 |
+
immediate: "Run PT130 decay test and compute leak rate"
|
| 99 |
+
long_term: "Establish DCV inspection/polishing interval based on cycling data"
|
| 100 |
+
system_priority: "MEDIUM"
|
| 101 |
+
occurrence_count: 1
|
| 102 |
+
|
| 103 |
+
- id: FM-003
|
| 104 |
+
name: "HP Seal Blowby / Energizer Degradation"
|
| 105 |
+
category: "seal"
|
| 106 |
+
symptoms:
|
| 107 |
+
primary: "Increased blowby leakage — reduced volumetric efficiency"
|
| 108 |
+
secondary:
|
| 109 |
+
- "Seal friction initially high after replacement (7-9 kgf)"
|
| 110 |
+
- "Friction breaks in to 4-5 kgf after ~1000 cycles"
|
| 111 |
+
- "Degradation rate very slow (8.2M cycle lower bound on seal life)"
|
| 112 |
+
sensor_signatures:
|
| 113 |
+
- tag: PT130
|
| 114 |
+
tagindex: 18
|
| 115 |
+
behavior: "Gradual pressure build degradation over thousands of cycles"
|
| 116 |
+
- tag: MC130_VFD_Speed
|
| 117 |
+
tagindex: 13
|
| 118 |
+
behavior: "Higher motor current at same speed = increased friction"
|
| 119 |
+
physical_mechanism: >
|
| 120 |
+
The HP seal energizer may degrade from repeated thermal cycling to 116K.
|
| 121 |
+
This reduces the radial sealing force against the cylinder bore, allowing
|
| 122 |
+
high-pressure fluid to leak past the piston (blowby). The seal life lower
|
| 123 |
+
bound from reliability testing is 8.2M cycles, equivalent to 3+ months
|
| 124 |
+
at 20 fills/day.
|
| 125 |
+
candidate_causes:
|
| 126 |
+
- cause: "Seal energizer thermal fatigue"
|
| 127 |
+
prior: 0.40
|
| 128 |
+
evidence: "Jul 18 — seal energizer possibly degraded from cryo cycling"
|
| 129 |
+
refs: ["2025-07-18"]
|
| 130 |
+
- cause: "Bore surface roughening"
|
| 131 |
+
prior: 0.30
|
| 132 |
+
evidence: "Theoretical — scoring increases blowby"
|
| 133 |
+
- cause: "Piston ring wear"
|
| 134 |
+
prior: 0.30
|
| 135 |
+
evidence: "Theoretical — progressive wear"
|
| 136 |
+
discriminating_tests:
|
| 137 |
+
- test: "Static blowby rate measurement at multiple pressures"
|
| 138 |
+
interpretation: "Higher blowby at higher P = seal gap increasing"
|
| 139 |
+
- test: "Track motor current at fixed speed over time"
|
| 140 |
+
interpretation: "Decreasing friction = seal wearing in (expected). Increasing = bore damage"
|
| 141 |
+
recommended_actions:
|
| 142 |
+
immediate: "Monitor blowby rate and motor current trends"
|
| 143 |
+
long_term: "Schedule seal inspection at cumulative stroke threshold"
|
| 144 |
+
system_priority: "LOW"
|
| 145 |
+
occurrence_count: 1
|
| 146 |
+
notes: "Seal life is very long — this is a slow degradation mode"
|
| 147 |
+
|
| 148 |
+
- id: FM-004
|
| 149 |
+
name: "Pressure Safety Loop Oscillation"
|
| 150 |
+
category: "controls"
|
| 151 |
+
symptoms:
|
| 152 |
+
primary: "Rapid pressure oscillations near setpoint"
|
| 153 |
+
secondary:
|
| 154 |
+
- "AOV141 opening/closing rapidly"
|
| 155 |
+
- "VFD speed oscillating"
|
| 156 |
+
- "Pressure overshoots followed by undershoots"
|
| 157 |
+
sensor_signatures:
|
| 158 |
+
- tag: PT130
|
| 159 |
+
tagindex: 18
|
| 160 |
+
behavior: "High-frequency oscillation (>0.5 Hz) near setpoint"
|
| 161 |
+
- tag: AOV140
|
| 162 |
+
tagindex: 1
|
| 163 |
+
behavior: "Rapid cycling between open and closed"
|
| 164 |
+
physical_mechanism: >
|
| 165 |
+
The pressure safety control loop has insufficient damping or incorrect
|
| 166 |
+
PID tuning. When pressure approaches the setpoint, the relief valve
|
| 167 |
+
(AOV141) opens and closes rapidly, causing pressure oscillation.
|
| 168 |
+
This stresses the valve mechanism and can cause premature wear.
|
| 169 |
+
candidate_causes:
|
| 170 |
+
- cause: "PID tuning too aggressive (insufficient damping)"
|
| 171 |
+
prior: 0.60
|
| 172 |
+
evidence: "Jul 22 — AOV141 rapid cycling observed at high pressure"
|
| 173 |
+
refs: ["2025-07-22"]
|
| 174 |
+
- cause: "Relief valve setpoint too close to operating pressure"
|
| 175 |
+
prior: 0.30
|
| 176 |
+
evidence: "Theoretical — narrow margin causes hunting"
|
| 177 |
+
- cause: "Valve actuator response delay"
|
| 178 |
+
prior: 0.10
|
| 179 |
+
evidence: "Theoretical"
|
| 180 |
+
discriminating_tests:
|
| 181 |
+
- test: "Ramp pressure slowly to setpoint and monitor AOV141 position"
|
| 182 |
+
interpretation: "Oscillation onset pressure identifies the trigger"
|
| 183 |
+
recommended_actions:
|
| 184 |
+
immediate: "Review safety loop PID parameters and increase damping"
|
| 185 |
+
long_term: "Tune safety loop with proper step-response testing"
|
| 186 |
+
system_priority: "MEDIUM"
|
| 187 |
+
occurrence_count: 1
|
| 188 |
+
|
| 189 |
+
- id: FM-005
|
| 190 |
+
name: "Dry Running"
|
| 191 |
+
category: "operational"
|
| 192 |
+
symptoms:
|
| 193 |
+
primary: "PT130/PT110 ratio drops below expected AND TT130 rises abnormally"
|
| 194 |
+
secondary:
|
| 195 |
+
- "Pump compressing gas instead of liquid"
|
| 196 |
+
- "Very damaging to seals and valves"
|
| 197 |
+
- "Mass flow drops to near zero"
|
| 198 |
+
sensor_signatures:
|
| 199 |
+
- tag: PT130
|
| 200 |
+
tagindex: 18
|
| 201 |
+
behavior: "Cannot reach target — pressure builds very slowly"
|
| 202 |
+
- tag: TT130
|
| 203 |
+
tagindex: 27
|
| 204 |
+
behavior: "Abnormal temperature rise (adiabatic gas compression)"
|
| 205 |
+
- tag: FT140
|
| 206 |
+
tagindex: 7
|
| 207 |
+
behavior: "Near zero flow"
|
| 208 |
+
physical_mechanism: >
|
| 209 |
+
When the inlet has insufficient liquid (boil-off, empty tank, blocked line),
|
| 210 |
+
the pump compresses gas instead of liquid. Gas has very low bulk modulus
|
| 211 |
+
compared to liquid, so pressure builds much slower. Adiabatic compression
|
| 212 |
+
of gas produces much higher discharge temperatures than liquid compression.
|
| 213 |
+
This is the most damaging operating condition — stop pump immediately.
|
| 214 |
+
candidate_causes:
|
| 215 |
+
- cause: "Insufficient cooldown (two-phase inlet)"
|
| 216 |
+
prior: 0.40
|
| 217 |
+
evidence: "D1 diagnostic signature"
|
| 218 |
+
- cause: "Cryotank empty/low liquid level"
|
| 219 |
+
prior: 0.30
|
| 220 |
+
evidence: "Check LT200"
|
| 221 |
+
- cause: "Inlet line blockage (ice formation)"
|
| 222 |
+
prior: 0.20
|
| 223 |
+
evidence: "Check TT110 for sudden changes"
|
| 224 |
+
- cause: "Valve stuck closed in inlet path"
|
| 225 |
+
prior: 0.10
|
| 226 |
+
evidence: "Check valve positions"
|
| 227 |
+
discriminating_tests:
|
| 228 |
+
- test: "Check TT110 — must be below saturation temperature at PT110"
|
| 229 |
+
interpretation: "TT110 > T_sat = two-phase or gas inlet"
|
| 230 |
+
recommended_actions:
|
| 231 |
+
immediate: "STOP PUMP. Verify inlet conditions before restart."
|
| 232 |
+
long_term: "Implement automatic dry-running protection based on D1"
|
| 233 |
+
system_priority: "CRITICAL"
|
| 234 |
+
occurrence_count: 0
|
| 235 |
+
notes: "Diagnostic D1 monitors for this continuously"
|
| 236 |
+
|
| 237 |
+
- id: FM-006
|
| 238 |
+
name: "ICV Thermal Backflow"
|
| 239 |
+
category: "check_valve"
|
| 240 |
+
symptoms:
|
| 241 |
+
primary: "TT110 rises while PT130 is high"
|
| 242 |
+
secondary:
|
| 243 |
+
- "Hot compressed fluid flowing backward through ICV"
|
| 244 |
+
- "Can cause inlet boiling — cascading failure"
|
| 245 |
+
sensor_signatures:
|
| 246 |
+
- tag: TT110
|
| 247 |
+
tagindex: 24
|
| 248 |
+
behavior: "Rising temperature correlated with high PT130"
|
| 249 |
+
- tag: PT130
|
| 250 |
+
tagindex: 18
|
| 251 |
+
behavior: "High pressure (normal operation)"
|
| 252 |
+
physical_mechanism: >
|
| 253 |
+
High-pressure, heated fluid leaks backward through the ICV during the
|
| 254 |
+
compression stroke. This hot fluid heats the inlet line, potentially
|
| 255 |
+
causing the subcooled inlet liquid to boil. Once boiling starts at the
|
| 256 |
+
inlet, the pump transitions to gas compression (FM-005 cascade).
|
| 257 |
+
candidate_causes:
|
| 258 |
+
- cause: "ICV not seating properly (same root cause as FM-001)"
|
| 259 |
+
prior: 0.70
|
| 260 |
+
evidence: "D3 diagnostic signature"
|
| 261 |
+
- cause: "ICV spring force insufficient at high pressure"
|
| 262 |
+
prior: 0.30
|
| 263 |
+
evidence: "Theoretical"
|
| 264 |
+
discriminating_tests:
|
| 265 |
+
- test: "Monitor TT110 rate of change during high-pressure operation"
|
| 266 |
+
interpretation: ">2 K/min rise = significant backflow"
|
| 267 |
+
recommended_actions:
|
| 268 |
+
immediate: "Check ICV seating. Same remediation as FM-001."
|
| 269 |
+
long_term: "Implement TT110 rate-of-rise protection"
|
| 270 |
+
system_priority: "HIGH"
|
| 271 |
+
occurrence_count: 0
|
| 272 |
+
notes: "Diagnostic D3 monitors this. Shares root cause with FM-001."
|
| 273 |
+
|
| 274 |
+
- id: FM-007
|
| 275 |
+
name: "Gas Entrainment / Insufficient Subcooling"
|
| 276 |
+
category: "thermal"
|
| 277 |
+
symptoms:
|
| 278 |
+
primary: "Reduced pump efficiency with two-phase inlet conditions"
|
| 279 |
+
secondary:
|
| 280 |
+
- "Inlet quality > 0 (not fully subcooled liquid)"
|
| 281 |
+
- "Volumetric efficiency reduced by gas fraction"
|
| 282 |
+
- "Less severe than full dry running (FM-005)"
|
| 283 |
+
sensor_signatures:
|
| 284 |
+
- tag: TT110
|
| 285 |
+
tagindex: 24
|
| 286 |
+
behavior: "Near or above saturation temperature at inlet pressure"
|
| 287 |
+
- tag: PT110
|
| 288 |
+
tagindex: 17
|
| 289 |
+
behavior: "May be low (insufficient supply pressure)"
|
| 290 |
+
physical_mechanism: >
|
| 291 |
+
When inlet liquid is not sufficiently subcooled (temperature close to
|
| 292 |
+
boiling point at inlet pressure), flash gas forms in the cylinder during
|
| 293 |
+
the suction stroke. This gas takes up volume that should be liquid,
|
| 294 |
+
reducing the effective displacement. The pump is 169x more efficient
|
| 295 |
+
with fully subcooled liquid than with gas (Technical PDF p.17).
|
| 296 |
+
candidate_causes:
|
| 297 |
+
- cause: "Cooldown incomplete"
|
| 298 |
+
prior: 0.40
|
| 299 |
+
evidence: "Check TT110 vs saturation temperature"
|
| 300 |
+
- cause: "Cryotank pressure too low (reduced subcooling margin)"
|
| 301 |
+
prior: 0.30
|
| 302 |
+
evidence: "Check PT110"
|
| 303 |
+
- cause: "Heat leak into inlet piping"
|
| 304 |
+
prior: 0.20
|
| 305 |
+
evidence: "Check insulation"
|
| 306 |
+
- cause: "Rapid pressure transients causing flash gas"
|
| 307 |
+
prior: 0.10
|
| 308 |
+
evidence: "Theoretical"
|
| 309 |
+
discriminating_tests:
|
| 310 |
+
- test: "Calculate subcooling margin: T_sat(PT110) - TT110"
|
| 311 |
+
interpretation: "< 2K margin = risk of flash gas"
|
| 312 |
+
recommended_actions:
|
| 313 |
+
immediate: "Verify adequate subcooling margin before fill"
|
| 314 |
+
long_term: "Add subcooling verification to pre-fill checklist"
|
| 315 |
+
system_priority: "MEDIUM"
|
| 316 |
+
occurrence_count: 0
|
| 317 |
+
|
| 318 |
+
- id: FM-008
|
| 319 |
+
name: "Cryotank Pressure Decay"
|
| 320 |
+
category: "system"
|
| 321 |
+
symptoms:
|
| 322 |
+
primary: "Pressure response drifts downward during extended fills"
|
| 323 |
+
secondary:
|
| 324 |
+
- "PT130 and flow rate decline over the fill duration"
|
| 325 |
+
- "Due to decreasing cryotank pressure (small tank)"
|
| 326 |
+
- "Not a pump fault — operational characteristic"
|
| 327 |
+
sensor_signatures:
|
| 328 |
+
- tag: PT130
|
| 329 |
+
tagindex: 18
|
| 330 |
+
behavior: "Gradual downward drift during extended fills"
|
| 331 |
+
- tag: PT110
|
| 332 |
+
tagindex: 17
|
| 333 |
+
behavior: "Decreasing during fill (cryotank depleting)"
|
| 334 |
+
- tag: LT200
|
| 335 |
+
tagindex: 12
|
| 336 |
+
behavior: "Decreasing (if available)"
|
| 337 |
+
physical_mechanism: >
|
| 338 |
+
The pump operates with a fixed pressure ratio. As cryotank pressure
|
| 339 |
+
drops (small tank being depleted), the inlet pressure drops, and since
|
| 340 |
+
the pump's pressure ratio is approximately constant, the discharge
|
| 341 |
+
pressure also decreases. This is documented in the Technical PDF p.22
|
| 342 |
+
and is an expected operational characteristic, not a fault.
|
| 343 |
+
candidate_causes:
|
| 344 |
+
- cause: "Normal tank depletion (small tank)"
|
| 345 |
+
prior: 0.80
|
| 346 |
+
evidence: "Technical PDF p.22 — documented behavior"
|
| 347 |
+
- cause: "Boil-off reducing tank pressure faster than expected"
|
| 348 |
+
prior: 0.15
|
| 349 |
+
evidence: "Check boil-off rate"
|
| 350 |
+
- cause: "Tank regulator malfunction"
|
| 351 |
+
prior: 0.05
|
| 352 |
+
evidence: "Theoretical"
|
| 353 |
+
discriminating_tests:
|
| 354 |
+
- test: "Plot PT110 and PT130 together over the fill"
|
| 355 |
+
interpretation: "Parallel decline = tank depletion. PT130 drops faster = pump issue"
|
| 356 |
+
recommended_actions:
|
| 357 |
+
immediate: "Expected behavior — no action unless decline rate is abnormal"
|
| 358 |
+
long_term: "Size production tank to maintain pressure over target fill profile"
|
| 359 |
+
system_priority: "LOW"
|
| 360 |
+
occurrence_count: 0
|
| 361 |
+
notes: "Not a fault — operational characteristic of small test tank"
|
analysis/knowledge/operational_context.yaml
ADDED
|
@@ -0,0 +1,154 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CSH2 Cryogenic Pump — Operational Context Knowledge
|
| 2 |
+
# Practical knowledge that operators have but specifications don't document.
|
| 3 |
+
# Derived from maintenance_log.xlsx, Technical PDF, and data analysis.
|
| 4 |
+
|
| 5 |
+
sensor_trustworthiness:
|
| 6 |
+
TT110:
|
| 7 |
+
notes: >
|
| 8 |
+
TT110 reads pipe wall temperature, not fluid temperature. After cooldown,
|
| 9 |
+
it needs 10-15 minutes to equilibrate with the flowing cryogen. During
|
| 10 |
+
transients (pump start/stop, flow changes), TT110 lags the actual fluid
|
| 11 |
+
temperature by 5-30 seconds depending on flow rate.
|
| 12 |
+
workaround: >
|
| 13 |
+
For precise inlet temperature, use virtual T_inlet back-calculated from
|
| 14 |
+
discharge state via ThermoProps (physics/thermo.py). The virtual sensor
|
| 15 |
+
uses PT110 + TT130 + isentropic relations to infer inlet conditions.
|
| 16 |
+
known_faults:
|
| 17 |
+
- "Jul 16, 2025: 0-32K readings are sensor fault, not real cryo temps"
|
| 18 |
+
- "Filter out readings during known warm periods where TT110 < 50K"
|
| 19 |
+
|
| 20 |
+
FT140:
|
| 21 |
+
notes: >
|
| 22 |
+
FT140 is a Coriolis mass flow meter calibrated for LN2. When running LH2,
|
| 23 |
+
the density difference (LN2: 808 kg/m³ vs LH2: 70.8 kg/m³) causes the
|
| 24 |
+
raw reading to be 100-200x too high. The FT140Calibrator applies a density
|
| 25 |
+
correction factor of approximately 11.4x.
|
| 26 |
+
workaround: >
|
| 27 |
+
Always apply FT140Calibrator (physics/ft140_calibration.py) density correction
|
| 28 |
+
before interpreting flow readings. Near-zero flow at low pump speeds is
|
| 29 |
+
expected and not a fault — the meter has a low-flow cutoff.
|
| 30 |
+
known_faults:
|
| 31 |
+
- "Raw FT140 reads ~100-200x too high for LH2 — apply correction"
|
| 32 |
+
- "Near-zero flow at idle is expected, not a sensor fault"
|
| 33 |
+
|
| 34 |
+
PT100:
|
| 35 |
+
notes: >
|
| 36 |
+
PT100 is stuck at -14.86 bar in the current dataset. This is not a
|
| 37 |
+
physical pressure — it's a sensor fault.
|
| 38 |
+
workaround: >
|
| 39 |
+
Use default_when_invalid=2.0 barg for any calculations requiring PT100
|
| 40 |
+
(e.g., Psat estimation). This is the saturation pressure approximation.
|
| 41 |
+
known_faults:
|
| 42 |
+
- "Stuck at -14.86 bar — use 2.0 barg default"
|
| 43 |
+
|
| 44 |
+
CalcMass:
|
| 45 |
+
notes: >
|
| 46 |
+
CalcMass is stuck at 0.18 kg in the current dataset. The PLC calculation
|
| 47 |
+
is not updating.
|
| 48 |
+
workaround: >
|
| 49 |
+
Use FT140 integration (corrected) for actual dispensed mass calculation.
|
| 50 |
+
Integrate corrected FT140 over the fill duration.
|
| 51 |
+
known_faults:
|
| 52 |
+
- "Stuck at 0.18 kg — use FT140 integration instead"
|
| 53 |
+
|
| 54 |
+
M130_Speed_vs_MC130:
|
| 55 |
+
notes: >
|
| 56 |
+
Tag 68 (M130_Speed) replaces tag 13 (MC130_VFD_Speed) after May 28, 2025.
|
| 57 |
+
Both map to motor speed percentage but are different register addresses.
|
| 58 |
+
TAG_ALIASES in config.py handles this mapping.
|
| 59 |
+
workaround: >
|
| 60 |
+
Check TAG_ALIASES. Use tag 68 for post-May data, tag 13 for pre-May.
|
| 61 |
+
Both are valid speed readings.
|
| 62 |
+
|
| 63 |
+
post_maintenance_expectations:
|
| 64 |
+
hp_seal_replacement:
|
| 65 |
+
description: "After HP seal replacement or energizer work"
|
| 66 |
+
expected_behavior:
|
| 67 |
+
- "Initial friction: 7-9 kgf (from Jul 18 maintenance entry)"
|
| 68 |
+
- "Break-in period: ~1000 cycles to reach 4-5 kgf steady state"
|
| 69 |
+
- "Motor current will be higher initially — this is normal"
|
| 70 |
+
- "Volumetric efficiency may be slightly higher initially (tighter seal)"
|
| 71 |
+
timeline: "1000 cycles (~2-3 days at normal testing rate)"
|
| 72 |
+
ref: "2025-07-18 maintenance event"
|
| 73 |
+
|
| 74 |
+
icv_replacement:
|
| 75 |
+
description: "After ICV replacement or reseating"
|
| 76 |
+
expected_behavior:
|
| 77 |
+
- "Immediate improvement in pressure build if ICV was the issue"
|
| 78 |
+
- "New valve may need seating-in (initial performance varies)"
|
| 79 |
+
- "Jul 22: orientation fix restored 700 bar immediately"
|
| 80 |
+
- "Jul 14: polished ICV helped but didn't fully fix — surface not flat"
|
| 81 |
+
timeline: "Immediate improvement expected"
|
| 82 |
+
ref: "2025-07-01, 2025-07-14, 2025-07-22 events"
|
| 83 |
+
|
| 84 |
+
dcv_polishing:
|
| 85 |
+
description: "After DCV sealing surface polishing"
|
| 86 |
+
expected_behavior:
|
| 87 |
+
- "Immediate improvement in leak test results"
|
| 88 |
+
- "Run PT130 decay test before and after to quantify improvement"
|
| 89 |
+
- "Jul 1: polishing improved seal but some residual leakage"
|
| 90 |
+
timeline: "Immediate improvement"
|
| 91 |
+
ref: "2025-07-01 maintenance event"
|
| 92 |
+
|
| 93 |
+
regime_caveats:
|
| 94 |
+
ln2_vs_lh2:
|
| 95 |
+
description: "LN2 and LH2 data are not directly comparable"
|
| 96 |
+
details: >
|
| 97 |
+
LN2 (77K, 808 kg/m³) and LH2 (20K, 70.8 kg/m³) have very different
|
| 98 |
+
fluid properties that shift all performance curves. Compression ratios,
|
| 99 |
+
volumetric efficiencies, heat transfer rates, and valve dynamics all
|
| 100 |
+
change significantly between fluids. The SPEED_SCALE_FACTOR in config.py
|
| 101 |
+
was calibrated for LN2 and needs adjustment for LH2.
|
| 102 |
+
impact:
|
| 103 |
+
- "Flow rate scales with density ratio"
|
| 104 |
+
- "Pressure build rate changes with bulk modulus"
|
| 105 |
+
- "Thermal effects are very different (77K vs 20K)"
|
| 106 |
+
|
| 107 |
+
pressure_drift:
|
| 108 |
+
description: "Discharge pressure drifts down during extended fills"
|
| 109 |
+
details: >
|
| 110 |
+
Documented in Technical PDF p.22. The pump operates at approximately
|
| 111 |
+
fixed pressure ratio. As the small test cryotank depletes, inlet
|
| 112 |
+
pressure (PT110) drops, and PT130 follows. This is NOT a fault —
|
| 113 |
+
it's an expected consequence of the small tank size.
|
| 114 |
+
impact:
|
| 115 |
+
- "PT130 and flow rate decline over extended fills"
|
| 116 |
+
- "Production tanks will be sized to maintain pressure"
|
| 117 |
+
|
| 118 |
+
subcooling_sensitivity:
|
| 119 |
+
description: "Pump performance is very sensitive to inlet subcooling"
|
| 120 |
+
details: >
|
| 121 |
+
From Technical PDF p.17: the pump is 169x more efficient with
|
| 122 |
+
subcooled liquid than with gas. Even small amounts of flash gas
|
| 123 |
+
at the inlet significantly reduce volumetric efficiency. Ensure
|
| 124 |
+
adequate subcooling margin (>5K) before starting fills.
|
| 125 |
+
impact:
|
| 126 |
+
- "Inlet quality > 0 = reduced efficiency"
|
| 127 |
+
- "Subcooling margin < 2K = risk zone"
|
| 128 |
+
|
| 129 |
+
system_personality:
|
| 130 |
+
recurring_issues:
|
| 131 |
+
- priority: 1
|
| 132 |
+
description: "ICV sealing is the #1 recurring issue"
|
| 133 |
+
details: "3 occurrences in July 2025. Check ICV first for any pressure build problem."
|
| 134 |
+
refs: ["2025-07-01", "2025-07-14", "2025-07-22"]
|
| 135 |
+
- priority: 2
|
| 136 |
+
description: "DCV sealing is the #2 issue"
|
| 137 |
+
details: "1 occurrence. DCV polishing was effective but residual leakage remained."
|
| 138 |
+
refs: ["2025-07-01"]
|
| 139 |
+
- priority: 3
|
| 140 |
+
description: "Safety loop tuning needs attention at high pressures"
|
| 141 |
+
details: "AOV141 hunting observed. PID tuning required."
|
| 142 |
+
refs: ["2025-07-22"]
|
| 143 |
+
|
| 144 |
+
strengths:
|
| 145 |
+
- "HP seal life is excellent (8.2M cycle lower bound = 3+ months at 20 fills/day)"
|
| 146 |
+
- "Phase 1B achieved 100% uptime across 6 fills"
|
| 147 |
+
- "Specific energy of 0.26 kWh/kg matches target"
|
| 148 |
+
- "Back-to-back fill capability demonstrated (4 consecutive fills)"
|
| 149 |
+
|
| 150 |
+
known_limitations:
|
| 151 |
+
- "Pump gas ratio 14.4% vs 2.7% target — efficiency gap exists"
|
| 152 |
+
- "Peak pressure 370 bar vs 700 bar H70 target — further testing needed"
|
| 153 |
+
- "Small test tank causes pressure drift during extended fills"
|
| 154 |
+
- "FT140 requires density correction for LH2 — raw readings are wrong"
|
analysis/knowledge/reasoning_chains.yaml
ADDED
|
@@ -0,0 +1,302 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CSH2 Cryogenic Pump — Diagnostic Reasoning Chains
|
| 2 |
+
# Step-by-step decision trees encoding how an expert reasons from observation to diagnosis.
|
| 3 |
+
# Each step specifies: what tool to call, what to look for, and branching rules.
|
| 4 |
+
|
| 5 |
+
reasoning_chains:
|
| 6 |
+
- id: RC-001
|
| 7 |
+
name: "Low Pressure Build Diagnostic"
|
| 8 |
+
trigger: "PT130 cannot reach target pressure"
|
| 9 |
+
related_failure_modes: [FM-001, FM-002, FM-003, FM-007]
|
| 10 |
+
steps:
|
| 11 |
+
- step: 1
|
| 12 |
+
action: "query_sensor_data"
|
| 13 |
+
description: "Pull PT130, PT110, TT110, VFD speed for the time window"
|
| 14 |
+
parameters:
|
| 15 |
+
tagindices: [18, 17, 24, 13]
|
| 16 |
+
look_for: "PT130 plateau level, PT110 stability, TT110 trend"
|
| 17 |
+
next:
|
| 18 |
+
if_pt130_below_50pct_target: "step_2"
|
| 19 |
+
if_pt130_slow_build: "step_5"
|
| 20 |
+
|
| 21 |
+
- step: 2
|
| 22 |
+
action: "run_simulation"
|
| 23 |
+
description: "Compute expected PT130 at current operating conditions"
|
| 24 |
+
parameters_from: "extracted operating conditions"
|
| 25 |
+
look_for: "Sim vs measured gap magnitude"
|
| 26 |
+
next:
|
| 27 |
+
if_gap_gt_20pct: "step_3"
|
| 28 |
+
if_gap_lt_20pct: "step_6"
|
| 29 |
+
|
| 30 |
+
- step: 3
|
| 31 |
+
action: "Run DCV leak test"
|
| 32 |
+
description: "Check PT130 decay rate with pump stopped"
|
| 33 |
+
look_for: "Decay rate in bar/min"
|
| 34 |
+
next:
|
| 35 |
+
if_decay_gt_5_bar_per_min: "conclude_FM002"
|
| 36 |
+
if_decay_lt_5_bar_per_min: "step_4"
|
| 37 |
+
|
| 38 |
+
- step: 4
|
| 39 |
+
action: "Compare pressure build at multiple speeds"
|
| 40 |
+
description: "Run pump at 10%, 30%, 50% speed and compare pressure"
|
| 41 |
+
look_for: "Does pressure gap worsen at higher speeds?"
|
| 42 |
+
next:
|
| 43 |
+
if_worse_at_higher_speed: "conclude_FM001"
|
| 44 |
+
if_similar_at_all_speeds: "step_5"
|
| 45 |
+
|
| 46 |
+
- step: 5
|
| 47 |
+
action: "Check inlet conditions"
|
| 48 |
+
description: "Verify subcooling margin: T_sat(PT110) - TT110"
|
| 49 |
+
look_for: "Subcooling margin in K"
|
| 50 |
+
next:
|
| 51 |
+
if_margin_lt_2K: "conclude_FM007"
|
| 52 |
+
if_margin_gt_2K: "step_6"
|
| 53 |
+
|
| 54 |
+
- step: 6
|
| 55 |
+
action: "search_domain_knowledge"
|
| 56 |
+
description: "Search maintenance history for recent ICV/DCV work"
|
| 57 |
+
parameters:
|
| 58 |
+
category: "maintenance_history"
|
| 59 |
+
component: "ICV"
|
| 60 |
+
look_for: "Recent maintenance events"
|
| 61 |
+
next:
|
| 62 |
+
if_recent_icv_work: "conclude_FM001_installation"
|
| 63 |
+
if_no_recent_work: "conclude_FM003"
|
| 64 |
+
|
| 65 |
+
conclusions:
|
| 66 |
+
conclude_FM001:
|
| 67 |
+
diagnosis: "ICV reverse flow — leak worsens with speed"
|
| 68 |
+
confidence: "high"
|
| 69 |
+
failure_mode: FM-001
|
| 70 |
+
conclude_FM001_installation:
|
| 71 |
+
diagnosis: "ICV installation issue — recent maintenance"
|
| 72 |
+
confidence: "high"
|
| 73 |
+
failure_mode: FM-001
|
| 74 |
+
conclude_FM002:
|
| 75 |
+
diagnosis: "DCV leakage — static pressure decay"
|
| 76 |
+
confidence: "high"
|
| 77 |
+
failure_mode: FM-002
|
| 78 |
+
conclude_FM003:
|
| 79 |
+
diagnosis: "HP seal blowby — gradual degradation"
|
| 80 |
+
confidence: "moderate"
|
| 81 |
+
failure_mode: FM-003
|
| 82 |
+
conclude_FM007:
|
| 83 |
+
diagnosis: "Insufficient subcooling — two-phase inlet"
|
| 84 |
+
confidence: "high"
|
| 85 |
+
failure_mode: FM-007
|
| 86 |
+
|
| 87 |
+
- id: RC-002
|
| 88 |
+
name: "Pressure Oscillation Diagnostic"
|
| 89 |
+
trigger: "PT130 showing high-frequency oscillations"
|
| 90 |
+
related_failure_modes: [FM-004]
|
| 91 |
+
steps:
|
| 92 |
+
- step: 1
|
| 93 |
+
action: "query_sensor_data"
|
| 94 |
+
description: "Pull PT130 at 1-second resolution for oscillation analysis"
|
| 95 |
+
parameters:
|
| 96 |
+
tagindices: [18, 1]
|
| 97 |
+
resolution: "1sec"
|
| 98 |
+
look_for: "Oscillation frequency and amplitude"
|
| 99 |
+
|
| 100 |
+
- step: 2
|
| 101 |
+
action: "Correlate with AOV141 cycling"
|
| 102 |
+
description: "Check if AOV140/141 is cycling in sync with PT130"
|
| 103 |
+
look_for: "Phase correlation between pressure and valve"
|
| 104 |
+
next:
|
| 105 |
+
if_correlated: "conclude_FM004"
|
| 106 |
+
if_not_correlated: "step_3"
|
| 107 |
+
|
| 108 |
+
- step: 3
|
| 109 |
+
action: "Check DCV dynamics"
|
| 110 |
+
description: "Analyze stroke timing — DCV may be bouncing"
|
| 111 |
+
look_for: "DCV open/close timing anomalies"
|
| 112 |
+
next:
|
| 113 |
+
default: "conclude_valve_dynamics"
|
| 114 |
+
|
| 115 |
+
conclusions:
|
| 116 |
+
conclude_FM004:
|
| 117 |
+
diagnosis: "Safety loop oscillation — AOV141 hunting"
|
| 118 |
+
confidence: "high"
|
| 119 |
+
failure_mode: FM-004
|
| 120 |
+
conclude_valve_dynamics:
|
| 121 |
+
diagnosis: "DCV dynamic behavior anomaly — investigate spring force"
|
| 122 |
+
confidence: "moderate"
|
| 123 |
+
failure_mode: FM-002
|
| 124 |
+
|
| 125 |
+
- id: RC-003
|
| 126 |
+
name: "Elevated Discharge Temperature Diagnostic"
|
| 127 |
+
trigger: "TT130 higher than isentropic prediction"
|
| 128 |
+
related_failure_modes: [FM-005, FM-006]
|
| 129 |
+
steps:
|
| 130 |
+
- step: 1
|
| 131 |
+
action: "query_sensor_data"
|
| 132 |
+
description: "Pull TT130, PT130, PT110, TT110"
|
| 133 |
+
parameters:
|
| 134 |
+
tagindices: [27, 18, 17, 24]
|
| 135 |
+
look_for: "TT130 excess above isentropic prediction"
|
| 136 |
+
|
| 137 |
+
- step: 2
|
| 138 |
+
action: "run_simulation"
|
| 139 |
+
description: "Compute isentropic discharge temperature"
|
| 140 |
+
look_for: "Measured TT130 vs predicted"
|
| 141 |
+
next:
|
| 142 |
+
if_excess_gt_20K: "step_3"
|
| 143 |
+
if_excess_lt_20K: "conclude_normal_friction"
|
| 144 |
+
|
| 145 |
+
- step: 3
|
| 146 |
+
action: "Check pressure ratio"
|
| 147 |
+
description: "PT130/PT110 ratio vs expected"
|
| 148 |
+
look_for: "Is pressure building normally?"
|
| 149 |
+
next:
|
| 150 |
+
if_ratio_low: "conclude_FM005"
|
| 151 |
+
if_ratio_normal: "step_4"
|
| 152 |
+
|
| 153 |
+
- step: 4
|
| 154 |
+
action: "Check TT110 trend"
|
| 155 |
+
description: "Is TT110 rising during operation?"
|
| 156 |
+
look_for: "TT110 rate of change"
|
| 157 |
+
next:
|
| 158 |
+
if_tt110_rising: "conclude_FM006"
|
| 159 |
+
if_tt110_stable: "conclude_friction"
|
| 160 |
+
|
| 161 |
+
conclusions:
|
| 162 |
+
conclude_FM005:
|
| 163 |
+
diagnosis: "Dry running — pump compressing gas"
|
| 164 |
+
confidence: "high"
|
| 165 |
+
failure_mode: FM-005
|
| 166 |
+
conclude_FM006:
|
| 167 |
+
diagnosis: "ICV thermal backflow — hot fluid heating inlet"
|
| 168 |
+
confidence: "high"
|
| 169 |
+
failure_mode: FM-006
|
| 170 |
+
conclude_normal_friction:
|
| 171 |
+
diagnosis: "Normal — friction heating within expected range"
|
| 172 |
+
confidence: "high"
|
| 173 |
+
conclude_friction:
|
| 174 |
+
diagnosis: "Elevated friction — possible seal break-in or bore damage"
|
| 175 |
+
confidence: "moderate"
|
| 176 |
+
failure_mode: FM-003
|
| 177 |
+
|
| 178 |
+
- id: RC-004
|
| 179 |
+
name: "Reduced Flow Rate Diagnostic"
|
| 180 |
+
trigger: "FT140 below expected mass flow rate"
|
| 181 |
+
related_failure_modes: [FM-001, FM-002, FM-007]
|
| 182 |
+
steps:
|
| 183 |
+
- step: 1
|
| 184 |
+
action: "query_sensor_data"
|
| 185 |
+
description: "Pull FT140, PT130, VFD speed"
|
| 186 |
+
parameters:
|
| 187 |
+
tagindices: [7, 18, 13]
|
| 188 |
+
look_for: "Flow rate vs expected at current speed"
|
| 189 |
+
|
| 190 |
+
- step: 2
|
| 191 |
+
action: "Check FT140 calibration"
|
| 192 |
+
description: "Verify FT140 density correction factor is applied"
|
| 193 |
+
look_for: "Is fluid H2 or N2? Apply correct density correction"
|
| 194 |
+
next:
|
| 195 |
+
if_calibration_wrong: "conclude_calibration_error"
|
| 196 |
+
if_calibration_correct: "step_3"
|
| 197 |
+
|
| 198 |
+
- step: 3
|
| 199 |
+
action: "run_simulation"
|
| 200 |
+
description: "Compute expected flow at current conditions"
|
| 201 |
+
look_for: "Vol efficiency and loss breakdown"
|
| 202 |
+
next:
|
| 203 |
+
if_icv_leak_dominant: "conclude_FM001"
|
| 204 |
+
if_dcv_leak_dominant: "conclude_FM002"
|
| 205 |
+
if_gas_inlet: "conclude_FM007"
|
| 206 |
+
|
| 207 |
+
conclusions:
|
| 208 |
+
conclude_FM001:
|
| 209 |
+
diagnosis: "ICV leakage reducing effective displacement"
|
| 210 |
+
confidence: "high"
|
| 211 |
+
failure_mode: FM-001
|
| 212 |
+
conclude_FM002:
|
| 213 |
+
diagnosis: "DCV leakage losing compressed fluid"
|
| 214 |
+
confidence: "high"
|
| 215 |
+
failure_mode: FM-002
|
| 216 |
+
conclude_FM007:
|
| 217 |
+
diagnosis: "Two-phase inlet reducing effective displacement"
|
| 218 |
+
confidence: "high"
|
| 219 |
+
failure_mode: FM-007
|
| 220 |
+
conclude_calibration_error:
|
| 221 |
+
diagnosis: "FT140 reading incorrect — apply density correction"
|
| 222 |
+
confidence: "high"
|
| 223 |
+
|
| 224 |
+
- id: RC-005
|
| 225 |
+
name: "Motor Power Anomaly Diagnostic"
|
| 226 |
+
trigger: "Motor power or current anomaly at given speed"
|
| 227 |
+
related_failure_modes: [FM-003]
|
| 228 |
+
steps:
|
| 229 |
+
- step: 1
|
| 230 |
+
action: "query_sensor_data"
|
| 231 |
+
description: "Pull VFD power, current, speed, torque"
|
| 232 |
+
parameters:
|
| 233 |
+
tagindices: [43, 45, 13, 48]
|
| 234 |
+
look_for: "Power/current at each speed point"
|
| 235 |
+
|
| 236 |
+
- step: 2
|
| 237 |
+
action: "Compare to baseline power-speed curve"
|
| 238 |
+
description: "Plot power vs speed, compare to reference"
|
| 239 |
+
look_for: "Offset direction"
|
| 240 |
+
next:
|
| 241 |
+
if_power_higher: "step_3"
|
| 242 |
+
if_power_lower: "conclude_reduced_load"
|
| 243 |
+
|
| 244 |
+
- step: 3
|
| 245 |
+
action: "search_domain_knowledge"
|
| 246 |
+
description: "Check seal friction baseline expectations"
|
| 247 |
+
parameters:
|
| 248 |
+
category: "operational_context"
|
| 249 |
+
look_for: "Post-maintenance friction expectations"
|
| 250 |
+
next:
|
| 251 |
+
if_recent_seal_work: "conclude_seal_break_in"
|
| 252 |
+
if_no_recent_work: "conclude_FM003"
|
| 253 |
+
|
| 254 |
+
conclusions:
|
| 255 |
+
conclude_FM003:
|
| 256 |
+
diagnosis: "HP seal friction increasing — possible bore damage"
|
| 257 |
+
confidence: "moderate"
|
| 258 |
+
failure_mode: FM-003
|
| 259 |
+
conclude_seal_break_in:
|
| 260 |
+
diagnosis: "Normal seal break-in after maintenance (7-9 kgf → 4-5 kgf)"
|
| 261 |
+
confidence: "high"
|
| 262 |
+
conclude_reduced_load:
|
| 263 |
+
diagnosis: "Reduced load — possible liquid starvation or low discharge pressure"
|
| 264 |
+
confidence: "moderate"
|
| 265 |
+
|
| 266 |
+
- id: RC-006
|
| 267 |
+
name: "Post-Maintenance Performance Verification"
|
| 268 |
+
trigger: "Performance check after maintenance event"
|
| 269 |
+
related_failure_modes: [FM-001, FM-002, FM-003]
|
| 270 |
+
steps:
|
| 271 |
+
- step: 1
|
| 272 |
+
action: "search_domain_knowledge"
|
| 273 |
+
description: "Look up the maintenance event details"
|
| 274 |
+
parameters:
|
| 275 |
+
category: "maintenance_history"
|
| 276 |
+
look_for: "What was done, what to expect"
|
| 277 |
+
|
| 278 |
+
- step: 2
|
| 279 |
+
action: "query_sensor_data"
|
| 280 |
+
description: "Pull first test run after maintenance"
|
| 281 |
+
look_for: "Compare to pre-maintenance baseline"
|
| 282 |
+
|
| 283 |
+
- step: 3
|
| 284 |
+
action: "run_diagnostics"
|
| 285 |
+
description: "Run full D1-D8 diagnostic sweep"
|
| 286 |
+
look_for: "Any new or resolved diagnostic flags"
|
| 287 |
+
|
| 288 |
+
- step: 4
|
| 289 |
+
action: "run_simulation"
|
| 290 |
+
description: "Compare sim prediction to post-maintenance performance"
|
| 291 |
+
look_for: "Has the residual changed after maintenance?"
|
| 292 |
+
|
| 293 |
+
conclusions:
|
| 294 |
+
conclude_improved:
|
| 295 |
+
diagnosis: "Maintenance successfully addressed the issue"
|
| 296 |
+
confidence: "high"
|
| 297 |
+
conclude_no_change:
|
| 298 |
+
diagnosis: "Maintenance did not resolve the underlying issue"
|
| 299 |
+
confidence: "moderate"
|
| 300 |
+
conclude_degraded:
|
| 301 |
+
diagnosis: "Performance worse after maintenance — investigate installation"
|
| 302 |
+
confidence: "high"
|
analysis/knowledge_retriever.py
ADDED
|
@@ -0,0 +1,432 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Domain Knowledge Retriever for CSH2 LLM Analysis
|
| 3 |
+
|
| 4 |
+
Pattern-based retrieval: maps sensor data patterns to relevant failure modes,
|
| 5 |
+
reasoning chains, and operational context from the YAML knowledge base.
|
| 6 |
+
|
| 7 |
+
At this scale (8 failure modes, 6 chains), rule-based retrieval is more precise
|
| 8 |
+
than embedding-based RAG — no vector DB needed.
|
| 9 |
+
"""
|
| 10 |
+
import logging
|
| 11 |
+
from pathlib import Path
|
| 12 |
+
from typing import Any, Dict, List, Optional, Set
|
| 13 |
+
|
| 14 |
+
try:
|
| 15 |
+
import yaml
|
| 16 |
+
except ImportError:
|
| 17 |
+
yaml = None
|
| 18 |
+
|
| 19 |
+
logger = logging.getLogger(__name__)
|
| 20 |
+
|
| 21 |
+
KNOWLEDGE_DIR = Path(__file__).parent / "knowledge"
|
| 22 |
+
|
| 23 |
+
# Trigger thresholds (physically meaningful boundaries)
|
| 24 |
+
LOW_PRESSURE_THRESHOLD_BAR = 350 # H35 target — below this is underperforming
|
| 25 |
+
HIGH_DISCHARGE_TEMP_K = 100 # Unusual for LH2 (inlet ~25K)
|
| 26 |
+
LOW_FLOW_THRESHOLD_KG_MIN = 0.5 # < half of demonstrated 1.13 kg/min
|
| 27 |
+
MOTOR_SPEED_ACTIVE_THRESHOLD = 83 # Scaled RPM (raw ~50 × 5/3 display multiplier) — pump is running
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
class DomainKnowledgeRetriever:
|
| 31 |
+
"""Retrieves structured domain knowledge from YAML files.
|
| 32 |
+
|
| 33 |
+
Loads failure_modes.yaml, reasoning_chains.yaml, operational_context.yaml
|
| 34 |
+
at startup. Uses pattern matching on cycle_stats to determine which
|
| 35 |
+
knowledge chunks to inject into LLM prompts.
|
| 36 |
+
"""
|
| 37 |
+
|
| 38 |
+
def __init__(self, knowledge_dir: Optional[Path] = None):
|
| 39 |
+
self._dir = Path(knowledge_dir) if knowledge_dir else KNOWLEDGE_DIR
|
| 40 |
+
|
| 41 |
+
if yaml is None:
|
| 42 |
+
raise ImportError("PyYAML required. Install with: pip install pyyaml")
|
| 43 |
+
|
| 44 |
+
self._failure_modes = self._load_yaml("failure_modes.yaml")
|
| 45 |
+
self._reasoning_chains = self._load_yaml("reasoning_chains.yaml")
|
| 46 |
+
self._operational_context = self._load_yaml("operational_context.yaml")
|
| 47 |
+
|
| 48 |
+
def _load_yaml(self, filename: str) -> Dict:
|
| 49 |
+
path = self._dir / filename
|
| 50 |
+
if not path.exists():
|
| 51 |
+
logger.warning(f"Knowledge file not found: {path}")
|
| 52 |
+
return {}
|
| 53 |
+
with open(path, "r") as f:
|
| 54 |
+
return yaml.safe_load(f) or {}
|
| 55 |
+
|
| 56 |
+
# ------------------------------------------------------------------
|
| 57 |
+
# Public API
|
| 58 |
+
# ------------------------------------------------------------------
|
| 59 |
+
|
| 60 |
+
def get_context_for_cycle(
|
| 61 |
+
self, cycle_stats: Dict, plateaus: Optional[Dict] = None
|
| 62 |
+
) -> Dict[str, Any]:
|
| 63 |
+
"""Determine which knowledge chunks are relevant based on cycle data.
|
| 64 |
+
|
| 65 |
+
Args:
|
| 66 |
+
cycle_stats: Dict from DataAnalyzer.analyze_cycle()
|
| 67 |
+
plateaus: Optional dict mapping tag names to plateau periods
|
| 68 |
+
|
| 69 |
+
Returns:
|
| 70 |
+
Dict with categorized knowledge chunks ready for formatting
|
| 71 |
+
"""
|
| 72 |
+
context: Dict[str, Any] = {
|
| 73 |
+
"failure_modes": [],
|
| 74 |
+
"reasoning_chains": [],
|
| 75 |
+
"sensor_notes": [],
|
| 76 |
+
"regime_caveats": [],
|
| 77 |
+
"system_personality": [],
|
| 78 |
+
"benchmarks": {},
|
| 79 |
+
}
|
| 80 |
+
|
| 81 |
+
# Always-include sections
|
| 82 |
+
context["system_personality"] = self._get_system_personality()
|
| 83 |
+
context["sensor_notes"] = self._get_sensor_notes_for_stats(cycle_stats)
|
| 84 |
+
context["benchmarks"] = self._get_phase1_benchmarks()
|
| 85 |
+
context["regime_caveats"] = self._get_regime_caveats()
|
| 86 |
+
|
| 87 |
+
# Pattern-based trigger detection
|
| 88 |
+
triggers = self._detect_triggers(cycle_stats, plateaus)
|
| 89 |
+
|
| 90 |
+
if "low_pressure_build" in triggers:
|
| 91 |
+
context["failure_modes"].extend(
|
| 92 |
+
self._get_fm_by_ids(["FM-001", "FM-002", "FM-007", "FM-008"])
|
| 93 |
+
)
|
| 94 |
+
context["reasoning_chains"].extend(
|
| 95 |
+
self._get_rc_by_ids(["RC-001"])
|
| 96 |
+
)
|
| 97 |
+
|
| 98 |
+
if "high_discharge_temp" in triggers:
|
| 99 |
+
context["failure_modes"].extend(
|
| 100 |
+
self._get_fm_by_ids(["FM-005", "FM-006"])
|
| 101 |
+
)
|
| 102 |
+
context["reasoning_chains"].extend(
|
| 103 |
+
self._get_rc_by_ids(["RC-003"])
|
| 104 |
+
)
|
| 105 |
+
|
| 106 |
+
if "flow_anomaly" in triggers:
|
| 107 |
+
context["failure_modes"].extend(
|
| 108 |
+
self._get_fm_by_ids(["FM-001", "FM-002", "FM-007"])
|
| 109 |
+
)
|
| 110 |
+
context["reasoning_chains"].extend(
|
| 111 |
+
self._get_rc_by_ids(["RC-004"])
|
| 112 |
+
)
|
| 113 |
+
|
| 114 |
+
if "pressure_oscillation" in triggers:
|
| 115 |
+
context["failure_modes"].extend(
|
| 116 |
+
self._get_fm_by_ids(["FM-004"])
|
| 117 |
+
)
|
| 118 |
+
context["reasoning_chains"].extend(
|
| 119 |
+
self._get_rc_by_ids(["RC-002"])
|
| 120 |
+
)
|
| 121 |
+
|
| 122 |
+
if "motor_anomaly" in triggers:
|
| 123 |
+
context["failure_modes"].extend(
|
| 124 |
+
self._get_fm_by_ids(["FM-003"])
|
| 125 |
+
)
|
| 126 |
+
context["reasoning_chains"].extend(
|
| 127 |
+
self._get_rc_by_ids(["RC-005"])
|
| 128 |
+
)
|
| 129 |
+
|
| 130 |
+
# Deduplicate (triggers may overlap)
|
| 131 |
+
context["failure_modes"] = _deduplicate(context["failure_modes"], "id")
|
| 132 |
+
context["reasoning_chains"] = _deduplicate(context["reasoning_chains"], "id")
|
| 133 |
+
|
| 134 |
+
return context
|
| 135 |
+
|
| 136 |
+
def get_context_for_comparison(
|
| 137 |
+
self, stats_a: Dict, stats_b: Dict
|
| 138 |
+
) -> Dict[str, Any]:
|
| 139 |
+
"""Get knowledge context for a two-cycle comparison.
|
| 140 |
+
|
| 141 |
+
Runs trigger detection on both cycles and takes the union.
|
| 142 |
+
"""
|
| 143 |
+
ctx_a = self.get_context_for_cycle(stats_a)
|
| 144 |
+
ctx_b = self.get_context_for_cycle(stats_b)
|
| 145 |
+
|
| 146 |
+
merged: Dict[str, Any] = {
|
| 147 |
+
"failure_modes": _deduplicate(
|
| 148 |
+
ctx_a["failure_modes"] + ctx_b["failure_modes"], "id"
|
| 149 |
+
),
|
| 150 |
+
"reasoning_chains": _deduplicate(
|
| 151 |
+
ctx_a["reasoning_chains"] + ctx_b["reasoning_chains"], "id"
|
| 152 |
+
),
|
| 153 |
+
"sensor_notes": _deduplicate(
|
| 154 |
+
ctx_a["sensor_notes"] + ctx_b["sensor_notes"], "sensor"
|
| 155 |
+
),
|
| 156 |
+
"regime_caveats": ctx_a["regime_caveats"], # Same for both
|
| 157 |
+
"system_personality": ctx_a["system_personality"],
|
| 158 |
+
"benchmarks": ctx_a["benchmarks"],
|
| 159 |
+
}
|
| 160 |
+
return merged
|
| 161 |
+
|
| 162 |
+
def format_for_prompt(self, context: Dict[str, Any]) -> str:
|
| 163 |
+
"""Format retrieved knowledge into text blocks for LLM prompt injection."""
|
| 164 |
+
sections = []
|
| 165 |
+
|
| 166 |
+
# System personality
|
| 167 |
+
if context.get("system_personality"):
|
| 168 |
+
lines = ["SYSTEM PERSONALITY (known recurring issues & strengths):"]
|
| 169 |
+
for item in context["system_personality"]:
|
| 170 |
+
prefix = f" [Priority {item['priority']}]" if "priority" in item else " -"
|
| 171 |
+
lines.append(f"{prefix} {item['description']}")
|
| 172 |
+
if item.get("details"):
|
| 173 |
+
lines.append(f" {item['details']}")
|
| 174 |
+
sections.append("\n".join(lines))
|
| 175 |
+
|
| 176 |
+
# Sensor trustworthiness
|
| 177 |
+
if context.get("sensor_notes"):
|
| 178 |
+
lines = ["SENSOR TRUSTWORTHINESS NOTES:"]
|
| 179 |
+
for note in context["sensor_notes"]:
|
| 180 |
+
lines.append(f" {note['sensor']}:")
|
| 181 |
+
lines.append(f" Issue: {note['notes']}")
|
| 182 |
+
lines.append(f" Workaround: {note['workaround']}")
|
| 183 |
+
sections.append("\n".join(lines))
|
| 184 |
+
|
| 185 |
+
# Failure modes (plain-language, no internal IDs)
|
| 186 |
+
if context.get("failure_modes"):
|
| 187 |
+
lines = ["RELEVANT FAILURE PATTERNS (from CSH2 knowledge base):"]
|
| 188 |
+
for fm in context["failure_modes"]:
|
| 189 |
+
priority = fm.get('priority', 'N/A')
|
| 190 |
+
lines.append(f" **{fm['name']}** ({priority} priority)")
|
| 191 |
+
lines.append(f" Primary symptom: {fm['primary_symptom']}")
|
| 192 |
+
mechanism = fm.get("mechanism", "")
|
| 193 |
+
if len(mechanism) > 250:
|
| 194 |
+
mechanism = mechanism[:250] + "..."
|
| 195 |
+
lines.append(f" Mechanism: {mechanism}")
|
| 196 |
+
if fm.get("candidate_causes"):
|
| 197 |
+
causes = ", ".join(
|
| 198 |
+
f"{c['cause']} ({c['prior']:.0%})"
|
| 199 |
+
for c in fm["candidate_causes"][:3]
|
| 200 |
+
)
|
| 201 |
+
lines.append(f" Top causes: {causes}")
|
| 202 |
+
if fm.get("recommended_action"):
|
| 203 |
+
lines.append(f" Action: {fm['recommended_action']}")
|
| 204 |
+
sections.append("\n".join(lines))
|
| 205 |
+
|
| 206 |
+
# Reasoning chains (plain-language, no internal IDs)
|
| 207 |
+
if context.get("reasoning_chains"):
|
| 208 |
+
lines = ["DIAGNOSTIC REASONING STEPS:"]
|
| 209 |
+
for rc in context["reasoning_chains"]:
|
| 210 |
+
lines.append(f" **{rc['name']}**")
|
| 211 |
+
lines.append(f" Trigger: {rc['trigger']}")
|
| 212 |
+
if rc.get("steps"):
|
| 213 |
+
steps_text = "; ".join(
|
| 214 |
+
f"Step {s['step']}: {s['description']}"
|
| 215 |
+
for s in rc["steps"][:4]
|
| 216 |
+
)
|
| 217 |
+
lines.append(f" Steps: {steps_text}")
|
| 218 |
+
sections.append("\n".join(lines))
|
| 219 |
+
|
| 220 |
+
# Regime caveats
|
| 221 |
+
if context.get("regime_caveats"):
|
| 222 |
+
lines = ["OPERATIONAL CAVEATS:"]
|
| 223 |
+
for caveat in context["regime_caveats"]:
|
| 224 |
+
lines.append(f" {caveat['name']}: {caveat['description']}")
|
| 225 |
+
sections.append("\n".join(lines))
|
| 226 |
+
|
| 227 |
+
# Benchmarks
|
| 228 |
+
benchmarks = context.get("benchmarks", {})
|
| 229 |
+
if benchmarks:
|
| 230 |
+
lines = [
|
| 231 |
+
"PHASE 1B BENCHMARKS (reference values — Sept 24-26 ccH2 fills):",
|
| 232 |
+
f" Specific energy: {benchmarks.get('specific_energy', '0.26 kWh/kg')}",
|
| 233 |
+
f" Flow rate: {benchmarks.get('flow_rate', '1.13-1.25 kg/min at 65% motor')}",
|
| 234 |
+
f" Peak pressure: {benchmarks.get('peak_pressure', '370 bar')}",
|
| 235 |
+
f" Delivered temp: {benchmarks.get('delivered_temp', '55K')}",
|
| 236 |
+
f" Uptime: {benchmarks.get('uptime', '100% (6 fills, 4 back-to-back)')}",
|
| 237 |
+
]
|
| 238 |
+
sections.append("\n".join(lines))
|
| 239 |
+
|
| 240 |
+
return "\n\n".join(sections)
|
| 241 |
+
|
| 242 |
+
# ------------------------------------------------------------------
|
| 243 |
+
# Trigger detection
|
| 244 |
+
# ------------------------------------------------------------------
|
| 245 |
+
|
| 246 |
+
def _detect_triggers(
|
| 247 |
+
self, stats: Dict, plateaus: Optional[Dict] = None
|
| 248 |
+
) -> Set[str]:
|
| 249 |
+
"""Detect which diagnostic triggers are active based on sensor patterns."""
|
| 250 |
+
triggers: Set[str] = set()
|
| 251 |
+
|
| 252 |
+
# --- Low pressure build ---
|
| 253 |
+
dp = stats.get("discharge_pressure", {})
|
| 254 |
+
peak_pressure = dp.get("peak", 0)
|
| 255 |
+
if 0 < peak_pressure < LOW_PRESSURE_THRESHOLD_BAR:
|
| 256 |
+
triggers.add("low_pressure_build")
|
| 257 |
+
|
| 258 |
+
# Also check if PT130 plateaus below target
|
| 259 |
+
if plateaus:
|
| 260 |
+
pt130_plateaus = plateaus.get("PT130", [])
|
| 261 |
+
if pt130_plateaus:
|
| 262 |
+
max_plateau = max(
|
| 263 |
+
(p.get("value", 0) for p in pt130_plateaus), default=0
|
| 264 |
+
)
|
| 265 |
+
if 0 < max_plateau < LOW_PRESSURE_THRESHOLD_BAR:
|
| 266 |
+
triggers.add("low_pressure_build")
|
| 267 |
+
|
| 268 |
+
# --- High discharge temperature ---
|
| 269 |
+
tt130 = stats.get("temperature_TT130", {})
|
| 270 |
+
tt130_peak = tt130.get("peak", 0)
|
| 271 |
+
tt130_avg = tt130.get("avg", 0)
|
| 272 |
+
if tt130_peak > HIGH_DISCHARGE_TEMP_K:
|
| 273 |
+
triggers.add("high_discharge_temp")
|
| 274 |
+
if tt130_avg > 0 and tt130_peak > tt130_avg * 1.3 and tt130_peak > 60:
|
| 275 |
+
triggers.add("high_discharge_temp")
|
| 276 |
+
|
| 277 |
+
# --- Flow anomaly ---
|
| 278 |
+
flow = stats.get("flow", {})
|
| 279 |
+
avg_flow = flow.get("avg_flow", -1)
|
| 280 |
+
motor = stats.get("motor", {})
|
| 281 |
+
avg_speed = motor.get("avg_speed", 0)
|
| 282 |
+
if avg_flow >= 0 and avg_flow < LOW_FLOW_THRESHOLD_KG_MIN and avg_speed > MOTOR_SPEED_ACTIVE_THRESHOLD:
|
| 283 |
+
triggers.add("flow_anomaly")
|
| 284 |
+
|
| 285 |
+
# --- Pressure oscillation ---
|
| 286 |
+
# Distinguish oscillation from a normal smooth ramp.
|
| 287 |
+
# Key insight: a ramp from ambient to 370 bar has peak >> initial.
|
| 288 |
+
# True oscillation happens when the system is already at high
|
| 289 |
+
# pressure and fluctuates. We trigger ONLY when:
|
| 290 |
+
# 1. Peak pressure > 200 bar (system was pressurized)
|
| 291 |
+
# 2. Initial pressure > 30% of peak (NOT a ramp from ambient)
|
| 292 |
+
# 3. Std is non-trivial (> 20 bar — real fluctuation)
|
| 293 |
+
dp_initial = dp.get("initial", 0)
|
| 294 |
+
dp_std = dp.get("std", 0)
|
| 295 |
+
if peak_pressure > 200 and dp_initial > peak_pressure * 0.3 and dp_std > 20:
|
| 296 |
+
triggers.add("pressure_oscillation")
|
| 297 |
+
|
| 298 |
+
return triggers
|
| 299 |
+
|
| 300 |
+
# ------------------------------------------------------------------
|
| 301 |
+
# Knowledge accessors
|
| 302 |
+
# ------------------------------------------------------------------
|
| 303 |
+
|
| 304 |
+
def _get_fm_by_ids(self, fm_ids: List[str]) -> List[Dict]:
|
| 305 |
+
"""Get failure modes by ID, formatted for context injection."""
|
| 306 |
+
modes = self._failure_modes.get("failure_modes", [])
|
| 307 |
+
results = []
|
| 308 |
+
for fm in modes:
|
| 309 |
+
if fm.get("id") in fm_ids:
|
| 310 |
+
results.append({
|
| 311 |
+
"id": fm.get("id"),
|
| 312 |
+
"name": fm.get("name"),
|
| 313 |
+
"priority": fm.get("system_priority"),
|
| 314 |
+
"primary_symptom": fm.get("symptoms", {}).get("primary", ""),
|
| 315 |
+
"mechanism": fm.get("physical_mechanism", ""),
|
| 316 |
+
"candidate_causes": [
|
| 317 |
+
{"cause": c["cause"], "prior": c["prior"]}
|
| 318 |
+
for c in fm.get("candidate_causes", [])
|
| 319 |
+
],
|
| 320 |
+
"recommended_action": fm.get("recommended_actions", {}).get("immediate", ""),
|
| 321 |
+
})
|
| 322 |
+
return results
|
| 323 |
+
|
| 324 |
+
def _get_rc_by_ids(self, rc_ids: List[str]) -> List[Dict]:
|
| 325 |
+
"""Get reasoning chains by ID, formatted for context injection."""
|
| 326 |
+
chains = self._reasoning_chains.get("reasoning_chains", [])
|
| 327 |
+
results = []
|
| 328 |
+
for chain in chains:
|
| 329 |
+
if chain.get("id") in rc_ids:
|
| 330 |
+
results.append({
|
| 331 |
+
"id": chain.get("id"),
|
| 332 |
+
"name": chain.get("name"),
|
| 333 |
+
"trigger": chain.get("trigger"),
|
| 334 |
+
"related_fms": chain.get("related_failure_modes", []),
|
| 335 |
+
"steps": [
|
| 336 |
+
{
|
| 337 |
+
"step": s.get("step"),
|
| 338 |
+
"action": s.get("action"),
|
| 339 |
+
"description": s.get("description"),
|
| 340 |
+
}
|
| 341 |
+
for s in chain.get("steps", [])
|
| 342 |
+
],
|
| 343 |
+
})
|
| 344 |
+
return results
|
| 345 |
+
|
| 346 |
+
def _get_system_personality(self) -> List[Dict]:
|
| 347 |
+
"""Get system personality: recurring issues + strengths."""
|
| 348 |
+
personality = self._operational_context.get("system_personality", {})
|
| 349 |
+
items = []
|
| 350 |
+
for issue in personality.get("recurring_issues", []):
|
| 351 |
+
items.append({
|
| 352 |
+
"priority": issue.get("priority"),
|
| 353 |
+
"description": issue.get("description"),
|
| 354 |
+
"details": issue.get("details"),
|
| 355 |
+
"type": "recurring_issue",
|
| 356 |
+
})
|
| 357 |
+
for strength in personality.get("strengths", []):
|
| 358 |
+
items.append({
|
| 359 |
+
"description": strength,
|
| 360 |
+
"details": None,
|
| 361 |
+
"type": "strength",
|
| 362 |
+
})
|
| 363 |
+
return items
|
| 364 |
+
|
| 365 |
+
def _get_sensor_notes_for_stats(self, stats: Dict) -> List[Dict]:
|
| 366 |
+
"""Get sensor trustworthiness notes for sensors present in the data."""
|
| 367 |
+
sensors = self._operational_context.get("sensor_trustworthiness", {})
|
| 368 |
+
present_sensors = set()
|
| 369 |
+
|
| 370 |
+
# Detect which sensors are in the stats
|
| 371 |
+
if "discharge_pressure" in stats:
|
| 372 |
+
present_sensors.add("PT130")
|
| 373 |
+
if "supply_pressure" in stats:
|
| 374 |
+
present_sensors.add("PT110")
|
| 375 |
+
if "temperature_TT110" in stats:
|
| 376 |
+
present_sensors.add("TT110")
|
| 377 |
+
if "temperature_TT130" in stats:
|
| 378 |
+
present_sensors.add("TT130")
|
| 379 |
+
if "flow" in stats:
|
| 380 |
+
present_sensors.add("FT140")
|
| 381 |
+
if "motor" in stats:
|
| 382 |
+
present_sensors.add("M130_Speed_vs_MC130")
|
| 383 |
+
|
| 384 |
+
results = []
|
| 385 |
+
for sensor_key, info in sensors.items():
|
| 386 |
+
# Match sensor key to present sensors
|
| 387 |
+
if sensor_key in present_sensors or any(
|
| 388 |
+
s in sensor_key for s in present_sensors
|
| 389 |
+
):
|
| 390 |
+
results.append({
|
| 391 |
+
"sensor": sensor_key,
|
| 392 |
+
"notes": info.get("notes", "").strip(),
|
| 393 |
+
"workaround": info.get("workaround", "").strip(),
|
| 394 |
+
})
|
| 395 |
+
return results
|
| 396 |
+
|
| 397 |
+
def _get_regime_caveats(self) -> List[Dict]:
|
| 398 |
+
"""Get all regime caveats."""
|
| 399 |
+
caveats = self._operational_context.get("regime_caveats", {})
|
| 400 |
+
return [
|
| 401 |
+
{
|
| 402 |
+
"name": name,
|
| 403 |
+
"description": info.get("description", ""),
|
| 404 |
+
}
|
| 405 |
+
for name, info in caveats.items()
|
| 406 |
+
]
|
| 407 |
+
|
| 408 |
+
def _get_phase1_benchmarks(self) -> Dict[str, str]:
|
| 409 |
+
"""Get Phase 1B benchmark values."""
|
| 410 |
+
return {
|
| 411 |
+
"specific_energy": "0.26 kWh/kg",
|
| 412 |
+
"flow_rate": "1.13-1.25 kg/min at 65% motor",
|
| 413 |
+
"peak_pressure": "370 bar (Phase 1B ccH2 fills)",
|
| 414 |
+
"delivered_temp": "55K",
|
| 415 |
+
"uptime": "100% (6 fills, 4 back-to-back)",
|
| 416 |
+
}
|
| 417 |
+
|
| 418 |
+
|
| 419 |
+
# ------------------------------------------------------------------
|
| 420 |
+
# Utility
|
| 421 |
+
# ------------------------------------------------------------------
|
| 422 |
+
|
| 423 |
+
def _deduplicate(items: List[Dict], key: str) -> List[Dict]:
|
| 424 |
+
"""Deduplicate list of dicts by a key field."""
|
| 425 |
+
seen: set = set()
|
| 426 |
+
result = []
|
| 427 |
+
for item in items:
|
| 428 |
+
val = item.get(key)
|
| 429 |
+
if val not in seen:
|
| 430 |
+
seen.add(val)
|
| 431 |
+
result.append(item)
|
| 432 |
+
return result
|
analysis/llm_analyzer.py
ADDED
|
@@ -0,0 +1,418 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
LLM-Powered Cycle Analysis Module
|
| 3 |
+
Uses Claude API for variation analysis and cycle comparison.
|
| 4 |
+
Injects domain knowledge from CSH2 YAML knowledge base via DomainKnowledgeRetriever.
|
| 5 |
+
"""
|
| 6 |
+
import os
|
| 7 |
+
import re
|
| 8 |
+
import json
|
| 9 |
+
from typing import Dict, List, Optional
|
| 10 |
+
from core import config
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
# ── Output Sanitizer ──────────────────────────────────────────────
|
| 14 |
+
# Catches any remaining internal reference codes the LLM generates
|
| 15 |
+
# despite prompt instructions not to use them.
|
| 16 |
+
|
| 17 |
+
# Pattern matches FM-001, RC-003, RM-001, D1-D8, etc.
|
| 18 |
+
_INTERNAL_CODE_PATTERN = re.compile(
|
| 19 |
+
r'\b(?:FM|RC|RM|D)-?\d{1,3}\b',
|
| 20 |
+
re.IGNORECASE,
|
| 21 |
+
)
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
def sanitize_llm_output(text: str) -> str:
|
| 25 |
+
"""Remove internal reference codes from LLM output.
|
| 26 |
+
|
| 27 |
+
Strips patterns like FM-001, RC-003, RM-001 that the engineering
|
| 28 |
+
team finds confusing. The surrounding text (which describes the
|
| 29 |
+
issue in plain language) is preserved.
|
| 30 |
+
"""
|
| 31 |
+
if not text:
|
| 32 |
+
return text
|
| 33 |
+
# Remove the code pattern, plus optional surrounding parens/brackets
|
| 34 |
+
# e.g., "(FM-001)" → "", "FM-001:" → ":"
|
| 35 |
+
cleaned = re.sub(
|
| 36 |
+
r'\s*[\(\[]*' + _INTERNAL_CODE_PATTERN.pattern + r'[\)\]]*\s*[:\-—]?\s*',
|
| 37 |
+
' ',
|
| 38 |
+
text,
|
| 39 |
+
flags=re.IGNORECASE,
|
| 40 |
+
)
|
| 41 |
+
# Clean up double spaces and leading/trailing whitespace per line
|
| 42 |
+
cleaned = re.sub(r' +', ' ', cleaned)
|
| 43 |
+
return cleaned.strip()
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
# ── Shared Murphy System Prompt ─────────────────────────────────
|
| 47 |
+
# Extracted as a module-level constant so it can be reused by
|
| 48 |
+
# result_interpreter.py and any future LLM callers.
|
| 49 |
+
|
| 50 |
+
MURPHY_SYSTEM_PROMPT = """You are Murphy, the diagnostic intelligence layer for CSH2's cryogenic hydrogen pump platform. You are a senior cryogenic systems engineer with deep expertise in:
|
| 51 |
+
|
| 52 |
+
- Reciprocating cryogenic pump physics (triplex design, 60mm stroke, ~40mm bore, ~75.4 cm3 swept volume)
|
| 53 |
+
- Cryogenic fluid behavior (LH2 at 20.3K, LN2 at 77.4K, phase transitions, subcooling margins)
|
| 54 |
+
- High-pressure hydrogen systems (up to 900 bar, H35/H70/ccH2 dispensing)
|
| 55 |
+
- Seal integrity, check valve dynamics (ICV/DCV), and thermal management
|
| 56 |
+
- Time-series sensor diagnostics and fault signature recognition
|
| 57 |
+
|
| 58 |
+
When you identify a pattern in the data, explain WHY it matters thermodynamically — not just that a number is high or low. Describe any matching failure patterns by their plain-English name and physical mechanism. Never use internal reference codes like FM-001 or RC-001 — always describe issues in terms the engineering team can immediately understand (e.g., "DCV seat wear" not "FM-001").
|
| 59 |
+
|
| 60 |
+
Always cite specific measured values from the data to support your claims. If the data summary says peak pressure was 371 bar, say "371 bar" — do not round, estimate, or invent values.
|
| 61 |
+
|
| 62 |
+
Key system specifications:
|
| 63 |
+
- Drive: Variable Frequency Drive (VFD), 0-100% speed, max ~300 RPM
|
| 64 |
+
- Pressure targets: H35 (350 bar), H70 (700 bar), max rated 900+ bar
|
| 65 |
+
- Phase 1B results: 0.26 kWh/kg, 1.13-1.25 kg/min flow, 370 bar, 55K delivered temp, 100% uptime
|
| 66 |
+
- Testing phases: LN2 mechanical validation (Mar-Aug 2025), LH2 Phase 1A (early Sep), Phase 1B ccH2 fills (Sep 24-26)
|
| 67 |
+
|
| 68 |
+
Be direct, methodical, and honest about uncertainty. Distinguish between "the data shows X" and "X could indicate Y but we'd need Z to confirm." """
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
class LLMCycleAnalyzer:
|
| 72 |
+
"""Claude-powered analysis for testing cycle interpretation.
|
| 73 |
+
|
| 74 |
+
Integrates DomainKnowledgeRetriever to inject CSH2-specific failure modes,
|
| 75 |
+
reasoning chains, and operational context into prompts.
|
| 76 |
+
"""
|
| 77 |
+
|
| 78 |
+
def __init__(self):
|
| 79 |
+
self.api_key = config.ANTHROPIC_API_KEY
|
| 80 |
+
self.api_available = bool(self.api_key and self.api_key != 'your_api_key_here')
|
| 81 |
+
self.model = "claude-sonnet-4-20250514"
|
| 82 |
+
self._client = None
|
| 83 |
+
self._retriever = None
|
| 84 |
+
|
| 85 |
+
@property
|
| 86 |
+
def client(self):
|
| 87 |
+
if self._client is None and self.api_available:
|
| 88 |
+
import anthropic
|
| 89 |
+
self._client = anthropic.Anthropic(api_key=self.api_key)
|
| 90 |
+
return self._client
|
| 91 |
+
|
| 92 |
+
@property
|
| 93 |
+
def retriever(self):
|
| 94 |
+
"""Lazy-load the domain knowledge retriever. Returns None on failure."""
|
| 95 |
+
if self._retriever is None:
|
| 96 |
+
try:
|
| 97 |
+
from analysis.knowledge_retriever import DomainKnowledgeRetriever
|
| 98 |
+
self._retriever = DomainKnowledgeRetriever()
|
| 99 |
+
except Exception:
|
| 100 |
+
self._retriever = None
|
| 101 |
+
return self._retriever
|
| 102 |
+
|
| 103 |
+
def _build_system_prompt(self) -> str:
|
| 104 |
+
"""Build the system prompt establishing Murphy's identity and expertise."""
|
| 105 |
+
return MURPHY_SYSTEM_PROMPT
|
| 106 |
+
|
| 107 |
+
def analyze_cycle_variations(
|
| 108 |
+
self,
|
| 109 |
+
cycle_stats: Dict,
|
| 110 |
+
plateaus: Dict[str, List[Dict]],
|
| 111 |
+
transitions: List[Dict] = None,
|
| 112 |
+
) -> str:
|
| 113 |
+
"""
|
| 114 |
+
Feature 4: Explain variations and stabilizations in a single testing cycle.
|
| 115 |
+
|
| 116 |
+
Args:
|
| 117 |
+
cycle_stats: Dict from DataAnalyzer.analyze_cycle()
|
| 118 |
+
plateaus: Dict mapping tag names to lists of plateau periods
|
| 119 |
+
transitions: Optional list of notable transitions
|
| 120 |
+
|
| 121 |
+
Returns:
|
| 122 |
+
LLM analysis text
|
| 123 |
+
"""
|
| 124 |
+
if not self.api_available:
|
| 125 |
+
return "LLM analysis unavailable. Add ANTHROPIC_API_KEY to .env file."
|
| 126 |
+
|
| 127 |
+
# Retrieve domain knowledge based on data patterns
|
| 128 |
+
knowledge_text = ""
|
| 129 |
+
if self.retriever:
|
| 130 |
+
try:
|
| 131 |
+
context = self.retriever.get_context_for_cycle(cycle_stats, plateaus)
|
| 132 |
+
knowledge_text = self.retriever.format_for_prompt(context)
|
| 133 |
+
except Exception:
|
| 134 |
+
knowledge_text = ""
|
| 135 |
+
|
| 136 |
+
prompt = self._build_variation_prompt(cycle_stats, plateaus, transitions, knowledge_text)
|
| 137 |
+
|
| 138 |
+
try:
|
| 139 |
+
message = self.client.messages.create(
|
| 140 |
+
model=self.model,
|
| 141 |
+
max_tokens=2000,
|
| 142 |
+
system=[
|
| 143 |
+
{
|
| 144 |
+
"type": "text",
|
| 145 |
+
"text": self._build_system_prompt(),
|
| 146 |
+
"cache_control": {"type": "ephemeral"},
|
| 147 |
+
}
|
| 148 |
+
],
|
| 149 |
+
messages=[{"role": "user", "content": prompt}],
|
| 150 |
+
)
|
| 151 |
+
return sanitize_llm_output(message.content[0].text)
|
| 152 |
+
except Exception as e:
|
| 153 |
+
return f"Analysis failed: {e}"
|
| 154 |
+
|
| 155 |
+
def compare_cycles(
|
| 156 |
+
self,
|
| 157 |
+
cycle_a_stats: Dict,
|
| 158 |
+
cycle_b_stats: Dict,
|
| 159 |
+
cycle_a_label: str = "Cycle A",
|
| 160 |
+
cycle_b_label: str = "Cycle B",
|
| 161 |
+
) -> str:
|
| 162 |
+
"""
|
| 163 |
+
Feature 5: Compare two testing cycles and explain differences.
|
| 164 |
+
|
| 165 |
+
Args:
|
| 166 |
+
cycle_a_stats: Dict from DataAnalyzer.analyze_cycle() for cycle A
|
| 167 |
+
cycle_b_stats: Dict from DataAnalyzer.analyze_cycle() for cycle B
|
| 168 |
+
cycle_a_label: Display label for cycle A
|
| 169 |
+
cycle_b_label: Display label for cycle B
|
| 170 |
+
|
| 171 |
+
Returns:
|
| 172 |
+
LLM comparison text
|
| 173 |
+
"""
|
| 174 |
+
if not self.api_available:
|
| 175 |
+
return "LLM analysis unavailable. Add ANTHROPIC_API_KEY to .env file."
|
| 176 |
+
|
| 177 |
+
# Retrieve domain knowledge based on both cycles
|
| 178 |
+
knowledge_text = ""
|
| 179 |
+
if self.retriever:
|
| 180 |
+
try:
|
| 181 |
+
context = self.retriever.get_context_for_comparison(cycle_a_stats, cycle_b_stats)
|
| 182 |
+
knowledge_text = self.retriever.format_for_prompt(context)
|
| 183 |
+
except Exception:
|
| 184 |
+
knowledge_text = ""
|
| 185 |
+
|
| 186 |
+
prompt = self._build_comparison_prompt(
|
| 187 |
+
cycle_a_stats, cycle_b_stats, cycle_a_label, cycle_b_label, knowledge_text
|
| 188 |
+
)
|
| 189 |
+
|
| 190 |
+
try:
|
| 191 |
+
message = self.client.messages.create(
|
| 192 |
+
model=self.model,
|
| 193 |
+
max_tokens=2500,
|
| 194 |
+
system=[
|
| 195 |
+
{
|
| 196 |
+
"type": "text",
|
| 197 |
+
"text": self._build_system_prompt(),
|
| 198 |
+
"cache_control": {"type": "ephemeral"},
|
| 199 |
+
}
|
| 200 |
+
],
|
| 201 |
+
messages=[{"role": "user", "content": prompt}],
|
| 202 |
+
)
|
| 203 |
+
return sanitize_llm_output(message.content[0].text)
|
| 204 |
+
except Exception as e:
|
| 205 |
+
return f"Comparison analysis failed: {e}"
|
| 206 |
+
|
| 207 |
+
def follow_up(
|
| 208 |
+
self,
|
| 209 |
+
conversation_history: List[Dict],
|
| 210 |
+
follow_up_question: str,
|
| 211 |
+
) -> str:
|
| 212 |
+
"""Multi-turn follow-up on a previous analysis.
|
| 213 |
+
|
| 214 |
+
Args:
|
| 215 |
+
conversation_history: List of {role, content} message dicts
|
| 216 |
+
representing the prior conversation (user prompt + assistant
|
| 217 |
+
analysis + any prior follow-ups).
|
| 218 |
+
follow_up_question: The engineer's follow-up question.
|
| 219 |
+
|
| 220 |
+
Returns:
|
| 221 |
+
LLM follow-up response text.
|
| 222 |
+
"""
|
| 223 |
+
if not self.api_available:
|
| 224 |
+
return "LLM analysis unavailable. Add ANTHROPIC_API_KEY to .env file."
|
| 225 |
+
|
| 226 |
+
messages = list(conversation_history) + [
|
| 227 |
+
{"role": "user", "content": follow_up_question},
|
| 228 |
+
]
|
| 229 |
+
|
| 230 |
+
try:
|
| 231 |
+
message = self.client.messages.create(
|
| 232 |
+
model=self.model,
|
| 233 |
+
max_tokens=1500,
|
| 234 |
+
system=[
|
| 235 |
+
{
|
| 236 |
+
"type": "text",
|
| 237 |
+
"text": self._build_system_prompt(),
|
| 238 |
+
"cache_control": {"type": "ephemeral"},
|
| 239 |
+
}
|
| 240 |
+
],
|
| 241 |
+
messages=messages,
|
| 242 |
+
)
|
| 243 |
+
return sanitize_llm_output(message.content[0].text)
|
| 244 |
+
except Exception as e:
|
| 245 |
+
return f"Follow-up failed: {e}"
|
| 246 |
+
|
| 247 |
+
def _build_variation_prompt(
|
| 248 |
+
self,
|
| 249 |
+
stats: Dict,
|
| 250 |
+
plateaus: Dict[str, List[Dict]],
|
| 251 |
+
transitions: List[Dict] = None,
|
| 252 |
+
knowledge_text: str = "",
|
| 253 |
+
) -> str:
|
| 254 |
+
"""Build the prompt for single-cycle variation analysis"""
|
| 255 |
+
|
| 256 |
+
# Format cycle summary
|
| 257 |
+
time_range = stats.get('time_range', {})
|
| 258 |
+
duration = time_range.get('duration_minutes', 0)
|
| 259 |
+
|
| 260 |
+
summary_lines = [f"Duration: {duration:.1f} minutes"]
|
| 261 |
+
|
| 262 |
+
if 'discharge_pressure' in stats:
|
| 263 |
+
dp = stats['discharge_pressure']
|
| 264 |
+
summary_lines.append(f"Peak Discharge Pressure: {dp['peak']:.1f} bar")
|
| 265 |
+
summary_lines.append(f"Average Discharge Pressure: {dp['avg']:.1f} bar")
|
| 266 |
+
summary_lines.append(f"Initial Pressure: {dp['initial']:.1f} bar -> Final: {dp['final']:.1f} bar")
|
| 267 |
+
|
| 268 |
+
if 'compression_ratio' in stats:
|
| 269 |
+
cr = stats['compression_ratio']
|
| 270 |
+
summary_lines.append(f"Compression Ratio: {cr['min']:.1f}x to {cr['max']:.1f}x (avg: {cr['avg']:.1f}x)")
|
| 271 |
+
|
| 272 |
+
for key in ['temperature_TT110', 'temperature_TT130']:
|
| 273 |
+
if key in stats:
|
| 274 |
+
t = stats[key]
|
| 275 |
+
tag = key.replace('temperature_', '')
|
| 276 |
+
summary_lines.append(f"{tag}: {t['min']:.1f} K to {t['peak']:.1f} K (avg: {t['avg']:.1f} K)")
|
| 277 |
+
|
| 278 |
+
if 'flow' in stats:
|
| 279 |
+
f = stats['flow']
|
| 280 |
+
summary_lines.append(f"Average Flow: {f['avg_flow']:.2f} kg/min, Max: {f['max_flow']:.2f} kg/min")
|
| 281 |
+
if 'total_mass_kg' in f:
|
| 282 |
+
summary_lines.append(f"Total Mass Delivered: {f['total_mass_kg']:.2f} kg")
|
| 283 |
+
|
| 284 |
+
if 'ramp_rate' in stats:
|
| 285 |
+
r = stats['ramp_rate']
|
| 286 |
+
summary_lines.append(f"Pressure Ramp Rate: avg {r['avg']:.1f} bar/min, max {r['max']:.1f} bar/min")
|
| 287 |
+
|
| 288 |
+
if 'motor' in stats:
|
| 289 |
+
m = stats['motor']
|
| 290 |
+
summary_lines.append(f"Motor Peak Speed: {m['peak_speed']:.0f} RPM, Avg: {m['avg_speed']:.0f} RPM")
|
| 291 |
+
|
| 292 |
+
cycle_summary = "\n".join(f" - {line}" for line in summary_lines)
|
| 293 |
+
|
| 294 |
+
# Format plateaus
|
| 295 |
+
plateau_lines = []
|
| 296 |
+
for tag, periods in plateaus.items():
|
| 297 |
+
if periods:
|
| 298 |
+
plateau_lines.append(f"\n {tag} plateaus ({len(periods)} detected):")
|
| 299 |
+
for i, p in enumerate(periods[:5], 1):
|
| 300 |
+
plateau_lines.append(
|
| 301 |
+
f" {i}. Value: {p['value']:.2f} | "
|
| 302 |
+
f"Duration: {p['duration_minutes']:.1f} min | "
|
| 303 |
+
f"From {p['start'].strftime('%H:%M:%S')} to {p['end'].strftime('%H:%M:%S')}"
|
| 304 |
+
)
|
| 305 |
+
plateau_text = "\n".join(plateau_lines) if plateau_lines else " No significant plateaus detected."
|
| 306 |
+
|
| 307 |
+
# Format performance vs targets
|
| 308 |
+
perf_lines = []
|
| 309 |
+
pvt = stats.get('performance_vs_targets', {})
|
| 310 |
+
if 'pressure' in pvt:
|
| 311 |
+
p = pvt['pressure']
|
| 312 |
+
status = "MET" if p['meets_phase1'] else "NOT MET"
|
| 313 |
+
perf_lines.append(f" - Phase 1 pressure target (500 bar): {status} (achieved {p['achieved']:.0f} bar)")
|
| 314 |
+
status_h70 = "MET" if p['meets_h70'] else "NOT MET"
|
| 315 |
+
perf_lines.append(f" - H70 pressure target (700 bar): {status_h70}")
|
| 316 |
+
perf_text = "\n".join(perf_lines) if perf_lines else " No targets evaluated."
|
| 317 |
+
|
| 318 |
+
prompt = f"""Analyze this testing cycle and explain the observed variations and stabilizations:
|
| 319 |
+
|
| 320 |
+
CYCLE SUMMARY:
|
| 321 |
+
{cycle_summary}
|
| 322 |
+
|
| 323 |
+
DETECTED PLATEAUS (constant-value periods):
|
| 324 |
+
{plateau_text}
|
| 325 |
+
|
| 326 |
+
PERFORMANCE vs TARGETS:
|
| 327 |
+
{perf_text}"""
|
| 328 |
+
|
| 329 |
+
if knowledge_text:
|
| 330 |
+
prompt += f"""
|
| 331 |
+
|
| 332 |
+
DOMAIN KNOWLEDGE (use this to inform your analysis — describe any matching patterns by name and mechanism):
|
| 333 |
+
{knowledge_text}"""
|
| 334 |
+
|
| 335 |
+
prompt += """
|
| 336 |
+
|
| 337 |
+
Provide a concise engineering analysis (3-5 paragraphs) covering:
|
| 338 |
+
1. What physical processes likely caused the observed pressure/temperature variations
|
| 339 |
+
2. Why plateaus (stabilizations) occurred at the specific values detected
|
| 340 |
+
3. Any diagnostic concerns — describe matching failure patterns by name and physical mechanism, and walk through the diagnostic reasoning
|
| 341 |
+
4. Recommendations for the next cycle or operational improvements
|
| 342 |
+
|
| 343 |
+
Use specific values from the data — cite exact numbers, do not round or estimate. If sensor trustworthiness notes are provided, factor them into your interpretation (e.g., TT110 reads pipe wall not fluid, FT140 needs density correction for LH2). Never use internal codes like FM-XXX or RC-XXX."""
|
| 344 |
+
|
| 345 |
+
return prompt
|
| 346 |
+
|
| 347 |
+
def _build_comparison_prompt(
|
| 348 |
+
self,
|
| 349 |
+
a: Dict,
|
| 350 |
+
b: Dict,
|
| 351 |
+
label_a: str,
|
| 352 |
+
label_b: str,
|
| 353 |
+
knowledge_text: str = "",
|
| 354 |
+
) -> str:
|
| 355 |
+
"""Build the prompt for cycle comparison"""
|
| 356 |
+
|
| 357 |
+
def summarize(stats: Dict, label: str) -> str:
|
| 358 |
+
lines = [f"{label}:"]
|
| 359 |
+
tr = stats.get('time_range', {})
|
| 360 |
+
lines.append(f" Duration: {tr.get('duration_minutes', 0):.1f} min")
|
| 361 |
+
|
| 362 |
+
if 'discharge_pressure' in stats:
|
| 363 |
+
dp = stats['discharge_pressure']
|
| 364 |
+
lines.append(f" Peak Pressure: {dp['peak']:.1f} bar, Avg: {dp['avg']:.1f} bar")
|
| 365 |
+
|
| 366 |
+
if 'compression_ratio' in stats:
|
| 367 |
+
cr = stats['compression_ratio']
|
| 368 |
+
lines.append(f" Compression Ratio: {cr['avg']:.1f}x (range: {cr['min']:.1f}x - {cr['max']:.1f}x)")
|
| 369 |
+
|
| 370 |
+
for key in ['temperature_TT110', 'temperature_TT130']:
|
| 371 |
+
if key in stats:
|
| 372 |
+
t = stats[key]
|
| 373 |
+
tag = key.replace('temperature_', '')
|
| 374 |
+
lines.append(f" {tag}: {t['min']:.1f} - {t['peak']:.1f} K (avg {t['avg']:.1f} K)")
|
| 375 |
+
|
| 376 |
+
if 'flow' in stats:
|
| 377 |
+
f = stats['flow']
|
| 378 |
+
lines.append(f" Flow: avg {f['avg_flow']:.2f} kg/min, max {f['max_flow']:.2f} kg/min")
|
| 379 |
+
if 'total_mass_kg' in f:
|
| 380 |
+
lines.append(f" Total Mass: {f['total_mass_kg']:.2f} kg")
|
| 381 |
+
|
| 382 |
+
if 'ramp_rate' in stats:
|
| 383 |
+
r = stats['ramp_rate']
|
| 384 |
+
lines.append(f" Ramp Rate: avg {r['avg']:.1f}, max {r['max']:.1f} bar/min")
|
| 385 |
+
|
| 386 |
+
if 'motor' in stats:
|
| 387 |
+
m = stats['motor']
|
| 388 |
+
lines.append(f" Motor: peak {m['peak_speed']:.0f} RPM, avg {m['avg_speed']:.0f} RPM")
|
| 389 |
+
|
| 390 |
+
return "\n".join(lines)
|
| 391 |
+
|
| 392 |
+
summary_a = summarize(a, label_a)
|
| 393 |
+
summary_b = summarize(b, label_b)
|
| 394 |
+
|
| 395 |
+
prompt = f"""Compare these two testing cycles and provide an engineering analysis:
|
| 396 |
+
|
| 397 |
+
{summary_a}
|
| 398 |
+
|
| 399 |
+
{summary_b}"""
|
| 400 |
+
|
| 401 |
+
if knowledge_text:
|
| 402 |
+
prompt += f"""
|
| 403 |
+
|
| 404 |
+
DOMAIN KNOWLEDGE (use this to inform your comparison — describe any matching patterns by name and mechanism):
|
| 405 |
+
{knowledge_text}"""
|
| 406 |
+
|
| 407 |
+
prompt += """
|
| 408 |
+
|
| 409 |
+
Provide a concise comparison (3-5 paragraphs) covering:
|
| 410 |
+
1. Key differences between the two cycles (pressure, temperature, flow, duration)
|
| 411 |
+
2. Likely physical or operational reasons for the differences — describe matching failure patterns by name and physical mechanism
|
| 412 |
+
3. Which cycle performed better and why (considering efficiency, stability, and Phase 1B benchmarks)
|
| 413 |
+
4. Any concerning trends or improvements noted between cycles
|
| 414 |
+
5. Recommendations based on the comparison, with diagnostic reasoning
|
| 415 |
+
|
| 416 |
+
Be technically precise. Cite exact values from both cycles — do not round or estimate. If sensor trustworthiness notes are provided, factor them into your interpretation. Never use internal codes like FM-XXX or RC-XXX."""
|
| 417 |
+
|
| 418 |
+
return prompt
|
analysis/nl2sql.py
ADDED
|
@@ -0,0 +1,217 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Natural Language Query Parser for Data Explorer
|
| 3 |
+
Uses Claude API to parse engineer queries into structured sensor + time range selections.
|
| 4 |
+
Adapted from clearskies_agent_v1.1/llm_parser.py
|
| 5 |
+
"""
|
| 6 |
+
import json
|
| 7 |
+
from typing import Dict, Optional, List
|
| 8 |
+
from datetime import datetime
|
| 9 |
+
from core import config
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
class NLQueryParser:
|
| 13 |
+
"""Parses natural language queries into structured sensor + time range selections"""
|
| 14 |
+
|
| 15 |
+
def __init__(self):
|
| 16 |
+
self.api_key = config.ANTHROPIC_API_KEY
|
| 17 |
+
self.api_available = bool(self.api_key and self.api_key != 'your_api_key_here')
|
| 18 |
+
self.model = "claude-sonnet-4-20250514"
|
| 19 |
+
self._client = None
|
| 20 |
+
|
| 21 |
+
@property
|
| 22 |
+
def client(self):
|
| 23 |
+
if self._client is None and self.api_available:
|
| 24 |
+
import anthropic
|
| 25 |
+
self._client = anthropic.Anthropic(api_key=self.api_key)
|
| 26 |
+
return self._client
|
| 27 |
+
|
| 28 |
+
def parse(
|
| 29 |
+
self,
|
| 30 |
+
user_query: str,
|
| 31 |
+
available_sensors: List[str],
|
| 32 |
+
) -> Optional[Dict]:
|
| 33 |
+
"""
|
| 34 |
+
Parse a natural language query into structured parameters.
|
| 35 |
+
|
| 36 |
+
Args:
|
| 37 |
+
user_query: Natural language query from the engineer
|
| 38 |
+
available_sensors: List of available sensor tag names in the database
|
| 39 |
+
|
| 40 |
+
Returns:
|
| 41 |
+
Dict with keys:
|
| 42 |
+
sensors: List[str] - matched sensor tag names
|
| 43 |
+
start_time: datetime - query start time
|
| 44 |
+
end_time: datetime - query end time
|
| 45 |
+
explanation: str - human-readable interpretation
|
| 46 |
+
Or None if parsing fails.
|
| 47 |
+
"""
|
| 48 |
+
if not self.api_available:
|
| 49 |
+
return None
|
| 50 |
+
|
| 51 |
+
static_instructions = self._build_static_instructions(available_sensors)
|
| 52 |
+
dynamic_query = f'User query: "{user_query}"\n\nReturn ONLY the JSON object, no other text.'
|
| 53 |
+
|
| 54 |
+
try:
|
| 55 |
+
message = self.client.messages.create(
|
| 56 |
+
model=self.model,
|
| 57 |
+
max_tokens=1000,
|
| 58 |
+
system=[
|
| 59 |
+
{
|
| 60 |
+
"type": "text",
|
| 61 |
+
"text": static_instructions,
|
| 62 |
+
"cache_control": {"type": "ephemeral"},
|
| 63 |
+
}
|
| 64 |
+
],
|
| 65 |
+
messages=[{"role": "user", "content": dynamic_query}],
|
| 66 |
+
)
|
| 67 |
+
response_text = message.content[0].text
|
| 68 |
+
parsed = self._extract_json(response_text)
|
| 69 |
+
|
| 70 |
+
if parsed:
|
| 71 |
+
return self._process_result(parsed, available_sensors)
|
| 72 |
+
return None
|
| 73 |
+
|
| 74 |
+
except Exception as e:
|
| 75 |
+
return {'error': str(e)}
|
| 76 |
+
|
| 77 |
+
def _build_static_instructions(self, available_sensors: List[str]) -> str:
|
| 78 |
+
"""Build the cacheable static instruction block with sensor context.
|
| 79 |
+
|
| 80 |
+
This is separated from the dynamic user query so the Anthropic API
|
| 81 |
+
can cache this large, stable prefix across repeated calls within
|
| 82 |
+
the same session (5-minute TTL).
|
| 83 |
+
"""
|
| 84 |
+
# Build sensor reference with descriptions
|
| 85 |
+
sensor_lines = []
|
| 86 |
+
for tag_name, info in config.SENSOR_TAGS.items():
|
| 87 |
+
desc = info.get('description', '')
|
| 88 |
+
units = info.get('units', '')
|
| 89 |
+
sensor_lines.append(f" {tag_name}: {desc} ({units})")
|
| 90 |
+
sensor_reference = "\n".join(sensor_lines)
|
| 91 |
+
|
| 92 |
+
# Build sensor group reference
|
| 93 |
+
group_lines = []
|
| 94 |
+
for group_name, tags in config.SENSOR_GROUPS.items():
|
| 95 |
+
group_lines.append(f" {group_name}: {', '.join(tags)}")
|
| 96 |
+
group_reference = "\n".join(group_lines)
|
| 97 |
+
|
| 98 |
+
current_date = datetime.now()
|
| 99 |
+
|
| 100 |
+
return f"""You are a query parser for a ClearSkies Hydrogen (CSH2) cryopump testing data system. Parse the engineer's natural language query into structured JSON for data retrieval.
|
| 101 |
+
|
| 102 |
+
Current date/time: {current_date.strftime('%Y-%m-%d %H:%M:%S')}
|
| 103 |
+
Data is available from March 2025 to September 2025.
|
| 104 |
+
|
| 105 |
+
AVAILABLE SENSORS (tag name: description (units)):
|
| 106 |
+
{sensor_reference}
|
| 107 |
+
|
| 108 |
+
SENSOR GROUPS (for category-based queries):
|
| 109 |
+
{group_reference}
|
| 110 |
+
|
| 111 |
+
COMMON ALIASES:
|
| 112 |
+
- "pressure" or "pressures" -> PT130 (Discharge), PT01T (Cryotank), PT110 (Inlet)
|
| 113 |
+
- "temperature" or "temperatures" -> TT110 (Pump Feed Line), TT130 (Discharge)
|
| 114 |
+
- "flow" -> FT140 (Flow Transmitter)
|
| 115 |
+
- "motor" or "speed" or "rpm" -> M130_Speed (Motor Speed)
|
| 116 |
+
- "power" -> AvgPower, PeakPower, PowerConsumption
|
| 117 |
+
- "discharge pressure" -> PT130
|
| 118 |
+
- "inlet pressure" or "supply pressure" -> PT110
|
| 119 |
+
- "tank pressure" or "cryotank pressure" -> PT01T
|
| 120 |
+
- "discharge temperature" -> TT130
|
| 121 |
+
- "feed temperature" or "pump temperature" -> TT110
|
| 122 |
+
- "valve" -> AOV140
|
| 123 |
+
- "density" -> N2Density, H2Density
|
| 124 |
+
- "level" -> LT200
|
| 125 |
+
- "all sensors" or "everything" -> PT130, PT01T, PT110, TT110, TT130, FT140, M130_Speed
|
| 126 |
+
|
| 127 |
+
Parse queries into a JSON object with these fields:
|
| 128 |
+
{{
|
| 129 |
+
"sensors": ["PT130", "TT110"],
|
| 130 |
+
"start_time": "2025-09-25T10:00:00",
|
| 131 |
+
"end_time": "2025-09-25T14:00:00",
|
| 132 |
+
"explanation": "Discharge pressure and pump feed temperature from Sept 25, 10 AM to 2 PM"
|
| 133 |
+
}}
|
| 134 |
+
|
| 135 |
+
RULES:
|
| 136 |
+
- Match sensor names to the available sensors list (case-insensitive)
|
| 137 |
+
- Use the aliases above to resolve common terms like "pressure", "temperature" etc.
|
| 138 |
+
- For relative dates like "yesterday", "last week", calculate based on current date
|
| 139 |
+
- For date-only queries without specific times, use 00:00:00 for start and 23:59:59 for end
|
| 140 |
+
- For "between X and Y" time references, parse both times
|
| 141 |
+
- Use ISO 8601 format for timestamps (YYYY-MM-DDTHH:MM:SS)
|
| 142 |
+
- If a sensor group is mentioned (e.g., "all pressures"), include all sensors from that group
|
| 143 |
+
- The explanation should be a concise human-readable summary of what was parsed
|
| 144 |
+
- If no sensors are explicitly mentioned, try to infer from context, or default to PT130, TT110, FT140
|
| 145 |
+
- If no time range is mentioned, default to the most recent day (today)"""
|
| 146 |
+
|
| 147 |
+
def _build_prompt(self, user_query: str, available_sensors: List[str]) -> str:
|
| 148 |
+
"""Build the full Claude prompt (used by fallback path).
|
| 149 |
+
|
| 150 |
+
The main parse() method uses _build_static_instructions() + dynamic query
|
| 151 |
+
separately for prompt caching. This method is kept for backward compatibility.
|
| 152 |
+
"""
|
| 153 |
+
static = self._build_static_instructions(available_sensors)
|
| 154 |
+
return f"""{static}
|
| 155 |
+
|
| 156 |
+
User query: "{user_query}"
|
| 157 |
+
|
| 158 |
+
Return ONLY the JSON object, no other text."""
|
| 159 |
+
|
| 160 |
+
def _extract_json(self, text: str) -> Optional[Dict]:
|
| 161 |
+
"""Extract JSON object from LLM response"""
|
| 162 |
+
try:
|
| 163 |
+
start = text.find('{')
|
| 164 |
+
end = text.rfind('}') + 1
|
| 165 |
+
|
| 166 |
+
if start != -1 and end > start:
|
| 167 |
+
json_str = text[start:end]
|
| 168 |
+
parsed = json.loads(json_str)
|
| 169 |
+
|
| 170 |
+
if 'sensors' in parsed:
|
| 171 |
+
return parsed
|
| 172 |
+
|
| 173 |
+
return None
|
| 174 |
+
except json.JSONDecodeError:
|
| 175 |
+
return None
|
| 176 |
+
|
| 177 |
+
def _process_result(self, parsed: Dict, available_sensors: List[str]) -> Optional[Dict]:
|
| 178 |
+
"""Validate and process the parsed result"""
|
| 179 |
+
try:
|
| 180 |
+
# Convert time strings to datetime
|
| 181 |
+
start_time = self._convert_to_datetime(parsed.get('start_time'))
|
| 182 |
+
end_time = self._convert_to_datetime(parsed.get('end_time'))
|
| 183 |
+
|
| 184 |
+
if not start_time or not end_time:
|
| 185 |
+
return {'error': 'Could not parse time range from query.'}
|
| 186 |
+
|
| 187 |
+
# Validate sensors exist in database
|
| 188 |
+
raw_sensors = parsed.get('sensors', [])
|
| 189 |
+
valid_sensors = []
|
| 190 |
+
# Case-insensitive match against available sensors
|
| 191 |
+
sensor_map = {s.upper(): s for s in available_sensors}
|
| 192 |
+
for s in raw_sensors:
|
| 193 |
+
matched = sensor_map.get(s.upper())
|
| 194 |
+
if matched:
|
| 195 |
+
valid_sensors.append(matched)
|
| 196 |
+
|
| 197 |
+
if not valid_sensors:
|
| 198 |
+
return {'error': f'No matching sensors found. Requested: {", ".join(raw_sensors)}'}
|
| 199 |
+
|
| 200 |
+
return {
|
| 201 |
+
'sensors': valid_sensors,
|
| 202 |
+
'start_time': start_time,
|
| 203 |
+
'end_time': end_time,
|
| 204 |
+
'explanation': parsed.get('explanation', ''),
|
| 205 |
+
}
|
| 206 |
+
|
| 207 |
+
except Exception as e:
|
| 208 |
+
return {'error': f'Processing failed: {e}'}
|
| 209 |
+
|
| 210 |
+
def _convert_to_datetime(self, iso_string: Optional[str]) -> Optional[datetime]:
|
| 211 |
+
"""Convert ISO string to datetime object"""
|
| 212 |
+
if not iso_string:
|
| 213 |
+
return None
|
| 214 |
+
try:
|
| 215 |
+
return datetime.fromisoformat(iso_string)
|
| 216 |
+
except (ValueError, TypeError):
|
| 217 |
+
return None
|
analysis/operations.py
ADDED
|
@@ -0,0 +1,657 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Operation Registry and Dispatcher for Advanced Data Explorer.
|
| 3 |
+
|
| 4 |
+
All post-processing operations the execution engine can dispatch.
|
| 5 |
+
The LLM QueryPlanner references this registry to build execution plans.
|
| 6 |
+
Operations are whitelisted — no arbitrary code execution.
|
| 7 |
+
"""
|
| 8 |
+
import pandas as pd
|
| 9 |
+
import numpy as np
|
| 10 |
+
from typing import Any, Dict, List, Optional
|
| 11 |
+
from dataclasses import dataclass, field
|
| 12 |
+
|
| 13 |
+
from analysis.advanced_analysis import AdvancedAnalyzer
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
# ── Result Dataclass ──────────────────────────────────────────────
|
| 17 |
+
|
| 18 |
+
@dataclass
|
| 19 |
+
class OperationOutput:
|
| 20 |
+
"""Result of a single operation step."""
|
| 21 |
+
op_name: str
|
| 22 |
+
label: str
|
| 23 |
+
data: Any # Primary result (dict, list, DataFrame)
|
| 24 |
+
metadata: Dict = field(default_factory=dict) # Stats about the operation
|
| 25 |
+
exportable_df: Optional[pd.DataFrame] = None # If this step produces downloadable data
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
# ── Operation Registry ────────────────────────────────────────────
|
| 29 |
+
# Each entry describes one whitelisted operation for the LLM prompt.
|
| 30 |
+
|
| 31 |
+
OPERATION_REGISTRY = {
|
| 32 |
+
"find_peak_pressure_blocks": {
|
| 33 |
+
"description": "Find contiguous time blocks where a pressure sensor exceeds a threshold",
|
| 34 |
+
"params": {
|
| 35 |
+
"pressure_tag": {"type": "str", "required": True, "description": "Sensor tag (e.g. PT130)"},
|
| 36 |
+
"threshold_bar": {"type": "float", "required": True, "description": "Pressure threshold in bar"},
|
| 37 |
+
},
|
| 38 |
+
"returns": "Dict with blocks list: start_time, end_time, duration, peak, avg for each block",
|
| 39 |
+
},
|
| 40 |
+
"detect_constant_periods": {
|
| 41 |
+
"description": "Find periods where a single sensor value is constant within tolerance for a minimum duration",
|
| 42 |
+
"params": {
|
| 43 |
+
"tag": {"type": "str", "required": True, "description": "Sensor tag name"},
|
| 44 |
+
"tolerance": {"type": "float", "required": False, "default": 0.01},
|
| 45 |
+
"min_duration_minutes": {"type": "float", "required": False, "default": 2.0},
|
| 46 |
+
},
|
| 47 |
+
"returns": "List of constant period dicts: start, end, duration_minutes, value",
|
| 48 |
+
},
|
| 49 |
+
"detect_constant_periods_multi": {
|
| 50 |
+
"description": "Find periods where multiple sensors are simultaneously constant (overlapping constant windows)",
|
| 51 |
+
"params": {
|
| 52 |
+
"tags": {"type": "List[str]", "required": True, "description": "Sensor tags that must all be constant"},
|
| 53 |
+
"tolerance": {"type": "float", "required": False, "default": 0.01},
|
| 54 |
+
"min_duration_minutes": {"type": "float", "required": False, "default": 2.0},
|
| 55 |
+
},
|
| 56 |
+
"returns": "List of overlapping constant period dicts",
|
| 57 |
+
},
|
| 58 |
+
"downsample_frequency": {
|
| 59 |
+
"description": "Reduce data frequency (e.g. 10Hz to 2Hz). Requires raw resolution data.",
|
| 60 |
+
"params": {
|
| 61 |
+
"target_hz": {"type": "float", "required": True, "description": "Target frequency in Hz"},
|
| 62 |
+
},
|
| 63 |
+
"returns": "Downsampled DataFrame",
|
| 64 |
+
},
|
| 65 |
+
"detect_pressure_ramps": {
|
| 66 |
+
"description": "Detect windows where pressure ramps from one level to another",
|
| 67 |
+
"params": {
|
| 68 |
+
"pressure_tag": {"type": "str", "required": True, "description": "Pressure sensor tag"},
|
| 69 |
+
"start_bar": {"type": "float", "required": True, "description": "Ramp start pressure in bar"},
|
| 70 |
+
"end_bar": {"type": "float", "required": True, "description": "Ramp end pressure in bar"},
|
| 71 |
+
"max_ramps": {"type": "int", "required": False, "default": None, "description": "Max ramps to return"},
|
| 72 |
+
},
|
| 73 |
+
"returns": "List of ramp dicts: start_time, end_time, start_pressure, peak_pressure, duration_minutes",
|
| 74 |
+
},
|
| 75 |
+
"extract_windows": {
|
| 76 |
+
"description": "Extract sub-DataFrames for each time window from a previous operation's result",
|
| 77 |
+
"params": {
|
| 78 |
+
"source": {"type": "str", "required": True, "description": "Must be 'previous_result'"},
|
| 79 |
+
},
|
| 80 |
+
"returns": "List of DataFrames, one per detected window",
|
| 81 |
+
},
|
| 82 |
+
"filter_by_condition": {
|
| 83 |
+
"description": "Filter data rows where a sensor meets a condition",
|
| 84 |
+
"params": {
|
| 85 |
+
"tag": {"type": "str", "required": True, "description": "Sensor tag"},
|
| 86 |
+
"operator": {"type": "str", "required": True, "description": "One of: gt, gte, lt, lte, eq"},
|
| 87 |
+
"value": {"type": "float", "required": True, "description": "Threshold value"},
|
| 88 |
+
},
|
| 89 |
+
"returns": "Filtered DataFrame",
|
| 90 |
+
},
|
| 91 |
+
"compute_statistics": {
|
| 92 |
+
"description": "Compute summary statistics (min, max, mean, std, count) for sensors",
|
| 93 |
+
"params": {
|
| 94 |
+
"tags": {"type": "List[str]", "required": False, "default": None, "description": "Tags to summarize (default: all)"},
|
| 95 |
+
},
|
| 96 |
+
"returns": "Statistics dict per sensor",
|
| 97 |
+
},
|
| 98 |
+
"detect_and_analyze_fills": {
|
| 99 |
+
"description": "Detect fill/test cycles using motor speed, then compute per-fill metrics: kWh/kg, total mass, pump strokes, peak pressure. Can filter by kWh/kg threshold or specific fill ID.",
|
| 100 |
+
"params": {
|
| 101 |
+
"kwh_per_kg_threshold": {"type": "float", "required": False, "default": None, "description": "Only return fills where kWh/kg exceeds this"},
|
| 102 |
+
"fill_id": {"type": "int", "required": False, "default": None, "description": "Analyze only this fill number"},
|
| 103 |
+
"include_strokes": {"type": "bool", "required": False, "default": True, "description": "Whether to compute stroke count"},
|
| 104 |
+
},
|
| 105 |
+
"returns": "List of fill dicts: fill_id, start_time, end_time, duration_min, peak_pressure, avg_flow, total_mass_kg, kwh_per_kg, total_strokes",
|
| 106 |
+
},
|
| 107 |
+
"count_pump_strokes": {
|
| 108 |
+
"description": "Count total pump strokes (motor revolutions) in the time window by integrating RPM over time",
|
| 109 |
+
"params": {
|
| 110 |
+
"speed_tag": {"type": "str", "required": False, "default": None, "description": "Motor speed tag (defaults to first available)"},
|
| 111 |
+
},
|
| 112 |
+
"returns": "Dict: total_revolutions, avg_rpm, active_minutes, peak_rpm, note",
|
| 113 |
+
},
|
| 114 |
+
}
|
| 115 |
+
|
| 116 |
+
|
| 117 |
+
def get_registry_for_prompt() -> str:
|
| 118 |
+
"""Serialize the operation registry as text for LLM prompt injection."""
|
| 119 |
+
lines = ["AVAILABLE OPERATIONS:"]
|
| 120 |
+
for op_name, info in OPERATION_REGISTRY.items():
|
| 121 |
+
lines.append(f"\n {op_name}: {info['description']}")
|
| 122 |
+
lines.append(" Parameters:")
|
| 123 |
+
for pname, pinfo in info["params"].items():
|
| 124 |
+
req = "REQUIRED" if pinfo.get("required") else f"optional, default={pinfo.get('default')}"
|
| 125 |
+
lines.append(f" {pname} ({pinfo['type']}, {req}): {pinfo.get('description', '')}")
|
| 126 |
+
lines.append(f" Returns: {info['returns']}")
|
| 127 |
+
return "\n".join(lines)
|
| 128 |
+
|
| 129 |
+
|
| 130 |
+
# ── Operation Dispatcher ──────────────────────────────────────────
|
| 131 |
+
|
| 132 |
+
OPERATOR_MAP = {"gt": ">", "gte": ">=", "lt": "<", "lte": "<=", "eq": "=="}
|
| 133 |
+
MOTOR_SPEED_TAGS = ["M130_Speed", "MC130_VFD_Speed", "M130_RPM"]
|
| 134 |
+
|
| 135 |
+
|
| 136 |
+
class OperationDispatcher:
|
| 137 |
+
"""Dispatch whitelisted operations against a DataFrame."""
|
| 138 |
+
|
| 139 |
+
def __init__(self):
|
| 140 |
+
self.analyzer = AdvancedAnalyzer()
|
| 141 |
+
|
| 142 |
+
def run(
|
| 143 |
+
self,
|
| 144 |
+
op_name: str,
|
| 145 |
+
df: pd.DataFrame,
|
| 146 |
+
params: Dict,
|
| 147 |
+
prev_result: Any = None,
|
| 148 |
+
db_connector=None,
|
| 149 |
+
) -> OperationOutput:
|
| 150 |
+
"""Dispatch a single operation by name."""
|
| 151 |
+
handler = getattr(self, f"_run_{op_name}", None)
|
| 152 |
+
if handler is None:
|
| 153 |
+
raise ValueError(f"Unknown operation: {op_name}")
|
| 154 |
+
|
| 155 |
+
# Some operations need extra context
|
| 156 |
+
if op_name in ("detect_and_analyze_fills",):
|
| 157 |
+
return handler(df, params, db_connector=db_connector)
|
| 158 |
+
if op_name in ("extract_windows",):
|
| 159 |
+
return handler(df, params, prev_result=prev_result)
|
| 160 |
+
|
| 161 |
+
return handler(df, params)
|
| 162 |
+
|
| 163 |
+
# ── Handlers ──────────────────────────────────────────────────
|
| 164 |
+
|
| 165 |
+
def _run_find_peak_pressure_blocks(self, df: pd.DataFrame, params: Dict) -> OperationOutput:
|
| 166 |
+
pressure_tag = params["pressure_tag"]
|
| 167 |
+
threshold = float(params["threshold_bar"])
|
| 168 |
+
result = self.analyzer.find_peak_pressure_blocks(df, pressure_tag, threshold)
|
| 169 |
+
|
| 170 |
+
# Build exportable table from blocks
|
| 171 |
+
export_df = None
|
| 172 |
+
blocks = result.get("blocks", [])
|
| 173 |
+
if blocks:
|
| 174 |
+
export_df = pd.DataFrame(blocks)
|
| 175 |
+
|
| 176 |
+
return OperationOutput(
|
| 177 |
+
op_name="find_peak_pressure_blocks",
|
| 178 |
+
label=f"{pressure_tag} > {threshold} bar",
|
| 179 |
+
data=result,
|
| 180 |
+
metadata={"num_blocks": result.get("num_blocks", 0), "total_points": result.get("total_points_above", 0)},
|
| 181 |
+
exportable_df=export_df,
|
| 182 |
+
)
|
| 183 |
+
|
| 184 |
+
def _run_detect_constant_periods(self, df: pd.DataFrame, params: Dict) -> OperationOutput:
|
| 185 |
+
tag = params["tag"]
|
| 186 |
+
tolerance = float(params.get("tolerance", 0.01))
|
| 187 |
+
min_dur = float(params.get("min_duration_minutes", 2.0))
|
| 188 |
+
periods = self.analyzer.detect_constant_periods(df, tag, tolerance, min_dur)
|
| 189 |
+
|
| 190 |
+
export_df = pd.DataFrame(periods) if periods else None
|
| 191 |
+
|
| 192 |
+
return OperationOutput(
|
| 193 |
+
op_name="detect_constant_periods",
|
| 194 |
+
label=f"{tag} constant periods (±{tolerance}, ≥{min_dur} min)",
|
| 195 |
+
data=periods,
|
| 196 |
+
metadata={"num_periods": len(periods)},
|
| 197 |
+
exportable_df=export_df,
|
| 198 |
+
)
|
| 199 |
+
|
| 200 |
+
def _run_detect_constant_periods_multi(self, df: pd.DataFrame, params: Dict) -> OperationOutput:
|
| 201 |
+
tags = params["tags"]
|
| 202 |
+
tolerance = float(params.get("tolerance", 0.01))
|
| 203 |
+
min_dur = float(params.get("min_duration_minutes", 2.0))
|
| 204 |
+
|
| 205 |
+
# Detect constant periods per tag
|
| 206 |
+
per_tag_periods = {}
|
| 207 |
+
for tag in tags:
|
| 208 |
+
if tag in df.columns:
|
| 209 |
+
per_tag_periods[tag] = self.analyzer.detect_constant_periods(df, tag, tolerance, min_dur)
|
| 210 |
+
else:
|
| 211 |
+
per_tag_periods[tag] = []
|
| 212 |
+
|
| 213 |
+
# Find overlapping intervals across all tags
|
| 214 |
+
overlapping = self._intersect_periods(per_tag_periods, min_dur)
|
| 215 |
+
|
| 216 |
+
export_df = pd.DataFrame(overlapping) if overlapping else None
|
| 217 |
+
|
| 218 |
+
return OperationOutput(
|
| 219 |
+
op_name="detect_constant_periods_multi",
|
| 220 |
+
label=f"Simultaneous constant: {', '.join(tags)} (≥{min_dur} min)",
|
| 221 |
+
data=overlapping,
|
| 222 |
+
metadata={"num_overlaps": len(overlapping), "per_tag_counts": {t: len(p) for t, p in per_tag_periods.items()}},
|
| 223 |
+
exportable_df=export_df,
|
| 224 |
+
)
|
| 225 |
+
|
| 226 |
+
def _intersect_periods(self, per_tag: Dict[str, List[Dict]], min_dur: float) -> List[Dict]:
|
| 227 |
+
"""Intersect constant periods across multiple tags — return only overlapping windows."""
|
| 228 |
+
tags = list(per_tag.keys())
|
| 229 |
+
if not tags or any(len(per_tag[t]) == 0 for t in tags):
|
| 230 |
+
return []
|
| 231 |
+
|
| 232 |
+
# Start with first tag's periods
|
| 233 |
+
current_intervals = [(p["start"], p["end"]) for p in per_tag[tags[0]]]
|
| 234 |
+
|
| 235 |
+
# Intersect with each subsequent tag
|
| 236 |
+
for tag in tags[1:]:
|
| 237 |
+
other = [(p["start"], p["end"]) for p in per_tag[tag]]
|
| 238 |
+
current_intervals = self._pairwise_intersect(current_intervals, other)
|
| 239 |
+
if not current_intervals:
|
| 240 |
+
return []
|
| 241 |
+
|
| 242 |
+
# Filter by minimum duration and build result dicts
|
| 243 |
+
result = []
|
| 244 |
+
for start, end in current_intervals:
|
| 245 |
+
dur_min = (end - start).total_seconds() / 60
|
| 246 |
+
if dur_min >= min_dur:
|
| 247 |
+
# Gather each tag's value during this window
|
| 248 |
+
tag_values = {}
|
| 249 |
+
for tag in tags:
|
| 250 |
+
for p in per_tag[tag]:
|
| 251 |
+
if p["start"] <= start and p["end"] >= end:
|
| 252 |
+
tag_values[tag] = p["value"]
|
| 253 |
+
break
|
| 254 |
+
|
| 255 |
+
result.append({
|
| 256 |
+
"start": start,
|
| 257 |
+
"end": end,
|
| 258 |
+
"duration_minutes": dur_min,
|
| 259 |
+
"tag_values": tag_values,
|
| 260 |
+
})
|
| 261 |
+
return result
|
| 262 |
+
|
| 263 |
+
@staticmethod
|
| 264 |
+
def _pairwise_intersect(intervals_a, intervals_b):
|
| 265 |
+
"""Compute pairwise intersection of two sorted interval lists."""
|
| 266 |
+
result = []
|
| 267 |
+
i, j = 0, 0
|
| 268 |
+
while i < len(intervals_a) and j < len(intervals_b):
|
| 269 |
+
a_start, a_end = intervals_a[i]
|
| 270 |
+
b_start, b_end = intervals_b[j]
|
| 271 |
+
# Overlap
|
| 272 |
+
start = max(a_start, b_start)
|
| 273 |
+
end = min(a_end, b_end)
|
| 274 |
+
if start < end:
|
| 275 |
+
result.append((start, end))
|
| 276 |
+
# Advance the interval that ends first
|
| 277 |
+
if a_end <= b_end:
|
| 278 |
+
i += 1
|
| 279 |
+
else:
|
| 280 |
+
j += 1
|
| 281 |
+
return result
|
| 282 |
+
|
| 283 |
+
def _run_downsample_frequency(self, df: pd.DataFrame, params: Dict) -> OperationOutput:
|
| 284 |
+
target_hz = float(params["target_hz"])
|
| 285 |
+
|
| 286 |
+
if "timestamp" not in df.columns or len(df) < 2:
|
| 287 |
+
return OperationOutput("downsample_frequency", f"Downsample to {target_hz} Hz", df, {"error": "Insufficient data"})
|
| 288 |
+
|
| 289 |
+
# Estimate current frequency
|
| 290 |
+
sorted_df = df.sort_values("timestamp")
|
| 291 |
+
dt = sorted_df["timestamp"].diff().dt.total_seconds().dropna()
|
| 292 |
+
current_hz = 1.0 / dt.median() if dt.median() > 0 else 1.0
|
| 293 |
+
|
| 294 |
+
if target_hz >= current_hz:
|
| 295 |
+
# Already at or below target frequency
|
| 296 |
+
return OperationOutput(
|
| 297 |
+
"downsample_frequency", f"Data already at {current_hz:.1f} Hz (target: {target_hz} Hz)",
|
| 298 |
+
sorted_df, {"current_hz": current_hz, "target_hz": target_hz, "rows_before": len(sorted_df), "rows_after": len(sorted_df)},
|
| 299 |
+
exportable_df=sorted_df,
|
| 300 |
+
)
|
| 301 |
+
|
| 302 |
+
step = max(1, int(round(current_hz / target_hz)))
|
| 303 |
+
downsampled = sorted_df.iloc[::step].reset_index(drop=True)
|
| 304 |
+
|
| 305 |
+
return OperationOutput(
|
| 306 |
+
op_name="downsample_frequency",
|
| 307 |
+
label=f"Downsampled {current_hz:.1f} Hz → {target_hz} Hz",
|
| 308 |
+
data=downsampled,
|
| 309 |
+
metadata={"current_hz": current_hz, "target_hz": target_hz, "step": step,
|
| 310 |
+
"rows_before": len(sorted_df), "rows_after": len(downsampled)},
|
| 311 |
+
exportable_df=downsampled,
|
| 312 |
+
)
|
| 313 |
+
|
| 314 |
+
def _run_detect_pressure_ramps(self, df: pd.DataFrame, params: Dict) -> OperationOutput:
|
| 315 |
+
pressure_tag = params["pressure_tag"]
|
| 316 |
+
start_bar = float(params["start_bar"])
|
| 317 |
+
end_bar = float(params["end_bar"])
|
| 318 |
+
max_ramps = params.get("max_ramps")
|
| 319 |
+
|
| 320 |
+
if pressure_tag not in df.columns:
|
| 321 |
+
return OperationOutput("detect_pressure_ramps", f"Ramps in {pressure_tag}", [],
|
| 322 |
+
{"error": f"Tag {pressure_tag} not found"})
|
| 323 |
+
|
| 324 |
+
sorted_df = df[["timestamp", pressure_tag]].dropna(subset=[pressure_tag]).sort_values("timestamp")
|
| 325 |
+
timestamps = sorted_df["timestamp"].values
|
| 326 |
+
values = sorted_df[pressure_tag].values
|
| 327 |
+
|
| 328 |
+
# State machine: IDLE → RAMPING → COMPLETE
|
| 329 |
+
ramps = []
|
| 330 |
+
state = "IDLE"
|
| 331 |
+
ramp_start_idx = 0
|
| 332 |
+
ramp_max_val = 0.0
|
| 333 |
+
|
| 334 |
+
for i in range(len(values)):
|
| 335 |
+
if state == "IDLE":
|
| 336 |
+
# Look for crossing start_bar from below
|
| 337 |
+
if values[i] >= start_bar * 0.9 and values[i] <= start_bar * 1.3:
|
| 338 |
+
state = "RAMPING"
|
| 339 |
+
ramp_start_idx = i
|
| 340 |
+
ramp_max_val = values[i]
|
| 341 |
+
elif state == "RAMPING":
|
| 342 |
+
ramp_max_val = max(ramp_max_val, values[i])
|
| 343 |
+
|
| 344 |
+
# Ramp complete: crossed end_bar
|
| 345 |
+
if values[i] >= end_bar:
|
| 346 |
+
ramps.append({
|
| 347 |
+
"start_time": pd.Timestamp(timestamps[ramp_start_idx]),
|
| 348 |
+
"end_time": pd.Timestamp(timestamps[i]),
|
| 349 |
+
"start_pressure": float(values[ramp_start_idx]),
|
| 350 |
+
"peak_pressure": float(ramp_max_val),
|
| 351 |
+
"duration_minutes": (pd.Timestamp(timestamps[i]) - pd.Timestamp(timestamps[ramp_start_idx])).total_seconds() / 60,
|
| 352 |
+
})
|
| 353 |
+
state = "IDLE"
|
| 354 |
+
continue
|
| 355 |
+
|
| 356 |
+
# Ramp aborted: pressure dropped significantly below start
|
| 357 |
+
if values[i] < start_bar * 0.5:
|
| 358 |
+
state = "IDLE"
|
| 359 |
+
|
| 360 |
+
if max_ramps and len(ramps) > int(max_ramps):
|
| 361 |
+
ramps = ramps[:int(max_ramps)]
|
| 362 |
+
|
| 363 |
+
export_df = pd.DataFrame(ramps) if ramps else None
|
| 364 |
+
|
| 365 |
+
return OperationOutput(
|
| 366 |
+
op_name="detect_pressure_ramps",
|
| 367 |
+
label=f"{pressure_tag} ramps {start_bar}→{end_bar} bar",
|
| 368 |
+
data=ramps,
|
| 369 |
+
metadata={"num_ramps": len(ramps)},
|
| 370 |
+
exportable_df=export_df,
|
| 371 |
+
)
|
| 372 |
+
|
| 373 |
+
def _run_extract_windows(self, df: pd.DataFrame, params: Dict, prev_result: Any = None) -> OperationOutput:
|
| 374 |
+
"""Slice DataFrame into sub-DataFrames per detected window from previous operation."""
|
| 375 |
+
windows = []
|
| 376 |
+
if isinstance(prev_result, OperationOutput):
|
| 377 |
+
data = prev_result.data
|
| 378 |
+
if isinstance(data, list):
|
| 379 |
+
windows = data
|
| 380 |
+
elif isinstance(data, dict):
|
| 381 |
+
windows = data.get("blocks", data.get("ramps", []))
|
| 382 |
+
|
| 383 |
+
if not windows:
|
| 384 |
+
return OperationOutput("extract_windows", "Extract windows", [],
|
| 385 |
+
{"error": "No windows found in previous result"})
|
| 386 |
+
|
| 387 |
+
dfs = []
|
| 388 |
+
labels = []
|
| 389 |
+
for i, w in enumerate(windows):
|
| 390 |
+
start = w.get("start_time", w.get("start"))
|
| 391 |
+
end = w.get("end_time", w.get("end"))
|
| 392 |
+
if start is None or end is None:
|
| 393 |
+
continue
|
| 394 |
+
|
| 395 |
+
mask = (df["timestamp"] >= pd.Timestamp(start)) & (df["timestamp"] <= pd.Timestamp(end))
|
| 396 |
+
window_df = df[mask].copy()
|
| 397 |
+
if not window_df.empty:
|
| 398 |
+
dfs.append(window_df)
|
| 399 |
+
labels.append(f"Window {i + 1}")
|
| 400 |
+
|
| 401 |
+
return OperationOutput(
|
| 402 |
+
op_name="extract_windows",
|
| 403 |
+
label=f"Extracted {len(dfs)} windows",
|
| 404 |
+
data=dfs,
|
| 405 |
+
metadata={"num_windows": len(dfs), "labels": labels,
|
| 406 |
+
"rows_per_window": [len(d) for d in dfs]},
|
| 407 |
+
exportable_df=None, # Multi-window: handled by export_utils
|
| 408 |
+
)
|
| 409 |
+
|
| 410 |
+
def _run_filter_by_condition(self, df: pd.DataFrame, params: Dict) -> OperationOutput:
|
| 411 |
+
tag = params["tag"]
|
| 412 |
+
operator = params["operator"]
|
| 413 |
+
value = float(params["value"])
|
| 414 |
+
|
| 415 |
+
if tag not in df.columns:
|
| 416 |
+
return OperationOutput("filter_by_condition", f"Filter {tag}", df,
|
| 417 |
+
{"error": f"Tag {tag} not found"})
|
| 418 |
+
|
| 419 |
+
op_str = OPERATOR_MAP.get(operator)
|
| 420 |
+
if not op_str:
|
| 421 |
+
return OperationOutput("filter_by_condition", f"Filter {tag}", df,
|
| 422 |
+
{"error": f"Unknown operator: {operator}. Use: gt, gte, lt, lte, eq"})
|
| 423 |
+
|
| 424 |
+
if operator == "gt":
|
| 425 |
+
mask = df[tag] > value
|
| 426 |
+
elif operator == "gte":
|
| 427 |
+
mask = df[tag] >= value
|
| 428 |
+
elif operator == "lt":
|
| 429 |
+
mask = df[tag] < value
|
| 430 |
+
elif operator == "lte":
|
| 431 |
+
mask = df[tag] <= value
|
| 432 |
+
else:
|
| 433 |
+
mask = df[tag] == value
|
| 434 |
+
|
| 435 |
+
filtered = df[mask].copy()
|
| 436 |
+
|
| 437 |
+
return OperationOutput(
|
| 438 |
+
op_name="filter_by_condition",
|
| 439 |
+
label=f"{tag} {op_str} {value}",
|
| 440 |
+
data=filtered,
|
| 441 |
+
metadata={"rows_before": len(df), "rows_after": len(filtered)},
|
| 442 |
+
exportable_df=filtered,
|
| 443 |
+
)
|
| 444 |
+
|
| 445 |
+
def _run_compute_statistics(self, df: pd.DataFrame, params: Dict) -> OperationOutput:
|
| 446 |
+
tags = params.get("tags")
|
| 447 |
+
numeric_cols = [c for c in df.columns if c != "timestamp" and pd.api.types.is_numeric_dtype(df[c])]
|
| 448 |
+
if tags:
|
| 449 |
+
numeric_cols = [c for c in numeric_cols if c in tags]
|
| 450 |
+
|
| 451 |
+
stats = {}
|
| 452 |
+
for col in numeric_cols:
|
| 453 |
+
data = df[col].dropna()
|
| 454 |
+
if len(data) > 0:
|
| 455 |
+
stats[col] = {
|
| 456 |
+
"count": int(len(data)),
|
| 457 |
+
"min": float(data.min()),
|
| 458 |
+
"max": float(data.max()),
|
| 459 |
+
"mean": float(data.mean()),
|
| 460 |
+
"std": float(data.std()),
|
| 461 |
+
}
|
| 462 |
+
|
| 463 |
+
export_df = pd.DataFrame(stats).T.reset_index().rename(columns={"index": "Sensor"}) if stats else None
|
| 464 |
+
|
| 465 |
+
return OperationOutput(
|
| 466 |
+
op_name="compute_statistics",
|
| 467 |
+
label=f"Statistics for {len(numeric_cols)} sensors",
|
| 468 |
+
data=stats,
|
| 469 |
+
metadata={"num_sensors": len(stats)},
|
| 470 |
+
exportable_df=export_df,
|
| 471 |
+
)
|
| 472 |
+
|
| 473 |
+
def _run_detect_and_analyze_fills(self, df: pd.DataFrame, params: Dict, db_connector=None) -> OperationOutput:
|
| 474 |
+
"""Detect fill cycles and compute per-fill metrics."""
|
| 475 |
+
from analysis.pump_cycle_detector import PumpCycleDetector
|
| 476 |
+
|
| 477 |
+
kwh_threshold = params.get("kwh_per_kg_threshold")
|
| 478 |
+
target_fill_id = params.get("fill_id")
|
| 479 |
+
include_strokes = params.get("include_strokes", True)
|
| 480 |
+
|
| 481 |
+
# Find motor speed tag in DataFrame
|
| 482 |
+
speed_tag = None
|
| 483 |
+
for tag in MOTOR_SPEED_TAGS:
|
| 484 |
+
if tag in df.columns:
|
| 485 |
+
speed_tag = tag
|
| 486 |
+
break
|
| 487 |
+
|
| 488 |
+
if not speed_tag:
|
| 489 |
+
return OperationOutput("detect_and_analyze_fills", "Detect fills", [],
|
| 490 |
+
{"error": "No motor speed tag found in data. Include M130_Speed or MC130_VFD_Speed."})
|
| 491 |
+
|
| 492 |
+
# Prepare motor speed series for cycle detector
|
| 493 |
+
motor_df = df[["timestamp", speed_tag]].dropna(subset=[speed_tag]).rename(columns={speed_tag: "value"})
|
| 494 |
+
|
| 495 |
+
detector = PumpCycleDetector()
|
| 496 |
+
raw_cycles = detector.detect_cycles(motor_df)
|
| 497 |
+
|
| 498 |
+
if not raw_cycles:
|
| 499 |
+
return OperationOutput("detect_and_analyze_fills", "Detect fills", [],
|
| 500 |
+
{"error": "No cycles detected in the time range"})
|
| 501 |
+
|
| 502 |
+
# Enrich with pressure/flow/temp from the fetched data (no additional DB query needed)
|
| 503 |
+
enriched = self._enrich_fills_from_df(raw_cycles, df, db_connector)
|
| 504 |
+
|
| 505 |
+
# Compute per-fill advanced metrics
|
| 506 |
+
fill_results = []
|
| 507 |
+
for c in enriched:
|
| 508 |
+
fill_dict = {
|
| 509 |
+
"fill_id": c["cycle_id"],
|
| 510 |
+
"start_time": c["start_time"],
|
| 511 |
+
"end_time": c["end_time"],
|
| 512 |
+
"duration_min": c.get("duration_minutes", 0),
|
| 513 |
+
"peak_pressure_bar": c.get("peak_pressure", None),
|
| 514 |
+
"avg_flow_kg_min": c.get("avg_flow", None),
|
| 515 |
+
"total_mass_kg": c.get("total_mass_kg", None),
|
| 516 |
+
"kwh_per_kg": None,
|
| 517 |
+
"total_revolutions": None,
|
| 518 |
+
}
|
| 519 |
+
|
| 520 |
+
# kWh/kg computation
|
| 521 |
+
if "avg_power_kw" in c and c.get("avg_flow") and c["avg_flow"] > 0:
|
| 522 |
+
# kWh/kg = AvgPower [kW] * 60 / FT140 [kg/min]
|
| 523 |
+
kwh_kg = (c["avg_power_kw"] * 60) / c["avg_flow"] if c["avg_flow"] > 0.001 else None
|
| 524 |
+
fill_dict["kwh_per_kg"] = round(kwh_kg, 4) if kwh_kg is not None else None
|
| 525 |
+
|
| 526 |
+
# Stroke count
|
| 527 |
+
if include_strokes and "avg_rpm" in c and c.get("duration_minutes"):
|
| 528 |
+
revolutions = c["avg_rpm"] * c["duration_minutes"]
|
| 529 |
+
fill_dict["total_revolutions"] = int(round(revolutions))
|
| 530 |
+
|
| 531 |
+
fill_results.append(fill_dict)
|
| 532 |
+
|
| 533 |
+
# Apply filters
|
| 534 |
+
if target_fill_id is not None:
|
| 535 |
+
target_id = int(target_fill_id)
|
| 536 |
+
matched = [f for f in fill_results if f["fill_id"] == target_id]
|
| 537 |
+
if not matched:
|
| 538 |
+
available = [f["fill_id"] for f in fill_results]
|
| 539 |
+
return OperationOutput("detect_and_analyze_fills", f"Fill {target_id}", [],
|
| 540 |
+
{"error": f"Fill {target_id} not found. Available fill IDs: {available}"})
|
| 541 |
+
fill_results = matched
|
| 542 |
+
|
| 543 |
+
if kwh_threshold is not None:
|
| 544 |
+
threshold = float(kwh_threshold)
|
| 545 |
+
fill_results = [f for f in fill_results if f.get("kwh_per_kg") is not None and f["kwh_per_kg"] > threshold]
|
| 546 |
+
|
| 547 |
+
export_df = pd.DataFrame(fill_results) if fill_results else None
|
| 548 |
+
|
| 549 |
+
note = "Revolutions = RPM × minutes. For simplex: strokes = revolutions. For triplex: strokes = 3 × revolutions."
|
| 550 |
+
|
| 551 |
+
return OperationOutput(
|
| 552 |
+
op_name="detect_and_analyze_fills",
|
| 553 |
+
label=f"Fill analysis ({len(fill_results)} fills)",
|
| 554 |
+
data=fill_results,
|
| 555 |
+
metadata={"num_fills": len(fill_results), "note": note},
|
| 556 |
+
exportable_df=export_df,
|
| 557 |
+
)
|
| 558 |
+
|
| 559 |
+
def _enrich_fills_from_df(self, cycles: List[Dict], df: pd.DataFrame, db_connector) -> List[Dict]:
|
| 560 |
+
"""Enrich cycle dicts with metrics computed from the already-fetched wide DataFrame."""
|
| 561 |
+
enriched = []
|
| 562 |
+
for c in cycles:
|
| 563 |
+
ec = c.copy()
|
| 564 |
+
mask = (df["timestamp"] >= c["start_time"]) & (df["timestamp"] <= c["end_time"])
|
| 565 |
+
cdf = df[mask]
|
| 566 |
+
|
| 567 |
+
if cdf.empty:
|
| 568 |
+
enriched.append(ec)
|
| 569 |
+
continue
|
| 570 |
+
|
| 571 |
+
# Peak pressure
|
| 572 |
+
if "PT130" in cdf.columns:
|
| 573 |
+
pt130 = cdf["PT130"].dropna()
|
| 574 |
+
if len(pt130) > 0:
|
| 575 |
+
ec["peak_pressure"] = float(pt130.max())
|
| 576 |
+
ec["initial_pressure"] = float(pt130.iloc[0])
|
| 577 |
+
|
| 578 |
+
# Average flow
|
| 579 |
+
if "FT140" in cdf.columns:
|
| 580 |
+
ft140 = cdf["FT140"].dropna()
|
| 581 |
+
if len(ft140) > 0:
|
| 582 |
+
ec["avg_flow"] = float(ft140.mean())
|
| 583 |
+
# Total mass via trapezoidal integration
|
| 584 |
+
flow_sorted = cdf[["timestamp", "FT140"]].dropna().sort_values("timestamp")
|
| 585 |
+
if len(flow_sorted) > 1:
|
| 586 |
+
dt_min = flow_sorted["timestamp"].diff().dt.total_seconds().fillna(0) / 60
|
| 587 |
+
ec["total_mass_kg"] = float((flow_sorted["FT140"] * dt_min).sum())
|
| 588 |
+
|
| 589 |
+
# Average power
|
| 590 |
+
if "AvgPower" in cdf.columns:
|
| 591 |
+
power = cdf["AvgPower"].dropna()
|
| 592 |
+
if len(power) > 0:
|
| 593 |
+
ec["avg_power_kw"] = float(power.mean())
|
| 594 |
+
|
| 595 |
+
# Motor speed
|
| 596 |
+
for tag in MOTOR_SPEED_TAGS:
|
| 597 |
+
if tag in cdf.columns:
|
| 598 |
+
speed = cdf[tag].dropna()
|
| 599 |
+
if len(speed) > 0:
|
| 600 |
+
ec["avg_rpm"] = float(speed.mean())
|
| 601 |
+
ec["peak_rpm"] = float(speed.max())
|
| 602 |
+
break
|
| 603 |
+
|
| 604 |
+
# Peak temperatures
|
| 605 |
+
for tag in ["TT110", "TT130"]:
|
| 606 |
+
if tag in cdf.columns:
|
| 607 |
+
data = cdf[tag].dropna()
|
| 608 |
+
if len(data) > 0:
|
| 609 |
+
ec[f"peak_{tag}"] = float(data.max())
|
| 610 |
+
|
| 611 |
+
enriched.append(ec)
|
| 612 |
+
return enriched
|
| 613 |
+
|
| 614 |
+
def _run_count_pump_strokes(self, df: pd.DataFrame, params: Dict) -> OperationOutput:
|
| 615 |
+
"""Count total pump revolutions by integrating RPM over time."""
|
| 616 |
+
speed_tag = params.get("speed_tag")
|
| 617 |
+
|
| 618 |
+
if not speed_tag:
|
| 619 |
+
for tag in MOTOR_SPEED_TAGS:
|
| 620 |
+
if tag in df.columns:
|
| 621 |
+
speed_tag = tag
|
| 622 |
+
break
|
| 623 |
+
|
| 624 |
+
if not speed_tag or speed_tag not in df.columns:
|
| 625 |
+
return OperationOutput("count_pump_strokes", "Pump strokes", {},
|
| 626 |
+
{"error": "No motor speed tag found. Include M130_Speed or MC130_VFD_Speed."})
|
| 627 |
+
|
| 628 |
+
speed_df = df[["timestamp", speed_tag]].dropna(subset=[speed_tag]).sort_values("timestamp")
|
| 629 |
+
|
| 630 |
+
if len(speed_df) < 2:
|
| 631 |
+
return OperationOutput("count_pump_strokes", "Pump strokes", {},
|
| 632 |
+
{"error": "Insufficient motor speed data"})
|
| 633 |
+
|
| 634 |
+
rpm_values = speed_df[speed_tag].values
|
| 635 |
+
timestamps = speed_df["timestamp"].values
|
| 636 |
+
|
| 637 |
+
# Integrate: revolutions = sum(RPM_i × Δt_i) where Δt in minutes
|
| 638 |
+
dt_minutes = pd.Series(timestamps).diff().dt.total_seconds().fillna(0).values / 60
|
| 639 |
+
total_revolutions = float(np.sum(rpm_values * dt_minutes))
|
| 640 |
+
active_mask = rpm_values > 10 # RPM > idle threshold
|
| 641 |
+
active_minutes = float(np.sum(dt_minutes[active_mask]))
|
| 642 |
+
|
| 643 |
+
result = {
|
| 644 |
+
"total_revolutions": int(round(total_revolutions)),
|
| 645 |
+
"avg_rpm": float(np.mean(rpm_values[active_mask])) if active_mask.any() else 0,
|
| 646 |
+
"peak_rpm": float(np.max(rpm_values)) if len(rpm_values) > 0 else 0,
|
| 647 |
+
"active_minutes": round(active_minutes, 1),
|
| 648 |
+
"speed_tag_used": speed_tag,
|
| 649 |
+
"note": "Revolutions = RPM × time. For simplex: strokes = revolutions. For triplex: strokes = 3 × revolutions.",
|
| 650 |
+
}
|
| 651 |
+
|
| 652 |
+
return OperationOutput(
|
| 653 |
+
op_name="count_pump_strokes",
|
| 654 |
+
label=f"Pump strokes ({speed_tag})",
|
| 655 |
+
data=result,
|
| 656 |
+
metadata={"total_revolutions": result["total_revolutions"]},
|
| 657 |
+
)
|
analysis/pump_cycle_detector.py
ADDED
|
@@ -0,0 +1,319 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Testing Cycle Detection Module
|
| 3 |
+
Detects testing operation cycles from motor speed time series data,
|
| 4 |
+
qualifies them by pressure/flow activity, and ranks by significance.
|
| 5 |
+
"""
|
| 6 |
+
import pandas as pd
|
| 7 |
+
import numpy as np
|
| 8 |
+
from typing import List, Dict, Optional
|
| 9 |
+
from datetime import datetime
|
| 10 |
+
from core import config
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
class PumpCycleDetector:
|
| 14 |
+
"""Detect and qualify testing cycles using motor speed state machine + multi-signal qualification.
|
| 15 |
+
|
| 16 |
+
Two-stage pipeline:
|
| 17 |
+
1. Candidate generation: motor speed thresholding finds all motor-on periods
|
| 18 |
+
2. Qualification: filters out brief jogs/false starts using pressure and flow data
|
| 19 |
+
"""
|
| 20 |
+
|
| 21 |
+
def __init__(self):
|
| 22 |
+
params = config.CYCLE_DETECTION
|
| 23 |
+
self.idle_threshold = params['idle_threshold']
|
| 24 |
+
self.min_cycle_seconds = params['min_cycle_seconds']
|
| 25 |
+
self.smoothing_window = params['smoothing_window']
|
| 26 |
+
self.max_gap_seconds = params['max_gap_seconds']
|
| 27 |
+
# Qualification thresholds
|
| 28 |
+
self.min_peak_pressure = params.get('min_peak_pressure_bar', 50.0)
|
| 29 |
+
self.min_pressure_rise = params.get('min_pressure_rise_bar', 10.0)
|
| 30 |
+
self.min_avg_flow = params.get('min_avg_flow_kg_min', 0.05)
|
| 31 |
+
self.max_cycle_duration_min = params.get('max_cycle_duration_hours', 8.0) * 60
|
| 32 |
+
|
| 33 |
+
def detect_cycles(self, motor_speed_df: pd.DataFrame) -> List[Dict]:
|
| 34 |
+
"""
|
| 35 |
+
Detect testing cycles from motor speed time series.
|
| 36 |
+
|
| 37 |
+
Args:
|
| 38 |
+
motor_speed_df: DataFrame with columns [timestamp, value]
|
| 39 |
+
where value is motor speed in RPM
|
| 40 |
+
|
| 41 |
+
Returns:
|
| 42 |
+
List of cycle dicts with keys:
|
| 43 |
+
cycle_id, start_time, end_time, duration_minutes, peak_speed
|
| 44 |
+
"""
|
| 45 |
+
if motor_speed_df.empty or len(motor_speed_df) < 10:
|
| 46 |
+
return []
|
| 47 |
+
|
| 48 |
+
df = motor_speed_df.sort_values('timestamp').copy()
|
| 49 |
+
df['speed_smooth'] = self._smooth_signal(df['value'])
|
| 50 |
+
df['is_running'] = df['speed_smooth'] > self.idle_threshold
|
| 51 |
+
|
| 52 |
+
# Find transitions
|
| 53 |
+
df['state_change'] = df['is_running'].astype(int).diff().fillna(0)
|
| 54 |
+
|
| 55 |
+
# Start events: state_change == 1 (idle -> running)
|
| 56 |
+
starts = df[df['state_change'] == 1]['timestamp'].tolist()
|
| 57 |
+
# Stop events: state_change == -1 (running -> idle)
|
| 58 |
+
stops = df[df['state_change'] == -1]['timestamp'].tolist()
|
| 59 |
+
|
| 60 |
+
if not starts and not stops:
|
| 61 |
+
# Check if entire period is running
|
| 62 |
+
if df['is_running'].any():
|
| 63 |
+
return [{
|
| 64 |
+
'cycle_id': 1,
|
| 65 |
+
'start_time': df['timestamp'].iloc[0],
|
| 66 |
+
'end_time': df['timestamp'].iloc[-1],
|
| 67 |
+
'duration_minutes': (df['timestamp'].iloc[-1] - df['timestamp'].iloc[0]).total_seconds() / 60,
|
| 68 |
+
'peak_speed': df['value'].max(),
|
| 69 |
+
}]
|
| 70 |
+
return []
|
| 71 |
+
|
| 72 |
+
# Pair starts and stops into cycles
|
| 73 |
+
cycles = self._pair_transitions(starts, stops, df)
|
| 74 |
+
|
| 75 |
+
# Filter by minimum duration
|
| 76 |
+
cycles = [c for c in cycles if c['duration_seconds'] >= self.min_cycle_seconds]
|
| 77 |
+
|
| 78 |
+
# Merge cycles with short gaps between them
|
| 79 |
+
cycles = self._merge_short_gaps(cycles)
|
| 80 |
+
|
| 81 |
+
# Compute peak speed for each cycle
|
| 82 |
+
for c in cycles:
|
| 83 |
+
mask = (df['timestamp'] >= c['start_time']) & (df['timestamp'] <= c['end_time'])
|
| 84 |
+
cycle_data = df[mask]
|
| 85 |
+
c['peak_speed'] = cycle_data['value'].max() if not cycle_data.empty else 0
|
| 86 |
+
|
| 87 |
+
# Assign sequential IDs
|
| 88 |
+
for i, c in enumerate(cycles, 1):
|
| 89 |
+
c['cycle_id'] = i
|
| 90 |
+
c['duration_minutes'] = c['duration_seconds'] / 60
|
| 91 |
+
del c['duration_seconds']
|
| 92 |
+
|
| 93 |
+
return cycles
|
| 94 |
+
|
| 95 |
+
def _smooth_signal(self, series: pd.Series) -> pd.Series:
|
| 96 |
+
"""Apply rolling median filter to reduce noise"""
|
| 97 |
+
return series.rolling(window=self.smoothing_window, center=True, min_periods=1).median()
|
| 98 |
+
|
| 99 |
+
def _pair_transitions(
|
| 100 |
+
self,
|
| 101 |
+
starts: List[datetime],
|
| 102 |
+
stops: List[datetime],
|
| 103 |
+
df: pd.DataFrame,
|
| 104 |
+
) -> List[Dict]:
|
| 105 |
+
"""Pair start/stop transitions into cycles"""
|
| 106 |
+
cycles = []
|
| 107 |
+
|
| 108 |
+
# If first data point is already running, prepend a synthetic start
|
| 109 |
+
if df['is_running'].iloc[0]:
|
| 110 |
+
starts = [df['timestamp'].iloc[0]] + starts
|
| 111 |
+
|
| 112 |
+
# If last data point is still running, append a synthetic stop
|
| 113 |
+
if df['is_running'].iloc[-1]:
|
| 114 |
+
stops = stops + [df['timestamp'].iloc[-1]]
|
| 115 |
+
|
| 116 |
+
# Match each start with the next stop
|
| 117 |
+
stop_idx = 0
|
| 118 |
+
for start in starts:
|
| 119 |
+
# Find the next stop after this start
|
| 120 |
+
while stop_idx < len(stops) and stops[stop_idx] <= start:
|
| 121 |
+
stop_idx += 1
|
| 122 |
+
|
| 123 |
+
if stop_idx < len(stops):
|
| 124 |
+
end = stops[stop_idx]
|
| 125 |
+
duration = (end - start).total_seconds()
|
| 126 |
+
cycles.append({
|
| 127 |
+
'start_time': start,
|
| 128 |
+
'end_time': end,
|
| 129 |
+
'duration_seconds': duration,
|
| 130 |
+
})
|
| 131 |
+
stop_idx += 1
|
| 132 |
+
|
| 133 |
+
return cycles
|
| 134 |
+
|
| 135 |
+
def _merge_short_gaps(self, cycles: List[Dict]) -> List[Dict]:
|
| 136 |
+
"""Merge cycles separated by brief motor stops"""
|
| 137 |
+
if len(cycles) <= 1:
|
| 138 |
+
return cycles
|
| 139 |
+
|
| 140 |
+
merged = [cycles[0].copy()]
|
| 141 |
+
for c in cycles[1:]:
|
| 142 |
+
gap = (c['start_time'] - merged[-1]['end_time']).total_seconds()
|
| 143 |
+
if gap <= self.max_gap_seconds:
|
| 144 |
+
# Merge: extend the previous cycle
|
| 145 |
+
merged[-1]['end_time'] = c['end_time']
|
| 146 |
+
merged[-1]['duration_seconds'] = (
|
| 147 |
+
merged[-1]['end_time'] - merged[-1]['start_time']
|
| 148 |
+
).total_seconds()
|
| 149 |
+
else:
|
| 150 |
+
merged.append(c.copy())
|
| 151 |
+
|
| 152 |
+
return merged
|
| 153 |
+
|
| 154 |
+
def _qualify_cycle(self, cycle: Dict) -> bool:
|
| 155 |
+
"""Check if a cycle is a meaningful testing event.
|
| 156 |
+
|
| 157 |
+
A cycle must show evidence of actual compression (pressure) or flow
|
| 158 |
+
to qualify. Motor speed and duration alone are insufficient — they
|
| 159 |
+
only confirm the motor ran, not that meaningful work happened.
|
| 160 |
+
|
| 161 |
+
Hard rejects cycles exceeding max_cycle_duration_min (noise spans).
|
| 162 |
+
"""
|
| 163 |
+
duration_min = cycle.get('duration_minutes', 0) or 0
|
| 164 |
+
|
| 165 |
+
# Hard reject: multi-day noise spans masquerading as cycles
|
| 166 |
+
if duration_min > self.max_cycle_duration_min:
|
| 167 |
+
return False
|
| 168 |
+
|
| 169 |
+
peak_pressure = cycle.get('peak_pressure', 0) or 0
|
| 170 |
+
initial_pressure = cycle.get('initial_pressure', 0) or 0
|
| 171 |
+
avg_flow = cycle.get('avg_flow', 0) or 0
|
| 172 |
+
|
| 173 |
+
# Criterion 1: Pressure built beyond ambient
|
| 174 |
+
if peak_pressure > self.min_peak_pressure:
|
| 175 |
+
return True
|
| 176 |
+
|
| 177 |
+
# Criterion 2: Meaningful pressure rise from initial value
|
| 178 |
+
if initial_pressure > 0 and (peak_pressure - initial_pressure) > self.min_pressure_rise:
|
| 179 |
+
return True
|
| 180 |
+
|
| 181 |
+
# Criterion 3: Flow was delivered
|
| 182 |
+
if avg_flow > self.min_avg_flow:
|
| 183 |
+
return True
|
| 184 |
+
|
| 185 |
+
return False
|
| 186 |
+
|
| 187 |
+
def _sort_and_renumber(self, cycles: List[Dict]) -> List[Dict]:
|
| 188 |
+
"""Sort cycles by peak pressure descending (most significant first),
|
| 189 |
+
then by duration as tiebreaker. Re-assign sequential cycle_ids."""
|
| 190 |
+
sorted_cycles = sorted(
|
| 191 |
+
cycles,
|
| 192 |
+
key=lambda c: (-(c.get('peak_pressure', 0) or 0), -(c.get('duration_minutes', 0) or 0)),
|
| 193 |
+
)
|
| 194 |
+
for i, c in enumerate(sorted_cycles, 1):
|
| 195 |
+
c['cycle_id'] = i
|
| 196 |
+
return sorted_cycles
|
| 197 |
+
|
| 198 |
+
def enrich_cycle_metadata(
|
| 199 |
+
self,
|
| 200 |
+
cycle: Dict,
|
| 201 |
+
db_connector,
|
| 202 |
+
tags: List[str] = None,
|
| 203 |
+
) -> Dict:
|
| 204 |
+
"""
|
| 205 |
+
Add peak pressure, peak temp, avg flow to a cycle's metadata.
|
| 206 |
+
|
| 207 |
+
Args:
|
| 208 |
+
cycle: cycle dict with start_time, end_time
|
| 209 |
+
db_connector: DatabaseConnector instance
|
| 210 |
+
tags: list of tags to query (defaults to CYCLE_DETAIL_TAGS)
|
| 211 |
+
|
| 212 |
+
Returns:
|
| 213 |
+
Enriched cycle dict with additional metrics
|
| 214 |
+
"""
|
| 215 |
+
if tags is None:
|
| 216 |
+
tags = config.CYCLE_DETAIL_TAGS
|
| 217 |
+
|
| 218 |
+
df = db_connector.get_sensor_data(
|
| 219 |
+
tags, cycle['start_time'], cycle['end_time'],
|
| 220 |
+
)
|
| 221 |
+
|
| 222 |
+
if df.empty:
|
| 223 |
+
return cycle
|
| 224 |
+
|
| 225 |
+
enriched = cycle.copy()
|
| 226 |
+
|
| 227 |
+
# Peak discharge pressure
|
| 228 |
+
pt130 = df[df['tag_name'] == 'PT130']
|
| 229 |
+
if not pt130.empty:
|
| 230 |
+
enriched['peak_pressure'] = pt130['value'].max()
|
| 231 |
+
|
| 232 |
+
# Peak temperatures
|
| 233 |
+
for tag in ['TT110', 'TT130']:
|
| 234 |
+
data = df[df['tag_name'] == tag]
|
| 235 |
+
if not data.empty:
|
| 236 |
+
enriched[f'peak_{tag}'] = data['value'].max()
|
| 237 |
+
|
| 238 |
+
# Average flow
|
| 239 |
+
ft140 = df[df['tag_name'] == 'FT140']
|
| 240 |
+
if not ft140.empty:
|
| 241 |
+
enriched['avg_flow'] = ft140['value'].mean()
|
| 242 |
+
|
| 243 |
+
return enriched
|
| 244 |
+
|
| 245 |
+
def batch_enrich_cycles(
|
| 246 |
+
self,
|
| 247 |
+
cycles: List[Dict],
|
| 248 |
+
db_connector,
|
| 249 |
+
tags: List[str] = None,
|
| 250 |
+
qualify: bool = True,
|
| 251 |
+
) -> List[Dict]:
|
| 252 |
+
"""
|
| 253 |
+
Enrich ALL cycles with metadata using a single DB query,
|
| 254 |
+
then optionally qualify and rank them.
|
| 255 |
+
|
| 256 |
+
Args:
|
| 257 |
+
cycles: list of cycle dicts with start_time, end_time
|
| 258 |
+
db_connector: DatabaseConnector instance
|
| 259 |
+
tags: list of tags to query (defaults to key enrichment tags)
|
| 260 |
+
qualify: if True, filter out non-meaningful cycles and sort by significance
|
| 261 |
+
|
| 262 |
+
Returns:
|
| 263 |
+
List of enriched (and optionally qualified + sorted) cycle dicts
|
| 264 |
+
"""
|
| 265 |
+
if not cycles:
|
| 266 |
+
return cycles
|
| 267 |
+
|
| 268 |
+
if tags is None:
|
| 269 |
+
tags = ['PT130', 'TT110', 'TT130', 'FT140']
|
| 270 |
+
|
| 271 |
+
# Single query covering the full time span of all cycles
|
| 272 |
+
earliest = min(c['start_time'] for c in cycles)
|
| 273 |
+
latest = max(c['end_time'] for c in cycles)
|
| 274 |
+
|
| 275 |
+
df = db_connector.get_sensor_data(
|
| 276 |
+
tags, earliest, latest,
|
| 277 |
+
table_override='procdatafloattable_utc_15sec',
|
| 278 |
+
)
|
| 279 |
+
|
| 280 |
+
if df.empty:
|
| 281 |
+
return cycles
|
| 282 |
+
|
| 283 |
+
enriched = []
|
| 284 |
+
for c in cycles:
|
| 285 |
+
ec = c.copy()
|
| 286 |
+
mask = (df['timestamp'] >= c['start_time']) & (df['timestamp'] <= c['end_time'])
|
| 287 |
+
cycle_df = df[mask]
|
| 288 |
+
|
| 289 |
+
if cycle_df.empty:
|
| 290 |
+
enriched.append(ec)
|
| 291 |
+
continue
|
| 292 |
+
|
| 293 |
+
# Peak discharge pressure
|
| 294 |
+
pt130 = cycle_df[cycle_df['tag_name'] == 'PT130']
|
| 295 |
+
if not pt130.empty:
|
| 296 |
+
pt130_sorted = pt130.sort_values('timestamp')
|
| 297 |
+
ec['peak_pressure'] = pt130_sorted['value'].max()
|
| 298 |
+
ec['initial_pressure'] = pt130_sorted['value'].iloc[0]
|
| 299 |
+
|
| 300 |
+
# Peak temperatures
|
| 301 |
+
for tag in ['TT110', 'TT130']:
|
| 302 |
+
data = cycle_df[cycle_df['tag_name'] == tag]
|
| 303 |
+
if not data.empty:
|
| 304 |
+
ec[f'peak_{tag}'] = data['value'].max()
|
| 305 |
+
|
| 306 |
+
# Average flow
|
| 307 |
+
ft140 = cycle_df[cycle_df['tag_name'] == 'FT140']
|
| 308 |
+
if not ft140.empty:
|
| 309 |
+
ec['avg_flow'] = ft140['value'].mean()
|
| 310 |
+
|
| 311 |
+
enriched.append(ec)
|
| 312 |
+
|
| 313 |
+
# Qualification: filter out non-meaningful cycles
|
| 314 |
+
if qualify:
|
| 315 |
+
qualified = [c for c in enriched if self._qualify_cycle(c)]
|
| 316 |
+
# Sort by significance and re-number
|
| 317 |
+
return self._sort_and_renumber(qualified) if qualified else []
|
| 318 |
+
|
| 319 |
+
return enriched
|
analysis/query_planner.py
ADDED
|
@@ -0,0 +1,459 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
LLM-Powered Query Planner for Advanced Data Explorer.
|
| 3 |
+
|
| 4 |
+
Classifies user queries (FETCH / ANALYZE / EXPORT), decomposes complex ones
|
| 5 |
+
into structured ExecutionPlan JSON referencing the whitelisted operation registry.
|
| 6 |
+
"""
|
| 7 |
+
import json
|
| 8 |
+
from typing import Dict, List, Optional
|
| 9 |
+
from datetime import datetime
|
| 10 |
+
from core import config
|
| 11 |
+
from analysis.operations import get_registry_for_prompt
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
SYSTEM_PROMPT = """You are a query planner for a cryogenic hydrogen pump testing data system (CSH2 MURPHY). Your job is to decompose an engineer's natural language query into a structured execution plan.
|
| 15 |
+
|
| 16 |
+
You classify queries into three types:
|
| 17 |
+
- FETCH: Simple data retrieval (sensors + time range, no post-processing). Use for "show me", "plot", "display" type queries.
|
| 18 |
+
- ANALYZE: Data retrieval + one or more analysis operations. Use for "find", "identify", "detect", "count", "calculate" queries.
|
| 19 |
+
- EXPORT: Data retrieval + transformation + CSV export. Use for "export", "download", "save" queries with transformations.
|
| 20 |
+
|
| 21 |
+
CRITICAL RULES:
|
| 22 |
+
1. You MUST output valid JSON matching the ExecutionPlan schema below. No other text.
|
| 23 |
+
2. You MUST only use operations from the AVAILABLE_OPERATIONS list.
|
| 24 |
+
3. You MUST NOT generate arbitrary SQL, Python code, or operations not in the registry.
|
| 25 |
+
4. For FETCH queries, set operations to an empty array [].
|
| 26 |
+
5. Always include the correct sensors needed for each operation in data_requirements.sensors.
|
| 27 |
+
6. For fill/cycle analysis, always include motor speed tags (M130_Speed, MC130_VFD_Speed), PT130, FT140, and AvgPower.
|
| 28 |
+
7. Choose resolution wisely:
|
| 29 |
+
- "raw": Only for sub-second analysis or frequency reduction from 10Hz. MAX 1 HOUR time range.
|
| 30 |
+
- "1sec": For detailed analysis within a single day.
|
| 31 |
+
- "15sec": For multi-day or full-range analysis.
|
| 32 |
+
- "auto": Let the system decide based on time range duration.
|
| 33 |
+
|
| 34 |
+
ExecutionPlan JSON schema:
|
| 35 |
+
{
|
| 36 |
+
"query_type": "FETCH | ANALYZE | EXPORT",
|
| 37 |
+
"explanation": "Human-readable summary of what will happen",
|
| 38 |
+
"data_requirements": {
|
| 39 |
+
"sensors": ["PT130", "TT110"],
|
| 40 |
+
"start_time": "YYYY-MM-DDTHH:MM:SS",
|
| 41 |
+
"end_time": "YYYY-MM-DDTHH:MM:SS",
|
| 42 |
+
"resolution": "raw | 1sec | 15sec | auto"
|
| 43 |
+
},
|
| 44 |
+
"operations": [
|
| 45 |
+
{
|
| 46 |
+
"op": "operation_name_from_registry",
|
| 47 |
+
"params": { ... },
|
| 48 |
+
"label": "Human-readable step description"
|
| 49 |
+
}
|
| 50 |
+
],
|
| 51 |
+
"export": {
|
| 52 |
+
"enabled": true,
|
| 53 |
+
"filename_hint": "descriptive_name"
|
| 54 |
+
}
|
| 55 |
+
}
|
| 56 |
+
"""
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
EXAMPLES = """
|
| 60 |
+
WORKED EXAMPLES:
|
| 61 |
+
|
| 62 |
+
Example 1 — Query: "Find time blocks where PT130 exceeded 900 bar"
|
| 63 |
+
{
|
| 64 |
+
"query_type": "ANALYZE",
|
| 65 |
+
"explanation": "Find contiguous time blocks where discharge pressure PT130 exceeded 900 bar across the full data range",
|
| 66 |
+
"data_requirements": {
|
| 67 |
+
"sensors": ["PT130"],
|
| 68 |
+
"start_time": "2025-03-14T00:00:00",
|
| 69 |
+
"end_time": "2025-09-25T23:59:59",
|
| 70 |
+
"resolution": "15sec"
|
| 71 |
+
},
|
| 72 |
+
"operations": [
|
| 73 |
+
{
|
| 74 |
+
"op": "find_peak_pressure_blocks",
|
| 75 |
+
"params": {"pressure_tag": "PT130", "threshold_bar": 900.0},
|
| 76 |
+
"label": "Find time blocks where PT130 > 900 bar"
|
| 77 |
+
}
|
| 78 |
+
],
|
| 79 |
+
"export": {"enabled": true, "filename_hint": "PT130_above_900bar"}
|
| 80 |
+
}
|
| 81 |
+
|
| 82 |
+
Example 2 — Query: "Find time blocks where both AOV140 and MC130 were constant for 5 minutes"
|
| 83 |
+
{
|
| 84 |
+
"query_type": "ANALYZE",
|
| 85 |
+
"explanation": "Find periods where both AOV140 (valve) and MC130_VFD_Speed (motor) were simultaneously constant for at least 5 minutes",
|
| 86 |
+
"data_requirements": {
|
| 87 |
+
"sensors": ["AOV140", "MC130_VFD_Speed"],
|
| 88 |
+
"start_time": "2025-03-14T00:00:00",
|
| 89 |
+
"end_time": "2025-09-25T23:59:59",
|
| 90 |
+
"resolution": "15sec"
|
| 91 |
+
},
|
| 92 |
+
"operations": [
|
| 93 |
+
{
|
| 94 |
+
"op": "detect_constant_periods_multi",
|
| 95 |
+
"params": {"tags": ["AOV140", "MC130_VFD_Speed"], "tolerance": 0.01, "min_duration_minutes": 5.0},
|
| 96 |
+
"label": "Find overlapping constant periods for AOV140 and MC130_VFD_Speed"
|
| 97 |
+
}
|
| 98 |
+
],
|
| 99 |
+
"export": {"enabled": true, "filename_hint": "constant_AOV140_MC130"}
|
| 100 |
+
}
|
| 101 |
+
|
| 102 |
+
Example 3 — Query: "Export PT130, TT110, FT140 at 2Hz as CSV for Sep 25 10am-11am"
|
| 103 |
+
{
|
| 104 |
+
"query_type": "EXPORT",
|
| 105 |
+
"explanation": "Export PT130, TT110, FT140 at 2Hz (downsampled from 10Hz raw data) for Sep 25 10-11am",
|
| 106 |
+
"data_requirements": {
|
| 107 |
+
"sensors": ["PT130", "TT110", "FT140"],
|
| 108 |
+
"start_time": "2025-09-25T10:00:00",
|
| 109 |
+
"end_time": "2025-09-25T11:00:00",
|
| 110 |
+
"resolution": "raw"
|
| 111 |
+
},
|
| 112 |
+
"operations": [
|
| 113 |
+
{
|
| 114 |
+
"op": "downsample_frequency",
|
| 115 |
+
"params": {"target_hz": 2.0},
|
| 116 |
+
"label": "Downsample from 10Hz to 2Hz"
|
| 117 |
+
}
|
| 118 |
+
],
|
| 119 |
+
"export": {"enabled": true, "filename_hint": "PT130_TT110_FT140_2Hz"}
|
| 120 |
+
}
|
| 121 |
+
|
| 122 |
+
Example 4 — Query: "Export the 3 testing windows May 5-6 where PT130 ramped from 100 to 800 bar"
|
| 123 |
+
{
|
| 124 |
+
"query_type": "EXPORT",
|
| 125 |
+
"explanation": "Find 3 testing windows in May 5-6 where PT130 ramped from 100 to 800 bar, export each window's data",
|
| 126 |
+
"data_requirements": {
|
| 127 |
+
"sensors": ["PT130", "TT110", "TT130", "FT140", "M130_Speed", "AOV140"],
|
| 128 |
+
"start_time": "2025-05-05T00:00:00",
|
| 129 |
+
"end_time": "2025-05-06T23:59:59",
|
| 130 |
+
"resolution": "1sec"
|
| 131 |
+
},
|
| 132 |
+
"operations": [
|
| 133 |
+
{
|
| 134 |
+
"op": "detect_pressure_ramps",
|
| 135 |
+
"params": {"pressure_tag": "PT130", "start_bar": 100.0, "end_bar": 800.0, "max_ramps": 3},
|
| 136 |
+
"label": "Detect pressure ramps from 100 to 800 bar"
|
| 137 |
+
},
|
| 138 |
+
{
|
| 139 |
+
"op": "extract_windows",
|
| 140 |
+
"params": {"source": "previous_result"},
|
| 141 |
+
"label": "Extract data for each ramp window"
|
| 142 |
+
}
|
| 143 |
+
],
|
| 144 |
+
"export": {"enabled": true, "filename_hint": "LN2_ramps_100_800bar"}
|
| 145 |
+
}
|
| 146 |
+
|
| 147 |
+
Example 5 — Query: "Identify fills where kWh per kg exceeded 0.6"
|
| 148 |
+
{
|
| 149 |
+
"query_type": "ANALYZE",
|
| 150 |
+
"explanation": "Detect all testing fills, compute kWh/kg for each, return those exceeding 0.6 kWh/kg",
|
| 151 |
+
"data_requirements": {
|
| 152 |
+
"sensors": ["M130_Speed", "MC130_VFD_Speed", "PT130", "FT140", "AvgPower", "TT110", "TT130"],
|
| 153 |
+
"start_time": "2025-03-14T00:00:00",
|
| 154 |
+
"end_time": "2025-09-25T23:59:59",
|
| 155 |
+
"resolution": "15sec"
|
| 156 |
+
},
|
| 157 |
+
"operations": [
|
| 158 |
+
{
|
| 159 |
+
"op": "detect_and_analyze_fills",
|
| 160 |
+
"params": {"kwh_per_kg_threshold": 0.6, "include_strokes": false},
|
| 161 |
+
"label": "Detect fills and filter by kWh/kg > 0.6"
|
| 162 |
+
}
|
| 163 |
+
],
|
| 164 |
+
"export": {"enabled": true, "filename_hint": "fills_above_0.6_kwhkg"}
|
| 165 |
+
}
|
| 166 |
+
|
| 167 |
+
Example 6 — Query: "In Fill 3, what was the total number of pump strokes?"
|
| 168 |
+
{
|
| 169 |
+
"query_type": "ANALYZE",
|
| 170 |
+
"explanation": "Detect fills, find Fill 3, compute its total pump strokes from motor RPM",
|
| 171 |
+
"data_requirements": {
|
| 172 |
+
"sensors": ["M130_Speed", "MC130_VFD_Speed", "PT130", "FT140", "AvgPower"],
|
| 173 |
+
"start_time": "2025-03-14T00:00:00",
|
| 174 |
+
"end_time": "2025-09-25T23:59:59",
|
| 175 |
+
"resolution": "15sec"
|
| 176 |
+
},
|
| 177 |
+
"operations": [
|
| 178 |
+
{
|
| 179 |
+
"op": "detect_and_analyze_fills",
|
| 180 |
+
"params": {"fill_id": 3, "include_strokes": true},
|
| 181 |
+
"label": "Find Fill 3 and compute pump strokes"
|
| 182 |
+
}
|
| 183 |
+
],
|
| 184 |
+
"export": {"enabled": false}
|
| 185 |
+
}
|
| 186 |
+
|
| 187 |
+
Example 7 — Query: "Between Sep 24 15:00 and 18:00, total pump strokes?"
|
| 188 |
+
{
|
| 189 |
+
"query_type": "ANALYZE",
|
| 190 |
+
"explanation": "Count total pump revolutions between Sep 24 15:00 and 18:00 UTC",
|
| 191 |
+
"data_requirements": {
|
| 192 |
+
"sensors": ["M130_Speed", "MC130_VFD_Speed"],
|
| 193 |
+
"start_time": "2025-09-24T15:00:00",
|
| 194 |
+
"end_time": "2025-09-24T18:00:00",
|
| 195 |
+
"resolution": "1sec"
|
| 196 |
+
},
|
| 197 |
+
"operations": [
|
| 198 |
+
{
|
| 199 |
+
"op": "count_pump_strokes",
|
| 200 |
+
"params": {},
|
| 201 |
+
"label": "Count total pump revolutions in 3-hour window"
|
| 202 |
+
}
|
| 203 |
+
],
|
| 204 |
+
"export": {"enabled": false}
|
| 205 |
+
}
|
| 206 |
+
"""
|
| 207 |
+
|
| 208 |
+
|
| 209 |
+
class QueryPlanner:
|
| 210 |
+
"""LLM-powered query classification and decomposition into ExecutionPlan JSON."""
|
| 211 |
+
|
| 212 |
+
def __init__(self):
|
| 213 |
+
self.api_key = config.ANTHROPIC_API_KEY
|
| 214 |
+
self.api_available = bool(self.api_key and self.api_key != 'your_api_key_here')
|
| 215 |
+
self.model = "claude-sonnet-4-20250514"
|
| 216 |
+
self._client = None
|
| 217 |
+
|
| 218 |
+
@property
|
| 219 |
+
def client(self):
|
| 220 |
+
if self._client is None and self.api_available:
|
| 221 |
+
import anthropic
|
| 222 |
+
self._client = anthropic.Anthropic(api_key=self.api_key)
|
| 223 |
+
return self._client
|
| 224 |
+
|
| 225 |
+
def plan(self, user_query: str, available_sensors: List[str]) -> Optional[Dict]:
|
| 226 |
+
"""
|
| 227 |
+
Decompose a natural language query into a structured execution plan.
|
| 228 |
+
|
| 229 |
+
Uses Anthropic prompt caching: the system prompt and static context
|
| 230 |
+
(sensor catalog, operation registry, worked examples) are cached for
|
| 231 |
+
5 minutes, so repeated queries only pay for the dynamic user query.
|
| 232 |
+
|
| 233 |
+
Returns:
|
| 234 |
+
Dict (ExecutionPlan) or dict with 'error' key on failure.
|
| 235 |
+
"""
|
| 236 |
+
if not self.api_available:
|
| 237 |
+
return {"error": "Claude API key not configured."}
|
| 238 |
+
|
| 239 |
+
static_context = self._build_static_context(available_sensors)
|
| 240 |
+
dynamic_query = f'User query: "{user_query}"\n\nReturn ONLY the JSON execution plan. No other text.'
|
| 241 |
+
|
| 242 |
+
try:
|
| 243 |
+
message = self.client.messages.create(
|
| 244 |
+
model=self.model,
|
| 245 |
+
max_tokens=2000,
|
| 246 |
+
system=[
|
| 247 |
+
{
|
| 248 |
+
"type": "text",
|
| 249 |
+
"text": SYSTEM_PROMPT,
|
| 250 |
+
"cache_control": {"type": "ephemeral"},
|
| 251 |
+
}
|
| 252 |
+
],
|
| 253 |
+
messages=[
|
| 254 |
+
{
|
| 255 |
+
"role": "user",
|
| 256 |
+
"content": [
|
| 257 |
+
{
|
| 258 |
+
"type": "text",
|
| 259 |
+
"text": static_context,
|
| 260 |
+
"cache_control": {"type": "ephemeral"},
|
| 261 |
+
},
|
| 262 |
+
{
|
| 263 |
+
"type": "text",
|
| 264 |
+
"text": dynamic_query,
|
| 265 |
+
},
|
| 266 |
+
],
|
| 267 |
+
}
|
| 268 |
+
],
|
| 269 |
+
)
|
| 270 |
+
response_text = message.content[0].text
|
| 271 |
+
plan = self._extract_json(response_text)
|
| 272 |
+
|
| 273 |
+
if plan:
|
| 274 |
+
validation = self._validate_plan(plan, available_sensors)
|
| 275 |
+
if validation:
|
| 276 |
+
return {"error": validation, "raw_plan": plan}
|
| 277 |
+
return self._process_plan(plan, available_sensors)
|
| 278 |
+
|
| 279 |
+
# Fallback: try V1 parser for simple queries
|
| 280 |
+
return self._fallback_parse(user_query, available_sensors)
|
| 281 |
+
|
| 282 |
+
except Exception as e:
|
| 283 |
+
return {"error": f"Planning failed: {e}"}
|
| 284 |
+
|
| 285 |
+
def _build_static_context(self, available_sensors: List[str]) -> str:
|
| 286 |
+
"""Build the cacheable static context block.
|
| 287 |
+
|
| 288 |
+
Contains sensor catalog, groups, aliases, operation registry, and
|
| 289 |
+
worked examples. This is ~25-30KB of stable text that changes only
|
| 290 |
+
when sensors or operations are added — ideal for Anthropic's prompt
|
| 291 |
+
caching (5-minute TTL, 90% cost reduction on cache hit).
|
| 292 |
+
"""
|
| 293 |
+
# Sensor reference with descriptions
|
| 294 |
+
sensor_lines = []
|
| 295 |
+
for tag_name, info in config.SENSOR_TAGS.items():
|
| 296 |
+
desc = info.get("description", "")
|
| 297 |
+
units = info.get("units", "")
|
| 298 |
+
sensor_lines.append(f" {tag_name}: {desc} ({units})")
|
| 299 |
+
sensor_reference = "\n".join(sensor_lines)
|
| 300 |
+
|
| 301 |
+
# Sensor group reference
|
| 302 |
+
group_lines = []
|
| 303 |
+
for group_name, tags in config.SENSOR_GROUPS.items():
|
| 304 |
+
group_lines.append(f" {group_name}: {', '.join(tags)}")
|
| 305 |
+
group_reference = "\n".join(group_lines)
|
| 306 |
+
|
| 307 |
+
# Operation registry
|
| 308 |
+
registry_text = get_registry_for_prompt()
|
| 309 |
+
|
| 310 |
+
current_date = datetime.now()
|
| 311 |
+
|
| 312 |
+
return f"""Current date/time: {current_date.strftime('%Y-%m-%d %H:%M:%S')}
|
| 313 |
+
Data is available from March 14, 2025 to September 25, 2025.
|
| 314 |
+
|
| 315 |
+
AVAILABLE SENSORS (tag name: description (units)):
|
| 316 |
+
{sensor_reference}
|
| 317 |
+
|
| 318 |
+
SENSOR GROUPS:
|
| 319 |
+
{group_reference}
|
| 320 |
+
|
| 321 |
+
COMMON ALIASES:
|
| 322 |
+
- "pressure" or "pressures" -> PT130 (Discharge), PT01T (Cryotank), PT110 (Inlet)
|
| 323 |
+
- "temperature" or "temperatures" -> TT110 (Pump Feed Line), TT130 (Discharge)
|
| 324 |
+
- "flow" -> FT140 (Flow Transmitter)
|
| 325 |
+
- "motor" or "speed" or "rpm" -> M130_Speed (Motor Speed)
|
| 326 |
+
- "power" -> AvgPower, PeakPower, PowerConsumption
|
| 327 |
+
- "discharge pressure" -> PT130
|
| 328 |
+
- "inlet pressure" or "supply pressure" -> PT110
|
| 329 |
+
- "tank pressure" or "cryotank pressure" -> PT01T
|
| 330 |
+
- "discharge temperature" -> TT130
|
| 331 |
+
- "feed temperature" or "pump temperature" -> TT110
|
| 332 |
+
- "valve" -> AOV140
|
| 333 |
+
- "fill" or "fills" or "test cycle" -> requires detect_and_analyze_fills operation
|
| 334 |
+
- "strokes" or "pump strokes" or "revolutions" -> requires count_pump_strokes or detect_and_analyze_fills
|
| 335 |
+
- "efficiency" or "kWh/kg" or "specific energy" -> requires detect_and_analyze_fills with AvgPower + FT140
|
| 336 |
+
|
| 337 |
+
{registry_text}
|
| 338 |
+
|
| 339 |
+
{EXAMPLES}"""
|
| 340 |
+
|
| 341 |
+
def _build_prompt(self, user_query: str, available_sensors: List[str]) -> str:
|
| 342 |
+
"""Build the full user prompt (used by fallback path).
|
| 343 |
+
|
| 344 |
+
The main plan() method uses _build_static_context() + dynamic query
|
| 345 |
+
separately for prompt caching. This method is kept for backward
|
| 346 |
+
compatibility with the fallback parser.
|
| 347 |
+
"""
|
| 348 |
+
static = self._build_static_context(available_sensors)
|
| 349 |
+
return f"""{static}
|
| 350 |
+
|
| 351 |
+
User query: "{user_query}"
|
| 352 |
+
|
| 353 |
+
Return ONLY the JSON execution plan. No other text."""
|
| 354 |
+
|
| 355 |
+
def _extract_json(self, text: str) -> Optional[Dict]:
|
| 356 |
+
"""Extract JSON object from LLM response."""
|
| 357 |
+
try:
|
| 358 |
+
start = text.find("{")
|
| 359 |
+
end = text.rfind("}") + 1
|
| 360 |
+
if start != -1 and end > start:
|
| 361 |
+
json_str = text[start:end]
|
| 362 |
+
return json.loads(json_str)
|
| 363 |
+
return None
|
| 364 |
+
except json.JSONDecodeError:
|
| 365 |
+
return None
|
| 366 |
+
|
| 367 |
+
def _validate_plan(self, plan: Dict, available_sensors: List[str]) -> Optional[str]:
|
| 368 |
+
"""Validate plan structure. Returns error string or None if valid."""
|
| 369 |
+
# Check required top-level fields
|
| 370 |
+
if "query_type" not in plan:
|
| 371 |
+
return "Missing 'query_type' field"
|
| 372 |
+
if plan["query_type"] not in ("FETCH", "ANALYZE", "EXPORT"):
|
| 373 |
+
return f"Invalid query_type: {plan['query_type']}"
|
| 374 |
+
if "data_requirements" not in plan:
|
| 375 |
+
return "Missing 'data_requirements' field"
|
| 376 |
+
|
| 377 |
+
dr = plan["data_requirements"]
|
| 378 |
+
if "sensors" not in dr or not dr["sensors"]:
|
| 379 |
+
return "No sensors specified"
|
| 380 |
+
if "start_time" not in dr or "end_time" not in dr:
|
| 381 |
+
return "Missing start_time or end_time"
|
| 382 |
+
|
| 383 |
+
# Validate resolution
|
| 384 |
+
resolution = dr.get("resolution", "auto")
|
| 385 |
+
if resolution not in ("raw", "1sec", "15sec", "auto"):
|
| 386 |
+
return f"Invalid resolution: {resolution}. Use: raw, 1sec, 15sec, auto"
|
| 387 |
+
|
| 388 |
+
# Safety: raw table time range limit
|
| 389 |
+
if resolution == "raw":
|
| 390 |
+
try:
|
| 391 |
+
start = datetime.fromisoformat(dr["start_time"])
|
| 392 |
+
end = datetime.fromisoformat(dr["end_time"])
|
| 393 |
+
duration_h = (end - start).total_seconds() / 3600
|
| 394 |
+
if duration_h > 1.0:
|
| 395 |
+
return f"Raw resolution limited to 1 hour. Requested {duration_h:.1f} hours. Use '1sec' or '15sec' for longer ranges."
|
| 396 |
+
except (ValueError, TypeError):
|
| 397 |
+
pass
|
| 398 |
+
|
| 399 |
+
# Validate operations exist in registry
|
| 400 |
+
from analysis.operations import OPERATION_REGISTRY
|
| 401 |
+
for op in plan.get("operations", []):
|
| 402 |
+
if op.get("op") not in OPERATION_REGISTRY:
|
| 403 |
+
known = ", ".join(OPERATION_REGISTRY.keys())
|
| 404 |
+
return f"Unknown operation: '{op['op']}'. Available: {known}"
|
| 405 |
+
|
| 406 |
+
return None
|
| 407 |
+
|
| 408 |
+
def _process_plan(self, plan: Dict, available_sensors: List[str]) -> Dict:
|
| 409 |
+
"""Process and clean up the plan: resolve sensors, parse datetimes."""
|
| 410 |
+
dr = plan.get("data_requirements", {})
|
| 411 |
+
|
| 412 |
+
# Validate sensors against available list (case-insensitive)
|
| 413 |
+
sensor_map = {s.upper(): s for s in available_sensors}
|
| 414 |
+
valid_sensors = []
|
| 415 |
+
for s in dr.get("sensors", []):
|
| 416 |
+
matched = sensor_map.get(s.upper())
|
| 417 |
+
if matched:
|
| 418 |
+
valid_sensors.append(matched)
|
| 419 |
+
|
| 420 |
+
if not valid_sensors:
|
| 421 |
+
return {"error": f"No matching sensors found. Requested: {dr.get('sensors', [])}"}
|
| 422 |
+
|
| 423 |
+
# Parse datetimes
|
| 424 |
+
try:
|
| 425 |
+
start_time = datetime.fromisoformat(dr["start_time"])
|
| 426 |
+
end_time = datetime.fromisoformat(dr["end_time"])
|
| 427 |
+
except (ValueError, TypeError, KeyError) as e:
|
| 428 |
+
return {"error": f"Invalid time format: {e}"}
|
| 429 |
+
|
| 430 |
+
# Clean up and return
|
| 431 |
+
plan["data_requirements"]["sensors"] = valid_sensors
|
| 432 |
+
plan["data_requirements"]["start_time"] = start_time
|
| 433 |
+
plan["data_requirements"]["end_time"] = end_time
|
| 434 |
+
plan["data_requirements"]["resolution"] = dr.get("resolution", "auto")
|
| 435 |
+
|
| 436 |
+
return plan
|
| 437 |
+
|
| 438 |
+
def _fallback_parse(self, user_query: str, available_sensors: List[str]) -> Optional[Dict]:
|
| 439 |
+
"""Fallback: attempt to parse as simple FETCH query via V1 NLQueryParser."""
|
| 440 |
+
try:
|
| 441 |
+
from analysis.nl2sql import NLQueryParser
|
| 442 |
+
parser = NLQueryParser()
|
| 443 |
+
result = parser.parse(user_query, available_sensors)
|
| 444 |
+
if result and "error" not in result:
|
| 445 |
+
return {
|
| 446 |
+
"query_type": "FETCH",
|
| 447 |
+
"explanation": result.get("explanation", ""),
|
| 448 |
+
"data_requirements": {
|
| 449 |
+
"sensors": result["sensors"],
|
| 450 |
+
"start_time": result["start_time"],
|
| 451 |
+
"end_time": result["end_time"],
|
| 452 |
+
"resolution": "auto",
|
| 453 |
+
},
|
| 454 |
+
"operations": [],
|
| 455 |
+
"export": {"enabled": False},
|
| 456 |
+
}
|
| 457 |
+
return result # Pass through error
|
| 458 |
+
except Exception as e:
|
| 459 |
+
return {"error": f"Fallback parse failed: {e}"}
|
analysis/result_interpreter.py
ADDED
|
@@ -0,0 +1,205 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Post-Execution Result Interpreter — "Murphy's Take"
|
| 3 |
+
|
| 4 |
+
After the Advanced Explorer executes an analytical plan and produces raw
|
| 5 |
+
tables/metrics, this module calls the LLM to generate a plain-English
|
| 6 |
+
interpretation grounded in cryogenic physics context.
|
| 7 |
+
"""
|
| 8 |
+
import pandas as pd
|
| 9 |
+
from typing import Optional
|
| 10 |
+
from core import config
|
| 11 |
+
from analysis.llm_analyzer import MURPHY_SYSTEM_PROMPT, sanitize_llm_output
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
class ResultInterpreter:
|
| 15 |
+
"""Generate plain-English interpretation of Advanced Explorer results.
|
| 16 |
+
|
| 17 |
+
Uses the shared Murphy system prompt with Anthropic prompt caching.
|
| 18 |
+
The system prompt is cached for 5 minutes across repeated calls,
|
| 19 |
+
so interpreting multiple query results in a session is fast.
|
| 20 |
+
"""
|
| 21 |
+
|
| 22 |
+
def __init__(self):
|
| 23 |
+
self.api_key = config.ANTHROPIC_API_KEY
|
| 24 |
+
self.api_available = bool(self.api_key and self.api_key != 'your_api_key_here')
|
| 25 |
+
self.model = "claude-sonnet-4-20250514"
|
| 26 |
+
self._client = None
|
| 27 |
+
|
| 28 |
+
@property
|
| 29 |
+
def client(self):
|
| 30 |
+
if self._client is None and self.api_available:
|
| 31 |
+
import anthropic
|
| 32 |
+
self._client = anthropic.Anthropic(api_key=self.api_key)
|
| 33 |
+
return self._client
|
| 34 |
+
|
| 35 |
+
def interpret(self, user_query: str, result) -> str:
|
| 36 |
+
"""Generate Murphy's interpretation of execution results.
|
| 37 |
+
|
| 38 |
+
Args:
|
| 39 |
+
user_query: The original natural language query from the user.
|
| 40 |
+
result: An ExecutionResult from the execution engine.
|
| 41 |
+
|
| 42 |
+
Returns:
|
| 43 |
+
Plain-English interpretation string, or an error message.
|
| 44 |
+
"""
|
| 45 |
+
if not self.api_available:
|
| 46 |
+
return "LLM interpretation unavailable. Add ANTHROPIC_API_KEY to .env."
|
| 47 |
+
|
| 48 |
+
prompt = self._build_interpretation_prompt(user_query, result)
|
| 49 |
+
|
| 50 |
+
try:
|
| 51 |
+
message = self.client.messages.create(
|
| 52 |
+
model=self.model,
|
| 53 |
+
max_tokens=1500,
|
| 54 |
+
system=[
|
| 55 |
+
{
|
| 56 |
+
"type": "text",
|
| 57 |
+
"text": MURPHY_SYSTEM_PROMPT,
|
| 58 |
+
"cache_control": {"type": "ephemeral"},
|
| 59 |
+
}
|
| 60 |
+
],
|
| 61 |
+
messages=[{"role": "user", "content": prompt}],
|
| 62 |
+
)
|
| 63 |
+
return sanitize_llm_output(message.content[0].text)
|
| 64 |
+
except Exception as e:
|
| 65 |
+
return f"Interpretation failed: {e}"
|
| 66 |
+
|
| 67 |
+
def _build_interpretation_prompt(self, user_query: str, result) -> str:
|
| 68 |
+
"""Build the prompt from execution results.
|
| 69 |
+
|
| 70 |
+
Serializes operation results compactly — key metrics, first N rows,
|
| 71 |
+
summary stats — to keep token usage reasonable while giving the LLM
|
| 72 |
+
enough context for a physics-grounded interpretation.
|
| 73 |
+
"""
|
| 74 |
+
sections = []
|
| 75 |
+
|
| 76 |
+
# Header
|
| 77 |
+
sections.append(f'The engineer asked: "{user_query}"')
|
| 78 |
+
sections.append(f"Query type: {result.query_type}")
|
| 79 |
+
sections.append(f"Plan explanation: {result.explanation}")
|
| 80 |
+
|
| 81 |
+
# Data context
|
| 82 |
+
sections.append(
|
| 83 |
+
f"\nData fetched: {result.row_count:,} rows from {result.table_used or 'unknown table'}"
|
| 84 |
+
)
|
| 85 |
+
|
| 86 |
+
# Source data summary (compact)
|
| 87 |
+
if result.source_data is not None and not result.source_data.empty:
|
| 88 |
+
src = result.source_data
|
| 89 |
+
if "timestamp" in src.columns:
|
| 90 |
+
ts = src["timestamp"]
|
| 91 |
+
sections.append(
|
| 92 |
+
f"Time range: {ts.min()} to {ts.max()}"
|
| 93 |
+
)
|
| 94 |
+
numeric_cols = [
|
| 95 |
+
c for c in src.columns
|
| 96 |
+
if c != "timestamp" and pd.api.types.is_numeric_dtype(src[c])
|
| 97 |
+
]
|
| 98 |
+
if numeric_cols:
|
| 99 |
+
stats_lines = []
|
| 100 |
+
for col in numeric_cols[:8]: # Cap at 8 sensors
|
| 101 |
+
col_data = src[col].dropna()
|
| 102 |
+
if len(col_data) > 0:
|
| 103 |
+
stats_lines.append(
|
| 104 |
+
f" {col}: min={col_data.min():.3f}, "
|
| 105 |
+
f"max={col_data.max():.3f}, "
|
| 106 |
+
f"mean={col_data.mean():.3f}, "
|
| 107 |
+
f"std={col_data.std():.3f}"
|
| 108 |
+
)
|
| 109 |
+
if stats_lines:
|
| 110 |
+
sections.append("Source data summary:\n" + "\n".join(stats_lines))
|
| 111 |
+
|
| 112 |
+
# Operation results (compact serialization)
|
| 113 |
+
for i, op_result in enumerate(result.operation_results):
|
| 114 |
+
op_section = [f"\n--- Step {i + 1}: {op_result.label} ---"]
|
| 115 |
+
|
| 116 |
+
if op_result.metadata.get("error"):
|
| 117 |
+
op_section.append(f" ERROR: {op_result.metadata['error']}")
|
| 118 |
+
sections.append("\n".join(op_section))
|
| 119 |
+
continue
|
| 120 |
+
|
| 121 |
+
# Key metadata (non-error)
|
| 122 |
+
meta = {k: v for k, v in op_result.metadata.items() if k != "error"}
|
| 123 |
+
if meta:
|
| 124 |
+
for k, v in meta.items():
|
| 125 |
+
op_section.append(f" {k}: {v}")
|
| 126 |
+
|
| 127 |
+
data = op_result.data
|
| 128 |
+
|
| 129 |
+
# Dict result (pump strokes, statistics, blocks summary)
|
| 130 |
+
if isinstance(data, dict):
|
| 131 |
+
for k, v in data.items():
|
| 132 |
+
if isinstance(v, (int, float, str, bool)):
|
| 133 |
+
op_section.append(f" {k}: {v}")
|
| 134 |
+
elif isinstance(v, list) and len(v) > 0:
|
| 135 |
+
op_section.append(f" {k}: {len(v)} items")
|
| 136 |
+
# Show first few items if they're dicts
|
| 137 |
+
if isinstance(v[0], dict):
|
| 138 |
+
for item in v[:5]:
|
| 139 |
+
op_section.append(f" {item}")
|
| 140 |
+
|
| 141 |
+
# List-of-dicts result (fills, periods, ramps)
|
| 142 |
+
elif isinstance(data, list) and data and isinstance(data[0], dict):
|
| 143 |
+
op_section.append(f" Found {len(data)} results")
|
| 144 |
+
# Show first 10 rows
|
| 145 |
+
for row in data[:10]:
|
| 146 |
+
row_str = ", ".join(
|
| 147 |
+
f"{k}={v}" for k, v in row.items()
|
| 148 |
+
)
|
| 149 |
+
op_section.append(f" {row_str}")
|
| 150 |
+
if len(data) > 10:
|
| 151 |
+
op_section.append(f" ... and {len(data) - 10} more")
|
| 152 |
+
|
| 153 |
+
# List-of-DataFrames result (extract_windows)
|
| 154 |
+
elif isinstance(data, list) and data and isinstance(data[0], pd.DataFrame):
|
| 155 |
+
op_section.append(f" Extracted {len(data)} windows")
|
| 156 |
+
for j, window_df in enumerate(data[:5]):
|
| 157 |
+
op_section.append(
|
| 158 |
+
f" Window {j + 1}: {len(window_df)} rows, "
|
| 159 |
+
f"cols: {', '.join(window_df.columns[:6])}"
|
| 160 |
+
)
|
| 161 |
+
|
| 162 |
+
# Single DataFrame result
|
| 163 |
+
elif isinstance(data, pd.DataFrame):
|
| 164 |
+
op_section.append(f" DataFrame: {len(data)} rows × {len(data.columns)} cols")
|
| 165 |
+
numeric = [
|
| 166 |
+
c for c in data.columns
|
| 167 |
+
if c != "timestamp" and pd.api.types.is_numeric_dtype(data[c])
|
| 168 |
+
]
|
| 169 |
+
for col in numeric[:5]:
|
| 170 |
+
col_data = data[col].dropna()
|
| 171 |
+
if len(col_data) > 0:
|
| 172 |
+
op_section.append(
|
| 173 |
+
f" {col}: min={col_data.min():.3f}, "
|
| 174 |
+
f"max={col_data.max():.3f}, "
|
| 175 |
+
f"mean={col_data.mean():.3f}"
|
| 176 |
+
)
|
| 177 |
+
|
| 178 |
+
sections.append("\n".join(op_section))
|
| 179 |
+
|
| 180 |
+
# Warnings
|
| 181 |
+
if result.warnings:
|
| 182 |
+
sections.append("\nWarnings: " + "; ".join(result.warnings))
|
| 183 |
+
|
| 184 |
+
# Errors
|
| 185 |
+
if result.errors:
|
| 186 |
+
sections.append("\nErrors: " + "; ".join(result.errors))
|
| 187 |
+
|
| 188 |
+
# Benchmarks reference
|
| 189 |
+
sections.append("""
|
| 190 |
+
REFERENCE BENCHMARKS (CSH2 Phase 1B):
|
| 191 |
+
- Specific energy: 0.26 kWh/kg (vs 4.5-5 kWh/kg conventional)
|
| 192 |
+
- Flow rate: 1.13-1.25 kg/min at 65% motor power
|
| 193 |
+
- Peak pressure: 370 bar achieved
|
| 194 |
+
- Delivered temperature: 55K minimum
|
| 195 |
+
- Uptime: 100% across 6 fills""")
|
| 196 |
+
|
| 197 |
+
# Final instruction
|
| 198 |
+
sections.append("""
|
| 199 |
+
Based on the above results, provide Murphy's interpretation in 3-5 concise sentences:
|
| 200 |
+
1. What do the results mean in physical/engineering terms?
|
| 201 |
+
2. Flag anything unusual or noteworthy.
|
| 202 |
+
3. Compare to Phase 1B benchmarks where applicable.
|
| 203 |
+
Be direct and technical. Focus on the "so what" — what should the engineer take away from these results.""")
|
| 204 |
+
|
| 205 |
+
return "\n".join(sections)
|
app.py
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
CSH2 Web Analytics Dashboard — DELPHI
|
| 3 |
+
Main entry point with multi-page navigation
|
| 4 |
+
"""
|
| 5 |
+
import streamlit as st
|
| 6 |
+
import base64
|
| 7 |
+
from pathlib import Path
|
| 8 |
+
|
| 9 |
+
st.set_page_config(
|
| 10 |
+
page_title="CSH2 // DELPHI",
|
| 11 |
+
page_icon="⚙️",
|
| 12 |
+
layout="wide",
|
| 13 |
+
initial_sidebar_state="expanded",
|
| 14 |
+
)
|
| 15 |
+
|
| 16 |
+
from ui.styles import inject_custom_css
|
| 17 |
+
from core.db_connector import get_db_connector
|
| 18 |
+
from ui.components import system_status_footer
|
| 19 |
+
|
| 20 |
+
# Inject HUD theme CSS
|
| 21 |
+
inject_custom_css()
|
| 22 |
+
|
| 23 |
+
# Initialize DB connection (cached as singleton)
|
| 24 |
+
db = get_db_connector()
|
| 25 |
+
|
| 26 |
+
# Store DB connector in session state for access from pages
|
| 27 |
+
if 'db' not in st.session_state:
|
| 28 |
+
st.session_state.db = db
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
@st.cache_data
|
| 32 |
+
def _get_logo_base64():
|
| 33 |
+
"""Load logo and encode as base64 for inline display"""
|
| 34 |
+
logo_path = Path(__file__).parent / "assets" / "csh2_logo.jpeg"
|
| 35 |
+
if not logo_path.exists():
|
| 36 |
+
return None
|
| 37 |
+
with open(logo_path, "rb") as f:
|
| 38 |
+
return base64.b64encode(f.read()).decode()
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
@st.cache_data(ttl=3600)
|
| 42 |
+
def _cached_time_range():
|
| 43 |
+
"""Cache the data time range (quasi-static, 1hr TTL)"""
|
| 44 |
+
_db = get_db_connector()
|
| 45 |
+
return _db.get_time_range()
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
# Sidebar branding
|
| 49 |
+
with st.sidebar:
|
| 50 |
+
# Logo with CSS filter for dark background
|
| 51 |
+
logo_b64 = _get_logo_base64()
|
| 52 |
+
if logo_b64:
|
| 53 |
+
st.markdown(f"""
|
| 54 |
+
<div style="text-align: center; margin-bottom: 0.5rem; padding-top: 0.5rem;">
|
| 55 |
+
<img src="data:image/jpeg;base64,{logo_b64}"
|
| 56 |
+
style="width: 120px; filter: invert(0.85) sepia(0.3) saturate(2) hue-rotate(15deg) brightness(1.1);" />
|
| 57 |
+
</div>
|
| 58 |
+
""", unsafe_allow_html=True)
|
| 59 |
+
|
| 60 |
+
st.markdown(
|
| 61 |
+
'<div style="text-align: center; color: #D4A04A; font-size: 0.7rem; letter-spacing: 2px; '
|
| 62 |
+
'text-transform: uppercase; margin-bottom: 12px;">PUMP ANALYTICS</div>',
|
| 63 |
+
unsafe_allow_html=True,
|
| 64 |
+
)
|
| 65 |
+
|
| 66 |
+
# System status
|
| 67 |
+
earliest, latest = _cached_time_range()
|
| 68 |
+
system_status_footer(
|
| 69 |
+
sensor_count=len(db.tag_cache),
|
| 70 |
+
data_start=earliest.strftime('%Y-%m-%d') if earliest else 'N/A',
|
| 71 |
+
data_end=latest.strftime('%Y-%m-%d') if latest else 'N/A',
|
| 72 |
+
connected=True,
|
| 73 |
+
)
|
| 74 |
+
|
| 75 |
+
st.markdown('<div style="height: 8px;"></div>', unsafe_allow_html=True)
|
| 76 |
+
|
| 77 |
+
# Page navigation
|
| 78 |
+
data_explorer = st.Page("pages/1_data_explorer.py", title="Data Explorer", default=True)
|
| 79 |
+
pump_cycles = st.Page("pages/2_pump_cycles.py", title="Testing Cycles")
|
| 80 |
+
cycle_detail = st.Page("pages/3_cycle_detail.py", title="Cycle Detail")
|
| 81 |
+
variation_analysis = st.Page("pages/4_variation_analysis.py", title="Variation Analysis")
|
| 82 |
+
cycle_comparison = st.Page("pages/5_cycle_comparison.py", title="Cycle Comparison")
|
| 83 |
+
advanced_explorer = st.Page("pages/6_advanced_explorer.py", title="Advanced Data Explorer")
|
| 84 |
+
search = st.Page("pages/7_search.py", title="Search")
|
| 85 |
+
|
| 86 |
+
pg = st.navigation([data_explorer, advanced_explorer, pump_cycles, cycle_detail, variation_analysis, cycle_comparison, search])
|
| 87 |
+
pg.run()
|
assets/csh2_logo.jpeg
ADDED
|
core/__init__.py
ADDED
|
File without changes
|
core/cached_queries.py
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Cached query wrappers for Streamlit performance.
|
| 3 |
+
Uses @st.cache_data to avoid re-fetching identical data on every rerun.
|
| 4 |
+
Tag names must be passed as tuples (not lists) for hashability.
|
| 5 |
+
"""
|
| 6 |
+
import streamlit as st
|
| 7 |
+
from typing import Optional
|
| 8 |
+
from datetime import datetime
|
| 9 |
+
from core.db_connector import get_db_connector
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
@st.cache_data(ttl=600)
|
| 13 |
+
def cached_get_pivot_data(
|
| 14 |
+
tag_names: tuple,
|
| 15 |
+
start_time: datetime,
|
| 16 |
+
end_time: datetime,
|
| 17 |
+
table_override: Optional[str] = None,
|
| 18 |
+
_cache_ver: int = 2,
|
| 19 |
+
):
|
| 20 |
+
"""Cached pivot data query (10-minute TTL). Supports table_override for resolution control."""
|
| 21 |
+
db = get_db_connector()
|
| 22 |
+
if table_override:
|
| 23 |
+
# Fetch via get_sensor_data with override, then pivot manually
|
| 24 |
+
df = db.get_sensor_data(list(tag_names), start_time, end_time, table_override=table_override)
|
| 25 |
+
if df.empty:
|
| 26 |
+
return df
|
| 27 |
+
pivot_df = df.pivot_table(index='timestamp', columns='tag_name', values='value').reset_index()
|
| 28 |
+
pivot_df.columns.name = None
|
| 29 |
+
return pivot_df
|
| 30 |
+
return db.get_pivot_data(list(tag_names), start_time, end_time)
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
@st.cache_data(ttl=600)
|
| 34 |
+
def cached_get_sensor_data(
|
| 35 |
+
tag_names: tuple,
|
| 36 |
+
start_time: datetime,
|
| 37 |
+
end_time: datetime,
|
| 38 |
+
table_override: Optional[str] = None,
|
| 39 |
+
_cache_ver: int = 2,
|
| 40 |
+
):
|
| 41 |
+
"""Cached sensor data query (10-minute TTL)"""
|
| 42 |
+
db = get_db_connector()
|
| 43 |
+
return db.get_sensor_data(list(tag_names), start_time, end_time, table_override=table_override)
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
@st.cache_data(ttl=3600)
|
| 47 |
+
def cached_get_available_months():
|
| 48 |
+
"""Cached available months query (1-hour TTL)"""
|
| 49 |
+
db = get_db_connector()
|
| 50 |
+
return db.get_available_months()
|
core/config.py
ADDED
|
@@ -0,0 +1,174 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Configuration for CSH2 Web Analytics Dashboard
|
| 3 |
+
Supports local (.env), Streamlit Cloud (st.secrets), and HuggingFace Spaces (env vars) deployment.
|
| 4 |
+
"""
|
| 5 |
+
import os
|
| 6 |
+
from datetime import timedelta
|
| 7 |
+
from zoneinfo import ZoneInfo
|
| 8 |
+
import streamlit as st
|
| 9 |
+
from dotenv import load_dotenv
|
| 10 |
+
|
| 11 |
+
load_dotenv()
|
| 12 |
+
|
| 13 |
+
# ── Timezone Configuration ──────────────────────────────────────────────────
|
| 14 |
+
# The PLC logs in US/Eastern. The DB's `dateandtime` column is Eastern local
|
| 15 |
+
# time, while `utc_full_timestamp` is the real UTC. The `cycle_periods` table
|
| 16 |
+
# stores Eastern times incorrectly labeled as UTC (+00:00), requiring a +4h
|
| 17 |
+
# correction for sensor data queries. All data (Mar-Sep 2025) falls within
|
| 18 |
+
# EDT (UTC-4). We display all user-facing times in Eastern.
|
| 19 |
+
DISPLAY_TZ = ZoneInfo("US/Eastern")
|
| 20 |
+
DISPLAY_TZ_LABEL = "ET" # label for axis titles and UI
|
| 21 |
+
CYCLE_PERIODS_UTC_OFFSET = timedelta(hours=4) # correction for cycle_periods timestamps
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
def _get_secret(key, default=None):
|
| 25 |
+
"""Read from st.secrets (Streamlit Cloud) or os.environ/.env (local dev)"""
|
| 26 |
+
try:
|
| 27 |
+
return st.secrets[key]
|
| 28 |
+
except (KeyError, FileNotFoundError, AttributeError):
|
| 29 |
+
return os.getenv(key, default)
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
# Database Configuration
|
| 33 |
+
DB_CONFIG = {
|
| 34 |
+
'host': _get_secret('DB_HOST'),
|
| 35 |
+
'port': _get_secret('DB_PORT'),
|
| 36 |
+
'database': _get_secret('DB_NAME'),
|
| 37 |
+
'user': _get_secret('DB_USER'),
|
| 38 |
+
'password': _get_secret('DB_PASSWORD'),
|
| 39 |
+
'connect_timeout': 10,
|
| 40 |
+
}
|
| 41 |
+
|
| 42 |
+
ANTHROPIC_API_KEY = _get_secret('ANTHROPIC_API_KEY')
|
| 43 |
+
|
| 44 |
+
# Database Tables
|
| 45 |
+
TABLES = {
|
| 46 |
+
'raw': 'procdatafloattable',
|
| 47 |
+
'agg_1sec': 'procdatafloattable_utc_1sec',
|
| 48 |
+
'agg_15sec': 'procdatafloattable_utc_15sec',
|
| 49 |
+
'tag_metadata': 'procdatatagtable',
|
| 50 |
+
'events': 'allevent',
|
| 51 |
+
'fill_reports': 'fillreport',
|
| 52 |
+
'cooldown_periods': 'cooldown_periods',
|
| 53 |
+
'cycle_periods': 'cycle_periods',
|
| 54 |
+
}
|
| 55 |
+
|
| 56 |
+
# Smart table routing thresholds (seconds)
|
| 57 |
+
TABLE_ROUTING = {
|
| 58 |
+
'raw_max_seconds': 3600, # <= 1 hour -> raw 10Hz
|
| 59 |
+
'agg_1sec_max_seconds': 86400, # <= 1 day -> 1sec aggregates
|
| 60 |
+
# > 1 day -> 15sec aggregates
|
| 61 |
+
}
|
| 62 |
+
|
| 63 |
+
MAX_ROWS_RETURN = 500000
|
| 64 |
+
|
| 65 |
+
# Resolution name to table mapping (used by Advanced Data Explorer execution engine)
|
| 66 |
+
RESOLUTION_MAP = {
|
| 67 |
+
'raw': TABLES['raw'],
|
| 68 |
+
'1sec': TABLES['agg_1sec'],
|
| 69 |
+
'15sec': TABLES['agg_15sec'],
|
| 70 |
+
}
|
| 71 |
+
|
| 72 |
+
# Key sensor tags with descriptions and units
|
| 73 |
+
SENSOR_TAGS = {
|
| 74 |
+
'AmbTempAvg': {'description': 'Ambient Temperature Average', 'units': 'K', 'tagindex': 0},
|
| 75 |
+
'AOV140': {'description': 'Valve Position AOV140', 'units': '%', 'tagindex': 1},
|
| 76 |
+
'AvgPower': {'description': 'Average Power', 'units': 'kW', 'tagindex': 2},
|
| 77 |
+
'CalcMass': {'description': 'Calculated Mass', 'units': 'kg', 'tagindex': 3},
|
| 78 |
+
'DPT01T': {'description': 'Differential Pressure', 'units': 'bar', 'tagindex': 4},
|
| 79 |
+
'DispTempAvg': {'description': 'Dispenser Temperature Average', 'units': 'K', 'tagindex': 5},
|
| 80 |
+
'FD100': {'description': 'Flow Detector', 'units': '', 'tagindex': 6},
|
| 81 |
+
'FT140': {'description': 'Flow Transmitter', 'units': 'kg/min', 'tagindex': 7},
|
| 82 |
+
'FTTMP': {'description': 'Flow Transmitter Temperature', 'units': 'K', 'tagindex': 8},
|
| 83 |
+
'GD100': {'description': 'Gas Detector', 'units': '', 'tagindex': 9},
|
| 84 |
+
'H201T': {'description': 'H2O Temperature', 'units': 'K', 'tagindex': 10},
|
| 85 |
+
'IR01T': {'description': 'IR Temperature', 'units': 'K', 'tagindex': 11},
|
| 86 |
+
'LT200': {'description': 'Level Transmitter', 'units': '%', 'tagindex': 12},
|
| 87 |
+
'MC130_VFD_Speed': {'description': 'Motor VFD Speed', 'units': 'RPM', 'tagindex': 13},
|
| 88 |
+
'PowerConsumption': {'description': 'Power Consumption', 'units': 'kW', 'tagindex': 14},
|
| 89 |
+
'PT01T': {'description': 'Cryotank Pressure', 'units': 'bar', 'tagindex': 15},
|
| 90 |
+
'PT100': {'description': 'Pressure PT100', 'units': 'bar', 'tagindex': 16},
|
| 91 |
+
'PT110': {'description': 'Inlet Pressure', 'units': 'bar', 'tagindex': 17},
|
| 92 |
+
'PT130': {'description': 'Discharge Pressure', 'units': 'bar', 'tagindex': 18},
|
| 93 |
+
'PT131': {'description': 'Discharge Pressure Alt', 'units': 'bar', 'tagindex': 19},
|
| 94 |
+
'PT150': {'description': 'Pressure PT150', 'units': 'bar', 'tagindex': 20},
|
| 95 |
+
'PT200': {'description': 'Pressure PT200', 'units': 'bar', 'tagindex': 21},
|
| 96 |
+
'PT310': {'description': 'Pressure PT310', 'units': 'bar', 'tagindex': 22},
|
| 97 |
+
'TT100': {'description': 'Temperature TT100', 'units': 'K', 'tagindex': 23},
|
| 98 |
+
'TT110': {'description': 'Pump Feed Line Temperature', 'units': 'K', 'tagindex': 24},
|
| 99 |
+
'TT120': {'description': 'Temperature TT120', 'units': 'K', 'tagindex': 25},
|
| 100 |
+
'TT121': {'description': 'Temperature TT121', 'units': 'K', 'tagindex': 26},
|
| 101 |
+
'TT130': {'description': 'Discharge Temperature', 'units': 'K', 'tagindex': 27},
|
| 102 |
+
'TT131': {'description': 'Discharge Temperature Alt', 'units': 'K', 'tagindex': 28},
|
| 103 |
+
'TT140': {'description': 'Temperature TT140', 'units': 'K', 'tagindex': 29},
|
| 104 |
+
'MC130_PwrConsumption': {'description': 'Motor Power Consumption', 'units': 'kW', 'tagindex': 30},
|
| 105 |
+
'PercentFill': {'description': 'Percent Fill', 'units': '%', 'tagindex': 31},
|
| 106 |
+
'PeakPower': {'description': 'Peak Power', 'units': 'kW', 'tagindex': 32},
|
| 107 |
+
'N2Density': {'description': 'N2 Density', 'units': 'kg/m3', 'tagindex': 33},
|
| 108 |
+
'PS100': {'description': 'Pump Status', 'units': '', 'tagindex': 34},
|
| 109 |
+
'H2Density': {'description': 'H2 Density', 'units': 'kg/m3', 'tagindex': 35},
|
| 110 |
+
'FTTMP2': {'description': 'Flow Temp 2', 'units': 'K', 'tagindex': 54},
|
| 111 |
+
'M130_BusV': {'description': 'Motor Bus Voltage', 'units': 'V', 'tagindex': 55},
|
| 112 |
+
'M130_Current': {'description': 'Motor Current', 'units': 'A', 'tagindex': 56},
|
| 113 |
+
'M130_Freq': {'description': 'Motor Frequency', 'units': 'Hz', 'tagindex': 58},
|
| 114 |
+
'M130_PF': {'description': 'Motor Power Factor', 'units': '', 'tagindex': 61},
|
| 115 |
+
'M130_Pwr': {'description': 'Motor Power', 'units': 'kW', 'tagindex': 62},
|
| 116 |
+
'M130_PwrCon': {'description': 'Motor Power Consumption', 'units': 'kW', 'tagindex': 63},
|
| 117 |
+
'M130_RPM': {'description': 'Motor RPM', 'units': 'RPM', 'tagindex': 66},
|
| 118 |
+
'M130_Speed': {'description': 'Motor Speed', 'units': 'RPM', 'tagindex': 68},
|
| 119 |
+
'M130_Torque': {'description': 'Motor Torque', 'units': 'Nm', 'tagindex': 70},
|
| 120 |
+
'PT010': {'description': 'Pressure PT010', 'units': 'bar', 'tagindex': 72},
|
| 121 |
+
'PT020': {'description': 'Pressure PT020', 'units': 'bar', 'tagindex': 73},
|
| 122 |
+
}
|
| 123 |
+
|
| 124 |
+
# Sensor groups for quick selection
|
| 125 |
+
SENSOR_GROUPS = {
|
| 126 |
+
'Pressure': ['PT01T', 'PT100', 'PT110', 'PT130', 'PT131', 'PT150', 'PT200', 'PT310', 'PT010', 'PT020'],
|
| 127 |
+
'Temperature': ['TT100', 'TT110', 'TT120', 'TT121', 'TT130', 'TT131', 'TT140', 'AmbTempAvg', 'DispTempAvg'],
|
| 128 |
+
'Flow': ['FT140', 'FD100', 'FTTMP', 'FTTMP2', 'CalcMass'],
|
| 129 |
+
'Motor/VFD': ['MC130_VFD_Speed', 'M130_Speed', 'M130_RPM', 'M130_Pwr', 'M130_Current', 'M130_Freq', 'M130_Torque', 'M130_BusV', 'M130_PF'],
|
| 130 |
+
'Power': ['AvgPower', 'PeakPower', 'PowerConsumption', 'MC130_PwrConsumption', 'M130_PwrCon'],
|
| 131 |
+
'Level & Density': ['LT200', 'PercentFill', 'N2Density', 'H2Density'],
|
| 132 |
+
'Status': ['PS100'],
|
| 133 |
+
}
|
| 134 |
+
|
| 135 |
+
# Motor speed tags (for testing cycle detection - try in order)
|
| 136 |
+
MOTOR_SPEED_TAGS = ['M130_Speed', 'MC130_VFD_Speed', 'M130_RPM']
|
| 137 |
+
|
| 138 |
+
# Motor speed display multiplier: M130_Speed (tag 68) reads as a % signal (0-100).
|
| 139 |
+
# Multiply by 5/3 to convert to actual RPM matching Grafana display.
|
| 140 |
+
# e.g., M130_Speed=180 raw → 180 × 5/3 = 300 RPM.
|
| 141 |
+
# Set to 1.0 to disable scaling.
|
| 142 |
+
MOTOR_SPEED_DISPLAY_MULTIPLIER = 5 / 3
|
| 143 |
+
|
| 144 |
+
# Testing cycle detection parameters
|
| 145 |
+
CYCLE_DETECTION = {
|
| 146 |
+
'idle_threshold': 10.0, # RPM below which motor is idle
|
| 147 |
+
'min_cycle_seconds': 60, # Minimum cycle duration
|
| 148 |
+
'smoothing_window': 5, # Rolling median window (in samples)
|
| 149 |
+
'max_gap_seconds': 120, # Max gap to merge adjacent cycles
|
| 150 |
+
# Qualification: cycle must show real compression or flow
|
| 151 |
+
'min_peak_pressure_bar': 50.0, # PT130 must exceed this (real fills reach 300+ bar)
|
| 152 |
+
'min_pressure_rise_bar': 10.0, # PT130 must rise this much from initial
|
| 153 |
+
'min_avg_flow_kg_min': 0.05, # FT140 average must exceed this (real fills do 1+ kg/min)
|
| 154 |
+
'max_cycle_duration_hours': 8.0, # Reject multi-day noise spans (no real test > 8h)
|
| 155 |
+
}
|
| 156 |
+
|
| 157 |
+
# Key tags for cycle detail analysis
|
| 158 |
+
CYCLE_DETAIL_TAGS = ['PT130', 'PT01T', 'PT110', 'TT110', 'TT130', 'FT140', 'M130_Speed', 'MC130_VFD_Speed', 'AOV140']
|
| 159 |
+
|
| 160 |
+
# Performance targets
|
| 161 |
+
PERFORMANCE_TARGETS = {
|
| 162 |
+
'pressure': {'phase1': 500, 'h70': 700, 'ultimate': 1000},
|
| 163 |
+
'flow': {'simplex': 2, 'triplex': 6},
|
| 164 |
+
'efficiency': {'pump_gas_ratio': 0.95, 'seal_blowby': 0.03},
|
| 165 |
+
'energy': {'specific_consumption': 0.5},
|
| 166 |
+
}
|
| 167 |
+
|
| 168 |
+
# Testing phases
|
| 169 |
+
TESTING_PHASES = {
|
| 170 |
+
'mechanical': {'description': 'Mechanical validation with LN2', 'status': 'completed', 'max_pressure': '937 bar'},
|
| 171 |
+
'phase1a': {'description': 'Phase 1A H2 testing with LH2', 'status': 'completed', 'max_pressure': '495 bar'},
|
| 172 |
+
'phase1b': {'description': 'Phase 1B ccH2 fills', 'status': 'completed', 'max_pressure': '495 bar'},
|
| 173 |
+
'phase2': {'description': 'Phase 2 H35/H70 fills', 'status': 'planned', 'target_pressure': '700+ bar'},
|
| 174 |
+
}
|
core/date_parser.py
ADDED
|
@@ -0,0 +1,341 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Natural language date/time parser for ClearSkies Analytics Agent
|
| 3 |
+
Converts user queries like "yesterday", "Sept 25", "between 10:00 and 12:00 on Sept 24" into datetime objects
|
| 4 |
+
NO EXTERNAL DEPENDENCIES - works with just standard library
|
| 5 |
+
"""
|
| 6 |
+
from datetime import datetime, timedelta
|
| 7 |
+
from typing import Tuple, Optional
|
| 8 |
+
import re
|
| 9 |
+
|
| 10 |
+
class DateTimeParser:
|
| 11 |
+
"""Parse natural language date/time expressions"""
|
| 12 |
+
|
| 13 |
+
def __init__(self, reference_date: Optional[datetime] = None):
|
| 14 |
+
"""
|
| 15 |
+
Initialize parser
|
| 16 |
+
|
| 17 |
+
Args:
|
| 18 |
+
reference_date: Reference date for relative expressions (default: now)
|
| 19 |
+
"""
|
| 20 |
+
self.reference_date = reference_date or datetime.now()
|
| 21 |
+
|
| 22 |
+
# Month name mapping
|
| 23 |
+
self.month_map = {
|
| 24 |
+
'jan': 1, 'january': 1,
|
| 25 |
+
'feb': 2, 'february': 2,
|
| 26 |
+
'mar': 3, 'march': 3,
|
| 27 |
+
'apr': 4, 'april': 4,
|
| 28 |
+
'may': 5,
|
| 29 |
+
'jun': 6, 'june': 6,
|
| 30 |
+
'jul': 7, 'july': 7,
|
| 31 |
+
'aug': 8, 'august': 8,
|
| 32 |
+
'sep': 9, 'sept': 9, 'september': 9,
|
| 33 |
+
'oct': 10, 'october': 10,
|
| 34 |
+
'nov': 11, 'november': 11,
|
| 35 |
+
'dec': 12, 'december': 12
|
| 36 |
+
}
|
| 37 |
+
|
| 38 |
+
def parse(self, text: str) -> Tuple[Optional[datetime], Optional[datetime]]:
|
| 39 |
+
"""
|
| 40 |
+
Parse natural language date/time into start and end datetime
|
| 41 |
+
|
| 42 |
+
Args:
|
| 43 |
+
text: User input text containing date/time references
|
| 44 |
+
|
| 45 |
+
Returns:
|
| 46 |
+
Tuple of (start_datetime, end_datetime)
|
| 47 |
+
"""
|
| 48 |
+
text = text.lower().strip()
|
| 49 |
+
|
| 50 |
+
# Priority 1: Time range on specific date (most specific)
|
| 51 |
+
result = self._try_time_range_on_date(text)
|
| 52 |
+
if result:
|
| 53 |
+
return result
|
| 54 |
+
|
| 55 |
+
# Priority 2: Specific date patterns
|
| 56 |
+
result = self._try_specific_patterns(text)
|
| 57 |
+
if result:
|
| 58 |
+
return result
|
| 59 |
+
|
| 60 |
+
# Default: return None if no date found
|
| 61 |
+
return None, None
|
| 62 |
+
|
| 63 |
+
def _try_time_range_on_date(self, text: str) -> Optional[Tuple[datetime, datetime]]:
|
| 64 |
+
"""
|
| 65 |
+
Handle patterns like:
|
| 66 |
+
- "between 10:00 and 12:00 on Sept 24"
|
| 67 |
+
- "from 10:00 to 12:00 on Sept 24"
|
| 68 |
+
- "10:00 to 12:00 on Sept 24"
|
| 69 |
+
"""
|
| 70 |
+
# Pattern 1: "between HH:MM and HH:MM on DATE"
|
| 71 |
+
pattern1 = r'between\s+(\d{1,2}):(\d{2})\s+and\s+(\d{1,2}):(\d{2})\s+on\s+(.+?)$'
|
| 72 |
+
match = re.search(pattern1, text, re.IGNORECASE)
|
| 73 |
+
|
| 74 |
+
if match:
|
| 75 |
+
start_hour = int(match.group(1))
|
| 76 |
+
start_min = int(match.group(2))
|
| 77 |
+
end_hour = int(match.group(3))
|
| 78 |
+
end_min = int(match.group(4))
|
| 79 |
+
date_str = match.group(5).strip()
|
| 80 |
+
|
| 81 |
+
# Parse the date part (just the date, ignore times in date string)
|
| 82 |
+
base_date = self._parse_date_only(date_str)
|
| 83 |
+
if base_date:
|
| 84 |
+
start = datetime(base_date.year, base_date.month, base_date.day, start_hour, start_min, 0, 0)
|
| 85 |
+
end = datetime(base_date.year, base_date.month, base_date.day, end_hour, end_min, 59, 999999)
|
| 86 |
+
return start, end
|
| 87 |
+
|
| 88 |
+
# Pattern 2: "from HH:MM to HH:MM on DATE"
|
| 89 |
+
pattern2 = r'from\s+(\d{1,2}):(\d{2})\s+to\s+(\d{1,2}):(\d{2})\s+on\s+(.+?)$'
|
| 90 |
+
match = re.search(pattern2, text, re.IGNORECASE)
|
| 91 |
+
|
| 92 |
+
if match:
|
| 93 |
+
start_hour = int(match.group(1))
|
| 94 |
+
start_min = int(match.group(2))
|
| 95 |
+
end_hour = int(match.group(3))
|
| 96 |
+
end_min = int(match.group(4))
|
| 97 |
+
date_str = match.group(5).strip()
|
| 98 |
+
|
| 99 |
+
base_date = self._parse_date_only(date_str)
|
| 100 |
+
if base_date:
|
| 101 |
+
start = datetime(base_date.year, base_date.month, base_date.day, start_hour, start_min, 0, 0)
|
| 102 |
+
end = datetime(base_date.year, base_date.month, base_date.day, end_hour, end_min, 59, 999999)
|
| 103 |
+
return start, end
|
| 104 |
+
|
| 105 |
+
# Pattern 3: "HH:MM to HH:MM on DATE" (without "from")
|
| 106 |
+
pattern3 = r'(\d{1,2}):(\d{2})\s+to\s+(\d{1,2}):(\d{2})\s+on\s+(.+?)$'
|
| 107 |
+
match = re.search(pattern3, text, re.IGNORECASE)
|
| 108 |
+
|
| 109 |
+
if match:
|
| 110 |
+
start_hour = int(match.group(1))
|
| 111 |
+
start_min = int(match.group(2))
|
| 112 |
+
end_hour = int(match.group(3))
|
| 113 |
+
end_min = int(match.group(4))
|
| 114 |
+
date_str = match.group(5).strip()
|
| 115 |
+
|
| 116 |
+
base_date = self._parse_date_only(date_str)
|
| 117 |
+
if base_date:
|
| 118 |
+
start = datetime(base_date.year, base_date.month, base_date.day, start_hour, start_min, 0, 0)
|
| 119 |
+
end = datetime(base_date.year, base_date.month, base_date.day, end_hour, end_min, 59, 999999)
|
| 120 |
+
return start, end
|
| 121 |
+
|
| 122 |
+
return None
|
| 123 |
+
|
| 124 |
+
def _parse_date_only(self, date_str: str) -> Optional[datetime]:
|
| 125 |
+
"""Parse a date string like 'Sept 24' or '9/24' into a datetime (time will be 00:00:00)"""
|
| 126 |
+
|
| 127 |
+
# Try month name + day
|
| 128 |
+
month_day_pattern = r'(jan|january|feb|february|mar|march|apr|april|may|jun|june|jul|july|aug|august|sep|sept|september|oct|october|nov|november|dec|december)[a-z]*\s+(\d{1,2})'
|
| 129 |
+
match = re.search(month_day_pattern, date_str, re.IGNORECASE)
|
| 130 |
+
if match:
|
| 131 |
+
month_str = match.group(1).lower()
|
| 132 |
+
month = self.month_map.get(month_str)
|
| 133 |
+
day = int(match.group(2))
|
| 134 |
+
year = self.reference_date.year
|
| 135 |
+
|
| 136 |
+
# If the date is in the future, assume it was last year
|
| 137 |
+
try:
|
| 138 |
+
test_date = datetime(year, month, day)
|
| 139 |
+
if test_date > self.reference_date:
|
| 140 |
+
year -= 1
|
| 141 |
+
return datetime(year, month, day, 0, 0, 0)
|
| 142 |
+
except ValueError:
|
| 143 |
+
return None
|
| 144 |
+
|
| 145 |
+
# Try MM/DD or M/D
|
| 146 |
+
md_pattern = r'\b(\d{1,2})[/-](\d{1,2})\b'
|
| 147 |
+
match = re.search(md_pattern, date_str)
|
| 148 |
+
if match:
|
| 149 |
+
month = int(match.group(1))
|
| 150 |
+
day = int(match.group(2))
|
| 151 |
+
year = self.reference_date.year
|
| 152 |
+
try:
|
| 153 |
+
return datetime(year, month, day, 0, 0, 0)
|
| 154 |
+
except ValueError:
|
| 155 |
+
return None
|
| 156 |
+
|
| 157 |
+
# Try YYYY-MM-DD
|
| 158 |
+
ymd_pattern = r'(\d{4})[/-](\d{1,2})[/-](\d{1,2})'
|
| 159 |
+
match = re.search(ymd_pattern, date_str)
|
| 160 |
+
if match:
|
| 161 |
+
year = int(match.group(1))
|
| 162 |
+
month = int(match.group(2))
|
| 163 |
+
day = int(match.group(3))
|
| 164 |
+
try:
|
| 165 |
+
return datetime(year, month, day, 0, 0, 0)
|
| 166 |
+
except ValueError:
|
| 167 |
+
return None
|
| 168 |
+
|
| 169 |
+
return None
|
| 170 |
+
|
| 171 |
+
def _try_specific_patterns(self, text: str) -> Optional[Tuple[datetime, datetime]]:
|
| 172 |
+
"""Try to match specific common patterns"""
|
| 173 |
+
|
| 174 |
+
# Month name + day (Sept 25, September 25th)
|
| 175 |
+
month_day_pattern = r'(jan|january|feb|february|mar|march|apr|april|may|jun|june|jul|july|aug|august|sep|sept|september|oct|october|nov|november|dec|december)[a-z]*\s+(\d{1,2})'
|
| 176 |
+
match = re.search(month_day_pattern, text, re.IGNORECASE)
|
| 177 |
+
if match:
|
| 178 |
+
month_str = match.group(1).lower()
|
| 179 |
+
month = self.month_map.get(month_str)
|
| 180 |
+
day = int(match.group(2))
|
| 181 |
+
year = self.reference_date.year
|
| 182 |
+
|
| 183 |
+
# If the date is in the future, assume it was last year
|
| 184 |
+
try:
|
| 185 |
+
test_date = datetime(year, month, day)
|
| 186 |
+
if test_date > self.reference_date:
|
| 187 |
+
year -= 1
|
| 188 |
+
|
| 189 |
+
start = datetime(year, month, day, 0, 0, 0)
|
| 190 |
+
end = datetime(year, month, day, 23, 59, 59)
|
| 191 |
+
return start, end
|
| 192 |
+
except ValueError:
|
| 193 |
+
pass
|
| 194 |
+
|
| 195 |
+
# YYYY-MM-DD or YYYY/MM/DD
|
| 196 |
+
ymd_pattern = r'(\d{4})[/-](\d{1,2})[/-](\d{1,2})'
|
| 197 |
+
match = re.search(ymd_pattern, text)
|
| 198 |
+
if match:
|
| 199 |
+
try:
|
| 200 |
+
year = int(match.group(1))
|
| 201 |
+
month = int(match.group(2))
|
| 202 |
+
day = int(match.group(3))
|
| 203 |
+
|
| 204 |
+
start = datetime(year, month, day, 0, 0, 0)
|
| 205 |
+
end = datetime(year, month, day, 23, 59, 59)
|
| 206 |
+
return start, end
|
| 207 |
+
except ValueError:
|
| 208 |
+
pass
|
| 209 |
+
|
| 210 |
+
# MM/DD or M/D
|
| 211 |
+
md_pattern = r'\b(\d{1,2})[/-](\d{1,2})\b'
|
| 212 |
+
match = re.search(md_pattern, text)
|
| 213 |
+
if match:
|
| 214 |
+
try:
|
| 215 |
+
month = int(match.group(1))
|
| 216 |
+
day = int(match.group(2))
|
| 217 |
+
year = self.reference_date.year
|
| 218 |
+
|
| 219 |
+
start = datetime(year, month, day, 0, 0, 0)
|
| 220 |
+
end = datetime(year, month, day, 23, 59, 59)
|
| 221 |
+
return start, end
|
| 222 |
+
except ValueError:
|
| 223 |
+
pass
|
| 224 |
+
|
| 225 |
+
# Today
|
| 226 |
+
if 'today' in text:
|
| 227 |
+
start = self.reference_date.replace(hour=0, minute=0, second=0, microsecond=0)
|
| 228 |
+
end = self.reference_date.replace(hour=23, minute=59, second=59, microsecond=999999)
|
| 229 |
+
return start, end
|
| 230 |
+
|
| 231 |
+
# Yesterday
|
| 232 |
+
if 'yesterday' in text:
|
| 233 |
+
yesterday = self.reference_date - timedelta(days=1)
|
| 234 |
+
start = yesterday.replace(hour=0, minute=0, second=0, microsecond=0)
|
| 235 |
+
end = yesterday.replace(hour=23, minute=59, second=59, microsecond=999999)
|
| 236 |
+
return start, end
|
| 237 |
+
|
| 238 |
+
# Last N hours
|
| 239 |
+
hours_match = re.search(r'last (\d+) hour', text)
|
| 240 |
+
if hours_match:
|
| 241 |
+
hours = int(hours_match.group(1))
|
| 242 |
+
end = self.reference_date
|
| 243 |
+
start = end - timedelta(hours=hours)
|
| 244 |
+
return start, end
|
| 245 |
+
|
| 246 |
+
# Last N days
|
| 247 |
+
days_match = re.search(r'last (\d+) day', text)
|
| 248 |
+
if days_match:
|
| 249 |
+
days = int(days_match.group(1))
|
| 250 |
+
end = self.reference_date
|
| 251 |
+
start = end - timedelta(days=days)
|
| 252 |
+
return start, end
|
| 253 |
+
|
| 254 |
+
# This week
|
| 255 |
+
if 'this week' in text:
|
| 256 |
+
start = self.reference_date - timedelta(days=self.reference_date.weekday())
|
| 257 |
+
start = start.replace(hour=0, minute=0, second=0, microsecond=0)
|
| 258 |
+
end = self.reference_date
|
| 259 |
+
return start, end
|
| 260 |
+
|
| 261 |
+
# Last week
|
| 262 |
+
if 'last week' in text:
|
| 263 |
+
end = self.reference_date - timedelta(days=self.reference_date.weekday())
|
| 264 |
+
start = end - timedelta(days=7)
|
| 265 |
+
start = start.replace(hour=0, minute=0, second=0, microsecond=0)
|
| 266 |
+
end = end.replace(hour=23, minute=59, second=59, microsecond=999999)
|
| 267 |
+
return start, end
|
| 268 |
+
|
| 269 |
+
# This month
|
| 270 |
+
if 'this month' in text:
|
| 271 |
+
start = self.reference_date.replace(day=1, hour=0, minute=0, second=0, microsecond=0)
|
| 272 |
+
end = self.reference_date
|
| 273 |
+
return start, end
|
| 274 |
+
|
| 275 |
+
# Time range patterns: "from X to Y"
|
| 276 |
+
range_pattern = r'from\s+(.+?)\s+to\s+(.+?)(?:\s|$)'
|
| 277 |
+
match = re.search(range_pattern, text, re.IGNORECASE)
|
| 278 |
+
if match:
|
| 279 |
+
start_str = match.group(1).strip()
|
| 280 |
+
end_str = match.group(2).strip()
|
| 281 |
+
|
| 282 |
+
# Try to parse each part recursively
|
| 283 |
+
start_result = self.parse(start_str)
|
| 284 |
+
end_result = self.parse(end_str)
|
| 285 |
+
|
| 286 |
+
if start_result[0] and end_result[1]:
|
| 287 |
+
return start_result[0], end_result[1]
|
| 288 |
+
|
| 289 |
+
return None
|
| 290 |
+
|
| 291 |
+
def extract_time_window(self, text: str) -> Optional[Tuple[datetime, datetime]]:
|
| 292 |
+
"""
|
| 293 |
+
Extract time window from query, with helpful error messages
|
| 294 |
+
|
| 295 |
+
Args:
|
| 296 |
+
text: User query
|
| 297 |
+
|
| 298 |
+
Returns:
|
| 299 |
+
(start_time, end_time) or None if no time found
|
| 300 |
+
"""
|
| 301 |
+
start, end = self.parse(text)
|
| 302 |
+
|
| 303 |
+
if start and end:
|
| 304 |
+
return start, end
|
| 305 |
+
|
| 306 |
+
return None
|
| 307 |
+
|
| 308 |
+
|
| 309 |
+
# Test function
|
| 310 |
+
if __name__ == "__main__":
|
| 311 |
+
print("Testing DateTimeParser...")
|
| 312 |
+
|
| 313 |
+
parser = DateTimeParser(reference_date=datetime(2025, 9, 26, 14, 30))
|
| 314 |
+
|
| 315 |
+
test_queries = [
|
| 316 |
+
"yesterday",
|
| 317 |
+
"today",
|
| 318 |
+
"last 3 hours",
|
| 319 |
+
"last 5 days",
|
| 320 |
+
"Sept 25",
|
| 321 |
+
"September 25th",
|
| 322 |
+
"between 10:00 and 12:00 on Sept 24",
|
| 323 |
+
"from 10:00 to 12:00 on Sept 24",
|
| 324 |
+
"10:00 to 12:00 on Sept 24",
|
| 325 |
+
"from Sept 24 to Sept 25",
|
| 326 |
+
"this week",
|
| 327 |
+
"last week",
|
| 328 |
+
"9/25",
|
| 329 |
+
"2025-09-25",
|
| 330 |
+
]
|
| 331 |
+
|
| 332 |
+
print("\nTest queries:")
|
| 333 |
+
for query in test_queries:
|
| 334 |
+
start, end = parser.parse(query)
|
| 335 |
+
if start and end:
|
| 336 |
+
print(f"✅ '{query}'")
|
| 337 |
+
print(f" Start: {start}")
|
| 338 |
+
print(f" End: {end}")
|
| 339 |
+
else:
|
| 340 |
+
print(f"❌ '{query}' - Could not parse")
|
| 341 |
+
print()
|
core/db_connector.py
ADDED
|
@@ -0,0 +1,492 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Enhanced Database Connector for CSH2 Web Dashboard
|
| 3 |
+
Smart table routing, connection health checks, pivot data support
|
| 4 |
+
"""
|
| 5 |
+
import psycopg2
|
| 6 |
+
from psycopg2.extras import RealDictCursor
|
| 7 |
+
import pandas as pd
|
| 8 |
+
from typing import List, Dict, Optional, Tuple
|
| 9 |
+
from datetime import datetime, timezone
|
| 10 |
+
import streamlit as st
|
| 11 |
+
from core import config
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
class DatabaseConnector:
|
| 15 |
+
"""Handles all database operations with smart table routing"""
|
| 16 |
+
|
| 17 |
+
def __init__(self):
|
| 18 |
+
self.conn = None
|
| 19 |
+
self.tag_cache = {}
|
| 20 |
+
self.index_to_tag = {}
|
| 21 |
+
self._connect()
|
| 22 |
+
self._load_tag_cache()
|
| 23 |
+
|
| 24 |
+
def _connect(self):
|
| 25 |
+
try:
|
| 26 |
+
self.conn = psycopg2.connect(**config.DB_CONFIG)
|
| 27 |
+
self.conn.autocommit = True
|
| 28 |
+
except Exception as e:
|
| 29 |
+
raise ConnectionError(f"Database connection failed: {e}")
|
| 30 |
+
|
| 31 |
+
def _ensure_connection(self):
|
| 32 |
+
"""Reconnect if the connection has gone stale"""
|
| 33 |
+
try:
|
| 34 |
+
with self.conn.cursor() as cur:
|
| 35 |
+
cur.execute("SELECT 1")
|
| 36 |
+
except Exception:
|
| 37 |
+
self._connect()
|
| 38 |
+
|
| 39 |
+
def _load_tag_cache(self):
|
| 40 |
+
try:
|
| 41 |
+
query = f"SELECT tagindex, tagname FROM {config.TABLES['tag_metadata']}"
|
| 42 |
+
df = pd.read_sql(query, self.conn)
|
| 43 |
+
self.tag_cache = {row['tagname']: row['tagindex'] for _, row in df.iterrows()}
|
| 44 |
+
self.index_to_tag = {row['tagindex']: row['tagname'] for _, row in df.iterrows()}
|
| 45 |
+
except Exception as e:
|
| 46 |
+
self.tag_cache = {}
|
| 47 |
+
self.index_to_tag = {}
|
| 48 |
+
|
| 49 |
+
def get_tag_index(self, tag_name: str) -> Optional[int]:
|
| 50 |
+
return self.tag_cache.get(tag_name) or self.tag_cache.get(tag_name.upper())
|
| 51 |
+
|
| 52 |
+
def get_tag_name(self, tag_index: int) -> Optional[str]:
|
| 53 |
+
return self.index_to_tag.get(tag_index)
|
| 54 |
+
|
| 55 |
+
def get_all_tag_names(self) -> List[str]:
|
| 56 |
+
return sorted(self.tag_cache.keys())
|
| 57 |
+
|
| 58 |
+
def _select_table(self, start_time: datetime, end_time: datetime) -> str:
|
| 59 |
+
"""Select the optimal table based on time range duration"""
|
| 60 |
+
duration_seconds = (end_time - start_time).total_seconds()
|
| 61 |
+
routing = config.TABLE_ROUTING
|
| 62 |
+
|
| 63 |
+
if duration_seconds <= routing['raw_max_seconds']:
|
| 64 |
+
return config.TABLES['raw']
|
| 65 |
+
elif duration_seconds <= routing['agg_1sec_max_seconds']:
|
| 66 |
+
return config.TABLES['agg_1sec']
|
| 67 |
+
else:
|
| 68 |
+
return config.TABLES['agg_15sec']
|
| 69 |
+
|
| 70 |
+
def get_sensor_data(
|
| 71 |
+
self,
|
| 72 |
+
tag_names: List[str],
|
| 73 |
+
start_time: datetime,
|
| 74 |
+
end_time: datetime,
|
| 75 |
+
table_override: Optional[str] = None,
|
| 76 |
+
limit: Optional[int] = None,
|
| 77 |
+
) -> pd.DataFrame:
|
| 78 |
+
"""
|
| 79 |
+
Fetch sensor data with smart table routing.
|
| 80 |
+
|
| 81 |
+
Returns DataFrame with columns: timestamp, tag_name, value
|
| 82 |
+
"""
|
| 83 |
+
self._ensure_connection()
|
| 84 |
+
|
| 85 |
+
tag_indices = []
|
| 86 |
+
valid_tags = []
|
| 87 |
+
for tag in tag_names:
|
| 88 |
+
idx = self.get_tag_index(tag)
|
| 89 |
+
if idx is not None:
|
| 90 |
+
tag_indices.append(idx)
|
| 91 |
+
valid_tags.append(tag)
|
| 92 |
+
|
| 93 |
+
if not tag_indices:
|
| 94 |
+
return pd.DataFrame(columns=['timestamp', 'tag_name', 'value'])
|
| 95 |
+
|
| 96 |
+
table = table_override or self._select_table(start_time, end_time)
|
| 97 |
+
tag_list = ', '.join(map(str, tag_indices))
|
| 98 |
+
limit_clause = f"LIMIT {limit}" if limit else f"LIMIT {config.MAX_ROWS_RETURN}"
|
| 99 |
+
|
| 100 |
+
query = f"""
|
| 101 |
+
SELECT
|
| 102 |
+
utc_full_timestamp as timestamp,
|
| 103 |
+
tagindex,
|
| 104 |
+
val as value
|
| 105 |
+
FROM {table}
|
| 106 |
+
WHERE tagindex IN ({tag_list})
|
| 107 |
+
AND utc_full_timestamp >= %s
|
| 108 |
+
AND utc_full_timestamp <= %s
|
| 109 |
+
ORDER BY utc_full_timestamp, tagindex
|
| 110 |
+
{limit_clause}
|
| 111 |
+
"""
|
| 112 |
+
|
| 113 |
+
try:
|
| 114 |
+
df = pd.read_sql(query, self.conn, params=(start_time, end_time))
|
| 115 |
+
df['tag_name'] = df['tagindex'].map(self.index_to_tag)
|
| 116 |
+
df = df[['timestamp', 'tag_name', 'value']]
|
| 117 |
+
return df
|
| 118 |
+
except Exception as e:
|
| 119 |
+
st.error(f"Query failed: {e}")
|
| 120 |
+
return pd.DataFrame(columns=['timestamp', 'tag_name', 'value'])
|
| 121 |
+
|
| 122 |
+
def get_pivot_data(
|
| 123 |
+
self,
|
| 124 |
+
tag_names: List[str],
|
| 125 |
+
start_time: datetime,
|
| 126 |
+
end_time: datetime,
|
| 127 |
+
) -> pd.DataFrame:
|
| 128 |
+
"""Fetch data and return in wide (pivoted) format"""
|
| 129 |
+
df = self.get_sensor_data(tag_names, start_time, end_time)
|
| 130 |
+
if df.empty:
|
| 131 |
+
return df
|
| 132 |
+
|
| 133 |
+
pivot_df = df.pivot_table(
|
| 134 |
+
index='timestamp',
|
| 135 |
+
columns='tag_name',
|
| 136 |
+
values='value',
|
| 137 |
+
).reset_index()
|
| 138 |
+
|
| 139 |
+
pivot_df.columns.name = None
|
| 140 |
+
return pivot_df
|
| 141 |
+
|
| 142 |
+
def get_available_months(self) -> List[str]:
|
| 143 |
+
"""Get distinct months where data exists (using 15sec table for speed)"""
|
| 144 |
+
self._ensure_connection()
|
| 145 |
+
query = f"""
|
| 146 |
+
SELECT DISTINCT date_trunc('month', utc_full_timestamp)::date as month
|
| 147 |
+
FROM {config.TABLES['agg_15sec']}
|
| 148 |
+
ORDER BY month
|
| 149 |
+
"""
|
| 150 |
+
try:
|
| 151 |
+
df = pd.read_sql(query, self.conn)
|
| 152 |
+
return [row['month'].strftime('%Y-%m') for _, row in df.iterrows()]
|
| 153 |
+
except Exception:
|
| 154 |
+
# Fallback: use raw table with a limit
|
| 155 |
+
try:
|
| 156 |
+
query = f"""
|
| 157 |
+
SELECT DISTINCT date_trunc('month', utc_full_timestamp)::date as month
|
| 158 |
+
FROM {config.TABLES['raw']}
|
| 159 |
+
ORDER BY month
|
| 160 |
+
"""
|
| 161 |
+
df = pd.read_sql(query, self.conn)
|
| 162 |
+
return [row['month'].strftime('%Y-%m') for _, row in df.iterrows()]
|
| 163 |
+
except Exception:
|
| 164 |
+
return []
|
| 165 |
+
|
| 166 |
+
def get_time_range(self) -> Tuple[Optional[datetime], Optional[datetime]]:
|
| 167 |
+
"""Get overall time range of available data"""
|
| 168 |
+
self._ensure_connection()
|
| 169 |
+
query = f"""
|
| 170 |
+
SELECT MIN(utc_full_timestamp) as earliest, MAX(utc_full_timestamp) as latest
|
| 171 |
+
FROM {config.TABLES['agg_15sec']}
|
| 172 |
+
"""
|
| 173 |
+
try:
|
| 174 |
+
df = pd.read_sql(query, self.conn)
|
| 175 |
+
return df['earliest'][0], df['latest'][0]
|
| 176 |
+
except Exception:
|
| 177 |
+
return None, None
|
| 178 |
+
|
| 179 |
+
def get_data_summary(
|
| 180 |
+
self,
|
| 181 |
+
tag_name: str,
|
| 182 |
+
start_time: datetime,
|
| 183 |
+
end_time: datetime,
|
| 184 |
+
) -> Dict:
|
| 185 |
+
"""Get statistical summary for a single tag"""
|
| 186 |
+
self._ensure_connection()
|
| 187 |
+
tag_index = self.get_tag_index(tag_name)
|
| 188 |
+
if tag_index is None:
|
| 189 |
+
return {'error': f'Tag {tag_name} not found'}
|
| 190 |
+
|
| 191 |
+
table = self._select_table(start_time, end_time)
|
| 192 |
+
query = f"""
|
| 193 |
+
SELECT COUNT(*) as count, MIN(val) as min_value, MAX(val) as max_value,
|
| 194 |
+
AVG(val) as avg_value, STDDEV(val) as std_value
|
| 195 |
+
FROM {table}
|
| 196 |
+
WHERE tagindex = %s AND utc_full_timestamp >= %s AND utc_full_timestamp <= %s
|
| 197 |
+
"""
|
| 198 |
+
try:
|
| 199 |
+
with self.conn.cursor(cursor_factory=RealDictCursor) as cur:
|
| 200 |
+
cur.execute(query, (tag_index, start_time, end_time))
|
| 201 |
+
result = cur.fetchone()
|
| 202 |
+
return dict(result)
|
| 203 |
+
except Exception as e:
|
| 204 |
+
return {'error': str(e)}
|
| 205 |
+
|
| 206 |
+
def check_table_exists(self, table_name: str) -> bool:
|
| 207 |
+
"""Check if a table exists in the database"""
|
| 208 |
+
self._ensure_connection()
|
| 209 |
+
query = """
|
| 210 |
+
SELECT EXISTS (
|
| 211 |
+
SELECT FROM information_schema.tables
|
| 212 |
+
WHERE table_schema = 'public' AND table_name = %s
|
| 213 |
+
)
|
| 214 |
+
"""
|
| 215 |
+
try:
|
| 216 |
+
with self.conn.cursor() as cur:
|
| 217 |
+
cur.execute(query, (table_name,))
|
| 218 |
+
return cur.fetchone()[0]
|
| 219 |
+
except Exception:
|
| 220 |
+
return False
|
| 221 |
+
|
| 222 |
+
def get_cooldown_periods(self, start_time: datetime, end_time: datetime) -> pd.DataFrame:
|
| 223 |
+
"""Query cooldown_periods table if it exists"""
|
| 224 |
+
self._ensure_connection()
|
| 225 |
+
if not self.check_table_exists('cooldown_periods'):
|
| 226 |
+
return pd.DataFrame()
|
| 227 |
+
|
| 228 |
+
query = """
|
| 229 |
+
SELECT cooldown_start, cooldown_end, start_temperature, end_temperature, duration_sec
|
| 230 |
+
FROM cooldown_periods
|
| 231 |
+
WHERE cooldown_start >= %s AND cooldown_start <= %s
|
| 232 |
+
ORDER BY cooldown_start
|
| 233 |
+
"""
|
| 234 |
+
try:
|
| 235 |
+
return pd.read_sql(query, self.conn, params=(start_time, end_time))
|
| 236 |
+
except Exception:
|
| 237 |
+
return pd.DataFrame()
|
| 238 |
+
|
| 239 |
+
def resolve_motor_speed_tag(self) -> Optional[str]:
|
| 240 |
+
"""Find which motor speed tag exists in the database"""
|
| 241 |
+
for tag in config.MOTOR_SPEED_TAGS:
|
| 242 |
+
if tag in self.tag_cache:
|
| 243 |
+
return tag
|
| 244 |
+
# Search for any tag containing 'speed' or 'rpm'
|
| 245 |
+
for tag_name in self.tag_cache:
|
| 246 |
+
if 'speed' in tag_name.lower() or 'rpm' in tag_name.lower():
|
| 247 |
+
return tag_name
|
| 248 |
+
return None
|
| 249 |
+
|
| 250 |
+
def get_cycle_periods(self, month_str: str = None) -> List[Dict]:
|
| 251 |
+
"""Fetch pre-computed cycles from the cycle_periods table.
|
| 252 |
+
|
| 253 |
+
Args:
|
| 254 |
+
month_str: Optional "YYYY-MM" filter. None returns all cycles.
|
| 255 |
+
|
| 256 |
+
Returns:
|
| 257 |
+
List of cycle dicts in the format expected by downstream pages
|
| 258 |
+
(cycle_id, start_time, end_time, duration_minutes, peak_speed,
|
| 259 |
+
peak_pressure, avg_flow, etc.), plus new fields from the DB.
|
| 260 |
+
"""
|
| 261 |
+
self._ensure_connection()
|
| 262 |
+
|
| 263 |
+
query = """
|
| 264 |
+
SELECT id, cycle_start, cycle_end, duration_sec,
|
| 265 |
+
max_motor_speed, max_flow_rate, avg_flow_rate,
|
| 266 |
+
total_kg_dispensed, min_discharge_pressure, max_discharge_pressure,
|
| 267 |
+
min_fill_temperature, max_fill_temperature, avg_ambient_temperature,
|
| 268 |
+
cryotank_start_pressure, cryotank_end_pressure, total_pump_strokes
|
| 269 |
+
FROM cycle_periods
|
| 270 |
+
"""
|
| 271 |
+
params = []
|
| 272 |
+
if month_str:
|
| 273 |
+
year, month = map(int, month_str.split('-'))
|
| 274 |
+
if month == 12:
|
| 275 |
+
next_year, next_month = year + 1, 1
|
| 276 |
+
else:
|
| 277 |
+
next_year, next_month = year, month + 1
|
| 278 |
+
query += " WHERE cycle_start >= %s AND cycle_start < %s"
|
| 279 |
+
params = [
|
| 280 |
+
datetime(year, month, 1, tzinfo=timezone.utc),
|
| 281 |
+
datetime(next_year, next_month, 1, tzinfo=timezone.utc),
|
| 282 |
+
]
|
| 283 |
+
query += " ORDER BY cycle_start"
|
| 284 |
+
|
| 285 |
+
try:
|
| 286 |
+
df = pd.read_sql(query, self.conn, params=params or None)
|
| 287 |
+
except Exception:
|
| 288 |
+
return []
|
| 289 |
+
|
| 290 |
+
if df.empty:
|
| 291 |
+
return []
|
| 292 |
+
|
| 293 |
+
from core.timezone import cycle_periods_to_utc, cycle_periods_to_eastern
|
| 294 |
+
|
| 295 |
+
speed_mult = getattr(config, 'MOTOR_SPEED_DISPLAY_MULTIPLIER', 1.0)
|
| 296 |
+
|
| 297 |
+
cycles = []
|
| 298 |
+
for _, row in df.iterrows():
|
| 299 |
+
raw_start = row['cycle_start'].to_pydatetime() if hasattr(row['cycle_start'], 'to_pydatetime') else row['cycle_start']
|
| 300 |
+
raw_end = row['cycle_end'].to_pydatetime() if hasattr(row['cycle_end'], 'to_pydatetime') else row['cycle_end']
|
| 301 |
+
|
| 302 |
+
cycles.append({
|
| 303 |
+
# Times corrected to real UTC — use these for all DB queries
|
| 304 |
+
'start_time': cycle_periods_to_utc(raw_start),
|
| 305 |
+
'end_time': cycle_periods_to_utc(raw_end),
|
| 306 |
+
# Eastern display times — use these for all UI rendering
|
| 307 |
+
'start_time_et': cycle_periods_to_eastern(raw_start),
|
| 308 |
+
'end_time_et': cycle_periods_to_eastern(raw_end),
|
| 309 |
+
# Standard fields
|
| 310 |
+
'cycle_id': int(row['id']),
|
| 311 |
+
'duration_minutes': float(row['duration_sec']) / 60.0,
|
| 312 |
+
'peak_speed': float(row['max_motor_speed'] or 0) * speed_mult,
|
| 313 |
+
'peak_pressure': float(row['max_discharge_pressure'] or 0),
|
| 314 |
+
'initial_pressure': float(row['min_discharge_pressure'] or 0),
|
| 315 |
+
'avg_flow': float(row['avg_flow_rate'] or 0),
|
| 316 |
+
'peak_TT110': float(row['max_fill_temperature'] or 0),
|
| 317 |
+
# Additional fields from cycle_periods table
|
| 318 |
+
'max_flow_rate': float(row['max_flow_rate'] or 0),
|
| 319 |
+
'total_kg_dispensed': float(row['total_kg_dispensed'] or 0),
|
| 320 |
+
'min_fill_temperature': float(row['min_fill_temperature'] or 0),
|
| 321 |
+
'avg_ambient_temperature': float(row['avg_ambient_temperature'] or 0),
|
| 322 |
+
'cryotank_start_pressure': float(row['cryotank_start_pressure'] or 0),
|
| 323 |
+
'cryotank_end_pressure': float(row['cryotank_end_pressure'] or 0),
|
| 324 |
+
'total_pump_strokes': float(row['total_pump_strokes'] or 0),
|
| 325 |
+
})
|
| 326 |
+
return cycles
|
| 327 |
+
|
| 328 |
+
def get_cycle_months(self) -> List[str]:
|
| 329 |
+
"""Get distinct months where cycles exist in cycle_periods table."""
|
| 330 |
+
self._ensure_connection()
|
| 331 |
+
query = """
|
| 332 |
+
SELECT DISTINCT date_trunc('month', cycle_start)::date as month
|
| 333 |
+
FROM cycle_periods
|
| 334 |
+
ORDER BY month
|
| 335 |
+
"""
|
| 336 |
+
try:
|
| 337 |
+
df = pd.read_sql(query, self.conn)
|
| 338 |
+
return [row['month'].strftime('%Y-%m') for _, row in df.iterrows()]
|
| 339 |
+
except Exception:
|
| 340 |
+
return []
|
| 341 |
+
|
| 342 |
+
# ── Text Search Methods ──
|
| 343 |
+
|
| 344 |
+
def search_memo_log(
|
| 345 |
+
self,
|
| 346 |
+
query: str,
|
| 347 |
+
limit: int = 20,
|
| 348 |
+
engineer: Optional[str] = None,
|
| 349 |
+
severity: Optional[str] = None,
|
| 350 |
+
) -> pd.DataFrame:
|
| 351 |
+
"""Full-text ranked search across engineer field notes in memo_log.
|
| 352 |
+
|
| 353 |
+
Uses PostgreSQL tsvector/tsquery with ts_rank for BM25-style relevance.
|
| 354 |
+
Searches across summary, issues_found, raw_transcript, maintenance_done,
|
| 355 |
+
and system_performance columns.
|
| 356 |
+
"""
|
| 357 |
+
self._ensure_connection()
|
| 358 |
+
sql = """
|
| 359 |
+
SELECT id, logged_at, engineer, severity, activity_type,
|
| 360 |
+
summary, issues_found, maintenance_done, system_performance,
|
| 361 |
+
components_affected,
|
| 362 |
+
ts_rank(
|
| 363 |
+
to_tsvector('english',
|
| 364 |
+
coalesce(summary, '') || ' ' ||
|
| 365 |
+
coalesce(issues_found, '') || ' ' ||
|
| 366 |
+
coalesce(raw_transcript, '') || ' ' ||
|
| 367 |
+
coalesce(maintenance_done, '') || ' ' ||
|
| 368 |
+
coalesce(system_performance, '')
|
| 369 |
+
),
|
| 370 |
+
plainto_tsquery('english', %(query)s)
|
| 371 |
+
) AS rank
|
| 372 |
+
FROM memo_log
|
| 373 |
+
WHERE to_tsvector('english',
|
| 374 |
+
coalesce(summary, '') || ' ' ||
|
| 375 |
+
coalesce(issues_found, '') || ' ' ||
|
| 376 |
+
coalesce(raw_transcript, '') || ' ' ||
|
| 377 |
+
coalesce(maintenance_done, '') || ' ' ||
|
| 378 |
+
coalesce(system_performance, '')
|
| 379 |
+
) @@ plainto_tsquery('english', %(query)s)
|
| 380 |
+
"""
|
| 381 |
+
params: Dict = {'query': query}
|
| 382 |
+
if engineer:
|
| 383 |
+
sql += " AND engineer = %(engineer)s"
|
| 384 |
+
params['engineer'] = engineer
|
| 385 |
+
if severity:
|
| 386 |
+
sql += " AND severity = %(severity)s"
|
| 387 |
+
params['severity'] = severity
|
| 388 |
+
sql += " ORDER BY rank DESC LIMIT %(limit)s"
|
| 389 |
+
params['limit'] = limit
|
| 390 |
+
|
| 391 |
+
try:
|
| 392 |
+
return pd.read_sql(sql, self.conn, params=params)
|
| 393 |
+
except Exception as e:
|
| 394 |
+
st.error(f"Memo search failed: {e}")
|
| 395 |
+
return pd.DataFrame()
|
| 396 |
+
|
| 397 |
+
def search_events(
|
| 398 |
+
self,
|
| 399 |
+
query: str,
|
| 400 |
+
condition: Optional[str] = None,
|
| 401 |
+
limit: int = 50,
|
| 402 |
+
) -> pd.DataFrame:
|
| 403 |
+
"""Search PLC alarm events using both full-text and pattern matching.
|
| 404 |
+
|
| 405 |
+
Alarm messages contain sensor tags in bracket notation (e.g. [AI_PT110_Alarm]),
|
| 406 |
+
so we use ILIKE alongside tsvector to catch tag name patterns.
|
| 407 |
+
"""
|
| 408 |
+
self._ensure_connection()
|
| 409 |
+
# Use ILIKE for pattern matching (catches sensor names in brackets)
|
| 410 |
+
# plus tsvector for any natural text content
|
| 411 |
+
sql = """
|
| 412 |
+
SELECT eventtimestamp, conditionname, message, inputvalue, limitvalue
|
| 413 |
+
FROM allevent
|
| 414 |
+
WHERE (
|
| 415 |
+
message ILIKE %(pattern)s
|
| 416 |
+
OR conditionname ILIKE %(pattern)s
|
| 417 |
+
)
|
| 418 |
+
"""
|
| 419 |
+
params: Dict = {'query': query, 'pattern': f'%{query}%', 'limit': limit}
|
| 420 |
+
if condition:
|
| 421 |
+
sql += " AND conditionname = %(condition)s"
|
| 422 |
+
params['condition'] = condition
|
| 423 |
+
sql += " ORDER BY eventtimestamp DESC LIMIT %(limit)s"
|
| 424 |
+
try:
|
| 425 |
+
return pd.read_sql(sql, self.conn, params=params)
|
| 426 |
+
except Exception as e:
|
| 427 |
+
st.error(f"Event search failed: {e}")
|
| 428 |
+
return pd.DataFrame()
|
| 429 |
+
|
| 430 |
+
def get_event_conditions(self) -> List[str]:
|
| 431 |
+
"""Get distinct condition types from allevent for filter dropdown."""
|
| 432 |
+
self._ensure_connection()
|
| 433 |
+
try:
|
| 434 |
+
with self.conn.cursor() as cur:
|
| 435 |
+
cur.execute(
|
| 436 |
+
"SELECT conditionname, COUNT(*) as cnt FROM allevent "
|
| 437 |
+
"GROUP BY conditionname ORDER BY cnt DESC"
|
| 438 |
+
)
|
| 439 |
+
return [row[0] for row in cur.fetchall() if row[0]]
|
| 440 |
+
except Exception:
|
| 441 |
+
return []
|
| 442 |
+
|
| 443 |
+
def search_tags_fuzzy(self, query: str, limit: int = 15) -> pd.DataFrame:
|
| 444 |
+
"""Fuzzy tag/sensor search using pg_trgm similarity on tagimport."""
|
| 445 |
+
self._ensure_connection()
|
| 446 |
+
sql = """
|
| 447 |
+
SELECT tagname, tagdesc, tagunit,
|
| 448 |
+
greatest(
|
| 449 |
+
similarity(lower(tagname), lower(%(query)s)),
|
| 450 |
+
similarity(lower(coalesce(tagdesc, '')), lower(%(query)s))
|
| 451 |
+
) AS sim
|
| 452 |
+
FROM tagimport
|
| 453 |
+
WHERE lower(tagname) %% lower(%(query)s)
|
| 454 |
+
OR lower(coalesce(tagdesc, '')) %% lower(%(query)s)
|
| 455 |
+
ORDER BY sim DESC
|
| 456 |
+
LIMIT %(limit)s
|
| 457 |
+
"""
|
| 458 |
+
try:
|
| 459 |
+
return pd.read_sql(sql, self.conn, params={'query': query, 'limit': limit})
|
| 460 |
+
except Exception as e:
|
| 461 |
+
st.error(f"Tag search failed: {e}")
|
| 462 |
+
return pd.DataFrame()
|
| 463 |
+
|
| 464 |
+
def get_memo_engineers(self) -> List[str]:
|
| 465 |
+
"""Get distinct engineers from memo_log for filter dropdown."""
|
| 466 |
+
self._ensure_connection()
|
| 467 |
+
try:
|
| 468 |
+
with self.conn.cursor() as cur:
|
| 469 |
+
cur.execute("SELECT DISTINCT engineer FROM memo_log WHERE engineer IS NOT NULL ORDER BY engineer")
|
| 470 |
+
return [row[0] for row in cur.fetchall()]
|
| 471 |
+
except Exception:
|
| 472 |
+
return []
|
| 473 |
+
|
| 474 |
+
def get_memo_severities(self) -> List[str]:
|
| 475 |
+
"""Get distinct severity levels from memo_log for filter dropdown."""
|
| 476 |
+
self._ensure_connection()
|
| 477 |
+
try:
|
| 478 |
+
with self.conn.cursor() as cur:
|
| 479 |
+
cur.execute("SELECT DISTINCT severity FROM memo_log WHERE severity IS NOT NULL ORDER BY severity")
|
| 480 |
+
return [row[0] for row in cur.fetchall()]
|
| 481 |
+
except Exception:
|
| 482 |
+
return []
|
| 483 |
+
|
| 484 |
+
def close(self):
|
| 485 |
+
if self.conn:
|
| 486 |
+
self.conn.close()
|
| 487 |
+
|
| 488 |
+
|
| 489 |
+
@st.cache_resource
|
| 490 |
+
def get_db_connector() -> DatabaseConnector:
|
| 491 |
+
"""Create a singleton database connector"""
|
| 492 |
+
return DatabaseConnector()
|
core/export_utils.py
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Export utilities for the Advanced Data Explorer.
|
| 3 |
+
Downsampling, multi-window CSV packaging, size estimation.
|
| 4 |
+
"""
|
| 5 |
+
import io
|
| 6 |
+
import zipfile
|
| 7 |
+
import pandas as pd
|
| 8 |
+
from typing import List, Optional
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
def downsample_dataframe(df: pd.DataFrame, target_hz: float) -> pd.DataFrame:
|
| 12 |
+
"""Downsample a time-indexed DataFrame to a target frequency.
|
| 13 |
+
|
| 14 |
+
Args:
|
| 15 |
+
df: DataFrame with a 'timestamp' column
|
| 16 |
+
target_hz: Target frequency in Hz
|
| 17 |
+
|
| 18 |
+
Returns:
|
| 19 |
+
Downsampled DataFrame
|
| 20 |
+
"""
|
| 21 |
+
if "timestamp" not in df.columns or len(df) < 2:
|
| 22 |
+
return df
|
| 23 |
+
|
| 24 |
+
sorted_df = df.sort_values("timestamp")
|
| 25 |
+
dt = sorted_df["timestamp"].diff().dt.total_seconds().dropna()
|
| 26 |
+
current_hz = 1.0 / dt.median() if dt.median() > 0 else 1.0
|
| 27 |
+
|
| 28 |
+
if target_hz >= current_hz:
|
| 29 |
+
return sorted_df
|
| 30 |
+
|
| 31 |
+
step = max(1, int(round(current_hz / target_hz)))
|
| 32 |
+
return sorted_df.iloc[::step].reset_index(drop=True)
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def package_multi_window_csv(
|
| 36 |
+
dfs: List[pd.DataFrame],
|
| 37 |
+
labels: Optional[List[str]] = None,
|
| 38 |
+
base_filename: str = "export",
|
| 39 |
+
) -> bytes:
|
| 40 |
+
"""Package multiple DataFrames into a zip of CSVs.
|
| 41 |
+
|
| 42 |
+
Args:
|
| 43 |
+
dfs: List of DataFrames to export
|
| 44 |
+
labels: Optional labels for each window (used in filenames)
|
| 45 |
+
base_filename: Base filename for the zip
|
| 46 |
+
|
| 47 |
+
Returns:
|
| 48 |
+
Bytes of the zip file
|
| 49 |
+
"""
|
| 50 |
+
buffer = io.BytesIO()
|
| 51 |
+
with zipfile.ZipFile(buffer, "w", zipfile.ZIP_DEFLATED) as zf:
|
| 52 |
+
for i, df in enumerate(dfs):
|
| 53 |
+
label = labels[i] if labels and i < len(labels) else f"window_{i + 1}"
|
| 54 |
+
safe_label = label.replace(" ", "_").replace("/", "-")
|
| 55 |
+
csv_name = f"{base_filename}_{safe_label}.csv"
|
| 56 |
+
csv_bytes = df.to_csv(index=False).encode("utf-8")
|
| 57 |
+
zf.writestr(csv_name, csv_bytes)
|
| 58 |
+
|
| 59 |
+
buffer.seek(0)
|
| 60 |
+
return buffer.getvalue()
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
def estimate_csv_size(df: pd.DataFrame) -> str:
|
| 64 |
+
"""Estimate CSV file size as a human-readable string."""
|
| 65 |
+
# Rough estimate: 50 bytes per cell
|
| 66 |
+
cells = df.shape[0] * df.shape[1]
|
| 67 |
+
bytes_est = cells * 50
|
| 68 |
+
if bytes_est < 1024:
|
| 69 |
+
return f"{bytes_est} B"
|
| 70 |
+
elif bytes_est < 1024 * 1024:
|
| 71 |
+
return f"{bytes_est / 1024:.0f} KB"
|
| 72 |
+
else:
|
| 73 |
+
return f"{bytes_est / (1024 * 1024):.1f} MB"
|
core/timezone.py
ADDED
|
@@ -0,0 +1,100 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Timezone utilities for CSH2 Web Dashboard.
|
| 3 |
+
|
| 4 |
+
The PLC logs in US/Eastern. The database has two time columns:
|
| 5 |
+
- dateandtime: Eastern local time (no timezone info)
|
| 6 |
+
- utc_full_timestamp: real UTC (timestamptz)
|
| 7 |
+
|
| 8 |
+
The cycle_periods table stores Eastern times incorrectly labeled as UTC.
|
| 9 |
+
This module provides consistent conversion and display helpers.
|
| 10 |
+
"""
|
| 11 |
+
from datetime import datetime, timedelta, timezone
|
| 12 |
+
from zoneinfo import ZoneInfo
|
| 13 |
+
import pandas as pd
|
| 14 |
+
from core.config import DISPLAY_TZ, DISPLAY_TZ_LABEL, CYCLE_PERIODS_UTC_OFFSET
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
def to_eastern(dt) -> datetime:
|
| 18 |
+
"""Convert a UTC datetime to Eastern time for display.
|
| 19 |
+
|
| 20 |
+
Accepts timezone-aware (UTC) or naive (assumed UTC) datetimes.
|
| 21 |
+
Returns a timezone-aware datetime in US/Eastern.
|
| 22 |
+
"""
|
| 23 |
+
if dt is None:
|
| 24 |
+
return None
|
| 25 |
+
if isinstance(dt, pd.Timestamp):
|
| 26 |
+
dt = dt.to_pydatetime()
|
| 27 |
+
if dt.tzinfo is None:
|
| 28 |
+
dt = dt.replace(tzinfo=timezone.utc)
|
| 29 |
+
return dt.astimezone(DISPLAY_TZ)
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def to_utc(dt) -> datetime:
|
| 33 |
+
"""Convert an Eastern datetime to UTC for database queries.
|
| 34 |
+
|
| 35 |
+
Accepts timezone-aware (Eastern) or naive (assumed Eastern) datetimes.
|
| 36 |
+
Returns a timezone-aware datetime in UTC.
|
| 37 |
+
"""
|
| 38 |
+
if dt is None:
|
| 39 |
+
return None
|
| 40 |
+
if isinstance(dt, pd.Timestamp):
|
| 41 |
+
dt = dt.to_pydatetime()
|
| 42 |
+
if dt.tzinfo is None:
|
| 43 |
+
dt = dt.replace(tzinfo=DISPLAY_TZ)
|
| 44 |
+
return dt.astimezone(timezone.utc)
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def cycle_periods_to_utc(dt) -> datetime:
|
| 48 |
+
"""Convert a cycle_periods timestamp (Eastern mislabeled as UTC) to real UTC.
|
| 49 |
+
|
| 50 |
+
The cycle_periods table stores Eastern local times with a +00:00 suffix.
|
| 51 |
+
Adding 4 hours converts to real UTC for sensor data queries.
|
| 52 |
+
"""
|
| 53 |
+
if dt is None:
|
| 54 |
+
return None
|
| 55 |
+
if isinstance(dt, pd.Timestamp):
|
| 56 |
+
dt = dt.to_pydatetime()
|
| 57 |
+
return dt + CYCLE_PERIODS_UTC_OFFSET
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
def cycle_periods_to_eastern(dt) -> datetime:
|
| 61 |
+
"""Convert a cycle_periods timestamp to a properly labeled Eastern datetime.
|
| 62 |
+
|
| 63 |
+
The stored values ARE Eastern — just strip the wrong UTC label and apply
|
| 64 |
+
the correct US/Eastern timezone.
|
| 65 |
+
"""
|
| 66 |
+
if dt is None:
|
| 67 |
+
return None
|
| 68 |
+
if isinstance(dt, pd.Timestamp):
|
| 69 |
+
dt = dt.to_pydatetime()
|
| 70 |
+
# Strip the incorrect UTC tzinfo, replace with Eastern
|
| 71 |
+
naive = dt.replace(tzinfo=None)
|
| 72 |
+
return naive.replace(tzinfo=DISPLAY_TZ)
|
| 73 |
+
|
| 74 |
+
|
| 75 |
+
def format_et(dt, fmt: str = '%b %d %H:%M') -> str:
|
| 76 |
+
"""Format a datetime for display in Eastern time.
|
| 77 |
+
|
| 78 |
+
If the datetime is UTC, converts to Eastern first.
|
| 79 |
+
Appends the timezone label.
|
| 80 |
+
"""
|
| 81 |
+
if dt is None:
|
| 82 |
+
return "—"
|
| 83 |
+
eastern = to_eastern(dt) if (dt.tzinfo and dt.tzinfo != DISPLAY_TZ) else dt
|
| 84 |
+
return f"{eastern.strftime(fmt)} {DISPLAY_TZ_LABEL}"
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
def convert_df_timestamps_to_eastern(df: pd.DataFrame, col: str = 'timestamp') -> pd.DataFrame:
|
| 88 |
+
"""Convert a DataFrame's timestamp column from UTC to Eastern for display.
|
| 89 |
+
|
| 90 |
+
Returns a new DataFrame (does not modify in place).
|
| 91 |
+
"""
|
| 92 |
+
if col not in df.columns:
|
| 93 |
+
return df
|
| 94 |
+
df = df.copy()
|
| 95 |
+
if df[col].dt.tz is not None:
|
| 96 |
+
df[col] = df[col].dt.tz_convert(DISPLAY_TZ)
|
| 97 |
+
else:
|
| 98 |
+
# Naive timestamps assumed UTC
|
| 99 |
+
df[col] = df[col].dt.tz_localize('UTC').dt.tz_convert(DISPLAY_TZ)
|
| 100 |
+
return df
|
pages/1_data_explorer.py
ADDED
|
@@ -0,0 +1,236 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Feature 1: Data Explorer - Pivot data view with interactive charts
|
| 3 |
+
Now with Natural Language query support powered by Claude
|
| 4 |
+
"""
|
| 5 |
+
import streamlit as st
|
| 6 |
+
import pandas as pd
|
| 7 |
+
from core.db_connector import get_db_connector
|
| 8 |
+
from core.cached_queries import cached_get_pivot_data
|
| 9 |
+
from analysis.nl2sql import NLQueryParser
|
| 10 |
+
from ui.components import page_header, tag_multiselect, date_range_picker, no_data_message
|
| 11 |
+
from ui.plotly_charts import create_timeseries_chart
|
| 12 |
+
from core.timezone import to_utc, convert_df_timestamps_to_eastern
|
| 13 |
+
|
| 14 |
+
page_header(
|
| 15 |
+
"Data Explorer",
|
| 16 |
+
"View sensor data in wide format with tags as columns. Use natural language or the sidebar controls to query."
|
| 17 |
+
)
|
| 18 |
+
|
| 19 |
+
db = get_db_connector()
|
| 20 |
+
nl_parser = NLQueryParser()
|
| 21 |
+
|
| 22 |
+
# ── Natural Language Query Section ──
|
| 23 |
+
st.subheader("Ask in Natural Language")
|
| 24 |
+
|
| 25 |
+
nl_col1, nl_col2 = st.columns([5, 1])
|
| 26 |
+
with nl_col1:
|
| 27 |
+
nl_query = st.text_input(
|
| 28 |
+
"Describe what data you want",
|
| 29 |
+
placeholder='e.g. "show me pressure and temperature from Sept 25 between 10am and 2pm"',
|
| 30 |
+
key="nl_query_input",
|
| 31 |
+
label_visibility="collapsed",
|
| 32 |
+
)
|
| 33 |
+
with nl_col2:
|
| 34 |
+
nl_btn = st.button("Query", type="primary", use_container_width=True, key="nl_query_btn")
|
| 35 |
+
|
| 36 |
+
# Process NL query
|
| 37 |
+
nl_parsed = None
|
| 38 |
+
if nl_btn and nl_query:
|
| 39 |
+
if not nl_parser.api_available:
|
| 40 |
+
st.error("Claude API key not configured. Add ANTHROPIC_API_KEY to your .env file for natural language queries.")
|
| 41 |
+
else:
|
| 42 |
+
with st.spinner("Interpreting your query..."):
|
| 43 |
+
all_sensors = db.get_all_tag_names()
|
| 44 |
+
nl_parsed = nl_parser.parse(nl_query, all_sensors)
|
| 45 |
+
|
| 46 |
+
if nl_parsed and 'error' in nl_parsed:
|
| 47 |
+
st.error(f"Could not parse query: {nl_parsed['error']}")
|
| 48 |
+
nl_parsed = None
|
| 49 |
+
elif nl_parsed:
|
| 50 |
+
# Store parsed result in session state for data fetch
|
| 51 |
+
st.session_state['nl_parsed'] = nl_parsed
|
| 52 |
+
else:
|
| 53 |
+
st.error("Could not understand the query. Try being more specific about sensors and time range.")
|
| 54 |
+
|
| 55 |
+
# Check for stored NL result (persists across reruns)
|
| 56 |
+
if 'nl_parsed' in st.session_state and st.session_state['nl_parsed']:
|
| 57 |
+
nl_parsed = st.session_state['nl_parsed']
|
| 58 |
+
|
| 59 |
+
# Show parsed interpretation
|
| 60 |
+
if nl_parsed and 'error' not in nl_parsed:
|
| 61 |
+
with st.container():
|
| 62 |
+
st.success(f"**Parsed:** {nl_parsed.get('explanation', '')}")
|
| 63 |
+
pcol1, pcol2, pcol3 = st.columns(3)
|
| 64 |
+
pcol1.markdown(f"**Sensors:** {', '.join(nl_parsed['sensors'])}")
|
| 65 |
+
pcol2.markdown(f"**From:** {nl_parsed['start_time'].strftime('%b %d, %Y %H:%M')}")
|
| 66 |
+
pcol3.markdown(f"**To:** {nl_parsed['end_time'].strftime('%b %d, %Y %H:%M')}")
|
| 67 |
+
|
| 68 |
+
col_fetch, col_clear = st.columns([1, 1])
|
| 69 |
+
with col_fetch:
|
| 70 |
+
nl_fetch_btn = st.button("Fetch Data", type="primary", use_container_width=True, key="nl_fetch_btn")
|
| 71 |
+
with col_clear:
|
| 72 |
+
nl_clear_btn = st.button("Clear", use_container_width=True, key="nl_clear_btn")
|
| 73 |
+
|
| 74 |
+
if nl_clear_btn:
|
| 75 |
+
del st.session_state['nl_parsed']
|
| 76 |
+
st.rerun()
|
| 77 |
+
|
| 78 |
+
if nl_fetch_btn:
|
| 79 |
+
selected_tags = nl_parsed['sensors']
|
| 80 |
+
# NL parser returns naive datetimes — treat as Eastern, convert to UTC for queries
|
| 81 |
+
start_dt = to_utc(nl_parsed['start_time'])
|
| 82 |
+
end_dt = to_utc(nl_parsed['end_time'])
|
| 83 |
+
|
| 84 |
+
duration_hours = (end_dt - start_dt).total_seconds() / 3600
|
| 85 |
+
table_used = db._select_table(start_dt, end_dt)
|
| 86 |
+
table_label = {
|
| 87 |
+
'procdatafloattable': 'Raw (10Hz)',
|
| 88 |
+
'procdatafloattable_utc_1sec': '1-second aggregates',
|
| 89 |
+
'procdatafloattable_utc_15sec': '15-second aggregates',
|
| 90 |
+
}.get(table_used, table_used)
|
| 91 |
+
|
| 92 |
+
with st.spinner(f"Querying {len(selected_tags)} sensors ({duration_hours:.1f} hours) from {table_label}..."):
|
| 93 |
+
df_pivot = cached_get_pivot_data(tuple(selected_tags), start_dt, end_dt)
|
| 94 |
+
|
| 95 |
+
if df_pivot.empty:
|
| 96 |
+
no_data_message()
|
| 97 |
+
else:
|
| 98 |
+
# Convert timestamps to Eastern for display
|
| 99 |
+
df_pivot = convert_df_timestamps_to_eastern(df_pivot)
|
| 100 |
+
|
| 101 |
+
# Summary metrics
|
| 102 |
+
cols = st.columns(4)
|
| 103 |
+
cols[0].metric("Rows", f"{len(df_pivot):,}")
|
| 104 |
+
cols[1].metric("Sensors", len(selected_tags))
|
| 105 |
+
cols[2].metric("Duration", f"{duration_hours:.1f} hrs")
|
| 106 |
+
cols[3].metric("Table", table_label)
|
| 107 |
+
|
| 108 |
+
st.divider()
|
| 109 |
+
|
| 110 |
+
# Interactive chart
|
| 111 |
+
tags_in_data = [t for t in selected_tags if t in df_pivot.columns]
|
| 112 |
+
if tags_in_data:
|
| 113 |
+
chart_title = f"{', '.join(tags_in_data[:3])}{'...' if len(tags_in_data) > 3 else ''}"
|
| 114 |
+
fig = create_timeseries_chart(df_pivot, tags_in_data, title=chart_title)
|
| 115 |
+
st.plotly_chart(fig, use_container_width=True)
|
| 116 |
+
|
| 117 |
+
st.divider()
|
| 118 |
+
|
| 119 |
+
# Summary statistics
|
| 120 |
+
with st.expander("Summary Statistics", expanded=True):
|
| 121 |
+
stats_data = []
|
| 122 |
+
for tag in tags_in_data:
|
| 123 |
+
col_data = df_pivot[tag].dropna()
|
| 124 |
+
if len(col_data) > 0:
|
| 125 |
+
stats_data.append({
|
| 126 |
+
'Sensor': tag,
|
| 127 |
+
'Count': len(col_data),
|
| 128 |
+
'Min': f"{col_data.min():.4f}",
|
| 129 |
+
'Max': f"{col_data.max():.4f}",
|
| 130 |
+
'Mean': f"{col_data.mean():.4f}",
|
| 131 |
+
'Std': f"{col_data.std():.4f}",
|
| 132 |
+
})
|
| 133 |
+
if stats_data:
|
| 134 |
+
st.dataframe(pd.DataFrame(stats_data), use_container_width=True, hide_index=True)
|
| 135 |
+
|
| 136 |
+
# Data table
|
| 137 |
+
with st.expander("Data Table", expanded=False):
|
| 138 |
+
st.dataframe(df_pivot, use_container_width=True, height=400)
|
| 139 |
+
|
| 140 |
+
# CSV download
|
| 141 |
+
csv = df_pivot.to_csv(index=False)
|
| 142 |
+
st.download_button(
|
| 143 |
+
label="Download CSV",
|
| 144 |
+
data=csv,
|
| 145 |
+
file_name=f"csh2_data_{start_dt.strftime('%Y%m%d_%H%M')}_{end_dt.strftime('%Y%m%d_%H%M')}.csv",
|
| 146 |
+
mime="text/csv",
|
| 147 |
+
)
|
| 148 |
+
|
| 149 |
+
st.divider()
|
| 150 |
+
|
| 151 |
+
st.caption("Or use the sidebar controls for manual selection:")
|
| 152 |
+
|
| 153 |
+
# ── Sidebar controls (manual mode) ──
|
| 154 |
+
with st.sidebar:
|
| 155 |
+
st.subheader("Manual Query")
|
| 156 |
+
selected_tags = tag_multiselect(db, key="explorer")
|
| 157 |
+
st.divider()
|
| 158 |
+
start_dt, end_dt = date_range_picker(key="explorer")
|
| 159 |
+
|
| 160 |
+
query_btn = st.button("Fetch Data", type="primary", use_container_width=True, key="manual_fetch_btn")
|
| 161 |
+
|
| 162 |
+
# Main content (manual mode)
|
| 163 |
+
if query_btn and selected_tags:
|
| 164 |
+
# Manual date picker returns naive datetimes — treat as Eastern, convert to UTC for queries
|
| 165 |
+
start_dt_utc = to_utc(start_dt)
|
| 166 |
+
end_dt_utc = to_utc(end_dt)
|
| 167 |
+
|
| 168 |
+
if start_dt >= end_dt:
|
| 169 |
+
st.error("Start time must be before end time.")
|
| 170 |
+
else:
|
| 171 |
+
duration_hours = (end_dt_utc - start_dt_utc).total_seconds() / 3600
|
| 172 |
+
table_used = db._select_table(start_dt_utc, end_dt_utc)
|
| 173 |
+
table_label = {
|
| 174 |
+
'procdatafloattable': 'Raw (10Hz)',
|
| 175 |
+
'procdatafloattable_utc_1sec': '1-second aggregates',
|
| 176 |
+
'procdatafloattable_utc_15sec': '15-second aggregates',
|
| 177 |
+
}.get(table_used, table_used)
|
| 178 |
+
|
| 179 |
+
with st.spinner(f"Querying {len(selected_tags)} sensors ({duration_hours:.1f} hours) from {table_label}..."):
|
| 180 |
+
df_pivot = cached_get_pivot_data(tuple(selected_tags), start_dt_utc, end_dt_utc)
|
| 181 |
+
|
| 182 |
+
if df_pivot.empty:
|
| 183 |
+
no_data_message()
|
| 184 |
+
else:
|
| 185 |
+
# Convert timestamps to Eastern for display
|
| 186 |
+
df_pivot = convert_df_timestamps_to_eastern(df_pivot)
|
| 187 |
+
# Summary metrics
|
| 188 |
+
cols = st.columns(4)
|
| 189 |
+
cols[0].metric("Rows", f"{len(df_pivot):,}")
|
| 190 |
+
cols[1].metric("Sensors", len(selected_tags))
|
| 191 |
+
cols[2].metric("Duration", f"{duration_hours:.1f} hrs")
|
| 192 |
+
cols[3].metric("Table", table_label)
|
| 193 |
+
|
| 194 |
+
st.divider()
|
| 195 |
+
|
| 196 |
+
# Interactive chart
|
| 197 |
+
tags_in_data = [t for t in selected_tags if t in df_pivot.columns]
|
| 198 |
+
if tags_in_data:
|
| 199 |
+
chart_title = f"{', '.join(tags_in_data[:3])}{'...' if len(tags_in_data) > 3 else ''}"
|
| 200 |
+
fig = create_timeseries_chart(df_pivot, tags_in_data, title=chart_title)
|
| 201 |
+
st.plotly_chart(fig, use_container_width=True)
|
| 202 |
+
|
| 203 |
+
st.divider()
|
| 204 |
+
|
| 205 |
+
# Summary statistics
|
| 206 |
+
with st.expander("Summary Statistics", expanded=True):
|
| 207 |
+
stats_data = []
|
| 208 |
+
for tag in tags_in_data:
|
| 209 |
+
col_data = df_pivot[tag].dropna()
|
| 210 |
+
if len(col_data) > 0:
|
| 211 |
+
stats_data.append({
|
| 212 |
+
'Sensor': tag,
|
| 213 |
+
'Count': len(col_data),
|
| 214 |
+
'Min': f"{col_data.min():.4f}",
|
| 215 |
+
'Max': f"{col_data.max():.4f}",
|
| 216 |
+
'Mean': f"{col_data.mean():.4f}",
|
| 217 |
+
'Std': f"{col_data.std():.4f}",
|
| 218 |
+
})
|
| 219 |
+
if stats_data:
|
| 220 |
+
st.dataframe(pd.DataFrame(stats_data), use_container_width=True, hide_index=True)
|
| 221 |
+
|
| 222 |
+
# Data table
|
| 223 |
+
with st.expander("Data Table", expanded=False):
|
| 224 |
+
st.dataframe(df_pivot, use_container_width=True, height=400)
|
| 225 |
+
|
| 226 |
+
# CSV download
|
| 227 |
+
csv = df_pivot.to_csv(index=False)
|
| 228 |
+
st.download_button(
|
| 229 |
+
label="Download CSV",
|
| 230 |
+
data=csv,
|
| 231 |
+
file_name=f"csh2_data_{start_dt.strftime('%Y%m%d_%H%M')}_{end_dt.strftime('%Y%m%d_%H%M')}.csv",
|
| 232 |
+
mime="text/csv",
|
| 233 |
+
)
|
| 234 |
+
|
| 235 |
+
elif query_btn and not selected_tags:
|
| 236 |
+
st.warning("Please select at least one sensor from the sidebar.")
|
pages/2_pump_cycles.py
ADDED
|
@@ -0,0 +1,117 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Feature 2: Testing Cycle Browser
|
| 3 |
+
Month-year filter with pre-computed cycles from the cycle_periods DB table.
|
| 4 |
+
"""
|
| 5 |
+
import streamlit as st
|
| 6 |
+
import pandas as pd
|
| 7 |
+
from core.db_connector import get_db_connector
|
| 8 |
+
from ui.components import page_header, metric_row
|
| 9 |
+
from ui.plotly_charts import create_cycle_timeline
|
| 10 |
+
|
| 11 |
+
page_header(
|
| 12 |
+
"Testing Cycles",
|
| 13 |
+
"Cycles are identified from motor activity — >0% starts a cycle and returning to 0% ends a cycle. "
|
| 14 |
+
"To qualify, a testing cycle must show either >5% motor speed OR >10 sec duration observed."
|
| 15 |
+
)
|
| 16 |
+
|
| 17 |
+
db = get_db_connector()
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
@st.cache_data(ttl=3600)
|
| 21 |
+
def get_available_months():
|
| 22 |
+
return db.get_available_months()
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
@st.cache_data(ttl=3600)
|
| 26 |
+
def get_cycle_months():
|
| 27 |
+
return db.get_cycle_months()
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
@st.cache_data(ttl=600)
|
| 31 |
+
def load_cycles_for_month(month_str: str):
|
| 32 |
+
"""Load pre-computed cycles from cycle_periods table for a given month."""
|
| 33 |
+
return db.get_cycle_periods(month_str)
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
@st.cache_data(ttl=600)
|
| 37 |
+
def load_all_cycles():
|
| 38 |
+
"""Load all pre-computed cycles (for cross-page use)."""
|
| 39 |
+
return db.get_cycle_periods()
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
# Sidebar: month selector
|
| 43 |
+
with st.sidebar:
|
| 44 |
+
st.subheader("Select Period")
|
| 45 |
+
sensor_months = get_available_months()
|
| 46 |
+
cycle_months = get_cycle_months()
|
| 47 |
+
months = sorted(set(sensor_months) | set(cycle_months))
|
| 48 |
+
|
| 49 |
+
if not months:
|
| 50 |
+
st.error("No data months found in database.")
|
| 51 |
+
st.stop()
|
| 52 |
+
|
| 53 |
+
selected_month = st.selectbox(
|
| 54 |
+
"Month",
|
| 55 |
+
months,
|
| 56 |
+
index=len(months) - 1, # Default to most recent
|
| 57 |
+
key="cycle_month",
|
| 58 |
+
)
|
| 59 |
+
|
| 60 |
+
# Load cycles for selected month
|
| 61 |
+
cycles = load_cycles_for_month(selected_month)
|
| 62 |
+
|
| 63 |
+
# Also cache all cycles for the comparison page
|
| 64 |
+
all_cycles = load_all_cycles()
|
| 65 |
+
st.session_state.all_cycles = all_cycles
|
| 66 |
+
|
| 67 |
+
if not cycles:
|
| 68 |
+
st.info(f"No testing cycles found in {selected_month}.")
|
| 69 |
+
else:
|
| 70 |
+
# Summary metrics
|
| 71 |
+
total_runtime = sum(c['duration_minutes'] for c in cycles)
|
| 72 |
+
peak_pressures = [c.get('peak_pressure', 0) for c in cycles if c.get('peak_pressure')]
|
| 73 |
+
max_pressure = max(peak_pressures) if peak_pressures else 0
|
| 74 |
+
total_dispensed = sum(c.get('total_kg_dispensed', 0) for c in cycles)
|
| 75 |
+
|
| 76 |
+
metrics = [
|
| 77 |
+
{'label': 'Cycles', 'value': len(cycles), 'unit': ''},
|
| 78 |
+
{'label': 'Total Runtime', 'value': f"{total_runtime:.0f}", 'unit': 'min'},
|
| 79 |
+
{'label': 'Peak Pressure (PT130)', 'value': f"{max_pressure:.0f}", 'unit': 'bar'},
|
| 80 |
+
{'label': 'Total Dispensed', 'value': f"{total_dispensed:.1f}", 'unit': 'kg'},
|
| 81 |
+
]
|
| 82 |
+
metric_row(metrics)
|
| 83 |
+
|
| 84 |
+
st.divider()
|
| 85 |
+
|
| 86 |
+
# Timeline chart
|
| 87 |
+
fig = create_cycle_timeline(cycles, selected_month)
|
| 88 |
+
st.plotly_chart(fig, use_container_width=True)
|
| 89 |
+
|
| 90 |
+
st.divider()
|
| 91 |
+
|
| 92 |
+
# Cycle table
|
| 93 |
+
st.subheader("Cycle Details")
|
| 94 |
+
|
| 95 |
+
table_data = []
|
| 96 |
+
for c in cycles:
|
| 97 |
+
display_start = c.get('start_time_et') or c['start_time']
|
| 98 |
+
display_end = c.get('end_time_et') or c['end_time']
|
| 99 |
+
table_data.append({
|
| 100 |
+
'Cycle': c['cycle_id'],
|
| 101 |
+
'Start (ET)': display_start.strftime('%b %d %H:%M'),
|
| 102 |
+
'End (ET)': display_end.strftime('%b %d %H:%M'),
|
| 103 |
+
'Duration (min)': f"{c['duration_minutes']:.1f}",
|
| 104 |
+
'Peak Speed (RPM)': f"{c.get('peak_speed', 0):.0f}",
|
| 105 |
+
'Peak Pressure (bar)': f"{c.get('peak_pressure', 0):.0f}" if c.get('peak_pressure') else 'N/A',
|
| 106 |
+
'Avg Flow (kg/min)': f"{c.get('avg_flow', 0):.2f}" if c.get('avg_flow') else 'N/A',
|
| 107 |
+
'Dispensed (kg)': f"{c.get('total_kg_dispensed', 0):.2f}" if c.get('total_kg_dispensed') else 'N/A',
|
| 108 |
+
'Pump Strokes': f"{c.get('total_pump_strokes', 0):.0f}" if c.get('total_pump_strokes') else 'N/A',
|
| 109 |
+
})
|
| 110 |
+
|
| 111 |
+
df_table = pd.DataFrame(table_data)
|
| 112 |
+
st.dataframe(df_table, use_container_width=True, hide_index=True)
|
| 113 |
+
|
| 114 |
+
# Store cycles in session state for other pages
|
| 115 |
+
st.session_state.detected_cycles = cycles
|
| 116 |
+
|
| 117 |
+
st.info("Select a cycle above, then navigate to **Cycle Detail** page for in-depth analysis.")
|
pages/3_cycle_detail.py
ADDED
|
@@ -0,0 +1,179 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Feature 3: Cycle Detail View
|
| 3 |
+
KPI cards, multi-axis chart, plateau detection
|
| 4 |
+
"""
|
| 5 |
+
import streamlit as st
|
| 6 |
+
import pandas as pd
|
| 7 |
+
from core.db_connector import get_db_connector
|
| 8 |
+
from core.cached_queries import cached_get_pivot_data
|
| 9 |
+
from core import config
|
| 10 |
+
from analysis.basic_analysis import DataAnalyzer
|
| 11 |
+
from analysis.advanced_analysis import AdvancedAnalyzer
|
| 12 |
+
from ui.components import page_header, metric_row, cycle_selector, no_data_message
|
| 13 |
+
from ui.plotly_charts import create_multi_axis_chart
|
| 14 |
+
from core.timezone import convert_df_timestamps_to_eastern
|
| 15 |
+
|
| 16 |
+
page_header(
|
| 17 |
+
"Cycle Detail",
|
| 18 |
+
"In-depth analysis of a selected testing cycle with KPIs, charts, and plateau detection."
|
| 19 |
+
)
|
| 20 |
+
|
| 21 |
+
db = get_db_connector()
|
| 22 |
+
analyzer = DataAnalyzer()
|
| 23 |
+
adv_analyzer = AdvancedAnalyzer()
|
| 24 |
+
|
| 25 |
+
# Check if cycles are loaded; fallback to DB if needed
|
| 26 |
+
cycles = st.session_state.get('detected_cycles', [])
|
| 27 |
+
|
| 28 |
+
if not cycles:
|
| 29 |
+
# Fallback: load all cycles from cycle_periods table
|
| 30 |
+
all_db_cycles = db.get_cycle_periods()
|
| 31 |
+
if all_db_cycles:
|
| 32 |
+
cycles = all_db_cycles
|
| 33 |
+
st.session_state.detected_cycles = cycles
|
| 34 |
+
st.info("Cycles loaded from database. Visit **Testing Cycles** page to filter by month.")
|
| 35 |
+
else:
|
| 36 |
+
st.warning("No cycles found. Check database connectivity or go to **Testing Cycles** page.")
|
| 37 |
+
st.stop()
|
| 38 |
+
|
| 39 |
+
# Sidebar: cycle selector
|
| 40 |
+
with st.sidebar:
|
| 41 |
+
st.subheader("Select Cycle")
|
| 42 |
+
month = st.session_state.get('cycle_month', '')
|
| 43 |
+
if month:
|
| 44 |
+
st.caption(f"Month: {month}")
|
| 45 |
+
|
| 46 |
+
selected_cycle = cycle_selector(cycles, key_prefix="detail")
|
| 47 |
+
|
| 48 |
+
if selected_cycle is None:
|
| 49 |
+
st.stop()
|
| 50 |
+
|
| 51 |
+
# Fetch cycle data
|
| 52 |
+
st.subheader(f"Cycle {selected_cycle['cycle_id']}")
|
| 53 |
+
|
| 54 |
+
# Determine which tags are available
|
| 55 |
+
detail_tags = [t for t in config.CYCLE_DETAIL_TAGS if db.get_tag_index(t) is not None]
|
| 56 |
+
|
| 57 |
+
with st.spinner("Loading cycle data..."):
|
| 58 |
+
df_pivot = cached_get_pivot_data(
|
| 59 |
+
tuple(detail_tags),
|
| 60 |
+
selected_cycle['start_time'],
|
| 61 |
+
selected_cycle['end_time'],
|
| 62 |
+
)
|
| 63 |
+
|
| 64 |
+
if df_pivot.empty:
|
| 65 |
+
no_data_message()
|
| 66 |
+
st.stop()
|
| 67 |
+
|
| 68 |
+
# Convert timestamps to Eastern for display
|
| 69 |
+
df_pivot = convert_df_timestamps_to_eastern(df_pivot)
|
| 70 |
+
|
| 71 |
+
# Run analysis
|
| 72 |
+
cycle_stats = analyzer.analyze_cycle(df_pivot)
|
| 73 |
+
|
| 74 |
+
# Store for LLM pages
|
| 75 |
+
st.session_state.current_cycle_stats = cycle_stats
|
| 76 |
+
st.session_state.current_cycle = selected_cycle
|
| 77 |
+
st.session_state.current_cycle_df = df_pivot
|
| 78 |
+
|
| 79 |
+
# KPI Cards
|
| 80 |
+
kpi_metrics = []
|
| 81 |
+
|
| 82 |
+
if 'discharge_pressure' in cycle_stats:
|
| 83 |
+
dp = cycle_stats['discharge_pressure']
|
| 84 |
+
kpi_metrics.append({'label': 'Peak Pressure (PT130)', 'value': f"{dp['peak']:.0f}", 'unit': 'bar'})
|
| 85 |
+
|
| 86 |
+
if 'time_range' in cycle_stats:
|
| 87 |
+
tr = cycle_stats['time_range']
|
| 88 |
+
kpi_metrics.append({'label': 'Duration', 'value': f"{tr['duration_minutes']:.0f}", 'unit': 'min'})
|
| 89 |
+
|
| 90 |
+
if 'compression_ratio' in cycle_stats:
|
| 91 |
+
cr = cycle_stats['compression_ratio']
|
| 92 |
+
kpi_metrics.append({'label': 'Compression Ratio', 'value': f"{cr['avg']:.1f}x", 'unit': ''})
|
| 93 |
+
|
| 94 |
+
if 'motor' in cycle_stats:
|
| 95 |
+
m = cycle_stats['motor']
|
| 96 |
+
kpi_metrics.append({'label': 'Peak Motor Speed', 'value': f"{m['peak_speed']:.0f}", 'unit': 'RPM'})
|
| 97 |
+
|
| 98 |
+
if 'flow' in cycle_stats:
|
| 99 |
+
f = cycle_stats['flow']
|
| 100 |
+
kpi_metrics.append({'label': 'Avg Flow', 'value': f"{f['avg_flow']:.2f}", 'unit': 'kg/min'})
|
| 101 |
+
|
| 102 |
+
for key in ['temperature_TT110', 'temperature_TT130']:
|
| 103 |
+
if key in cycle_stats:
|
| 104 |
+
t = cycle_stats[key]
|
| 105 |
+
tag = key.replace('temperature_', '')
|
| 106 |
+
kpi_metrics.append({'label': f'Peak {tag}', 'value': f"{t['peak']:.1f}", 'unit': 'K'})
|
| 107 |
+
|
| 108 |
+
if kpi_metrics:
|
| 109 |
+
# Display in rows of 4
|
| 110 |
+
for i in range(0, len(kpi_metrics), 4):
|
| 111 |
+
metric_row(kpi_metrics[i:i+4])
|
| 112 |
+
|
| 113 |
+
st.divider()
|
| 114 |
+
|
| 115 |
+
# Plateau Detection
|
| 116 |
+
pressure_tags_available = [t for t in ['PT130', 'PT01T', 'PT110'] if t in df_pivot.columns]
|
| 117 |
+
temp_tags_available = [t for t in ['TT110', 'TT130'] if t in df_pivot.columns]
|
| 118 |
+
other_tags_available = [t for t in ['FT140', 'M130_Speed', 'MC130_VFD_Speed', 'AOV140'] if t in df_pivot.columns]
|
| 119 |
+
|
| 120 |
+
all_plateau_tags = pressure_tags_available + temp_tags_available
|
| 121 |
+
|
| 122 |
+
|
| 123 |
+
@st.cache_data(ttl=600)
|
| 124 |
+
def _detect_plateaus(_df_pivot, tags: tuple, tolerance: float, min_duration: float):
|
| 125 |
+
"""Cached plateau detection to avoid repeated CPU-bound ops on rerun"""
|
| 126 |
+
_adv = AdvancedAnalyzer()
|
| 127 |
+
result = {}
|
| 128 |
+
for tag in tags:
|
| 129 |
+
periods = _adv.detect_constant_periods(_df_pivot, tag, tolerance, min_duration)
|
| 130 |
+
if periods:
|
| 131 |
+
result[tag] = periods
|
| 132 |
+
return result
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
plateaus = _detect_plateaus(df_pivot, tuple(all_plateau_tags), 0.02, 1.0)
|
| 136 |
+
|
| 137 |
+
# Store plateaus for LLM page
|
| 138 |
+
st.session_state.current_plateaus = plateaus
|
| 139 |
+
|
| 140 |
+
# Multi-axis chart with plateau highlights
|
| 141 |
+
# Use Eastern times for chart display
|
| 142 |
+
display_start = selected_cycle.get('start_time_et') or selected_cycle['start_time']
|
| 143 |
+
display_end = selected_cycle.get('end_time_et') or selected_cycle['end_time']
|
| 144 |
+
|
| 145 |
+
fig = create_multi_axis_chart(
|
| 146 |
+
df_pivot,
|
| 147 |
+
pressure_tags=pressure_tags_available,
|
| 148 |
+
temp_tags=temp_tags_available,
|
| 149 |
+
other_tags=other_tags_available,
|
| 150 |
+
title=f"Cycle {selected_cycle['cycle_id']} — {display_start.strftime('%b %d %H:%M')} ET",
|
| 151 |
+
height=600,
|
| 152 |
+
plateaus=plateaus,
|
| 153 |
+
time_range=(display_start, display_end),
|
| 154 |
+
)
|
| 155 |
+
st.plotly_chart(fig, use_container_width=True)
|
| 156 |
+
|
| 157 |
+
st.divider()
|
| 158 |
+
|
| 159 |
+
# Plateau tables
|
| 160 |
+
if plateaus:
|
| 161 |
+
st.subheader("Detected Plateaus (Constant Periods)")
|
| 162 |
+
|
| 163 |
+
tabs = st.tabs(list(plateaus.keys()))
|
| 164 |
+
for tab, (tag, periods) in zip(tabs, plateaus.items()):
|
| 165 |
+
with tab:
|
| 166 |
+
rows = []
|
| 167 |
+
for i, p in enumerate(periods, 1):
|
| 168 |
+
rows.append({
|
| 169 |
+
'#': i,
|
| 170 |
+
'Start': p['start'].strftime('%H:%M:%S'),
|
| 171 |
+
'End': p['end'].strftime('%H:%M:%S'),
|
| 172 |
+
'Duration (min)': f"{p['duration_minutes']:.1f}",
|
| 173 |
+
'Value': f"{p['value']:.2f}",
|
| 174 |
+
})
|
| 175 |
+
st.dataframe(pd.DataFrame(rows), use_container_width=True, hide_index=True)
|
| 176 |
+
else:
|
| 177 |
+
st.info("No significant plateaus detected in this cycle.")
|
| 178 |
+
|
| 179 |
+
st.caption("Navigate to **Variation Analysis** for LLM-powered interpretation of this cycle.")
|
pages/4_variation_analysis.py
ADDED
|
@@ -0,0 +1,113 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Feature 4: LLM-Driven Variation Analysis
|
| 3 |
+
Claude explains variations and stabilizations in a testing cycle
|
| 4 |
+
"""
|
| 5 |
+
import streamlit as st
|
| 6 |
+
from core.db_connector import get_db_connector
|
| 7 |
+
from analysis.llm_analyzer import LLMCycleAnalyzer
|
| 8 |
+
from ui.components import page_header, analysis_card, follow_up_section
|
| 9 |
+
|
| 10 |
+
page_header(
|
| 11 |
+
"Variation Analysis",
|
| 12 |
+
"AI-powered interpretation of pressure/temperature variations and stabilizations in a testing cycle."
|
| 13 |
+
)
|
| 14 |
+
|
| 15 |
+
db = get_db_connector()
|
| 16 |
+
llm = LLMCycleAnalyzer()
|
| 17 |
+
|
| 18 |
+
# Check prerequisites
|
| 19 |
+
cycle_stats = st.session_state.get('current_cycle_stats')
|
| 20 |
+
current_cycle = st.session_state.get('current_cycle')
|
| 21 |
+
plateaus = st.session_state.get('current_plateaus', {})
|
| 22 |
+
|
| 23 |
+
if not cycle_stats or not current_cycle:
|
| 24 |
+
st.warning(
|
| 25 |
+
"No cycle selected. Please go to **Testing Cycles** to detect cycles, "
|
| 26 |
+
"then **Cycle Detail** to select and analyze a specific cycle."
|
| 27 |
+
)
|
| 28 |
+
st.stop()
|
| 29 |
+
|
| 30 |
+
# Header info
|
| 31 |
+
st.subheader(f"Cycle {current_cycle['cycle_id']} - {current_cycle['start_time'].strftime('%b %d %H:%M')}")
|
| 32 |
+
|
| 33 |
+
col1, col2, col3 = st.columns(3)
|
| 34 |
+
col1.metric("Duration", f"{cycle_stats['time_range']['duration_minutes']:.0f} min")
|
| 35 |
+
|
| 36 |
+
if 'discharge_pressure' in cycle_stats:
|
| 37 |
+
col2.metric("Peak PT130", f"{cycle_stats['discharge_pressure']['peak']:.0f} bar")
|
| 38 |
+
|
| 39 |
+
plateau_count = sum(len(p) for p in plateaus.values())
|
| 40 |
+
col3.metric("Plateaus", plateau_count)
|
| 41 |
+
|
| 42 |
+
st.divider()
|
| 43 |
+
|
| 44 |
+
# LLM availability check
|
| 45 |
+
if not llm.api_available:
|
| 46 |
+
st.error("Claude API key not configured. Add ANTHROPIC_API_KEY to your .env file to enable AI analysis.")
|
| 47 |
+
st.stop()
|
| 48 |
+
|
| 49 |
+
# Check for cached result
|
| 50 |
+
cache_key = f"variation_{current_cycle['cycle_id']}_{current_cycle['start_time']}"
|
| 51 |
+
prompt_key = f"{cache_key}_prompt"
|
| 52 |
+
followup_key = f"{cache_key}_followups"
|
| 53 |
+
cached_result = st.session_state.get(cache_key)
|
| 54 |
+
|
| 55 |
+
if cached_result:
|
| 56 |
+
analysis_card("AI Analysis", cached_result)
|
| 57 |
+
|
| 58 |
+
# Follow-up section
|
| 59 |
+
original_prompt = st.session_state.get(prompt_key, "")
|
| 60 |
+
follow_up_section(
|
| 61 |
+
session_key=followup_key,
|
| 62 |
+
llm_analyzer=llm,
|
| 63 |
+
original_prompt=original_prompt,
|
| 64 |
+
original_analysis=cached_result,
|
| 65 |
+
)
|
| 66 |
+
|
| 67 |
+
if st.button("Re-analyze", key="reanalyze_variation"):
|
| 68 |
+
del st.session_state[cache_key]
|
| 69 |
+
st.session_state.pop(prompt_key, None)
|
| 70 |
+
st.session_state.pop(followup_key, None)
|
| 71 |
+
st.rerun()
|
| 72 |
+
else:
|
| 73 |
+
st.info("Click below to generate an AI-powered analysis of this cycle's variations and stabilizations.")
|
| 74 |
+
|
| 75 |
+
if st.button("Analyze Cycle", type="primary", use_container_width=True):
|
| 76 |
+
with st.spinner("Claude is analyzing the cycle data..."):
|
| 77 |
+
result = llm.analyze_cycle_variations(
|
| 78 |
+
cycle_stats=cycle_stats,
|
| 79 |
+
plateaus=plateaus,
|
| 80 |
+
)
|
| 81 |
+
|
| 82 |
+
# Store the prompt that was used (for follow-up context)
|
| 83 |
+
st.session_state[prompt_key] = llm._build_variation_prompt(
|
| 84 |
+
cycle_stats, plateaus, None,
|
| 85 |
+
llm.retriever.format_for_prompt(
|
| 86 |
+
llm.retriever.get_context_for_cycle(cycle_stats, plateaus)
|
| 87 |
+
) if llm.retriever else "",
|
| 88 |
+
)
|
| 89 |
+
st.session_state[cache_key] = result
|
| 90 |
+
st.rerun()
|
| 91 |
+
|
| 92 |
+
# Show input data summary
|
| 93 |
+
with st.expander("Data sent to AI", expanded=False):
|
| 94 |
+
st.json({
|
| 95 |
+
'time_range': {
|
| 96 |
+
'start': str(cycle_stats['time_range']['start']),
|
| 97 |
+
'end': str(cycle_stats['time_range']['end']),
|
| 98 |
+
'duration_minutes': cycle_stats['time_range']['duration_minutes'],
|
| 99 |
+
},
|
| 100 |
+
'discharge_pressure': cycle_stats.get('discharge_pressure'),
|
| 101 |
+
'compression_ratio': cycle_stats.get('compression_ratio'),
|
| 102 |
+
'flow': cycle_stats.get('flow'),
|
| 103 |
+
'ramp_rate': cycle_stats.get('ramp_rate'),
|
| 104 |
+
'motor': cycle_stats.get('motor'),
|
| 105 |
+
'plateaus': {
|
| 106 |
+
tag: [
|
| 107 |
+
{'value': p['value'], 'duration_min': p['duration_minutes']}
|
| 108 |
+
for p in periods
|
| 109 |
+
]
|
| 110 |
+
for tag, periods in plateaus.items()
|
| 111 |
+
},
|
| 112 |
+
'performance_vs_targets': cycle_stats.get('performance_vs_targets'),
|
| 113 |
+
})
|
pages/5_cycle_comparison.py
ADDED
|
@@ -0,0 +1,245 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Feature 5: LLM-Driven Cycle Comparison
|
| 3 |
+
Side-by-side comparison of two testing cycles with AI narrative.
|
| 4 |
+
Supports cross-month comparison — cycles loaded directly from cycle_periods DB table.
|
| 5 |
+
"""
|
| 6 |
+
import streamlit as st
|
| 7 |
+
import pandas as pd
|
| 8 |
+
from core.db_connector import get_db_connector
|
| 9 |
+
from core import config
|
| 10 |
+
from analysis.basic_analysis import DataAnalyzer
|
| 11 |
+
from analysis.llm_analyzer import LLMCycleAnalyzer
|
| 12 |
+
from ui.components import page_header, cycle_selector, metric_row, analysis_card, follow_up_section
|
| 13 |
+
from ui.plotly_charts import create_comparison_chart
|
| 14 |
+
from core.timezone import convert_df_timestamps_to_eastern
|
| 15 |
+
|
| 16 |
+
page_header(
|
| 17 |
+
"Cycle Comparison",
|
| 18 |
+
"Compare two testing cycles side-by-side with AI-powered analysis of differences."
|
| 19 |
+
)
|
| 20 |
+
|
| 21 |
+
db = get_db_connector()
|
| 22 |
+
analyzer = DataAnalyzer()
|
| 23 |
+
llm = LLMCycleAnalyzer()
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
# ── Load all cycles directly from DB ────────────────────────────
|
| 27 |
+
|
| 28 |
+
@st.cache_data(ttl=600)
|
| 29 |
+
def load_all_cycles():
|
| 30 |
+
return db.get_cycle_periods()
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
@st.cache_data(ttl=3600)
|
| 34 |
+
def get_available_months():
|
| 35 |
+
return db.get_available_months()
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
@st.cache_data(ttl=3600)
|
| 39 |
+
def get_cycle_months():
|
| 40 |
+
return db.get_cycle_months()
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
all_cycles = load_all_cycles()
|
| 44 |
+
sensor_months = get_available_months()
|
| 45 |
+
cycle_months = get_cycle_months()
|
| 46 |
+
available_months = sorted(set(sensor_months) | set(cycle_months))
|
| 47 |
+
|
| 48 |
+
if len(all_cycles) < 2:
|
| 49 |
+
st.warning("Need at least 2 cycles in the database for comparison.")
|
| 50 |
+
st.stop()
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
# ── Cycle Selection with Independent Month Filters ──────────────
|
| 54 |
+
|
| 55 |
+
@st.cache_data(ttl=600)
|
| 56 |
+
def load_cycle_data(start_time, end_time):
|
| 57 |
+
"""Load and analyze a cycle's data"""
|
| 58 |
+
detail_tags = [t for t in config.CYCLE_DETAIL_TAGS if db.get_tag_index(t) is not None]
|
| 59 |
+
df_pivot = db.get_pivot_data(detail_tags, start_time, end_time)
|
| 60 |
+
if df_pivot.empty:
|
| 61 |
+
return None, None
|
| 62 |
+
# Convert timestamps to Eastern for display
|
| 63 |
+
df_pivot = convert_df_timestamps_to_eastern(df_pivot)
|
| 64 |
+
stats = analyzer.analyze_cycle(df_pivot)
|
| 65 |
+
return df_pivot, stats
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
month_options = ["All Months"] + available_months
|
| 69 |
+
|
| 70 |
+
col_a, col_b = st.columns(2)
|
| 71 |
+
|
| 72 |
+
with col_a:
|
| 73 |
+
st.subheader("Cycle A")
|
| 74 |
+
month_a = st.selectbox("Filter by month", month_options, key="comp_month_a")
|
| 75 |
+
if month_a == "All Months":
|
| 76 |
+
filtered_a = all_cycles
|
| 77 |
+
else:
|
| 78 |
+
filtered_a = [c for c in all_cycles if (c.get('start_time_et') or c['start_time']).strftime('%Y-%m') == month_a]
|
| 79 |
+
cycle_a = cycle_selector(filtered_a, key_prefix="comp_a")
|
| 80 |
+
|
| 81 |
+
with col_b:
|
| 82 |
+
st.subheader("Cycle B")
|
| 83 |
+
month_b = st.selectbox("Filter by month", month_options, key="comp_month_b")
|
| 84 |
+
if month_b == "All Months":
|
| 85 |
+
filtered_b = all_cycles
|
| 86 |
+
else:
|
| 87 |
+
filtered_b = [c for c in all_cycles if (c.get('start_time_et') or c['start_time']).strftime('%Y-%m') == month_b]
|
| 88 |
+
cycle_b = cycle_selector(filtered_b, key_prefix="comp_b")
|
| 89 |
+
|
| 90 |
+
if cycle_a is None or cycle_b is None:
|
| 91 |
+
st.stop()
|
| 92 |
+
|
| 93 |
+
if cycle_a['cycle_id'] == cycle_b['cycle_id']:
|
| 94 |
+
st.warning("Please select two different cycles to compare.")
|
| 95 |
+
st.stop()
|
| 96 |
+
|
| 97 |
+
# Load data for both cycles
|
| 98 |
+
with st.spinner("Loading cycle data..."):
|
| 99 |
+
df_a, stats_a = load_cycle_data(cycle_a['start_time'], cycle_a['end_time'])
|
| 100 |
+
df_b, stats_b = load_cycle_data(cycle_b['start_time'], cycle_b['end_time'])
|
| 101 |
+
|
| 102 |
+
if df_a is None or df_b is None:
|
| 103 |
+
st.error("Could not load data for one or both cycles.")
|
| 104 |
+
st.stop()
|
| 105 |
+
|
| 106 |
+
st.divider()
|
| 107 |
+
|
| 108 |
+
# Side-by-side metrics
|
| 109 |
+
st.subheader("Key Metrics Comparison")
|
| 110 |
+
|
| 111 |
+
metrics_table = []
|
| 112 |
+
|
| 113 |
+
|
| 114 |
+
def add_metric(label, key_path_a, key_path_b, fmt=".1f", unit=""):
|
| 115 |
+
"""Helper to extract nested dict values for comparison"""
|
| 116 |
+
val_a = stats_a
|
| 117 |
+
val_b = stats_b
|
| 118 |
+
for key in key_path_a:
|
| 119 |
+
val_a = val_a.get(key, {}) if isinstance(val_a, dict) else None
|
| 120 |
+
for key in key_path_b:
|
| 121 |
+
val_b = val_b.get(key, {}) if isinstance(val_b, dict) else None
|
| 122 |
+
|
| 123 |
+
# Ensure final values are numeric, not residual dicts from missing keys
|
| 124 |
+
if not isinstance(val_a, (int, float)):
|
| 125 |
+
val_a = None
|
| 126 |
+
if not isinstance(val_b, (int, float)):
|
| 127 |
+
val_b = None
|
| 128 |
+
|
| 129 |
+
if val_a is not None and val_b is not None:
|
| 130 |
+
diff = val_a - val_b
|
| 131 |
+
metrics_table.append({
|
| 132 |
+
'Metric': label,
|
| 133 |
+
f'Cycle {cycle_a["cycle_id"]}': f"{val_a:{fmt}} {unit}".strip(),
|
| 134 |
+
f'Cycle {cycle_b["cycle_id"]}': f"{val_b:{fmt}} {unit}".strip(),
|
| 135 |
+
'Difference': f"{diff:+{fmt}} {unit}".strip(),
|
| 136 |
+
})
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
add_metric("Duration (min)", ['time_range', 'duration_minutes'], ['time_range', 'duration_minutes'], ".0f", "min")
|
| 140 |
+
add_metric("Peak Pressure", ['discharge_pressure', 'peak'], ['discharge_pressure', 'peak'], ".0f", "bar")
|
| 141 |
+
add_metric("Avg Pressure", ['discharge_pressure', 'avg'], ['discharge_pressure', 'avg'], ".1f", "bar")
|
| 142 |
+
add_metric("Compression Ratio (avg)", ['compression_ratio', 'avg'], ['compression_ratio', 'avg'], ".1f", "x")
|
| 143 |
+
add_metric("Avg Flow", ['flow', 'avg_flow'], ['flow', 'avg_flow'], ".2f", "kg/min")
|
| 144 |
+
add_metric("Peak TT110", ['temperature_TT110', 'peak'], ['temperature_TT110', 'peak'], ".1f", "K")
|
| 145 |
+
add_metric("Peak TT130", ['temperature_TT130', 'peak'], ['temperature_TT130', 'peak'], ".1f", "K")
|
| 146 |
+
|
| 147 |
+
if 'motor' in stats_a and 'motor' in stats_b:
|
| 148 |
+
add_metric("Peak Motor Speed", ['motor', 'peak_speed'], ['motor', 'peak_speed'], ".0f", "RPM")
|
| 149 |
+
|
| 150 |
+
if 'ramp_rate' in stats_a and 'ramp_rate' in stats_b:
|
| 151 |
+
add_metric("Max Ramp Rate", ['ramp_rate', 'max'], ['ramp_rate', 'max'], ".1f", "bar/min")
|
| 152 |
+
|
| 153 |
+
if metrics_table:
|
| 154 |
+
st.dataframe(pd.DataFrame(metrics_table), use_container_width=True, hide_index=True)
|
| 155 |
+
|
| 156 |
+
# Cycle metadata from DB (new fields)
|
| 157 |
+
st.divider()
|
| 158 |
+
st.subheader("Cycle Metadata (from DB)")
|
| 159 |
+
meta_cols = st.columns(2)
|
| 160 |
+
|
| 161 |
+
for meta_col, cycle, label in [(meta_cols[0], cycle_a, "A"), (meta_cols[1], cycle_b, "B")]:
|
| 162 |
+
with meta_col:
|
| 163 |
+
_et = (cycle.get('start_time_et') or cycle['start_time']).strftime('%b %d %H:%M')
|
| 164 |
+
st.markdown(f"**Cycle {cycle['cycle_id']}** ({_et} ET)")
|
| 165 |
+
m_cols = st.columns(2)
|
| 166 |
+
m_cols[0].metric("Dispensed", f"{cycle.get('total_kg_dispensed', 0):.2f} kg")
|
| 167 |
+
m_cols[1].metric("Pump Strokes", f"{cycle.get('total_pump_strokes', 0):.0f}")
|
| 168 |
+
|
| 169 |
+
st.divider()
|
| 170 |
+
|
| 171 |
+
# Comparison charts
|
| 172 |
+
st.subheader("Overlaid Comparison Charts")
|
| 173 |
+
|
| 174 |
+
# Find common tags
|
| 175 |
+
common_tags = [t for t in df_a.columns if t in df_b.columns and t != 'timestamp']
|
| 176 |
+
chart_tags = [t for t in ['PT130', 'TT110', 'TT130', 'FT140', 'M130_Speed', 'MC130_VFD_Speed'] if t in common_tags]
|
| 177 |
+
|
| 178 |
+
if chart_tags:
|
| 179 |
+
selected_chart_tag = st.selectbox(
|
| 180 |
+
"Select sensor to compare",
|
| 181 |
+
chart_tags,
|
| 182 |
+
key="comparison_tag",
|
| 183 |
+
)
|
| 184 |
+
|
| 185 |
+
_et_a = (cycle_a.get('start_time_et') or cycle_a['start_time']).strftime('%b %d %H:%M')
|
| 186 |
+
_et_b = (cycle_b.get('start_time_et') or cycle_b['start_time']).strftime('%b %d %H:%M')
|
| 187 |
+
label_a = f"Cycle {cycle_a['cycle_id']} ({_et_a} ET)"
|
| 188 |
+
label_b = f"Cycle {cycle_b['cycle_id']} ({_et_b} ET)"
|
| 189 |
+
|
| 190 |
+
fig = create_comparison_chart(df_a, df_b, selected_chart_tag, label_a, label_b)
|
| 191 |
+
st.plotly_chart(fig, use_container_width=True)
|
| 192 |
+
|
| 193 |
+
st.divider()
|
| 194 |
+
|
| 195 |
+
# LLM Comparison
|
| 196 |
+
st.subheader("AI Comparison Analysis")
|
| 197 |
+
|
| 198 |
+
if not llm.api_available:
|
| 199 |
+
st.error("Claude API key not configured. Add ANTHROPIC_API_KEY to your .env file.")
|
| 200 |
+
st.stop()
|
| 201 |
+
|
| 202 |
+
cache_key = f"comparison_{cycle_a['cycle_id']}_{cycle_b['cycle_id']}"
|
| 203 |
+
prompt_key = f"{cache_key}_prompt"
|
| 204 |
+
followup_key = f"{cache_key}_followups"
|
| 205 |
+
cached_result = st.session_state.get(cache_key)
|
| 206 |
+
|
| 207 |
+
if cached_result:
|
| 208 |
+
analysis_card("AI Comparison", cached_result)
|
| 209 |
+
|
| 210 |
+
# Follow-up section
|
| 211 |
+
original_prompt = st.session_state.get(prompt_key, "")
|
| 212 |
+
follow_up_section(
|
| 213 |
+
session_key=followup_key,
|
| 214 |
+
llm_analyzer=llm,
|
| 215 |
+
original_prompt=original_prompt,
|
| 216 |
+
original_analysis=cached_result,
|
| 217 |
+
)
|
| 218 |
+
|
| 219 |
+
if st.button("Re-analyze", key="reanalyze_comparison"):
|
| 220 |
+
del st.session_state[cache_key]
|
| 221 |
+
st.session_state.pop(prompt_key, None)
|
| 222 |
+
st.session_state.pop(followup_key, None)
|
| 223 |
+
st.rerun()
|
| 224 |
+
else:
|
| 225 |
+
if st.button("Compare with AI", type="primary", use_container_width=True):
|
| 226 |
+
label_a = f"Cycle {cycle_a['cycle_id']} ({(cycle_a.get('start_time_et') or cycle_a['start_time']).strftime('%b %d')} ET)"
|
| 227 |
+
label_b = f"Cycle {cycle_b['cycle_id']} ({(cycle_b.get('start_time_et') or cycle_b['start_time']).strftime('%b %d')} ET)"
|
| 228 |
+
|
| 229 |
+
# Build and store the prompt for follow-up context
|
| 230 |
+
knowledge_text = ""
|
| 231 |
+
if llm.retriever:
|
| 232 |
+
try:
|
| 233 |
+
context = llm.retriever.get_context_for_comparison(stats_a, stats_b)
|
| 234 |
+
knowledge_text = llm.retriever.format_for_prompt(context)
|
| 235 |
+
except Exception:
|
| 236 |
+
pass
|
| 237 |
+
st.session_state[prompt_key] = llm._build_comparison_prompt(
|
| 238 |
+
stats_a, stats_b, label_a, label_b, knowledge_text
|
| 239 |
+
)
|
| 240 |
+
|
| 241 |
+
with st.spinner("Claude is comparing the two cycles..."):
|
| 242 |
+
result = llm.compare_cycles(stats_a, stats_b, label_a, label_b)
|
| 243 |
+
|
| 244 |
+
st.session_state[cache_key] = result
|
| 245 |
+
st.rerun()
|
pages/6_advanced_explorer.py
ADDED
|
@@ -0,0 +1,437 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Advanced Data Explorer — LLM-Powered Analytical Queries
|
| 3 |
+
Decomposes natural language into structured execution plans,
|
| 4 |
+
runs whitelisted operations, and presents results with exports.
|
| 5 |
+
"""
|
| 6 |
+
import json
|
| 7 |
+
import copy
|
| 8 |
+
import streamlit as st
|
| 9 |
+
import pandas as pd
|
| 10 |
+
from datetime import datetime
|
| 11 |
+
|
| 12 |
+
from core.db_connector import get_db_connector
|
| 13 |
+
from analysis.query_planner import QueryPlanner
|
| 14 |
+
from analysis.execution_engine import ExecutionEngine
|
| 15 |
+
from analysis.result_interpreter import ResultInterpreter
|
| 16 |
+
from analysis.operations import OperationOutput
|
| 17 |
+
from core.export_utils import package_multi_window_csv, estimate_csv_size
|
| 18 |
+
from ui.components import page_header, analysis_card, follow_up_section
|
| 19 |
+
from analysis.llm_analyzer import LLMCycleAnalyzer
|
| 20 |
+
from ui.plotly_charts import create_timeseries_chart
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
# ── Helper Functions (defined first for Streamlit top-to-bottom flow) ──
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def _plan_to_serializable(plan: dict) -> dict:
|
| 27 |
+
"""Convert plan with datetimes to JSON-serializable dict."""
|
| 28 |
+
p = copy.deepcopy(plan)
|
| 29 |
+
dr = p.get("data_requirements", {})
|
| 30 |
+
for key in ("start_time", "end_time"):
|
| 31 |
+
if isinstance(dr.get(key), datetime):
|
| 32 |
+
dr[key] = dr[key].isoformat()
|
| 33 |
+
return p
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def _render_dict_result(data: dict, op_result: OperationOutput):
|
| 37 |
+
"""Render a dict result (e.g., pump strokes, statistics)."""
|
| 38 |
+
# Special rendering for pump stroke results
|
| 39 |
+
if "total_revolutions" in data:
|
| 40 |
+
cols = st.columns(4)
|
| 41 |
+
cols[0].metric("Total Revolutions", f"{data.get('total_revolutions', 0):,}")
|
| 42 |
+
cols[1].metric("Avg RPM", f"{data.get('avg_rpm', 0):.1f}")
|
| 43 |
+
cols[2].metric("Peak RPM", f"{data.get('peak_rpm', 0):.1f}")
|
| 44 |
+
cols[3].metric("Active Time", f"{data.get('active_minutes', 0):.1f} min")
|
| 45 |
+
|
| 46 |
+
if data.get("note"):
|
| 47 |
+
st.info(data["note"])
|
| 48 |
+
return
|
| 49 |
+
|
| 50 |
+
# Blocks result from find_peak_pressure_blocks
|
| 51 |
+
if "blocks" in data:
|
| 52 |
+
blocks = data["blocks"]
|
| 53 |
+
if blocks:
|
| 54 |
+
st.markdown(f"Found **{len(blocks)}** time blocks")
|
| 55 |
+
block_df = pd.DataFrame(blocks)
|
| 56 |
+
st.dataframe(block_df, use_container_width=True, hide_index=True)
|
| 57 |
+
else:
|
| 58 |
+
st.info("No blocks found matching the criteria.")
|
| 59 |
+
return
|
| 60 |
+
|
| 61 |
+
# Generic dict
|
| 62 |
+
for key, val in data.items():
|
| 63 |
+
st.markdown(f"**{key}:** {val}")
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
def _render_list_result(data: list, op_result: OperationOutput):
|
| 67 |
+
"""Render a list-of-dicts result (fills, periods, ramps)."""
|
| 68 |
+
st.markdown(f"Found **{len(data)}** results")
|
| 69 |
+
|
| 70 |
+
result_df = pd.DataFrame(data)
|
| 71 |
+
|
| 72 |
+
# Format datetime columns for display
|
| 73 |
+
for col in result_df.columns:
|
| 74 |
+
if pd.api.types.is_datetime64_any_dtype(result_df[col]):
|
| 75 |
+
result_df[col] = result_df[col].dt.strftime("%Y-%m-%d %H:%M:%S")
|
| 76 |
+
|
| 77 |
+
st.dataframe(result_df, use_container_width=True, hide_index=True)
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
def _render_multi_df_result(data: list, op_result: OperationOutput):
|
| 81 |
+
"""Render multiple DataFrames (extract_windows)."""
|
| 82 |
+
labels = op_result.metadata.get("labels", [f"Window {i+1}" for i in range(len(data))])
|
| 83 |
+
rows = op_result.metadata.get("rows_per_window", [len(d) for d in data])
|
| 84 |
+
|
| 85 |
+
st.markdown(f"Extracted **{len(data)}** windows")
|
| 86 |
+
|
| 87 |
+
for i, (window_df, label) in enumerate(zip(data, labels)):
|
| 88 |
+
with st.expander(f"{label} ({rows[i] if i < len(rows) else len(window_df):,} rows)"):
|
| 89 |
+
tags_in_data = [c for c in window_df.columns if c != "timestamp" and pd.api.types.is_numeric_dtype(window_df[c])]
|
| 90 |
+
if "timestamp" in window_df.columns and tags_in_data:
|
| 91 |
+
fig = create_timeseries_chart(window_df, tags_in_data[:5], title=label)
|
| 92 |
+
st.plotly_chart(fig, use_container_width=True)
|
| 93 |
+
st.dataframe(window_df, use_container_width=True, height=200)
|
| 94 |
+
|
| 95 |
+
# Multi-window zip download
|
| 96 |
+
if data:
|
| 97 |
+
zip_bytes = package_multi_window_csv(data, labels)
|
| 98 |
+
st.download_button(
|
| 99 |
+
label=f"Download All Windows (ZIP, {len(data)} files)",
|
| 100 |
+
data=zip_bytes,
|
| 101 |
+
file_name="csh2_windows_export.zip",
|
| 102 |
+
mime="application/zip",
|
| 103 |
+
key="v2_download_zip",
|
| 104 |
+
)
|
| 105 |
+
|
| 106 |
+
|
| 107 |
+
def _render_df_result(data: pd.DataFrame, op_result: OperationOutput):
|
| 108 |
+
"""Render a single DataFrame result."""
|
| 109 |
+
st.markdown(f"**{len(data):,}** rows")
|
| 110 |
+
|
| 111 |
+
tags_in_data = [c for c in data.columns if c != "timestamp" and pd.api.types.is_numeric_dtype(data[c])]
|
| 112 |
+
|
| 113 |
+
if "timestamp" in data.columns and tags_in_data:
|
| 114 |
+
fig = create_timeseries_chart(data, tags_in_data[:5], title=op_result.label)
|
| 115 |
+
st.plotly_chart(fig, use_container_width=True)
|
| 116 |
+
|
| 117 |
+
with st.expander("Data Preview"):
|
| 118 |
+
st.dataframe(data.head(500), use_container_width=True, height=300)
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
def _render_operation_result(op_result: OperationOutput, index: int):
|
| 122 |
+
"""Render a single operation's results."""
|
| 123 |
+
st.subheader(f"Step {index + 1}: {op_result.label}")
|
| 124 |
+
|
| 125 |
+
if op_result.metadata.get("error"):
|
| 126 |
+
st.warning(f"Issue: {op_result.metadata['error']}")
|
| 127 |
+
return
|
| 128 |
+
|
| 129 |
+
data = op_result.data
|
| 130 |
+
|
| 131 |
+
if isinstance(data, dict):
|
| 132 |
+
_render_dict_result(data, op_result)
|
| 133 |
+
elif isinstance(data, list) and data and isinstance(data[0], dict):
|
| 134 |
+
_render_list_result(data, op_result)
|
| 135 |
+
elif isinstance(data, list) and data and isinstance(data[0], pd.DataFrame):
|
| 136 |
+
_render_multi_df_result(data, op_result)
|
| 137 |
+
elif isinstance(data, pd.DataFrame):
|
| 138 |
+
_render_df_result(data, op_result)
|
| 139 |
+
|
| 140 |
+
# Metadata display
|
| 141 |
+
meta = {k: v for k, v in op_result.metadata.items() if k != "error"}
|
| 142 |
+
if meta:
|
| 143 |
+
with st.expander("Details"):
|
| 144 |
+
for k, v in meta.items():
|
| 145 |
+
st.markdown(f"**{k}:** {v}")
|
| 146 |
+
|
| 147 |
+
# Export button for this operation
|
| 148 |
+
if op_result.exportable_df is not None and not op_result.exportable_df.empty:
|
| 149 |
+
csv = op_result.exportable_df.to_csv(index=False)
|
| 150 |
+
size = estimate_csv_size(op_result.exportable_df)
|
| 151 |
+
safe_name = op_result.op_name.replace(" ", "_")
|
| 152 |
+
st.download_button(
|
| 153 |
+
label=f"Download {safe_name}.csv ({size})",
|
| 154 |
+
data=csv,
|
| 155 |
+
file_name=f"csh2_{safe_name}.csv",
|
| 156 |
+
mime="text/csv",
|
| 157 |
+
key=f"v2_download_{index}",
|
| 158 |
+
)
|
| 159 |
+
|
| 160 |
+
|
| 161 |
+
# ── Page Header ──────────────────────────────────────────────────
|
| 162 |
+
|
| 163 |
+
page_header(
|
| 164 |
+
"Advanced Data Explorer",
|
| 165 |
+
"LLM-powered analytical queries with post-processing and export"
|
| 166 |
+
)
|
| 167 |
+
|
| 168 |
+
db = get_db_connector()
|
| 169 |
+
planner = QueryPlanner()
|
| 170 |
+
|
| 171 |
+
|
| 172 |
+
# ── Example Queries ──────────────────────────────────────────────
|
| 173 |
+
|
| 174 |
+
EXAMPLE_QUERIES = [
|
| 175 |
+
"Find time blocks where PT130 exceeded 900 bar",
|
| 176 |
+
"Find where both AOV140 and MC130_VFD_Speed were constant for 5+ minutes",
|
| 177 |
+
"Export PT130, TT110, FT140 at 2Hz as CSV for Sep 25 10am-11am",
|
| 178 |
+
"Identify fills where kWh per kg dispensed exceeded 0.6",
|
| 179 |
+
"In Fill 3, what was the total number of pump strokes?",
|
| 180 |
+
"Between Sep 24 15:00 and Sep 24 18:00, total pump strokes?",
|
| 181 |
+
]
|
| 182 |
+
|
| 183 |
+
|
| 184 |
+
# ── Session State Initialization ─────────────────────────────────
|
| 185 |
+
|
| 186 |
+
for key in ("v2_plan", "v2_result", "v2_query", "v2_run_plan", "v2_interpretation", "v2_interp_prompt"):
|
| 187 |
+
if key not in st.session_state:
|
| 188 |
+
st.session_state[key] = None
|
| 189 |
+
# v2_run_plan is a bool flag, default False
|
| 190 |
+
if st.session_state.v2_run_plan is None:
|
| 191 |
+
st.session_state.v2_run_plan = False
|
| 192 |
+
|
| 193 |
+
|
| 194 |
+
# ── Query Input Section ──────────────────────────────────────────
|
| 195 |
+
|
| 196 |
+
|
| 197 |
+
def _set_example_query(text: str):
|
| 198 |
+
"""Callback: set the widget's own key so text appears on next render."""
|
| 199 |
+
st.session_state.v2_query_input = text
|
| 200 |
+
st.session_state.v2_query = text
|
| 201 |
+
|
| 202 |
+
|
| 203 |
+
st.subheader("Ask a Complex Question")
|
| 204 |
+
|
| 205 |
+
query_col, btn_col = st.columns([5, 1])
|
| 206 |
+
with query_col:
|
| 207 |
+
user_query = st.text_input(
|
| 208 |
+
"Describe your analysis",
|
| 209 |
+
value=st.session_state.v2_query or "",
|
| 210 |
+
placeholder='e.g. "Find time blocks where PT130 exceeded 900 bar"',
|
| 211 |
+
key="v2_query_input",
|
| 212 |
+
label_visibility="collapsed",
|
| 213 |
+
)
|
| 214 |
+
# Sync widget value back to logical query state
|
| 215 |
+
st.session_state.v2_query = user_query
|
| 216 |
+
with btn_col:
|
| 217 |
+
analyze_btn = st.button("Analyze", type="primary", use_container_width=True, key="v2_analyze_btn")
|
| 218 |
+
|
| 219 |
+
# Example query chips
|
| 220 |
+
st.caption("Try an example:")
|
| 221 |
+
example_cols = st.columns(3)
|
| 222 |
+
for i, example in enumerate(EXAMPLE_QUERIES):
|
| 223 |
+
col = example_cols[i % 3]
|
| 224 |
+
with col:
|
| 225 |
+
st.button(
|
| 226 |
+
example,
|
| 227 |
+
key=f"v2_example_{i}",
|
| 228 |
+
use_container_width=True,
|
| 229 |
+
on_click=_set_example_query,
|
| 230 |
+
args=(example,),
|
| 231 |
+
)
|
| 232 |
+
|
| 233 |
+
st.divider()
|
| 234 |
+
|
| 235 |
+
|
| 236 |
+
# ── Planning Phase ───────────────────────────────────────────────
|
| 237 |
+
|
| 238 |
+
should_plan = (analyze_btn and user_query) or st.session_state.v2_run_plan
|
| 239 |
+
if should_plan:
|
| 240 |
+
query_to_plan = user_query or st.session_state.v2_query or ""
|
| 241 |
+
st.session_state.v2_run_plan = False # Reset flag immediately
|
| 242 |
+
st.session_state.v2_query = query_to_plan
|
| 243 |
+
st.session_state.v2_plan = None
|
| 244 |
+
st.session_state.v2_result = None
|
| 245 |
+
st.session_state.v2_interpretation = None
|
| 246 |
+
st.session_state.v2_interp_prompt = None
|
| 247 |
+
st.session_state.pop("v2_followups", None)
|
| 248 |
+
|
| 249 |
+
if not query_to_plan:
|
| 250 |
+
st.warning("Please enter a query or click an example.")
|
| 251 |
+
elif not planner.api_available:
|
| 252 |
+
st.error("Claude API key not configured. Add ANTHROPIC_API_KEY to your .env or Streamlit secrets.")
|
| 253 |
+
else:
|
| 254 |
+
with st.spinner("Planning execution..."):
|
| 255 |
+
all_sensors = db.get_all_tag_names()
|
| 256 |
+
plan = planner.plan(query_to_plan, all_sensors)
|
| 257 |
+
|
| 258 |
+
if plan and "error" in plan:
|
| 259 |
+
st.error(f"Planning failed: {plan['error']}")
|
| 260 |
+
elif plan:
|
| 261 |
+
st.session_state.v2_plan = plan
|
| 262 |
+
else:
|
| 263 |
+
st.error("Could not create an execution plan. Try rephrasing your query.")
|
| 264 |
+
|
| 265 |
+
|
| 266 |
+
# ── Plan Review Card ─────────────────────────────────────────────
|
| 267 |
+
|
| 268 |
+
plan = st.session_state.v2_plan
|
| 269 |
+
|
| 270 |
+
if plan and "error" not in plan and st.session_state.v2_result is None:
|
| 271 |
+
qt = plan.get("query_type", "FETCH")
|
| 272 |
+
explanation = plan.get("explanation", "")
|
| 273 |
+
dr = plan.get("data_requirements", {})
|
| 274 |
+
ops = plan.get("operations", [])
|
| 275 |
+
export_info = plan.get("export", {})
|
| 276 |
+
|
| 277 |
+
# Query type badge
|
| 278 |
+
badge_colors = {"FETCH": "#5BA3B5", "ANALYZE": "#D4A04A", "EXPORT": "#5B9A6E"}
|
| 279 |
+
badge_color = badge_colors.get(qt, "#6B6358")
|
| 280 |
+
|
| 281 |
+
st.markdown(f"""
|
| 282 |
+
<div style="background: #1A1714; border: 1px solid #2A2520; border-radius: 8px; padding: 16px; margin-bottom: 16px;">
|
| 283 |
+
<div style="display: flex; align-items: center; gap: 12px; margin-bottom: 12px;">
|
| 284 |
+
<span style="background: {badge_color}; color: #0F0D0B; font-weight: 700; padding: 3px 10px;
|
| 285 |
+
border-radius: 4px; font-size: 0.75rem; letter-spacing: 1px;">{qt}</span>
|
| 286 |
+
<span style="color: #B0A898; font-size: 0.9rem;">{explanation}</span>
|
| 287 |
+
</div>
|
| 288 |
+
</div>
|
| 289 |
+
""", unsafe_allow_html=True)
|
| 290 |
+
|
| 291 |
+
# Data requirements
|
| 292 |
+
sensors = dr.get("sensors", [])
|
| 293 |
+
resolution = dr.get("resolution", "auto")
|
| 294 |
+
start_time = dr.get("start_time", "")
|
| 295 |
+
end_time = dr.get("end_time", "")
|
| 296 |
+
|
| 297 |
+
if isinstance(start_time, datetime):
|
| 298 |
+
start_str = start_time.strftime("%b %d, %Y %H:%M")
|
| 299 |
+
else:
|
| 300 |
+
start_str = str(start_time)
|
| 301 |
+
if isinstance(end_time, datetime):
|
| 302 |
+
end_str = end_time.strftime("%b %d, %Y %H:%M")
|
| 303 |
+
else:
|
| 304 |
+
end_str = str(end_time)
|
| 305 |
+
|
| 306 |
+
info_cols = st.columns(4)
|
| 307 |
+
info_cols[0].markdown(f"**Sensors:** {', '.join(sensors[:6])}{'...' if len(sensors) > 6 else ''}")
|
| 308 |
+
info_cols[1].markdown(f"**From:** {start_str}")
|
| 309 |
+
info_cols[2].markdown(f"**To:** {end_str}")
|
| 310 |
+
info_cols[3].markdown(f"**Resolution:** `{resolution}`")
|
| 311 |
+
|
| 312 |
+
if ops:
|
| 313 |
+
st.markdown("**Execution Steps:**")
|
| 314 |
+
for i, op in enumerate(ops, 1):
|
| 315 |
+
op_label = op.get("label", op.get("op", ""))
|
| 316 |
+
st.markdown(f" `{i}.` {op_label}")
|
| 317 |
+
|
| 318 |
+
if export_info.get("enabled"):
|
| 319 |
+
st.markdown(f"**Export:** `{export_info.get('filename_hint', 'export')}.csv`")
|
| 320 |
+
|
| 321 |
+
# Action buttons
|
| 322 |
+
exec_col, cancel_col = st.columns([2, 1])
|
| 323 |
+
with exec_col:
|
| 324 |
+
execute_btn = st.button("Execute Plan", type="primary", use_container_width=True, key="v2_execute_btn")
|
| 325 |
+
with cancel_col:
|
| 326 |
+
cancel_btn = st.button("Cancel", use_container_width=True, key="v2_cancel_btn")
|
| 327 |
+
|
| 328 |
+
with st.expander("Debug JSON"):
|
| 329 |
+
display_plan = _plan_to_serializable(plan)
|
| 330 |
+
st.code(json.dumps(display_plan, indent=2), language="json")
|
| 331 |
+
|
| 332 |
+
if cancel_btn:
|
| 333 |
+
st.session_state.v2_plan = None
|
| 334 |
+
st.session_state.v2_result = None
|
| 335 |
+
st.rerun()
|
| 336 |
+
|
| 337 |
+
if execute_btn:
|
| 338 |
+
engine = ExecutionEngine()
|
| 339 |
+
with st.spinner("Executing plan..."):
|
| 340 |
+
exec_result = engine.execute(plan)
|
| 341 |
+
st.session_state.v2_result = exec_result
|
| 342 |
+
st.rerun()
|
| 343 |
+
|
| 344 |
+
|
| 345 |
+
# ── Results Display ──────────────────────────────────────────────
|
| 346 |
+
|
| 347 |
+
result = st.session_state.v2_result
|
| 348 |
+
|
| 349 |
+
if result is not None:
|
| 350 |
+
st.divider()
|
| 351 |
+
|
| 352 |
+
if result.success:
|
| 353 |
+
st.success(f"Execution completed in {result.execution_time_seconds:.1f}s")
|
| 354 |
+
else:
|
| 355 |
+
st.error(f"Execution failed ({result.execution_time_seconds:.1f}s)")
|
| 356 |
+
for err in result.errors:
|
| 357 |
+
st.error(err)
|
| 358 |
+
|
| 359 |
+
# Warnings
|
| 360 |
+
for warn in (result.warnings or []):
|
| 361 |
+
st.warning(warn)
|
| 362 |
+
|
| 363 |
+
# Summary metrics
|
| 364 |
+
metric_cols = st.columns(4)
|
| 365 |
+
metric_cols[0].metric("Rows Fetched", f"{result.row_count:,}")
|
| 366 |
+
metric_cols[1].metric("Table", result.table_used or "N/A")
|
| 367 |
+
metric_cols[2].metric("Operations", len(result.operation_results))
|
| 368 |
+
metric_cols[3].metric("Time", f"{result.execution_time_seconds:.1f}s")
|
| 369 |
+
|
| 370 |
+
# Explanation
|
| 371 |
+
if result.explanation:
|
| 372 |
+
analysis_card("Plan Summary", result.explanation)
|
| 373 |
+
|
| 374 |
+
# Render each operation result
|
| 375 |
+
for i, op_result in enumerate(result.operation_results):
|
| 376 |
+
st.divider()
|
| 377 |
+
_render_operation_result(op_result, i)
|
| 378 |
+
|
| 379 |
+
# ── Murphy's Interpretation ──
|
| 380 |
+
if result.success and result.operation_results:
|
| 381 |
+
st.divider()
|
| 382 |
+
if st.session_state.v2_interpretation:
|
| 383 |
+
analysis_card("Murphy's Take", st.session_state.v2_interpretation)
|
| 384 |
+
|
| 385 |
+
# Follow-up section for Murphy's Take
|
| 386 |
+
interp_prompt = st.session_state.get("v2_interp_prompt", "")
|
| 387 |
+
_llm_for_followup = LLMCycleAnalyzer()
|
| 388 |
+
follow_up_section(
|
| 389 |
+
session_key="v2_followups",
|
| 390 |
+
llm_analyzer=_llm_for_followup,
|
| 391 |
+
original_prompt=interp_prompt,
|
| 392 |
+
original_analysis=st.session_state.v2_interpretation,
|
| 393 |
+
)
|
| 394 |
+
else:
|
| 395 |
+
if st.button("Get Murphy's Take", type="primary", key="v2_interpret_btn"):
|
| 396 |
+
interpreter = ResultInterpreter()
|
| 397 |
+
if interpreter.api_available:
|
| 398 |
+
# Store the prompt for follow-up context
|
| 399 |
+
interp_prompt = interpreter._build_interpretation_prompt(
|
| 400 |
+
st.session_state.v2_query or "", result
|
| 401 |
+
)
|
| 402 |
+
st.session_state.v2_interp_prompt = interp_prompt
|
| 403 |
+
|
| 404 |
+
with st.spinner("Murphy is analyzing the results..."):
|
| 405 |
+
interpretation = interpreter.interpret(
|
| 406 |
+
st.session_state.v2_query or "", result
|
| 407 |
+
)
|
| 408 |
+
st.session_state.v2_interpretation = interpretation
|
| 409 |
+
st.rerun()
|
| 410 |
+
else:
|
| 411 |
+
st.error("Claude API key not configured for interpretation.")
|
| 412 |
+
|
| 413 |
+
# Source data overview
|
| 414 |
+
if result.source_data is not None and not result.source_data.empty:
|
| 415 |
+
st.divider()
|
| 416 |
+
with st.expander("Source Data Overview"):
|
| 417 |
+
src = result.source_data
|
| 418 |
+
st.markdown(f"**{len(src):,}** rows, **{len(src.columns)}** columns")
|
| 419 |
+
tags_in_src = [c for c in src.columns if c != "timestamp" and pd.api.types.is_numeric_dtype(src[c])]
|
| 420 |
+
if "timestamp" in src.columns and tags_in_src:
|
| 421 |
+
fig = create_timeseries_chart(src, tags_in_src[:5], title="Source Data")
|
| 422 |
+
st.plotly_chart(fig, use_container_width=True)
|
| 423 |
+
st.dataframe(src.head(200), use_container_width=True, height=200)
|
| 424 |
+
|
| 425 |
+
|
| 426 |
+
# ── Reset Button ─────────────────────────────────────────────────
|
| 427 |
+
|
| 428 |
+
if st.session_state.v2_plan or st.session_state.v2_result:
|
| 429 |
+
st.divider()
|
| 430 |
+
if st.button("New Query", use_container_width=True, key="v2_reset"):
|
| 431 |
+
st.session_state.v2_plan = None
|
| 432 |
+
st.session_state.v2_result = None
|
| 433 |
+
st.session_state.v2_query = None
|
| 434 |
+
st.session_state.v2_interpretation = None
|
| 435 |
+
st.session_state.v2_interp_prompt = None
|
| 436 |
+
st.session_state.pop("v2_followups", None)
|
| 437 |
+
st.rerun()
|
pages/7_search.py
ADDED
|
@@ -0,0 +1,187 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Feature 7: Search — Full-text search across engineer notes, PLC events, and sensor tags
|
| 3 |
+
Uses PostgreSQL tsvector/tsquery (BM25-style ranking) and pg_trgm (fuzzy matching)
|
| 4 |
+
"""
|
| 5 |
+
import streamlit as st
|
| 6 |
+
import pandas as pd
|
| 7 |
+
from core.db_connector import get_db_connector
|
| 8 |
+
from ui.components import page_header
|
| 9 |
+
|
| 10 |
+
page_header(
|
| 11 |
+
"Search",
|
| 12 |
+
"Full-text search across engineer logs, PLC alarm events, and sensor metadata."
|
| 13 |
+
)
|
| 14 |
+
|
| 15 |
+
db = get_db_connector()
|
| 16 |
+
|
| 17 |
+
# ── Search input ──
|
| 18 |
+
search_col, btn_col = st.columns([5, 1])
|
| 19 |
+
with search_col:
|
| 20 |
+
query = st.text_input(
|
| 21 |
+
"Search query",
|
| 22 |
+
placeholder='e.g. "seal leak", "pressure spike valve", "cooldown temperature"',
|
| 23 |
+
key="search_query",
|
| 24 |
+
label_visibility="collapsed",
|
| 25 |
+
)
|
| 26 |
+
with btn_col:
|
| 27 |
+
search_btn = st.button("Search", type="primary", use_container_width=True)
|
| 28 |
+
|
| 29 |
+
# ── Tabs for search targets ──
|
| 30 |
+
tab_memos, tab_events, tab_tags = st.tabs([
|
| 31 |
+
"Engineer Logs (memo_log)",
|
| 32 |
+
"PLC Events (allevent)",
|
| 33 |
+
"Sensor Lookup (tags)",
|
| 34 |
+
])
|
| 35 |
+
|
| 36 |
+
# ── Tab 1: Memo Log Search ──
|
| 37 |
+
with tab_memos:
|
| 38 |
+
# Filters
|
| 39 |
+
filter_col1, filter_col2 = st.columns(2)
|
| 40 |
+
with filter_col1:
|
| 41 |
+
engineers = db.get_memo_engineers()
|
| 42 |
+
engineer_filter = st.selectbox(
|
| 43 |
+
"Filter by Engineer",
|
| 44 |
+
["All"] + engineers,
|
| 45 |
+
key="memo_engineer_filter",
|
| 46 |
+
)
|
| 47 |
+
with filter_col2:
|
| 48 |
+
severities = db.get_memo_severities()
|
| 49 |
+
severity_filter = st.selectbox(
|
| 50 |
+
"Filter by Severity",
|
| 51 |
+
["All"] + severities,
|
| 52 |
+
key="memo_severity_filter",
|
| 53 |
+
)
|
| 54 |
+
|
| 55 |
+
if (search_btn or query) and query:
|
| 56 |
+
eng = engineer_filter if engineer_filter != "All" else None
|
| 57 |
+
sev = severity_filter if severity_filter != "All" else None
|
| 58 |
+
|
| 59 |
+
with st.spinner("Searching engineer logs..."):
|
| 60 |
+
df = db.search_memo_log(query, limit=25, engineer=eng, severity=sev)
|
| 61 |
+
|
| 62 |
+
if df.empty:
|
| 63 |
+
st.info(f'No engineer logs matching "{query}".')
|
| 64 |
+
else:
|
| 65 |
+
st.markdown(f"**{len(df)} results** — ranked by relevance")
|
| 66 |
+
|
| 67 |
+
for _, row in df.iterrows():
|
| 68 |
+
# Build header
|
| 69 |
+
date_str = row['logged_at'].strftime('%b %d, %Y') if pd.notna(row['logged_at']) else 'N/A'
|
| 70 |
+
engineer = row.get('engineer', 'Unknown')
|
| 71 |
+
severity = row.get('severity', '')
|
| 72 |
+
rank = row.get('rank', 0)
|
| 73 |
+
|
| 74 |
+
sev_color = {
|
| 75 |
+
'High': '#CC3030', 'Medium': '#CC8F00', 'Low': '#5B9A6E',
|
| 76 |
+
}.get(severity, '#6B6358')
|
| 77 |
+
|
| 78 |
+
# Card for each result
|
| 79 |
+
parts = []
|
| 80 |
+
if pd.notna(row.get('summary')) and row['summary']:
|
| 81 |
+
parts.append(f"**Summary:** {row['summary']}")
|
| 82 |
+
if pd.notna(row.get('issues_found')) and row['issues_found']:
|
| 83 |
+
parts.append(f"**Issues:** {row['issues_found']}")
|
| 84 |
+
if pd.notna(row.get('maintenance_done')) and row['maintenance_done']:
|
| 85 |
+
parts.append(f"**Maintenance:** {row['maintenance_done']}")
|
| 86 |
+
if pd.notna(row.get('system_performance')) and row['system_performance']:
|
| 87 |
+
parts.append(f"**Performance:** {row['system_performance']}")
|
| 88 |
+
if pd.notna(row.get('components_affected')) and row['components_affected']:
|
| 89 |
+
parts.append(f"**Components:** {row['components_affected']}")
|
| 90 |
+
|
| 91 |
+
body = "\n\n".join(parts) if parts else "(no text content)"
|
| 92 |
+
|
| 93 |
+
with st.expander(
|
| 94 |
+
f"#{row['id']} — {date_str} — {engineer} — "
|
| 95 |
+
f"Severity: {severity or 'N/A'} — Rank: {rank:.3f}",
|
| 96 |
+
expanded=(rank > 0.5),
|
| 97 |
+
):
|
| 98 |
+
st.markdown(body)
|
| 99 |
+
else:
|
| 100 |
+
st.caption("Enter a search query above to search across 392 engineer field notes.")
|
| 101 |
+
|
| 102 |
+
# ── Tab 2: PLC Event Search ──
|
| 103 |
+
with tab_events:
|
| 104 |
+
event_col1, event_col2 = st.columns([3, 1])
|
| 105 |
+
with event_col1:
|
| 106 |
+
event_query = st.text_input(
|
| 107 |
+
"Event search",
|
| 108 |
+
value=query or "",
|
| 109 |
+
placeholder='e.g. "PT110", "PT130", "MainPumpFault", "SafetyRelay"',
|
| 110 |
+
key="event_query",
|
| 111 |
+
label_visibility="collapsed",
|
| 112 |
+
)
|
| 113 |
+
with event_col2:
|
| 114 |
+
conditions = db.get_event_conditions()
|
| 115 |
+
condition_filter = st.selectbox(
|
| 116 |
+
"Condition",
|
| 117 |
+
["All"] + conditions,
|
| 118 |
+
key="event_condition_filter",
|
| 119 |
+
)
|
| 120 |
+
|
| 121 |
+
event_btn = st.button("Search Events", type="primary", key="event_search_btn")
|
| 122 |
+
|
| 123 |
+
if (event_btn or event_query) and event_query:
|
| 124 |
+
cond = condition_filter if condition_filter != "All" else None
|
| 125 |
+
with st.spinner("Searching PLC events..."):
|
| 126 |
+
df_events = db.search_events(event_query, condition=cond, limit=100)
|
| 127 |
+
|
| 128 |
+
if df_events.empty:
|
| 129 |
+
st.info(f'No PLC events matching "{event_query}".')
|
| 130 |
+
else:
|
| 131 |
+
st.markdown(f"**{len(df_events)} results** (showing up to 100)")
|
| 132 |
+
|
| 133 |
+
display_df = df_events[['eventtimestamp', 'conditionname', 'message']].copy()
|
| 134 |
+
display_df.columns = ['Timestamp', 'Condition', 'Message']
|
| 135 |
+
# Trim message for readability
|
| 136 |
+
display_df['Message'] = display_df['Message'].str[:120]
|
| 137 |
+
|
| 138 |
+
st.dataframe(display_df, use_container_width=True, height=500, hide_index=True)
|
| 139 |
+
else:
|
| 140 |
+
st.caption(
|
| 141 |
+
"Search across 385K PLC alarm events by sensor tag name (PT110, TT130, etc.) "
|
| 142 |
+
"or event type (MainPumpFault, SafetyRelay). Filter by condition: HI, LO, TRIP, HIHI, LOLO."
|
| 143 |
+
)
|
| 144 |
+
|
| 145 |
+
# ── Tab 3: Fuzzy Tag Lookup ──
|
| 146 |
+
with tab_tags:
|
| 147 |
+
tag_query = st.text_input(
|
| 148 |
+
"Sensor name or description",
|
| 149 |
+
placeholder='e.g. "discharge pressure", "temperature inlet", "motor speed"',
|
| 150 |
+
key="tag_search_query",
|
| 151 |
+
label_visibility="collapsed",
|
| 152 |
+
)
|
| 153 |
+
tag_btn = st.button("Find Sensors", type="primary", key="tag_search_btn")
|
| 154 |
+
|
| 155 |
+
if (tag_btn or tag_query) and tag_query:
|
| 156 |
+
with st.spinner("Fuzzy matching sensors..."):
|
| 157 |
+
df_tags = db.search_tags_fuzzy(tag_query, limit=15)
|
| 158 |
+
|
| 159 |
+
if df_tags.empty:
|
| 160 |
+
st.info(f'No sensors matching "{tag_query}". Try broader terms.')
|
| 161 |
+
else:
|
| 162 |
+
st.markdown(f"**{len(df_tags)} matches** — ranked by similarity")
|
| 163 |
+
|
| 164 |
+
# Color-code similarity
|
| 165 |
+
def fmt_sim(val):
|
| 166 |
+
if val > 0.5:
|
| 167 |
+
return f"🟢 {val:.3f}"
|
| 168 |
+
elif val > 0.3:
|
| 169 |
+
return f"🟡 {val:.3f}"
|
| 170 |
+
else:
|
| 171 |
+
return f"🔴 {val:.3f}"
|
| 172 |
+
|
| 173 |
+
display = df_tags[['tagname', 'tagdesc', 'tagunit', 'sim']].copy()
|
| 174 |
+
display.columns = ['Tag Name', 'Description', 'Unit', 'Similarity']
|
| 175 |
+
display['Similarity'] = display['Similarity'].apply(fmt_sim)
|
| 176 |
+
|
| 177 |
+
st.dataframe(display, use_container_width=True, hide_index=True)
|
| 178 |
+
|
| 179 |
+
st.caption(
|
| 180 |
+
"Similarity scores use pg_trgm trigram matching. "
|
| 181 |
+
"🟢 >0.5 = strong match, 🟡 >0.3 = partial, 🔴 <0.3 = weak."
|
| 182 |
+
)
|
| 183 |
+
else:
|
| 184 |
+
st.caption(
|
| 185 |
+
"Enter a sensor name or description to fuzzy-search across 468 tags. "
|
| 186 |
+
"Works with partial names and typos."
|
| 187 |
+
)
|
pages/__init__.py
ADDED
|
File without changes
|
requirements.txt
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
streamlit==1.42.0
|
| 2 |
+
psycopg2-binary==2.9.10
|
| 3 |
+
pandas==2.2.3
|
| 4 |
+
numpy==1.26.4
|
| 5 |
+
plotly==5.24.1
|
| 6 |
+
anthropic==0.44.0
|
| 7 |
+
python-dotenv==1.0.1
|
| 8 |
+
python-dateutil==2.9.0
|
| 9 |
+
pyyaml==6.0.2
|
ui/__init__.py
ADDED
|
File without changes
|
ui/components.py
ADDED
|
@@ -0,0 +1,260 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Reusable UI components for CSH2 Web Dashboard — HUD Theme
|
| 3 |
+
"""
|
| 4 |
+
import streamlit as st
|
| 5 |
+
import pandas as pd
|
| 6 |
+
from typing import List, Dict, Optional
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
def page_header(title: str, subtitle: str = ""):
|
| 10 |
+
"""Render a HUD-style page header with gradient accent lines"""
|
| 11 |
+
subtitle_html = f'<div class="hud-subtitle">{subtitle}</div>' if subtitle else ''
|
| 12 |
+
st.markdown(f"""
|
| 13 |
+
<div class="hud-page-header">
|
| 14 |
+
<div class="hud-header-line"></div>
|
| 15 |
+
<h1>{title}</h1>
|
| 16 |
+
{subtitle_html}
|
| 17 |
+
<div class="hud-header-line"></div>
|
| 18 |
+
</div>
|
| 19 |
+
""", unsafe_allow_html=True)
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def metric_row(metrics: List[Dict]):
|
| 23 |
+
"""
|
| 24 |
+
Render a row of HUD-style metric cards.
|
| 25 |
+
|
| 26 |
+
metrics: list of dicts with keys: label, value, unit, delta (optional), status (optional)
|
| 27 |
+
status can be: 'nominal', 'warning', 'alarm' (defaults to no special styling)
|
| 28 |
+
"""
|
| 29 |
+
cols = st.columns(len(metrics))
|
| 30 |
+
for col, m in zip(cols, metrics):
|
| 31 |
+
with col:
|
| 32 |
+
display_value = f"{m['value']}"
|
| 33 |
+
if m.get('unit'):
|
| 34 |
+
display_value = f"{m['value']} {m['unit']}"
|
| 35 |
+
st.metric(
|
| 36 |
+
label=m['label'],
|
| 37 |
+
value=display_value,
|
| 38 |
+
delta=m.get('delta'),
|
| 39 |
+
)
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def analysis_card(title: str, content: str):
|
| 43 |
+
"""Render an LLM analysis result in a HUD terminal-style card"""
|
| 44 |
+
st.markdown(f"""
|
| 45 |
+
<div class="analysis-card">
|
| 46 |
+
<h4>// {title.upper()}</h4>
|
| 47 |
+
<div style="color: #B0A898; line-height: 1.7; font-size: 0.85rem;">
|
| 48 |
+
{content.replace(chr(10), '<br>')}
|
| 49 |
+
</div>
|
| 50 |
+
</div>
|
| 51 |
+
""", unsafe_allow_html=True)
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
def system_status_footer(sensor_count: int, data_start: str, data_end: str, connected: bool = True):
|
| 55 |
+
"""Render a HUD-style system status panel in the sidebar"""
|
| 56 |
+
conn_color = '#5B9A6E' if connected else '#CC3030'
|
| 57 |
+
conn_text = 'CONNECTED' if connected else 'OFFLINE'
|
| 58 |
+
st.markdown(f"""
|
| 59 |
+
<div style="
|
| 60 |
+
border-top: 1px solid #2A2520;
|
| 61 |
+
padding-top: 12px;
|
| 62 |
+
margin-top: 12px;
|
| 63 |
+
font-size: 0.7rem;
|
| 64 |
+
color: #6B6358;
|
| 65 |
+
letter-spacing: 0.5px;
|
| 66 |
+
">
|
| 67 |
+
<div style="margin-bottom: 4px;">
|
| 68 |
+
<span class="hud-indicator" style="background-color: {conn_color}; box-shadow: 0 0 6px {conn_color};"></span>
|
| 69 |
+
<span style="color: {conn_color};">{conn_text}</span>
|
| 70 |
+
</div>
|
| 71 |
+
<div>SENSORS: {sensor_count}</div>
|
| 72 |
+
<div>DATA: {data_start} — {data_end}</div>
|
| 73 |
+
<div style="margin-top: 6px; color: #2A2520;">CSH2 DELPHI v1.0</div>
|
| 74 |
+
</div>
|
| 75 |
+
""", unsafe_allow_html=True)
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
def cycle_selector(cycles: List[Dict], key_prefix: str = "cycle") -> Optional[Dict]:
|
| 79 |
+
"""
|
| 80 |
+
Render a cycle selector dropdown.
|
| 81 |
+
|
| 82 |
+
cycles: list of dicts with keys: cycle_id, start_time, end_time, duration_minutes, peak_pressure
|
| 83 |
+
Returns selected cycle dict or None
|
| 84 |
+
"""
|
| 85 |
+
if not cycles:
|
| 86 |
+
st.info("No testing cycles detected for this period.")
|
| 87 |
+
return None
|
| 88 |
+
|
| 89 |
+
options = []
|
| 90 |
+
for c in cycles:
|
| 91 |
+
# Use Eastern display time if available, fall back to start_time
|
| 92 |
+
display_time = c.get('start_time_et') or c['start_time']
|
| 93 |
+
start_str = display_time.strftime('%b %d %H:%M') + " ET"
|
| 94 |
+
duration = f"{c['duration_minutes']:.0f} min"
|
| 95 |
+
peak = f"{c.get('peak_pressure', 0):.0f} bar" if c.get('peak_pressure') else "N/A"
|
| 96 |
+
options.append(f"Cycle {c['cycle_id']} | {start_str} | {duration} | Peak (PT130): {peak}")
|
| 97 |
+
|
| 98 |
+
selected_idx = st.selectbox(
|
| 99 |
+
"Select a testing cycle",
|
| 100 |
+
range(len(options)),
|
| 101 |
+
format_func=lambda i: options[i],
|
| 102 |
+
key=f"{key_prefix}_selector",
|
| 103 |
+
)
|
| 104 |
+
|
| 105 |
+
return cycles[selected_idx] if selected_idx is not None else None
|
| 106 |
+
|
| 107 |
+
|
| 108 |
+
def tag_multiselect(db_connector, key: str = "tags", default_group: str = None) -> List[str]:
|
| 109 |
+
"""
|
| 110 |
+
Render a tag multi-select with group filtering.
|
| 111 |
+
|
| 112 |
+
Selecting a sensor group filters the available options to only show
|
| 113 |
+
sensors in that group. "All Sensors" shows every tag in the database.
|
| 114 |
+
|
| 115 |
+
Returns list of selected tag names.
|
| 116 |
+
"""
|
| 117 |
+
from core.config import SENSOR_GROUPS
|
| 118 |
+
|
| 119 |
+
group_names = ["All Sensors"] + list(SENSOR_GROUPS.keys())
|
| 120 |
+
|
| 121 |
+
def _clear_selection():
|
| 122 |
+
"""Reset the multiselect when group changes to avoid stale selections."""
|
| 123 |
+
st.session_state.pop(f"{key}_select", None)
|
| 124 |
+
|
| 125 |
+
selected_group = st.selectbox(
|
| 126 |
+
"Sensor Group",
|
| 127 |
+
group_names,
|
| 128 |
+
key=f"{key}_group",
|
| 129 |
+
on_change=_clear_selection,
|
| 130 |
+
)
|
| 131 |
+
|
| 132 |
+
all_tags = db_connector.get_all_tag_names()
|
| 133 |
+
|
| 134 |
+
if selected_group == "All Sensors":
|
| 135 |
+
available_tags = all_tags
|
| 136 |
+
default_tags = []
|
| 137 |
+
else:
|
| 138 |
+
group_tags = SENSOR_GROUPS.get(selected_group, [])
|
| 139 |
+
available_tags = [t for t in group_tags if t in all_tags]
|
| 140 |
+
default_tags = available_tags # Pre-select all sensors in the chosen group
|
| 141 |
+
|
| 142 |
+
selected_tags = st.multiselect(
|
| 143 |
+
"Select Sensors",
|
| 144 |
+
options=available_tags,
|
| 145 |
+
default=default_tags,
|
| 146 |
+
key=f"{key}_select",
|
| 147 |
+
)
|
| 148 |
+
|
| 149 |
+
return selected_tags
|
| 150 |
+
|
| 151 |
+
|
| 152 |
+
def date_range_picker(key: str = "dates"):
|
| 153 |
+
"""Render start/end date and time pickers. Returns (start_datetime, end_datetime)."""
|
| 154 |
+
from datetime import datetime, time, timedelta
|
| 155 |
+
|
| 156 |
+
col1, col2 = st.columns(2)
|
| 157 |
+
with col1:
|
| 158 |
+
start_date = st.date_input(
|
| 159 |
+
"Start Date",
|
| 160 |
+
value=datetime(2025, 9, 25),
|
| 161 |
+
key=f"{key}_start_date",
|
| 162 |
+
)
|
| 163 |
+
start_time = st.time_input("Start Time", value=time(0, 0), key=f"{key}_start_time")
|
| 164 |
+
|
| 165 |
+
with col2:
|
| 166 |
+
end_date = st.date_input(
|
| 167 |
+
"End Date",
|
| 168 |
+
value=datetime(2025, 9, 25),
|
| 169 |
+
key=f"{key}_end_date",
|
| 170 |
+
)
|
| 171 |
+
end_time = st.time_input("End Time", value=time(23, 59), key=f"{key}_end_time")
|
| 172 |
+
|
| 173 |
+
start_dt = datetime.combine(start_date, start_time)
|
| 174 |
+
end_dt = datetime.combine(end_date, end_time)
|
| 175 |
+
|
| 176 |
+
return start_dt, end_dt
|
| 177 |
+
|
| 178 |
+
|
| 179 |
+
def no_data_message():
|
| 180 |
+
"""Show a styled no-data message"""
|
| 181 |
+
st.warning("No data found for the selected sensors and time range. Try adjusting your filters.")
|
| 182 |
+
|
| 183 |
+
|
| 184 |
+
def follow_up_section(
|
| 185 |
+
session_key: str,
|
| 186 |
+
llm_analyzer,
|
| 187 |
+
original_prompt: str,
|
| 188 |
+
original_analysis: str,
|
| 189 |
+
max_turns: int = 3,
|
| 190 |
+
):
|
| 191 |
+
"""Render a follow-up Q&A section below an analysis card.
|
| 192 |
+
|
| 193 |
+
Maintains a multi-turn conversation thread stored in
|
| 194 |
+
``st.session_state[session_key]`` as a list of ``{role, content}`` dicts.
|
| 195 |
+
|
| 196 |
+
Args:
|
| 197 |
+
session_key: Unique session state key for this conversation thread.
|
| 198 |
+
llm_analyzer: An ``LLMCycleAnalyzer`` instance with ``follow_up()`` method.
|
| 199 |
+
original_prompt: The original user prompt that generated the analysis.
|
| 200 |
+
original_analysis: The original assistant analysis text.
|
| 201 |
+
max_turns: Maximum follow-up exchanges (default 3).
|
| 202 |
+
"""
|
| 203 |
+
# Initialize conversation history if needed
|
| 204 |
+
if session_key not in st.session_state:
|
| 205 |
+
st.session_state[session_key] = []
|
| 206 |
+
|
| 207 |
+
followups = st.session_state[session_key]
|
| 208 |
+
|
| 209 |
+
# Display existing follow-up thread
|
| 210 |
+
if followups:
|
| 211 |
+
for msg in followups:
|
| 212 |
+
if msg["role"] == "user":
|
| 213 |
+
st.markdown(f"""
|
| 214 |
+
<div style="background: #1A1714; border-left: 3px solid #5BA3B5;
|
| 215 |
+
padding: 8px 12px; margin: 8px 0; border-radius: 4px;
|
| 216 |
+
font-size: 0.85rem; color: #B0A898;">
|
| 217 |
+
<strong style="color: #5BA3B5;">You:</strong> {msg['content']}
|
| 218 |
+
</div>
|
| 219 |
+
""", unsafe_allow_html=True)
|
| 220 |
+
else:
|
| 221 |
+
analysis_card("Murphy Follow-up", msg["content"])
|
| 222 |
+
|
| 223 |
+
# Show input if under max turns
|
| 224 |
+
turn_count = len([m for m in followups if m["role"] == "user"])
|
| 225 |
+
if turn_count >= max_turns:
|
| 226 |
+
st.caption(f"Maximum {max_turns} follow-up questions reached.")
|
| 227 |
+
return
|
| 228 |
+
|
| 229 |
+
# Follow-up input
|
| 230 |
+
input_col, btn_col = st.columns([5, 1])
|
| 231 |
+
with input_col:
|
| 232 |
+
followup_q = st.text_input(
|
| 233 |
+
"Ask a follow-up question",
|
| 234 |
+
placeholder="e.g. 'What about the temperature trend?' or 'Explain the plateau at 370 bar'",
|
| 235 |
+
key=f"{session_key}_input",
|
| 236 |
+
label_visibility="collapsed",
|
| 237 |
+
)
|
| 238 |
+
with btn_col:
|
| 239 |
+
followup_btn = st.button(
|
| 240 |
+
"Ask",
|
| 241 |
+
type="primary",
|
| 242 |
+
use_container_width=True,
|
| 243 |
+
key=f"{session_key}_btn",
|
| 244 |
+
)
|
| 245 |
+
|
| 246 |
+
if followup_btn and followup_q:
|
| 247 |
+
# Build full conversation history for the API call
|
| 248 |
+
conversation = [
|
| 249 |
+
{"role": "user", "content": original_prompt},
|
| 250 |
+
{"role": "assistant", "content": original_analysis},
|
| 251 |
+
] + followups
|
| 252 |
+
|
| 253 |
+
with st.spinner("Murphy is thinking..."):
|
| 254 |
+
response = llm_analyzer.follow_up(conversation, followup_q)
|
| 255 |
+
|
| 256 |
+
# Append to thread
|
| 257 |
+
followups.append({"role": "user", "content": followup_q})
|
| 258 |
+
followups.append({"role": "assistant", "content": response})
|
| 259 |
+
st.session_state[session_key] = followups
|
| 260 |
+
st.rerun()
|
ui/plotly_charts.py
ADDED
|
@@ -0,0 +1,383 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Plotly chart builders for CSH2 Web Dashboard — HUD Theme
|
| 3 |
+
"""
|
| 4 |
+
import plotly.graph_objects as go
|
| 5 |
+
from plotly.subplots import make_subplots
|
| 6 |
+
import pandas as pd
|
| 7 |
+
import numpy as np
|
| 8 |
+
from typing import List, Dict, Optional, Tuple
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
# Warm amber industrial palette
|
| 12 |
+
COLORS = [
|
| 13 |
+
'#D4A04A', # Warm amber (primary)
|
| 14 |
+
'#5B9A6E', # Muted green (nominal)
|
| 15 |
+
'#5BA3B5', # Muted teal (informational)
|
| 16 |
+
'#CC3030', # Muted red (alarm)
|
| 17 |
+
'#E8C47A', # Light amber
|
| 18 |
+
'#7DB88E', # Light green
|
| 19 |
+
'#7BBFCC', # Light teal
|
| 20 |
+
'#E06060', # Light red
|
| 21 |
+
'#C4A35A', # Gold
|
| 22 |
+
'#A0856A', # Warm brown
|
| 23 |
+
]
|
| 24 |
+
|
| 25 |
+
# Downsample threshold for charts (browser performance)
|
| 26 |
+
MAX_CHART_POINTS = 5000
|
| 27 |
+
|
| 28 |
+
# Reusable HUD layout config for all Plotly charts
|
| 29 |
+
HUD_LAYOUT = dict(
|
| 30 |
+
template='plotly_dark',
|
| 31 |
+
paper_bgcolor='#0C0A08',
|
| 32 |
+
plot_bgcolor='#0C0A08',
|
| 33 |
+
font=dict(family='JetBrains Mono, monospace', color='#B0A898', size=11),
|
| 34 |
+
title_font=dict(family='JetBrains Mono, monospace', color='#D4A04A', size=14),
|
| 35 |
+
hovermode='x unified',
|
| 36 |
+
hoverlabel=dict(
|
| 37 |
+
bgcolor='#12100E',
|
| 38 |
+
bordercolor='#D4A04A',
|
| 39 |
+
font=dict(family='JetBrains Mono, monospace', color='#E8E0D4', size=11),
|
| 40 |
+
),
|
| 41 |
+
legend=dict(
|
| 42 |
+
orientation='h',
|
| 43 |
+
yanchor='bottom',
|
| 44 |
+
y=1.02,
|
| 45 |
+
xanchor='right',
|
| 46 |
+
x=1,
|
| 47 |
+
font=dict(size=10, color='#B0A898'),
|
| 48 |
+
bgcolor='rgba(12, 10, 8, 0.8)',
|
| 49 |
+
bordercolor='#2A2520',
|
| 50 |
+
borderwidth=1,
|
| 51 |
+
),
|
| 52 |
+
)
|
| 53 |
+
|
| 54 |
+
# Reusable HUD axis config
|
| 55 |
+
HUD_AXIS = dict(
|
| 56 |
+
gridcolor='#201C18',
|
| 57 |
+
zerolinecolor='#2A2520',
|
| 58 |
+
tickfont=dict(color='#6B6358', size=10),
|
| 59 |
+
title_font=dict(color='#B0A898', size=11),
|
| 60 |
+
)
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
def _downsample_df(
|
| 64 |
+
df: pd.DataFrame,
|
| 65 |
+
tags: List[str],
|
| 66 |
+
max_points: int = MAX_CHART_POINTS,
|
| 67 |
+
) -> Tuple[pd.DataFrame, bool]:
|
| 68 |
+
"""
|
| 69 |
+
Downsample a DataFrame for charting using min/max bucketing.
|
| 70 |
+
Preserves peaks and valleys while reducing point count.
|
| 71 |
+
|
| 72 |
+
Returns (downsampled_df, was_downsampled).
|
| 73 |
+
"""
|
| 74 |
+
if len(df) <= max_points:
|
| 75 |
+
return df, False
|
| 76 |
+
|
| 77 |
+
n_buckets = max_points // 2 # each bucket yields ~2 rows (min + max)
|
| 78 |
+
bucket_size = max(len(df) // n_buckets, 2)
|
| 79 |
+
|
| 80 |
+
df = df.reset_index(drop=True)
|
| 81 |
+
df['_bucket'] = df.index // bucket_size
|
| 82 |
+
|
| 83 |
+
result_indices = {0, len(df) - 1} # always keep first and last
|
| 84 |
+
|
| 85 |
+
# Find the primary tag to drive bucket selection
|
| 86 |
+
primary_tag = None
|
| 87 |
+
for tag in tags:
|
| 88 |
+
if tag in df.columns:
|
| 89 |
+
primary_tag = tag
|
| 90 |
+
break
|
| 91 |
+
|
| 92 |
+
if primary_tag is None:
|
| 93 |
+
# Fallback: uniform sampling
|
| 94 |
+
step = max(len(df) // max_points, 1)
|
| 95 |
+
return df.iloc[::step].drop(columns=['_bucket'], errors='ignore'), True
|
| 96 |
+
|
| 97 |
+
for _, group in df.groupby('_bucket'):
|
| 98 |
+
if len(group) == 0:
|
| 99 |
+
continue
|
| 100 |
+
col = group[primary_tag].dropna()
|
| 101 |
+
if len(col) > 0:
|
| 102 |
+
result_indices.add(col.idxmin())
|
| 103 |
+
result_indices.add(col.idxmax())
|
| 104 |
+
|
| 105 |
+
result = df.loc[sorted(result_indices)].drop(columns=['_bucket'])
|
| 106 |
+
return result, True
|
| 107 |
+
|
| 108 |
+
|
| 109 |
+
def create_timeseries_chart(
|
| 110 |
+
df_pivot: pd.DataFrame,
|
| 111 |
+
tags: List[str],
|
| 112 |
+
title: str = "Sensor Data Over Time",
|
| 113 |
+
height: int = 500,
|
| 114 |
+
time_range: Optional[Tuple] = None,
|
| 115 |
+
) -> go.Figure:
|
| 116 |
+
"""Create an interactive time series chart from pivoted data"""
|
| 117 |
+
fig = go.Figure()
|
| 118 |
+
|
| 119 |
+
# Downsample for browser performance
|
| 120 |
+
df_plot, downsampled = _downsample_df(df_pivot, tags)
|
| 121 |
+
|
| 122 |
+
for i, tag in enumerate(tags):
|
| 123 |
+
if tag in df_plot.columns:
|
| 124 |
+
fig.add_trace(go.Scatter(
|
| 125 |
+
x=df_plot['timestamp'],
|
| 126 |
+
y=df_plot[tag],
|
| 127 |
+
mode='lines',
|
| 128 |
+
name=tag,
|
| 129 |
+
line=dict(color=COLORS[i % len(COLORS)], width=1.5),
|
| 130 |
+
hovertemplate=f'{tag}: %{{y:.2f}}<br>%{{x}}<extra></extra>',
|
| 131 |
+
))
|
| 132 |
+
|
| 133 |
+
ds_note = f" (downsampled {len(df_pivot):,} \u2192 {len(df_plot):,} pts)" if downsampled else ""
|
| 134 |
+
|
| 135 |
+
# Compute explicit x-axis range
|
| 136 |
+
xaxis_range = None
|
| 137 |
+
if time_range is not None:
|
| 138 |
+
_x_min, _x_max = pd.Timestamp(time_range[0]), pd.Timestamp(time_range[1])
|
| 139 |
+
duration = (_x_max - _x_min).total_seconds()
|
| 140 |
+
margin = pd.Timedelta(seconds=max(duration * 0.02, 1))
|
| 141 |
+
xaxis_range = [_x_min - margin, _x_max + margin]
|
| 142 |
+
elif 'timestamp' in df_plot.columns and len(df_plot) > 0:
|
| 143 |
+
_x_min = df_plot['timestamp'].min()
|
| 144 |
+
_x_max = df_plot['timestamp'].max()
|
| 145 |
+
duration = (_x_max - _x_min).total_seconds()
|
| 146 |
+
margin = pd.Timedelta(seconds=max(duration * 0.02, 1))
|
| 147 |
+
xaxis_range = [_x_min - margin, _x_max + margin]
|
| 148 |
+
|
| 149 |
+
xaxis_cfg = dict(
|
| 150 |
+
title='Time (ET)',
|
| 151 |
+
rangeslider=dict(visible=True, thickness=0.05, bgcolor='#12100E'),
|
| 152 |
+
type='date',
|
| 153 |
+
**HUD_AXIS,
|
| 154 |
+
)
|
| 155 |
+
if xaxis_range is not None:
|
| 156 |
+
xaxis_cfg['range'] = xaxis_range
|
| 157 |
+
|
| 158 |
+
fig.update_layout(
|
| 159 |
+
**HUD_LAYOUT,
|
| 160 |
+
title=dict(text=title + ds_note, font=HUD_LAYOUT['title_font']),
|
| 161 |
+
xaxis=xaxis_cfg,
|
| 162 |
+
yaxis=dict(title='Value', **HUD_AXIS),
|
| 163 |
+
height=height,
|
| 164 |
+
margin=dict(l=60, r=20, t=60, b=40),
|
| 165 |
+
)
|
| 166 |
+
|
| 167 |
+
return fig
|
| 168 |
+
|
| 169 |
+
|
| 170 |
+
def create_multi_axis_chart(
|
| 171 |
+
df: pd.DataFrame,
|
| 172 |
+
pressure_tags: List[str],
|
| 173 |
+
temp_tags: List[str],
|
| 174 |
+
other_tags: List[str] = None,
|
| 175 |
+
title: str = "Cycle Detail",
|
| 176 |
+
height: int = 600,
|
| 177 |
+
plateaus: Dict[str, List[Dict]] = None,
|
| 178 |
+
time_range: Optional[Tuple] = None,
|
| 179 |
+
) -> go.Figure:
|
| 180 |
+
"""
|
| 181 |
+
Create multi-axis chart for cycle detail view.
|
| 182 |
+
Left Y: Pressure, Right Y: Temperature, Subplot: Other sensors
|
| 183 |
+
"""
|
| 184 |
+
has_other = other_tags and any(t in df.columns for t in other_tags)
|
| 185 |
+
rows = 2 if has_other else 1
|
| 186 |
+
|
| 187 |
+
# Downsample for browser performance
|
| 188 |
+
all_chart_tags = pressure_tags + temp_tags + (other_tags or [])
|
| 189 |
+
df_plot, _ = _downsample_df(df, all_chart_tags)
|
| 190 |
+
|
| 191 |
+
fig = make_subplots(
|
| 192 |
+
rows=rows, cols=1,
|
| 193 |
+
shared_xaxes=True,
|
| 194 |
+
vertical_spacing=0.12,
|
| 195 |
+
specs=[[{"secondary_y": True}]] + ([[{"secondary_y": False}]] if has_other else []),
|
| 196 |
+
)
|
| 197 |
+
|
| 198 |
+
color_idx = 0
|
| 199 |
+
|
| 200 |
+
# Pressure traces (left Y)
|
| 201 |
+
for tag in pressure_tags:
|
| 202 |
+
if tag in df_plot.columns:
|
| 203 |
+
fig.add_trace(go.Scatter(
|
| 204 |
+
x=df_plot['timestamp'], y=df_plot[tag],
|
| 205 |
+
mode='lines', name=tag,
|
| 206 |
+
line=dict(color=COLORS[color_idx % len(COLORS)], width=2),
|
| 207 |
+
hovertemplate=f'{tag}: %{{y:.1f}} bar<extra></extra>',
|
| 208 |
+
), row=1, col=1, secondary_y=False)
|
| 209 |
+
color_idx += 1
|
| 210 |
+
|
| 211 |
+
# Temperature traces (right Y)
|
| 212 |
+
for tag in temp_tags:
|
| 213 |
+
if tag in df_plot.columns:
|
| 214 |
+
fig.add_trace(go.Scatter(
|
| 215 |
+
x=df_plot['timestamp'], y=df_plot[tag],
|
| 216 |
+
mode='lines', name=tag,
|
| 217 |
+
line=dict(color=COLORS[color_idx % len(COLORS)], width=2, dash='dash'),
|
| 218 |
+
hovertemplate=f'{tag}: %{{y:.1f}} K<extra></extra>',
|
| 219 |
+
), row=1, col=1, secondary_y=True)
|
| 220 |
+
color_idx += 1
|
| 221 |
+
|
| 222 |
+
# Other sensor traces (subplot 2)
|
| 223 |
+
if has_other:
|
| 224 |
+
for tag in other_tags:
|
| 225 |
+
if tag in df_plot.columns:
|
| 226 |
+
fig.add_trace(go.Scatter(
|
| 227 |
+
x=df_plot['timestamp'], y=df_plot[tag],
|
| 228 |
+
mode='lines', name=tag,
|
| 229 |
+
line=dict(color=COLORS[color_idx % len(COLORS)], width=1.5),
|
| 230 |
+
hovertemplate=f'{tag}: %{{y:.2f}}<extra></extra>',
|
| 231 |
+
), row=2, col=1)
|
| 232 |
+
color_idx += 1
|
| 233 |
+
|
| 234 |
+
# Compute x-axis bounds for clamping vrects and setting explicit range
|
| 235 |
+
if time_range is not None:
|
| 236 |
+
_x_min, _x_max = pd.Timestamp(time_range[0]), pd.Timestamp(time_range[1])
|
| 237 |
+
elif 'timestamp' in df_plot.columns and len(df_plot) > 0:
|
| 238 |
+
_x_min = df_plot['timestamp'].min()
|
| 239 |
+
_x_max = df_plot['timestamp'].max()
|
| 240 |
+
else:
|
| 241 |
+
_x_min = _x_max = None
|
| 242 |
+
|
| 243 |
+
# Add plateau highlights (clamped to data range)
|
| 244 |
+
if plateaus:
|
| 245 |
+
for tag, periods in plateaus.items():
|
| 246 |
+
color = 'rgba(212, 160, 74, 0.1)' if 'PT' in tag or 'pressure' in tag.lower() else 'rgba(91, 154, 110, 0.1)'
|
| 247 |
+
for p in periods:
|
| 248 |
+
vx0, vx1 = pd.Timestamp(p['start']), pd.Timestamp(p['end'])
|
| 249 |
+
# Normalize timezone awareness to match _x_min/_x_max
|
| 250 |
+
if _x_min is not None:
|
| 251 |
+
if _x_min.tzinfo is not None and vx0.tzinfo is None:
|
| 252 |
+
vx0 = vx0.tz_localize(_x_min.tzinfo)
|
| 253 |
+
vx1 = vx1.tz_localize(_x_min.tzinfo)
|
| 254 |
+
elif _x_min.tzinfo is None and vx0.tzinfo is not None:
|
| 255 |
+
vx0 = vx0.tz_localize(None)
|
| 256 |
+
vx1 = vx1.tz_localize(None)
|
| 257 |
+
# Clamp vrect bounds to prevent x-axis expansion
|
| 258 |
+
if _x_min is not None:
|
| 259 |
+
vx0 = max(vx0, _x_min)
|
| 260 |
+
if _x_max is not None:
|
| 261 |
+
vx1 = min(vx1, _x_max)
|
| 262 |
+
if _x_min is not None and vx1 < _x_min:
|
| 263 |
+
continue
|
| 264 |
+
if _x_max is not None and vx0 > _x_max:
|
| 265 |
+
continue
|
| 266 |
+
fig.add_vrect(
|
| 267 |
+
x0=vx0, x1=vx1,
|
| 268 |
+
fillcolor=color,
|
| 269 |
+
layer='below',
|
| 270 |
+
line_width=0,
|
| 271 |
+
row=1, col=1,
|
| 272 |
+
annotation_text=f"{tag} plateau",
|
| 273 |
+
annotation_position="top left",
|
| 274 |
+
annotation_font_size=9,
|
| 275 |
+
annotation_font_color='#6B6358',
|
| 276 |
+
)
|
| 277 |
+
|
| 278 |
+
fig.update_layout(
|
| 279 |
+
**HUD_LAYOUT,
|
| 280 |
+
title=dict(text=title, font=HUD_LAYOUT['title_font']),
|
| 281 |
+
height=height,
|
| 282 |
+
margin=dict(l=60, r=60, t=120, b=40),
|
| 283 |
+
)
|
| 284 |
+
# Override legend position separately to avoid duplicate 'legend' kwarg
|
| 285 |
+
# (HUD_LAYOUT already contains 'legend', so passing it again raises TypeError)
|
| 286 |
+
fig.update_layout(legend_y=1.10)
|
| 287 |
+
|
| 288 |
+
fig.update_yaxes(title_text="Pressure (bar)", row=1, col=1, secondary_y=False, **HUD_AXIS)
|
| 289 |
+
fig.update_yaxes(title_text="Temperature (K)", row=1, col=1, secondary_y=True, **HUD_AXIS)
|
| 290 |
+
if has_other:
|
| 291 |
+
fig.update_yaxes(title_text="Motor & Flow", row=2, col=1, **HUD_AXIS)
|
| 292 |
+
|
| 293 |
+
# Set explicit x-axis range to prevent Plotly auto-expansion
|
| 294 |
+
if _x_min is not None and _x_max is not None:
|
| 295 |
+
duration = (_x_max - _x_min).total_seconds()
|
| 296 |
+
margin = pd.Timedelta(seconds=max(duration * 0.02, 1))
|
| 297 |
+
fig.update_xaxes(range=[_x_min - margin, _x_max + margin], **HUD_AXIS)
|
| 298 |
+
else:
|
| 299 |
+
fig.update_xaxes(**HUD_AXIS)
|
| 300 |
+
|
| 301 |
+
return fig
|
| 302 |
+
|
| 303 |
+
|
| 304 |
+
def create_cycle_timeline(cycles: List[Dict], month: str, height: int = 300) -> go.Figure:
|
| 305 |
+
"""Create a Gantt-style timeline of testing cycles in a month"""
|
| 306 |
+
fig = go.Figure()
|
| 307 |
+
|
| 308 |
+
for i, c in enumerate(cycles):
|
| 309 |
+
color = COLORS[i % len(COLORS)]
|
| 310 |
+
# Use Eastern display times for hover, fall back to start_time
|
| 311 |
+
_start_et = c.get('start_time_et') or c['start_time']
|
| 312 |
+
_end_et = c.get('end_time_et') or c['end_time']
|
| 313 |
+
fig.add_trace(go.Bar(
|
| 314 |
+
x=[c['duration_minutes']],
|
| 315 |
+
y=[f"Cycle {c['cycle_id']}"],
|
| 316 |
+
orientation='h',
|
| 317 |
+
marker=dict(color=color, line=dict(width=1, color='#2A2520')),
|
| 318 |
+
text=f"{c['duration_minutes']:.0f} min | Peak (PT130): {c.get('peak_pressure', 0):.0f} bar",
|
| 319 |
+
textposition='inside',
|
| 320 |
+
textfont=dict(color='#0C0A08', size=10),
|
| 321 |
+
hovertemplate=(
|
| 322 |
+
f"Cycle {c['cycle_id']}<br>"
|
| 323 |
+
f"Start: {_start_et.strftime('%b %d %H:%M')} ET<br>"
|
| 324 |
+
f"End: {_end_et.strftime('%b %d %H:%M')} ET<br>"
|
| 325 |
+
f"Duration: {c['duration_minutes']:.1f} min<br>"
|
| 326 |
+
f"Peak Pressure (PT130): {c.get('peak_pressure', 0):.0f} bar"
|
| 327 |
+
"<extra></extra>"
|
| 328 |
+
),
|
| 329 |
+
showlegend=False,
|
| 330 |
+
))
|
| 331 |
+
|
| 332 |
+
fig.update_layout(
|
| 333 |
+
**HUD_LAYOUT,
|
| 334 |
+
title=dict(text=f"TESTING CYCLES — {month}", font=HUD_LAYOUT['title_font']),
|
| 335 |
+
xaxis=dict(title='Duration (minutes)', **HUD_AXIS),
|
| 336 |
+
yaxis=dict(autorange='reversed', **HUD_AXIS),
|
| 337 |
+
height=height,
|
| 338 |
+
margin=dict(l=80, r=20, t=50, b=40),
|
| 339 |
+
)
|
| 340 |
+
|
| 341 |
+
return fig
|
| 342 |
+
|
| 343 |
+
|
| 344 |
+
def create_comparison_chart(
|
| 345 |
+
df_a: pd.DataFrame,
|
| 346 |
+
df_b: pd.DataFrame,
|
| 347 |
+
tag: str,
|
| 348 |
+
label_a: str = "Cycle A",
|
| 349 |
+
label_b: str = "Cycle B",
|
| 350 |
+
height: int = 400,
|
| 351 |
+
) -> go.Figure:
|
| 352 |
+
"""Overlay two cycles on normalized time axis for comparison"""
|
| 353 |
+
fig = go.Figure()
|
| 354 |
+
|
| 355 |
+
if tag in df_a.columns:
|
| 356 |
+
# Normalize time to minutes from start
|
| 357 |
+
t0_a = df_a['timestamp'].min()
|
| 358 |
+
minutes_a = (df_a['timestamp'] - t0_a).dt.total_seconds() / 60
|
| 359 |
+
fig.add_trace(go.Scatter(
|
| 360 |
+
x=minutes_a, y=df_a[tag],
|
| 361 |
+
mode='lines', name=f"{label_a} — {tag}",
|
| 362 |
+
line=dict(color=COLORS[0], width=2),
|
| 363 |
+
))
|
| 364 |
+
|
| 365 |
+
if tag in df_b.columns:
|
| 366 |
+
t0_b = df_b['timestamp'].min()
|
| 367 |
+
minutes_b = (df_b['timestamp'] - t0_b).dt.total_seconds() / 60
|
| 368 |
+
fig.add_trace(go.Scatter(
|
| 369 |
+
x=minutes_b, y=df_b[tag],
|
| 370 |
+
mode='lines', name=f"{label_b} — {tag}",
|
| 371 |
+
line=dict(color=COLORS[1], width=2),
|
| 372 |
+
))
|
| 373 |
+
|
| 374 |
+
fig.update_layout(
|
| 375 |
+
**HUD_LAYOUT,
|
| 376 |
+
title=dict(text=f"{tag} — CYCLE COMPARISON", font=HUD_LAYOUT['title_font']),
|
| 377 |
+
xaxis=dict(title='Minutes from Cycle Start', **HUD_AXIS),
|
| 378 |
+
yaxis=dict(title=tag, **HUD_AXIS),
|
| 379 |
+
height=height,
|
| 380 |
+
margin=dict(l=60, r=20, t=50, b=40),
|
| 381 |
+
)
|
| 382 |
+
|
| 383 |
+
return fig
|
ui/styles.py
ADDED
|
@@ -0,0 +1,442 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
Custom CSS styles for CSH2 Web Dashboard — Warm Amber Industrial Theme
|
| 3 |
+
"""
|
| 4 |
+
import streamlit as st
|
| 5 |
+
|
| 6 |
+
|
| 7 |
+
def inject_custom_css():
|
| 8 |
+
"""Inject Warm Amber HUD / Mission Control CSS theme"""
|
| 9 |
+
st.markdown("""
|
| 10 |
+
<style>
|
| 11 |
+
/* ============================================================
|
| 12 |
+
A. FONT IMPORT & GLOBAL OVERRIDE
|
| 13 |
+
============================================================ */
|
| 14 |
+
@import url('https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@300;400;500;600;700&display=swap');
|
| 15 |
+
|
| 16 |
+
* {
|
| 17 |
+
font-family: 'JetBrains Mono', 'Fira Code', 'Source Code Pro', 'Courier New', monospace !important;
|
| 18 |
+
}
|
| 19 |
+
|
| 20 |
+
/* Restore Material Icons/Symbols for Streamlit's internal UI elements
|
| 21 |
+
(sidebar collapse button, nav icons) — prevents ligature text leak */
|
| 22 |
+
[data-testid="stSidebarCollapseButton"],
|
| 23 |
+
[data-testid="stSidebarCollapseButton"] *,
|
| 24 |
+
[data-testid="stSidebarNavLink"] span[data-testid="stIconMaterial"],
|
| 25 |
+
[data-testid="stSidebarNavLink"] span[data-testid="stIconMaterial"] *,
|
| 26 |
+
.material-symbols-rounded,
|
| 27 |
+
.material-icons {
|
| 28 |
+
font-family: 'Material Symbols Rounded', 'Material Icons', sans-serif !important;
|
| 29 |
+
font-variant-ligatures: normal !important;
|
| 30 |
+
}
|
| 31 |
+
|
| 32 |
+
/* ============================================================
|
| 33 |
+
B. MAIN CONTAINER (scanline removed)
|
| 34 |
+
============================================================ */
|
| 35 |
+
.main .block-container {
|
| 36 |
+
padding-top: 2rem;
|
| 37 |
+
padding-bottom: 2rem;
|
| 38 |
+
max-width: 1400px;
|
| 39 |
+
}
|
| 40 |
+
|
| 41 |
+
/* ============================================================
|
| 42 |
+
C. TYPOGRAPHY
|
| 43 |
+
============================================================ */
|
| 44 |
+
h1 {
|
| 45 |
+
color: #D4A04A !important;
|
| 46 |
+
font-weight: 700 !important;
|
| 47 |
+
letter-spacing: 1px;
|
| 48 |
+
text-transform: uppercase;
|
| 49 |
+
font-size: 1.5rem !important;
|
| 50 |
+
}
|
| 51 |
+
|
| 52 |
+
h2 {
|
| 53 |
+
color: #B0A898 !important;
|
| 54 |
+
font-weight: 600 !important;
|
| 55 |
+
border-bottom: 1px solid #2A2520;
|
| 56 |
+
padding-bottom: 0.3rem;
|
| 57 |
+
text-transform: uppercase;
|
| 58 |
+
font-size: 1.1rem !important;
|
| 59 |
+
letter-spacing: 0.5px;
|
| 60 |
+
}
|
| 61 |
+
|
| 62 |
+
h3 {
|
| 63 |
+
color: #B0A898 !important;
|
| 64 |
+
font-weight: 500 !important;
|
| 65 |
+
font-size: 0.95rem !important;
|
| 66 |
+
letter-spacing: 0.3px;
|
| 67 |
+
}
|
| 68 |
+
|
| 69 |
+
/* ============================================================
|
| 70 |
+
D. METRIC CARDS — HUD PANELS WITH CORNER BRACKETS
|
| 71 |
+
============================================================ */
|
| 72 |
+
div[data-testid="stMetric"] {
|
| 73 |
+
background-color: #12100E;
|
| 74 |
+
border: 1px solid #2A2520;
|
| 75 |
+
border-radius: 2px;
|
| 76 |
+
padding: 12px 16px;
|
| 77 |
+
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.3);
|
| 78 |
+
position: relative;
|
| 79 |
+
transition: box-shadow 0.3s ease;
|
| 80 |
+
}
|
| 81 |
+
|
| 82 |
+
div[data-testid="stMetric"]:hover {
|
| 83 |
+
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.4);
|
| 84 |
+
}
|
| 85 |
+
|
| 86 |
+
/* Corner bracket — top left */
|
| 87 |
+
div[data-testid="stMetric"]::before {
|
| 88 |
+
content: '';
|
| 89 |
+
position: absolute;
|
| 90 |
+
top: 0;
|
| 91 |
+
left: 0;
|
| 92 |
+
width: 12px;
|
| 93 |
+
height: 12px;
|
| 94 |
+
border-top: 2px solid #D4A04A;
|
| 95 |
+
border-left: 2px solid #D4A04A;
|
| 96 |
+
pointer-events: none;
|
| 97 |
+
}
|
| 98 |
+
|
| 99 |
+
/* Corner bracket — bottom right */
|
| 100 |
+
div[data-testid="stMetric"]::after {
|
| 101 |
+
content: '';
|
| 102 |
+
position: absolute;
|
| 103 |
+
bottom: 0;
|
| 104 |
+
right: 0;
|
| 105 |
+
width: 12px;
|
| 106 |
+
height: 12px;
|
| 107 |
+
border-bottom: 2px solid #D4A04A;
|
| 108 |
+
border-right: 2px solid #D4A04A;
|
| 109 |
+
pointer-events: none;
|
| 110 |
+
}
|
| 111 |
+
|
| 112 |
+
div[data-testid="stMetric"] label {
|
| 113 |
+
color: #6B6358 !important;
|
| 114 |
+
font-size: 0.7rem !important;
|
| 115 |
+
text-transform: uppercase;
|
| 116 |
+
letter-spacing: 1.5px;
|
| 117 |
+
}
|
| 118 |
+
|
| 119 |
+
div[data-testid="stMetric"] [data-testid="stMetricValue"] {
|
| 120 |
+
color: #FFFFFF !important;
|
| 121 |
+
font-size: 1.5rem !important;
|
| 122 |
+
font-weight: 700 !important;
|
| 123 |
+
}
|
| 124 |
+
|
| 125 |
+
div[data-testid="stMetric"] [data-testid="stMetricDelta"] svg {
|
| 126 |
+
fill: #5B9A6E;
|
| 127 |
+
}
|
| 128 |
+
|
| 129 |
+
/* ============================================================
|
| 130 |
+
E. SIDEBAR
|
| 131 |
+
============================================================ */
|
| 132 |
+
section[data-testid="stSidebar"] {
|
| 133 |
+
background-color: #0C0A08;
|
| 134 |
+
border-right: 1px solid #2A2520;
|
| 135 |
+
}
|
| 136 |
+
|
| 137 |
+
section[data-testid="stSidebar"]::after {
|
| 138 |
+
content: '';
|
| 139 |
+
position: absolute;
|
| 140 |
+
top: 0;
|
| 141 |
+
left: 0;
|
| 142 |
+
right: 0;
|
| 143 |
+
height: 2px;
|
| 144 |
+
background: linear-gradient(90deg, transparent, #D4A04A, transparent);
|
| 145 |
+
pointer-events: none;
|
| 146 |
+
}
|
| 147 |
+
|
| 148 |
+
section[data-testid="stSidebar"] h1,
|
| 149 |
+
section[data-testid="stSidebar"] h2,
|
| 150 |
+
section[data-testid="stSidebar"] h3 {
|
| 151 |
+
color: #D4A04A !important;
|
| 152 |
+
font-size: 1rem !important;
|
| 153 |
+
}
|
| 154 |
+
|
| 155 |
+
section[data-testid="stSidebar"] .stCaption {
|
| 156 |
+
color: #6B6358 !important;
|
| 157 |
+
}
|
| 158 |
+
|
| 159 |
+
/* ============================================================
|
| 160 |
+
F. BUTTONS — OUTLINE / HUD STYLE
|
| 161 |
+
============================================================ */
|
| 162 |
+
.stButton > button {
|
| 163 |
+
border-radius: 2px;
|
| 164 |
+
border: 1px solid #D4A04A;
|
| 165 |
+
background-color: transparent;
|
| 166 |
+
color: #D4A04A !important;
|
| 167 |
+
font-weight: 600;
|
| 168 |
+
text-transform: uppercase;
|
| 169 |
+
letter-spacing: 1px;
|
| 170 |
+
font-size: 0.8rem;
|
| 171 |
+
transition: all 0.2s;
|
| 172 |
+
}
|
| 173 |
+
|
| 174 |
+
.stButton > button:hover {
|
| 175 |
+
background-color: rgba(212, 160, 74, 0.1);
|
| 176 |
+
border-color: #D4A04A;
|
| 177 |
+
box-shadow: 0 0 12px rgba(212, 160, 74, 0.2);
|
| 178 |
+
color: #D4A04A !important;
|
| 179 |
+
}
|
| 180 |
+
|
| 181 |
+
.stButton > button:focus {
|
| 182 |
+
color: #D4A04A !important;
|
| 183 |
+
}
|
| 184 |
+
|
| 185 |
+
/* Primary buttons */
|
| 186 |
+
.stButton > button[kind="primary"],
|
| 187 |
+
.stButton > button[data-testid="stBaseButton-primary"] {
|
| 188 |
+
background-color: rgba(212, 160, 74, 0.15);
|
| 189 |
+
border-color: #D4A04A;
|
| 190 |
+
color: #D4A04A !important;
|
| 191 |
+
}
|
| 192 |
+
|
| 193 |
+
.stButton > button[kind="primary"]:hover,
|
| 194 |
+
.stButton > button[data-testid="stBaseButton-primary"]:hover {
|
| 195 |
+
background-color: rgba(212, 160, 74, 0.25);
|
| 196 |
+
}
|
| 197 |
+
|
| 198 |
+
/* Download button — muted teal accent */
|
| 199 |
+
.stDownloadButton > button {
|
| 200 |
+
background-color: transparent !important;
|
| 201 |
+
border: 1px solid #5BA3B5 !important;
|
| 202 |
+
color: #5BA3B5 !important;
|
| 203 |
+
border-radius: 2px;
|
| 204 |
+
text-transform: uppercase;
|
| 205 |
+
letter-spacing: 1px;
|
| 206 |
+
font-size: 0.8rem;
|
| 207 |
+
}
|
| 208 |
+
|
| 209 |
+
.stDownloadButton > button:hover {
|
| 210 |
+
background-color: rgba(91, 163, 181, 0.1) !important;
|
| 211 |
+
box-shadow: 0 0 12px rgba(91, 163, 181, 0.2) !important;
|
| 212 |
+
}
|
| 213 |
+
|
| 214 |
+
/* ============================================================
|
| 215 |
+
G. ANALYSIS CARDS — TERMINAL OUTPUT AESTHETIC
|
| 216 |
+
============================================================ */
|
| 217 |
+
.analysis-card {
|
| 218 |
+
background-color: #0C0A08;
|
| 219 |
+
border: 1px solid #2A2520;
|
| 220 |
+
border-left: 3px solid #D4A04A;
|
| 221 |
+
border-radius: 2px;
|
| 222 |
+
padding: 20px 24px;
|
| 223 |
+
margin: 16px 0;
|
| 224 |
+
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.2);
|
| 225 |
+
position: relative;
|
| 226 |
+
}
|
| 227 |
+
|
| 228 |
+
.analysis-card::before {
|
| 229 |
+
content: '\25B8 ANALYSIS OUTPUT';
|
| 230 |
+
display: block;
|
| 231 |
+
color: #D4A04A;
|
| 232 |
+
font-size: 0.65rem;
|
| 233 |
+
letter-spacing: 2px;
|
| 234 |
+
text-transform: uppercase;
|
| 235 |
+
margin-bottom: 12px;
|
| 236 |
+
padding-bottom: 8px;
|
| 237 |
+
border-bottom: 1px solid #2A2520;
|
| 238 |
+
opacity: 0.7;
|
| 239 |
+
}
|
| 240 |
+
|
| 241 |
+
.analysis-card h4 {
|
| 242 |
+
color: #D4A04A !important;
|
| 243 |
+
margin-top: 0;
|
| 244 |
+
font-size: 0.9rem !important;
|
| 245 |
+
text-transform: uppercase;
|
| 246 |
+
letter-spacing: 1px;
|
| 247 |
+
}
|
| 248 |
+
|
| 249 |
+
.analysis-card div {
|
| 250 |
+
color: #B0A898 !important;
|
| 251 |
+
line-height: 1.7;
|
| 252 |
+
font-size: 0.85rem;
|
| 253 |
+
}
|
| 254 |
+
|
| 255 |
+
/* ============================================================
|
| 256 |
+
H. TABS
|
| 257 |
+
============================================================ */
|
| 258 |
+
.stTabs [data-baseweb="tab-list"] {
|
| 259 |
+
gap: 0px;
|
| 260 |
+
border-bottom: 1px solid #2A2520;
|
| 261 |
+
}
|
| 262 |
+
|
| 263 |
+
.stTabs [data-baseweb="tab"] {
|
| 264 |
+
border-radius: 0;
|
| 265 |
+
padding: 8px 20px;
|
| 266 |
+
color: #6B6358;
|
| 267 |
+
border-bottom: 2px solid transparent;
|
| 268 |
+
}
|
| 269 |
+
|
| 270 |
+
.stTabs [data-baseweb="tab"][aria-selected="true"] {
|
| 271 |
+
color: #D4A04A !important;
|
| 272 |
+
border-bottom: 2px solid #D4A04A !important;
|
| 273 |
+
background-color: rgba(212, 160, 74, 0.05);
|
| 274 |
+
}
|
| 275 |
+
|
| 276 |
+
/* ============================================================
|
| 277 |
+
I. EXPANDERS, SELECTBOX, DIVIDERS, DATAFRAME
|
| 278 |
+
============================================================ */
|
| 279 |
+
.streamlit-expanderHeader {
|
| 280 |
+
background-color: #12100E !important;
|
| 281 |
+
border-radius: 2px;
|
| 282 |
+
border: 1px solid #2A2520;
|
| 283 |
+
color: #B0A898 !important;
|
| 284 |
+
}
|
| 285 |
+
|
| 286 |
+
.stSelectbox > div > div {
|
| 287 |
+
border-radius: 2px;
|
| 288 |
+
border-color: #2A2520;
|
| 289 |
+
}
|
| 290 |
+
|
| 291 |
+
hr {
|
| 292 |
+
border-color: #2A2520 !important;
|
| 293 |
+
}
|
| 294 |
+
|
| 295 |
+
.stDataFrame {
|
| 296 |
+
border-radius: 2px;
|
| 297 |
+
overflow: hidden;
|
| 298 |
+
border: 1px solid #2A2520;
|
| 299 |
+
}
|
| 300 |
+
|
| 301 |
+
/* ============================================================
|
| 302 |
+
J. SPINNER
|
| 303 |
+
============================================================ */
|
| 304 |
+
.stSpinner > div {
|
| 305 |
+
border-color: #D4A04A !important;
|
| 306 |
+
}
|
| 307 |
+
|
| 308 |
+
/* ============================================================
|
| 309 |
+
K. ANIMATIONS — PULSE (scanline removed)
|
| 310 |
+
============================================================ */
|
| 311 |
+
@keyframes pulse-nominal {
|
| 312 |
+
0%, 100% { box-shadow: 0 0 6px rgba(91, 154, 110, 0.1); }
|
| 313 |
+
50% { box-shadow: 0 0 12px rgba(91, 154, 110, 0.2); }
|
| 314 |
+
}
|
| 315 |
+
|
| 316 |
+
@keyframes pulse-amber {
|
| 317 |
+
0%, 100% { box-shadow: 0 0 6px rgba(204, 143, 0, 0.1); }
|
| 318 |
+
50% { box-shadow: 0 0 12px rgba(204, 143, 0, 0.2); }
|
| 319 |
+
}
|
| 320 |
+
|
| 321 |
+
@keyframes pulse-red {
|
| 322 |
+
0%, 100% { box-shadow: 0 0 6px rgba(204, 48, 48, 0.1); }
|
| 323 |
+
50% { box-shadow: 0 0 12px rgba(204, 48, 48, 0.2); }
|
| 324 |
+
}
|
| 325 |
+
|
| 326 |
+
/* ============================================================
|
| 327 |
+
L. STATUS CLASSES (for dynamic component use)
|
| 328 |
+
============================================================ */
|
| 329 |
+
.status-nominal {
|
| 330 |
+
border-color: #5B9A6E !important;
|
| 331 |
+
}
|
| 332 |
+
.status-nominal .hud-indicator {
|
| 333 |
+
background-color: #5B9A6E;
|
| 334 |
+
box-shadow: 0 0 6px #5B9A6E;
|
| 335 |
+
}
|
| 336 |
+
|
| 337 |
+
.status-warning {
|
| 338 |
+
border-color: #CC8F00 !important;
|
| 339 |
+
animation: pulse-amber 2s ease-in-out infinite;
|
| 340 |
+
}
|
| 341 |
+
.status-warning .hud-indicator {
|
| 342 |
+
background-color: #CC8F00;
|
| 343 |
+
box-shadow: 0 0 6px #CC8F00;
|
| 344 |
+
}
|
| 345 |
+
|
| 346 |
+
.status-alarm {
|
| 347 |
+
border-color: #CC3030 !important;
|
| 348 |
+
animation: pulse-red 1.5s ease-in-out infinite;
|
| 349 |
+
}
|
| 350 |
+
.status-alarm .hud-indicator {
|
| 351 |
+
background-color: #CC3030;
|
| 352 |
+
box-shadow: 0 0 6px #CC3030;
|
| 353 |
+
}
|
| 354 |
+
|
| 355 |
+
.hud-indicator {
|
| 356 |
+
display: inline-block;
|
| 357 |
+
width: 6px;
|
| 358 |
+
height: 6px;
|
| 359 |
+
border-radius: 50%;
|
| 360 |
+
margin-right: 6px;
|
| 361 |
+
vertical-align: middle;
|
| 362 |
+
}
|
| 363 |
+
|
| 364 |
+
/* ============================================================
|
| 365 |
+
M. HUD PAGE HEADER (used by components.py)
|
| 366 |
+
============================================================ */
|
| 367 |
+
.hud-page-header {
|
| 368 |
+
margin-bottom: 1.5rem;
|
| 369 |
+
}
|
| 370 |
+
|
| 371 |
+
.hud-subtitle {
|
| 372 |
+
color: #6B6358;
|
| 373 |
+
font-size: 0.75rem;
|
| 374 |
+
letter-spacing: 0.5px;
|
| 375 |
+
margin-bottom: 0.5rem;
|
| 376 |
+
}
|
| 377 |
+
|
| 378 |
+
.hud-header-line {
|
| 379 |
+
height: 1px;
|
| 380 |
+
background: linear-gradient(90deg, #D4A04A, #2A2520 60%, transparent);
|
| 381 |
+
}
|
| 382 |
+
|
| 383 |
+
/* ============================================================
|
| 384 |
+
N. HIDE ERROR HELPER LINKS
|
| 385 |
+
============================================================ */
|
| 386 |
+
[data-testid="stException"] a[href*="google"],
|
| 387 |
+
[data-testid="stException"] a[href*="chatgpt"],
|
| 388 |
+
[data-testid="stException"] a[href*="copy"],
|
| 389 |
+
div[data-testid="stException"] div:last-child {
|
| 390 |
+
display: none !important;
|
| 391 |
+
}
|
| 392 |
+
|
| 393 |
+
/* ============================================================
|
| 394 |
+
O. BRAND FOOTER
|
| 395 |
+
============================================================ */
|
| 396 |
+
.brand-footer {
|
| 397 |
+
position: fixed;
|
| 398 |
+
bottom: 0;
|
| 399 |
+
left: 0;
|
| 400 |
+
width: 100%;
|
| 401 |
+
text-align: center;
|
| 402 |
+
padding: 6px;
|
| 403 |
+
background-color: #0C0A08;
|
| 404 |
+
color: #6B6358;
|
| 405 |
+
font-size: 0.65rem;
|
| 406 |
+
border-top: 1px solid #2A2520;
|
| 407 |
+
z-index: 999;
|
| 408 |
+
letter-spacing: 1px;
|
| 409 |
+
text-transform: uppercase;
|
| 410 |
+
}
|
| 411 |
+
|
| 412 |
+
/* ============================================================
|
| 413 |
+
P. TEXT INPUT / TEXTAREA STYLING
|
| 414 |
+
============================================================ */
|
| 415 |
+
.stTextInput > div > div > input,
|
| 416 |
+
.stTextArea > div > div > textarea {
|
| 417 |
+
border-color: #2A2520;
|
| 418 |
+
border-radius: 2px;
|
| 419 |
+
color: #E8E0D4;
|
| 420 |
+
}
|
| 421 |
+
|
| 422 |
+
.stTextInput > div > div > input:focus,
|
| 423 |
+
.stTextArea > div > div > textarea:focus {
|
| 424 |
+
border-color: #D4A04A;
|
| 425 |
+
box-shadow: 0 0 6px rgba(212, 160, 74, 0.15);
|
| 426 |
+
}
|
| 427 |
+
|
| 428 |
+
/* ============================================================
|
| 429 |
+
Q. MULTISELECT / DATE INPUT
|
| 430 |
+
============================================================ */
|
| 431 |
+
.stMultiSelect > div > div {
|
| 432 |
+
border-radius: 2px;
|
| 433 |
+
border-color: #2A2520;
|
| 434 |
+
}
|
| 435 |
+
|
| 436 |
+
.stDateInput > div > div > input {
|
| 437 |
+
border-radius: 2px;
|
| 438 |
+
border-color: #2A2520;
|
| 439 |
+
}
|
| 440 |
+
|
| 441 |
+
</style>
|
| 442 |
+
""", unsafe_allow_html=True)
|