ARKAISW commited on
Commit
d64fd01
·
0 Parent(s):

Finalizing AUTOPSY for Hackathon submission

Browse files
.env.example ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ FRED_API_KEY=your_fred_api_key_here
2
+
3
+ # ── LLM Config (any OpenAI-compatible provider) ─────────────────────────────
4
+ # Pick ONE provider below and uncomment its lines. All are free-tier compatible.
5
+
6
+ # --- Groq (free, fast) ---
7
+ # LLM_BASE_URL=https://api.groq.com/openai/v1
8
+ # LLM_API_KEY=your_groq_api_key
9
+ # LLM_MODEL=llama-3.3-70b-versatile
10
+
11
+ # --- OpenRouter (free models available) ---
12
+ # LLM_BASE_URL=https://openrouter.ai/api/v1
13
+ # LLM_API_KEY=your_openrouter_api_key
14
+ # LLM_MODEL=meta-llama/llama-3.3-70b-instruct:free
15
+
16
+ # --- Google Gemini (free) ---
17
+ # LLM_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai
18
+ # LLM_API_KEY=your_gemini_api_key
19
+ # LLM_MODEL=gemini-2.0-flash
20
+
21
+ # --- Together AI (free tier) ---
22
+ # LLM_BASE_URL=https://api.together.xyz/v1
23
+ # LLM_API_KEY=your_together_api_key
24
+ # LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo-Free
25
+
26
+ # --- Ollama (local, no key needed) ---
27
+ LLM_BASE_URL=http://localhost:11434/v1
28
+ LLM_API_KEY=ollama
29
+ LLM_MODEL=qwen2.5:7b
30
+
31
+ # --- LM Studio (local, no key needed) ---
32
+ # LLM_BASE_URL=http://localhost:1234/v1
33
+ # LLM_API_KEY=lm-studio
34
+ # LLM_MODEL=your-loaded-model
35
+
36
+ # --- OpenAI (paid) ---
37
+ # LLM_BASE_URL=https://api.openai.com/v1
38
+ # LLM_API_KEY=your_openai_api_key
39
+ # LLM_MODEL=gpt-4o-mini
.gitignore ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ .env
2
+ __pycache__/
3
+ *.pyc
4
+ .DS_Store
5
+ venv/
6
+ .venv/
7
+ *.egg-info/
8
+ dist/
9
+ build/
.vscode/settings.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "python.defaultInterpreterPath": "/run/media/arka/New Volume/DEVELOPMENT/AUTOPSY/.venv/bin/python",
3
+ "python.interpreterPath": "/run/media/arka/New Volume/DEVELOPMENT/AUTOPSY/.venv/bin/python",
4
+ "python.analysis.extraPaths": [
5
+ "/run/media/arka/New Volume/DEVELOPMENT/AUTOPSY/.venv/lib64/python3.14/site-packages"
6
+ ]
7
+ }
GEMINI.md ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # AUTOPSY — Market Dislocation Fingerprint Intelligence System
2
+
3
+ AUTOPSY is a real-time market stress intelligence system designed to identify historical crisis analogues by analyzing the "structural fingerprint" of current market conditions.
4
+
5
+ ## Project Overview
6
+
7
+ - **Purpose:** Answers "Which historical crisis does the current market structure most resemble — and what happened next?"
8
+ - **Core Methodology:**
9
+ 1. Fetches ~40 live market indicators across 5 dimensions (Liquidity, Volatility, Correlation, Credit, Positioning).
10
+ 2. Computes a "fingerprint vector" representing the current structural state.
11
+ 3. Compares the vector against fingerprints of 10 historical crises using cosine similarity.
12
+ 4. Projects fingerprints into 2D space using UMAP for visualization.
13
+ 5. Generates an AI-powered risk narrative using Claude (Anthropic API).
14
+ - **Tech Stack:**
15
+ - **Language:** Python 3.11+ (Current environment: 3.14)
16
+ - **Frontend:** Streamlit, Plotly
17
+ - **Data/Math:** Pandas, NumPy, Scipy, Scikit-learn, UMAP-learn
18
+ - **APIs:** `yfinance` (market data), `fredapi` (macro data), `anthropic` (AI agent)
19
+
20
+ ## Directory Structure
21
+
22
+ - `agent/`: Contains `analyst.py` for interacting with the Anthropic API.
23
+ - `dashboard/`: Streamlit UI implementation.
24
+ - `app.py`: Main entry point.
25
+ - `charts.py`: Plotly visualization logic.
26
+ - `state.py`: Session state management.
27
+ - `data/`: Data layer.
28
+ - `pipeline.py`: Fetches raw data from FRED and Yahoo Finance.
29
+ - `indicators.py`: Computes 40 structural indicators from raw data.
30
+ - `crisis_library.py`: Metadata for 10 historical crisis events.
31
+ - `fingerprint/`: Core analytical engine.
32
+ - `engine.py`: Vector normalization and similarity computation.
33
+ - `embedding.py`: UMAP/PCA dimensionality reduction.
34
+ - `tests/`: Project tests (e.g., `test_pipeline.py`).
35
+
36
+ ## Getting Started
37
+
38
+ ### Environment Setup
39
+
40
+ 1. **Install Dependencies:**
41
+ ```bash
42
+ pip install -r requirements.txt
43
+ ```
44
+
45
+ 2. **Configuration:**
46
+ Create a `.env` file in the root directory with your API keys:
47
+ ```env
48
+ FRED_API_KEY=your_fred_api_key
49
+ ANTHROPIC_API_KEY=your_anthropic_api_key
50
+ ```
51
+
52
+ ### Running the Application
53
+
54
+ Launch the Streamlit dashboard:
55
+ ```bash
56
+ streamlit run dashboard/app.py
57
+ ```
58
+
59
+ ### Running Tests
60
+
61
+ Execute tests using `pytest`:
62
+ ```bash
63
+ pytest
64
+ ```
65
+
66
+ ## Development Conventions
67
+
68
+ - **Data Alignment:** All financial data is resampled to Business Days (`B`) and missing values are forward-filled.
69
+ - **Normalization:** Fingerprint vectors are normalized using `RobustScaler` to handle outliers in market data.
70
+ - **Type Hinting:** Use type hints for function signatures to maintain clarity in the analytical pipeline.
71
+ - **Defensive Data Fetching:** `data/pipeline.py` includes logic to handle both MultiIndex and flat column formats from `yfinance`.
72
+ - **Caching:** Heavy data operations (fetching and indicator computation) in the dashboard are cached using `@st.cache_data`.
73
+
74
+ ## Key Indicators (5 Dimensions)
75
+
76
+ 1. **Liquidity:** TED spread, HY/IG ratios, Yield Curve velocity.
77
+ 2. **Volatility:** VIX levels, Term structure (VIX9D/VIX3M), Vol-of-Vol.
78
+ 3. **Correlation:** Equity-Bond correlation, Cross-sector dispersion.
79
+ 4. **Credit:** HY/IG Oasis spreads, Credit-Equity dislocations.
80
+ 5. **Positioning:** Safe haven flows (Gold, JPY, CHF), Defensive sector rotations.
README.md ADDED
@@ -0,0 +1,194 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Autopsy Dashboard
3
+ emoji: 🔬
4
+ colorFrom: pink
5
+ colorTo: purple
6
+ sdk: streamlit
7
+ sdk_version: 1.32.0
8
+ app_file: dashboard/app.py
9
+ pinned: false
10
+ ---
11
+
12
+ # 🔬 AUTOPSY
13
+ ### Market Dislocation Fingerprint Intelligence System
14
+
15
+ > *"Which historical crisis does the current market structure most resemble — and what happened next?"*
16
+
17
+ **TechEx Intelligent Enterprise Solutions Hackathon 2026 — Track 4: Data & Intelligence**
18
+
19
+ ---
20
+
21
+ ## What Is AUTOPSY?
22
+
23
+ AUTOPSY is a real-time market stress intelligence system that detects structural fingerprints in financial markets and matches them against 10 historical crises — before price-level stress makes the danger obvious.
24
+
25
+ It analyzes **40 market indicators** across 5 structural dimensions:
26
+
27
+ | Dimension | What It Captures |
28
+ |---|---|
29
+ | 🔴 Liquidity | TED spread, credit spread velocity, yield curve shape |
30
+ | 🟠 Volatility | VIX level, term structure inversion, vol-of-vol |
31
+ | 🟡 Correlation | Cross-asset correlation breakdown, safe haven flows |
32
+ | 🟣 Credit | HY/IG spread divergence, financials vs market |
33
+ | 🔵 Positioning | Gold/Yen/CHF flows, defensive rotation, EM outflows |
34
+
35
+ These are compared against pre-crisis fingerprints from:
36
+ **LTCM 1998 · Dot-Com 2000 · GFC 2008 · Flash Crash 2010 · Eurozone 2011 · Taper Tantrum 2013 · China/Oil 2015 · Volmageddon 2018 · COVID 2020 · SVB 2023**
37
+
38
+ An AI analyst then reads the fingerprint match and writes a structured institutional risk narrative using any OpenAI-compatible LLM (Groq, Gemini, OpenRouter, Ollama, etc.).
39
+
40
+ **This is not a price prediction tool. It is a market structure stress detector.**
41
+
42
+ ---
43
+
44
+ ## How It Works
45
+
46
+ ```
47
+ Live Market Data (FRED + Yahoo Finance)
48
+
49
+
50
+ 40 Indicator Computation (rolling z-scores, velocities, correlations)
51
+
52
+
53
+ Fingerprint Vector (RobustScaler normalized, 40-dimensional)
54
+
55
+
56
+ Cosine Similarity vs 10 Historical Crisis Fingerprints
57
+
58
+
59
+ AI Narrative (LLM reads pre-computed results, writes risk brief)
60
+
61
+
62
+ Live Dashboard (Radar · Analogues · Embedding Space · Heatmap · Rewind)
63
+ ```
64
+
65
+ The math does the analysis. The AI writes the explanation. Every number is traceable to a public data source.
66
+
67
+ ---
68
+
69
+ ## Features
70
+
71
+ - **Live Mode** — current market fingerprint updated daily
72
+ - **Rewind Mode** — set any historical date and see what AUTOPSY would have shown
73
+ - **Heatmap Mode** — 40-indicator stress heatmap over 30/60/90/252 day windows
74
+ - **Embedding Space** — 2D PCA projection showing where "NOW" sits relative to historical crises
75
+ - **AI Risk Narrative** — structured 4-section brief: Assessment · Analogues · Divergences · Risk Posture
76
+
77
+ ---
78
+
79
+ ## Quickstart
80
+
81
+ ### 1. Clone
82
+ ```bash
83
+ git clone https://github.com/ARKAISW/autopsy.git
84
+ cd autopsy
85
+ ```
86
+
87
+ ### 2. Install
88
+ ```bash
89
+ python -m venv .venv
90
+ source .venv/bin/activate # Windows: .venv\Scripts\activate
91
+ pip install -r requirements.txt
92
+ ```
93
+
94
+ ### 3. API Keys
95
+ Create an environment file for local use:
96
+
97
+ ```bash
98
+ cp .env.example .env
99
+ # Open .env and fill in your keys
100
+ ```
101
+
102
+ **⚠️ IMPORTANT:** Never commit or upload your `.env` file to GitHub or Hugging Face. Add it to your `.gitignore`.
103
+
104
+ ### 4. Run
105
+ ```bash
106
+ streamlit run dashboard/app.py
107
+ ```
108
+
109
+ ---
110
+
111
+ ## Getting API Keys (Both Free)
112
+
113
+ | Key | Where to Get It | Cost |
114
+ |---|---|---|
115
+ | `FRED_API_KEY` | [fred.stlouisfed.org/docs/api/api_key.html](https://fred.stlouisfed.org/docs/api/api_key.html) | Free, instant |
116
+ | `LLM_API_KEY` | [Groq](https://console.groq.com) · [Gemini](https://aistudio.google.com) · [OpenRouter](https://openrouter.ai) | Free tier available |
117
+
118
+ Configure your LLM provider in `.env`:
119
+ ```env
120
+ LLM_BASE_URL=https://api.groq.com/openai/v1
121
+ LLM_API_KEY=your_key_here
122
+ LLM_MODEL=llama-3.3-70b-versatile
123
+ ```
124
+
125
+ Any OpenAI-compatible API works — Groq, Google Gemini, OpenRouter, Together AI, Ollama (local), LM Studio (local), or OpenAI.
126
+
127
+ ---
128
+
129
+ ## Data Sources
130
+
131
+ ### FRED (Federal Reserve Economic Data)
132
+ | Series | FRED ID | Link |
133
+ |---|---|---|
134
+ | TED Spread | TEDRATE | [→](https://fred.stlouisfed.org/series/TEDRATE) |
135
+ | HY Credit Spread (OAS) | BAMLH0A0HYM2 | [→](https://fred.stlouisfed.org/series/BAMLH0A0HYM2) |
136
+ | IG Credit Spread (OAS) | BAMLC0A0CM | [→](https://fred.stlouisfed.org/series/BAMLC0A0CM) |
137
+ | 10Y-2Y Yield Curve | T10Y2Y | [→](https://fred.stlouisfed.org/series/T10Y2Y) |
138
+ | 10Y-3M Yield Curve | T10Y3M | [→](https://fred.stlouisfed.org/series/T10Y3M) |
139
+ | VIX Index | VIXCLS | [→](https://fred.stlouisfed.org/series/VIXCLS) |
140
+
141
+ ### Yahoo Finance
142
+ SPY · QQQ · IEF · LQD · HYG · GLD · USO · UUP · EEM · XLF · XLE · XLU · ^VIX · ^VIX3M · ^VIX9D · EURUSD=X · JPY=X · CHF=X
143
+
144
+ ---
145
+
146
+ ## Project Structure
147
+
148
+ ```
149
+ autopsy/
150
+ ├── data/
151
+ │ ├── pipeline.py # FRED + yfinance data fetching
152
+ │ ├── indicators.py # 40 indicator computations
153
+ │ └── crisis_library.py # 10 historical crisis definitions
154
+ ├── fingerprint/
155
+ │ ├── engine.py # Fingerprint extraction + similarity
156
+ │ └── embedding.py # PCA embedding for visualization
157
+ ├── agent/
158
+ │ └── analyst.py # Universal LLM analyst (OpenAI-compatible)
159
+ ├── dashboard/
160
+ │ ├── app.py # Main Streamlit app
161
+ │ ├── charts.py # All Plotly chart functions
162
+ │ └── state.py # Session state management
163
+ └── requirements.txt
164
+ ```
165
+
166
+ ---
167
+
168
+ ## Deployment (Hugging Face Spaces)
169
+
170
+ 1. Create a new Space on [huggingface.co/spaces](https://huggingface.co/spaces) with **Streamlit** SDK
171
+ 2. Push this repo to the Space
172
+ 3. Go to Space **Settings → Repository Secrets** and add:
173
+ - `FRED_API_KEY`
174
+ - `LLM_BASE_URL`
175
+ - `LLM_API_KEY`
176
+ - `LLM_MODEL`
177
+ 4. The app will launch automatically
178
+
179
+ ---
180
+
181
+ ## Disclaimer
182
+
183
+ AUTOPSY is a research and educational tool. It does not constitute investment advice. All data is sourced from public APIs. Past crisis structural patterns do not guarantee future market behavior.
184
+
185
+ ---
186
+
187
+ ## Author
188
+
189
+ **Arka Sarkar**
190
+ [GitHub](https://github.com/ARKAISW) · TechEx Intelligent Enterprise Solutions Hackathon 2026
191
+
192
+ ---
193
+
194
+ *Data: FRED (Federal Reserve Bank of St. Louis) + Yahoo Finance · AI: Any OpenAI-compatible LLM*
agent/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ # agent package
agent/analyst.py ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # agent/analyst.py
2
+
3
+ """
4
+ Universal LLM analyst — works with ANY OpenAI-compatible API:
5
+ - Groq (free) → GROQ_API_KEY + https://api.groq.com/openai/v1
6
+ - OpenRouter (free) → OPENROUTER_API_KEY + https://openrouter.ai/api/v1
7
+ - Together (free) → TOGETHER_API_KEY + https://api.together.xyz/v1
8
+ - Ollama (local) → no key + http://localhost:11434/v1
9
+ - LM Studio (local) → no key + http://localhost:1234/v1
10
+ - OpenAI → OPENAI_API_KEY + https://api.openai.com/v1
11
+ - Google Gemini → GEMINI_API_KEY + https://generativelanguage.googleapis.com/v1beta/openai
12
+ - Mistral → MISTRAL_API_KEY + https://api.mistral.ai/v1
13
+
14
+ Just set LLM_BASE_URL, LLM_API_KEY, and LLM_MODEL in your .env file.
15
+ """
16
+
17
+ import os
18
+ import requests
19
+ from dotenv import load_dotenv
20
+
21
+ load_dotenv()
22
+
23
+ # ── Universal config ─────────────────────────────────────────────────────────
24
+ LLM_BASE_URL = os.getenv("LLM_BASE_URL", "http://localhost:11434/v1") # default: Ollama
25
+ LLM_API_KEY = os.getenv("LLM_API_KEY", "ollama") # some providers need a non-empty string
26
+ LLM_MODEL = os.getenv("LLM_MODEL", "qwen2.5:7b") # default: Ollama model
27
+
28
+
29
+ def _call_llm(prompt: str) -> str:
30
+ """
31
+ Call any OpenAI-compatible chat completions API.
32
+ This single function works with every provider listed above.
33
+ """
34
+ url = f"{LLM_BASE_URL.rstrip('/')}/chat/completions"
35
+ headers = {
36
+ "Content-Type": "application/json",
37
+ "Authorization": f"Bearer {LLM_API_KEY}",
38
+ }
39
+ payload = {
40
+ "model": LLM_MODEL,
41
+ "messages": [{"role": "user", "content": prompt}],
42
+ "max_tokens": 600,
43
+ "temperature": 0.3,
44
+ }
45
+
46
+ try:
47
+ resp = requests.post(url, json=payload, headers=headers, timeout=120)
48
+ resp.raise_for_status()
49
+ data = resp.json()
50
+ return data["choices"][0]["message"]["content"]
51
+ except Exception as e:
52
+ return f"[AI analyst unavailable: {e}]"
53
+
54
+
55
+ def build_prompt(
56
+ similarity_results: list,
57
+ dimension_scores: dict,
58
+ available_indicators: list,
59
+ live_vector,
60
+ query_date: str = "today"
61
+ ) -> str:
62
+ """Builds the structured prompt for the analyst agent."""
63
+ top_3 = similarity_results[:3]
64
+
65
+ indicator_stress = [(available_indicators[i], abs(live_vector[i]))
66
+ for i in range(len(available_indicators))]
67
+ indicator_stress.sort(key=lambda x: x[1], reverse=True)
68
+ top_indicators = indicator_stress[:5]
69
+
70
+ top_analogue_text = "\n".join([
71
+ f" {i+1}. {r['name']} ({r['short']}): {r['similarity']:.1f}% similarity\n"
72
+ f" Key signature: {r['key_signature']}\n"
73
+ f" Peak date: {r['peak_date']}"
74
+ for i, r in enumerate(top_3)
75
+ ])
76
+
77
+ dimension_text = "\n".join([
78
+ f" {dim}: {score:.1f}/100 stress"
79
+ for dim, score in sorted(dimension_scores.items(), key=lambda x: x[1], reverse=True)
80
+ ])
81
+
82
+ indicator_text = "\n".join([
83
+ f" {ind.replace('_', ' ')}: {val:.2f}\u03c3 deviation"
84
+ for ind, val in top_indicators
85
+ ])
86
+
87
+ prompt = f"""You are AUTOPSY, a quantitative market risk analyst system.
88
+ Your job is to analyze current market structure and produce a concise, precise risk narrative.
89
+
90
+ ## Current Market Snapshot (as of {query_date})
91
+
92
+ ### Top Crisis Structural Analogues:
93
+ {top_analogue_text}
94
+
95
+ ### Stress by Dimension (0-100 scale):
96
+ {dimension_text}
97
+
98
+ ### Most Stressed Indicators:
99
+ {indicator_text}
100
+
101
+ ## Your Task
102
+
103
+ Write a structured risk narrative with EXACTLY these four sections:
104
+
105
+ **STRUCTURAL ASSESSMENT** (2-3 sentences)
106
+ Describe what the current market structure fingerprint reveals.
107
+
108
+ **HISTORICAL ANALOGUES** (3-4 sentences)
109
+ Explain what the top 1-2 analogues share with the current fingerprint.
110
+
111
+ **KEY DIVERGENCES** (2-3 sentences)
112
+ What aspects of the current fingerprint explicitly differ from the top analogue?
113
+
114
+ **RISK POSTURE** (2-3 sentences)
115
+ What should a risk-aware institutional investor monitor closely?
116
+
117
+ Keep the total response under 350 words. Be precise. Write as a senior quant risk officer would brief a CIO."""
118
+
119
+ return prompt
120
+
121
+
122
+ def run_analyst(
123
+ similarity_results: list,
124
+ dimension_scores: dict,
125
+ available_indicators: list,
126
+ live_vector,
127
+ query_date: str = "today"
128
+ ) -> str:
129
+ """Calls the LLM and returns the structured narrative string."""
130
+ prompt = build_prompt(
131
+ similarity_results, dimension_scores, available_indicators, live_vector, query_date
132
+ )
133
+
134
+ result = _call_llm(prompt)
135
+
136
+ if result.startswith("[AI analyst unavailable"):
137
+ if similarity_results:
138
+ result += f"\n\nTop analogue: {similarity_results[0]['name']} ({similarity_results[0]['similarity']:.1f}% similarity)"
139
+ return result
dashboard/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ # dashboard package
dashboard/app.py ADDED
@@ -0,0 +1,348 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # dashboard/app.py
2
+
3
+ import streamlit as st
4
+ import pandas as pd
5
+ import numpy as np
6
+ from datetime import datetime, timedelta
7
+ import sys
8
+ import os
9
+ import re
10
+
11
+ # Ensure imports work from root
12
+ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
13
+
14
+ from data.pipeline import fetch_all_data
15
+ from data.indicators import compute_all_indicators
16
+ from fingerprint.engine import (
17
+ extract_crisis_fingerprints, extract_live_fingerprint,
18
+ extract_historical_fingerprint, compute_similarity_scores, compute_dimension_scores
19
+ )
20
+ from fingerprint.embedding import build_umap_embedding
21
+ from agent.analyst import run_analyst
22
+ from dashboard.charts import (
23
+ make_radar_chart, make_analogue_bar_chart, make_embedding_scatter,
24
+ make_indicator_heatmap, make_dimension_timeseries
25
+ )
26
+ from dashboard.state import initialize_session_state
27
+
28
+ # ── Page Config ───────────────────────────────────────────────────────────────
29
+ # FIX #1: page_name -> page_title (page_name is not a valid Streamlit arg)
30
+ st.set_page_config(
31
+ page_title="AUTOPSY",
32
+ page_icon="\U0001f52c",
33
+ layout="wide",
34
+ initial_sidebar_state="expanded"
35
+ )
36
+
37
+
38
+ # ── FIX #14: Cache heavy data fetching ────────────────────────────────────────
39
+ @st.cache_data(ttl=3600, show_spinner=False)
40
+ def load_all_data():
41
+ """Cached wrapper around the heavy fetch + compute pipeline."""
42
+ raw_df = fetch_all_data(start_date="2000-01-01")
43
+ indicator_df = compute_all_indicators(raw_df)
44
+ return raw_df, indicator_df
45
+
46
+
47
+ # ── Custom CSS ────────────────────────────────────────────────────────────────
48
+ st.markdown("""
49
+ <style>
50
+ .stApp { background-color: #0E1117; }
51
+ .main-title {
52
+ font-size: 2.8rem; font-weight: 900; color: #E84393;
53
+ letter-spacing: -1px; margin-bottom: 0;
54
+ }
55
+ .subtitle {
56
+ font-size: 1.0rem; color: #888;
57
+ margin-top: -8px; margin-bottom: 24px;
58
+ }
59
+ .metric-card {
60
+ background: #1A1D23; border-radius: 10px;
61
+ padding: 16px 20px; border: 1px solid #2D3139;
62
+ }
63
+ .analogue-card {
64
+ background: #1A1D23; border-radius: 8px;
65
+ padding: 14px 16px; border-left: 4px solid #E84393;
66
+ margin-bottom: 10px;
67
+ }
68
+ .narrative-box {
69
+ background: #1A1D23; border-radius: 10px;
70
+ padding: 20px 24px; border: 1px solid #2D3139;
71
+ font-size: 0.92rem; line-height: 1.7; color: #D0D0D0;
72
+ }
73
+ .section-header {
74
+ font-size: 1.1rem; font-weight: 700; color: #FAFAFA;
75
+ margin: 20px 0 8px 0; border-bottom: 1px solid #2D3139;
76
+ padding-bottom: 4px;
77
+ }
78
+ .stress-badge-high { color: #E74C3C; font-weight: bold; }
79
+ .stress-badge-med { color: #E67E22; font-weight: bold; }
80
+ .stress-badge-low { color: #27AE60; font-weight: bold; }
81
+ </style>
82
+ """, unsafe_allow_html=True)
83
+
84
+ initialize_session_state()
85
+
86
+
87
+ # ── Sidebar ───────────────────────────────────────────────────────────────────
88
+ with st.sidebar:
89
+ st.markdown("## \U0001f52c AUTOPSY")
90
+ st.markdown("*Market Dislocation Fingerprint Intelligence*")
91
+
92
+ st.divider()
93
+
94
+ mode = st.radio(
95
+ "Mode",
96
+ options=["\U0001f534 Live Market", "\u23ea Rewind", "\U0001f5fa Heatmap"],
97
+ index=0
98
+ )
99
+
100
+ st.divider()
101
+
102
+ rewind_date = None
103
+ heatmap_days = 60
104
+
105
+ if mode == "\u23ea Rewind":
106
+ rewind_date = st.date_input(
107
+ "Select Historical Date",
108
+ value=pd.to_datetime("2020-03-01"),
109
+ min_value=pd.to_datetime("2005-01-01"),
110
+ max_value=pd.to_datetime("today")
111
+ )
112
+ st.caption("See what AUTOPSY would have shown on this date.")
113
+
114
+ if mode == "\U0001f5fa Heatmap":
115
+ heatmap_days = st.selectbox("Lookback Window", [30, 60, 90, 252], index=1)
116
+
117
+ st.divider()
118
+
119
+ load_button = st.button("\U0001f504 Load / Refresh Data", use_container_width=True)
120
+
121
+ if st.session_state.data_loaded:
122
+ st.success("\u2705 Data loaded")
123
+ if st.session_state.indicator_df is not None:
124
+ st.caption(f"Latest: {st.session_state.indicator_df.index[-1].strftime('%Y-%m-%d')}")
125
+ else:
126
+ st.warning("\u26a0\ufe0f Click to load data")
127
+
128
+ st.divider()
129
+ st.caption("Data: FRED + Yahoo Finance\nAI: Claude (Anthropic)")
130
+
131
+
132
+ # ── Data Loading ──────────────────────────────────────────────────────────────
133
+ # FIX #11: Only trigger on explicit button press, not on first load
134
+ if load_button:
135
+ with st.spinner("Fetching market data (this takes ~30 seconds)..."):
136
+ try:
137
+ raw_df, indicator_df = load_all_data()
138
+ st.session_state.raw_df = raw_df
139
+ st.session_state.indicator_df = indicator_df
140
+
141
+ with st.spinner("Building crisis fingerprint library..."):
142
+ crisis_fp, scaler, avail_ind = extract_crisis_fingerprints(indicator_df)
143
+ st.session_state.crisis_fingerprints = crisis_fp
144
+ st.session_state.scaler = scaler
145
+ st.session_state.available_indicators = avail_ind
146
+
147
+ with st.spinner("Computing live fingerprint..."):
148
+ live_vector = extract_live_fingerprint(indicator_df, scaler, avail_ind)
149
+ st.session_state.live_vector = live_vector
150
+ sim_results = compute_similarity_scores(live_vector, crisis_fp)
151
+ st.session_state.similarity_results = sim_results
152
+ dim_scores = compute_dimension_scores(live_vector, avail_ind)
153
+ st.session_state.dimension_scores = dim_scores
154
+
155
+ with st.spinner("Building embedding space..."):
156
+ crisis_coords, live_coords = build_umap_embedding(crisis_fp, live_vector)
157
+ st.session_state.embedding_coords = crisis_coords
158
+ st.session_state.live_embedding_coords = live_coords
159
+
160
+ with st.spinner("Running AI analyst..."):
161
+ narrative = run_analyst(
162
+ sim_results, dim_scores, avail_ind,
163
+ live_vector, query_date=datetime.today().strftime("%B %d, %Y")
164
+ )
165
+ st.session_state.analyst_narrative = narrative
166
+
167
+ st.session_state.data_loaded = True
168
+ st.rerun()
169
+
170
+ except Exception as e:
171
+ st.error(f"Data loading failed: {str(e)}")
172
+ st.exception(e)
173
+
174
+
175
+ # ── Main Content ──────────────────────────────────────────────────────────────
176
+ if not st.session_state.data_loaded:
177
+ st.markdown('<div class="main-title">AUTOPSY</div>', unsafe_allow_html=True)
178
+ st.markdown('<div class="subtitle">Market Dislocation Fingerprint Intelligence System</div>',
179
+ unsafe_allow_html=True)
180
+ st.info("\U0001f448 Click **Load / Refresh Data** in the sidebar to begin.")
181
+
182
+ st.markdown("### What is AUTOPSY?")
183
+ st.markdown("""
184
+ AUTOPSY analyzes the **structural fingerprint** of current market conditions across five dimensions:
185
+ **Liquidity \u00b7 Volatility \u00b7 Correlation \u00b7 Credit \u00b7 Positioning**
186
+
187
+ It compares this fingerprint against pre-crisis signatures from 10 historical dislocations,
188
+ identifies the closest structural analogues, and generates an AI-powered risk narrative.
189
+
190
+ This is not a price prediction tool. It is a **market structure stress detector**.
191
+ """)
192
+ st.stop()
193
+
194
+
195
+ # ── Determine what vector/results to display ──────────────────────────────────
196
+ # FIX #6: replaced 'rewind_date' in dir() with proper variable check
197
+ if mode == "\u23ea Rewind" and rewind_date is not None:
198
+ rewind_str = rewind_date.strftime("%Y-%m-%d")
199
+ display_vector = extract_historical_fingerprint(
200
+ st.session_state.indicator_df,
201
+ st.session_state.scaler,
202
+ st.session_state.available_indicators,
203
+ rewind_str
204
+ )
205
+ if display_vector is None:
206
+ st.error(f"No data available for {rewind_str}")
207
+ st.stop()
208
+ display_similarity = compute_similarity_scores(display_vector, st.session_state.crisis_fingerprints)
209
+ display_dimension = compute_dimension_scores(display_vector, st.session_state.available_indicators)
210
+ display_date_label = rewind_date.strftime("%B %d, %Y")
211
+ rewind_narrative = run_analyst(
212
+ display_similarity, display_dimension, st.session_state.available_indicators,
213
+ display_vector, query_date=display_date_label
214
+ )
215
+ display_narrative = rewind_narrative
216
+ rewind_coords, rewind_live_coords = build_umap_embedding(
217
+ st.session_state.crisis_fingerprints, display_vector
218
+ )
219
+ display_embedding_coords = rewind_coords
220
+ display_live_coords = rewind_live_coords
221
+ is_rewind = True
222
+ else:
223
+ display_vector = st.session_state.live_vector
224
+ display_similarity = st.session_state.similarity_results
225
+ display_dimension = st.session_state.dimension_scores
226
+ display_date_label = datetime.today().strftime("%B %d, %Y")
227
+ display_narrative = st.session_state.analyst_narrative
228
+ display_embedding_coords = st.session_state.embedding_coords
229
+ display_live_coords = st.session_state.live_embedding_coords
230
+ is_rewind = False
231
+
232
+
233
+ # ── Header ────────────────────────────────────────────────────────────────────
234
+ col_title, col_date = st.columns([3, 1])
235
+ with col_title:
236
+ label = f"\u23ea REWIND: {display_date_label}" if is_rewind else "\U0001f534 LIVE"
237
+ st.markdown('<div class="main-title">AUTOPSY</div>', unsafe_allow_html=True)
238
+ st.markdown(f'<div class="subtitle">Market Dislocation Fingerprint Intelligence \u00b7 {label}</div>',
239
+ unsafe_allow_html=True)
240
+ with col_date:
241
+ top_analogue = display_similarity[0]
242
+ st.metric(
243
+ label="Top Analogue",
244
+ value=top_analogue["short"],
245
+ delta=f"{top_analogue['similarity']:.1f}% match"
246
+ )
247
+
248
+
249
+ # ── Heatmap Mode ──────────────────────────────────────────────────────────────
250
+ if mode == "\U0001f5fa Heatmap":
251
+ st.markdown('<div class="section-header">Indicator Stress Heatmap</div>', unsafe_allow_html=True)
252
+ fig_heatmap = make_indicator_heatmap(
253
+ st.session_state.indicator_df,
254
+ st.session_state.available_indicators,
255
+ days=heatmap_days
256
+ )
257
+ st.plotly_chart(fig_heatmap, use_container_width=True)
258
+
259
+ st.markdown('<div class="section-header">Dimension Stress History</div>', unsafe_allow_html=True)
260
+ fig_ts = make_dimension_timeseries(
261
+ st.session_state.indicator_df,
262
+ st.session_state.available_indicators,
263
+ days=heatmap_days
264
+ )
265
+ st.plotly_chart(fig_ts, use_container_width=True)
266
+ st.stop()
267
+
268
+
269
+ # ── AI Analyst Narrative ──────────────────────────────────────────────────────
270
+ st.markdown('<div class="section-header">AI Risk Analyst</div>', unsafe_allow_html=True)
271
+ if display_narrative:
272
+ styled_narrative = re.sub(r'\*\*(.*?)\*\*', r'<span style="color: #E84393; font-weight: 700; font-size: 1.0rem;">\1</span>', display_narrative)
273
+ st.markdown(f'<div class="narrative-box">{styled_narrative.replace(chr(10), "<br>")}</div>',
274
+ unsafe_allow_html=True)
275
+ else:
276
+ st.info("AI narrative will appear after data is loaded.")
277
+
278
+
279
+ # ── Main Dashboard Layout (Live + Rewind) ─────────────────────────────────────
280
+ col_left, col_right = st.columns([1.1, 0.9])
281
+
282
+ with col_left:
283
+ st.markdown('<div class="section-header">Structural Stress Profile</div>', unsafe_allow_html=True)
284
+ fig_radar = make_radar_chart(display_dimension)
285
+ st.plotly_chart(fig_radar, use_container_width=True)
286
+
287
+ st.markdown('<div class="section-header">Crisis Analogue Similarity</div>', unsafe_allow_html=True)
288
+ fig_bar = make_analogue_bar_chart(display_similarity)
289
+ st.plotly_chart(fig_bar, use_container_width=True)
290
+
291
+ with col_right:
292
+ st.markdown('<div class="section-header">Top Structural Analogues</div>', unsafe_allow_html=True)
293
+ for r in display_similarity[:3]:
294
+ border_color = r["color"]
295
+ st.markdown(f"""
296
+ <div style="background:#1A1D23; border-radius:8px; padding:14px 16px;
297
+ border-left:4px solid {border_color}; margin-bottom:10px;">
298
+ <div style="font-weight:700; color:#FAFAFA; font-size:1.0rem;">
299
+ {r['name']}
300
+ <span style="float:right; color:{border_color}; font-size:1.1rem;">{r['similarity']:.1f}%</span>
301
+ </div>
302
+ <div style="color:#888; font-size:0.82rem; margin-top:4px;">{r['key_signature']}</div>
303
+ <div style="color:#666; font-size:0.78rem; margin-top:2px;">Peak: {r['peak_date']}</div>
304
+ </div>
305
+ """, unsafe_allow_html=True)
306
+
307
+ st.markdown('<div class="section-header">Fingerprint Space</div>', unsafe_allow_html=True)
308
+ fig_embed = make_embedding_scatter(display_embedding_coords, display_live_coords)
309
+ st.plotly_chart(fig_embed, use_container_width=True)
310
+
311
+
312
+ # ── Dimension Timeseries ──────────────────────────────────────────────────────
313
+ st.markdown('<div class="section-header">Stress Dimension History (1 Year)</div>', unsafe_allow_html=True)
314
+ fig_ts = make_dimension_timeseries(
315
+ st.session_state.indicator_df,
316
+ st.session_state.available_indicators,
317
+ days=252
318
+ )
319
+ st.plotly_chart(fig_ts, use_container_width=True)
320
+
321
+
322
+ # ── Data Sources ──────────────────────────────────────────────────────────────
323
+ with st.expander("📚 Data Sources & Methodology"):
324
+ from data.pipeline import FRED_SERIES, YF_TICKERS
325
+
326
+ st.markdown("### FRED Data Series (Federal Reserve Economic Data)")
327
+ fred_cols = st.columns(2)
328
+ fred_items = [(name, series_id) for name, series_id in FRED_SERIES.items() if series_id is not None]
329
+ for i, (name, series_id) in enumerate(fred_items):
330
+ url = f"https://fred.stlouisfed.org/series/{series_id}"
331
+ fred_cols[i % 2].markdown(f"- [{name.replace('_', ' ').title()}]({url}) — `{series_id}`")
332
+
333
+ st.markdown("### Yahoo Finance Tickers")
334
+ yf_cols = st.columns(3)
335
+ for i, (name, ticker) in enumerate(YF_TICKERS.items()):
336
+ clean_ticker = ticker.replace("^", "%5E")
337
+ url = f"https://finance.yahoo.com/quote/{clean_ticker}"
338
+ yf_cols[i % 2].markdown(f"- [{ticker}]({url}) — {name.replace('_', ' ').title()}")
339
+
340
+
341
+ # ── Footer ────────────────────────────────────────────────────────────────────
342
+ st.divider()
343
+ st.caption(
344
+ "AUTOPSY | Market Dislocation Fingerprint Intelligence | "
345
+ "TechEx Intelligent Enterprise Solutions Hackathon 2026 | "
346
+ "Data: FRED, Yahoo Finance | AI: Claude (Anthropic) | "
347
+ "Not investment advice."
348
+ )
dashboard/charts.py ADDED
@@ -0,0 +1,199 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # dashboard/charts.py
2
+
3
+ import plotly.graph_objects as go
4
+ import plotly.express as px
5
+ import pandas as pd
6
+ import numpy as np
7
+ from data.indicators import INDICATOR_GROUPS
8
+ from data.crisis_library import CRISIS_LIBRARY
9
+
10
+
11
+ BACKGROUND_COLOR = "#0E1117"
12
+ CARD_COLOR = "#1A1D23"
13
+ TEXT_COLOR = "#FAFAFA"
14
+ ACCENT_COLOR = "#E84393"
15
+ GRID_COLOR = "#2D3139"
16
+
17
+
18
+ def make_radar_chart(dimension_scores: dict, title: str = "Current Market Stress") -> go.Figure:
19
+ """Radar chart showing stress level across 5 dimensions."""
20
+ dims = list(dimension_scores.keys())
21
+ scores = list(dimension_scores.values())
22
+ dims_plot = dims + [dims[0]]
23
+ scores_plot = scores + [scores[0]]
24
+
25
+ fig = go.Figure()
26
+ fig.add_trace(go.Scatterpolar(
27
+ r=scores_plot, theta=dims_plot, fill='toself',
28
+ fillcolor='rgba(232, 67, 147, 0.2)',
29
+ line=dict(color=ACCENT_COLOR, width=2), name="Current Stress"
30
+ ))
31
+ fig.update_layout(
32
+ polar=dict(
33
+ bgcolor=CARD_COLOR,
34
+ radialaxis=dict(visible=True, range=[0, 100],
35
+ tickfont=dict(color=TEXT_COLOR, size=10), gridcolor=GRID_COLOR,
36
+ ticksuffix="%"),
37
+ angularaxis=dict(tickfont=dict(color=TEXT_COLOR, size=12), gridcolor=GRID_COLOR),
38
+ ),
39
+ showlegend=True, paper_bgcolor=BACKGROUND_COLOR, plot_bgcolor=BACKGROUND_COLOR,
40
+ legend=dict(font=dict(color=TEXT_COLOR, size=11), bgcolor=CARD_COLOR,
41
+ bordercolor=GRID_COLOR, borderwidth=1, x=0.02, y=-0.05),
42
+ title=dict(text=title, font=dict(color=TEXT_COLOR, size=14)),
43
+ margin=dict(l=60, r=60, t=60, b=80), height=400,
44
+ annotations=[dict(text="0 = No stress · 100 = Extreme stress",
45
+ xref="paper", yref="paper", x=0.5, y=-0.08,
46
+ showarrow=False, font=dict(color="#666", size=10))],
47
+ )
48
+ return fig
49
+
50
+
51
+ def make_analogue_bar_chart(similarity_results: list) -> go.Figure:
52
+ """Horizontal bar chart of crisis analogue similarity scores."""
53
+ top_n = similarity_results[:10]
54
+ names = [r["short"] for r in reversed(top_n)]
55
+ scores = [r["similarity"] for r in reversed(top_n)]
56
+ colors = [r["color"] for r in reversed(top_n)]
57
+
58
+ fig = go.Figure(go.Bar(
59
+ x=scores, y=names, orientation='h',
60
+ marker=dict(color=colors, opacity=0.85),
61
+ text=[f"{s:.1f}%" for s in scores], textposition='outside',
62
+ textfont=dict(color=TEXT_COLOR),
63
+ name="Cosine Similarity",
64
+ ))
65
+ fig.update_layout(
66
+ paper_bgcolor=BACKGROUND_COLOR, plot_bgcolor=BACKGROUND_COLOR,
67
+ xaxis=dict(range=[0, 110], gridcolor=GRID_COLOR,
68
+ tickfont=dict(color=TEXT_COLOR), ticksuffix="%",
69
+ title=dict(text="Structural Similarity (Cosine, 0–100%)", font=dict(color=TEXT_COLOR))),
70
+ yaxis=dict(tickfont=dict(color=TEXT_COLOR),
71
+ title=dict(text="Historical Crisis", font=dict(color=TEXT_COLOR))),
72
+ showlegend=False,
73
+ margin=dict(l=20, r=80, t=30, b=50), height=380,
74
+ annotations=[dict(text="Higher % = current market structure more closely resembles this crisis",
75
+ xref="paper", yref="paper", x=0.5, y=-0.12,
76
+ showarrow=False, font=dict(color="#666", size=10))],
77
+ )
78
+ return fig
79
+
80
+
81
+ def make_embedding_scatter(crisis_coords: dict, live_coords: tuple = None) -> go.Figure:
82
+ """2D PCA scatter plot of crisis fingerprints + current market."""
83
+ fig = go.Figure()
84
+
85
+ for crisis_key, (x, y) in crisis_coords.items():
86
+ info = CRISIS_LIBRARY.get(crisis_key, {})
87
+ fig.add_trace(go.Scatter(
88
+ x=[x], y=[y], mode='markers',
89
+ marker=dict(size=14, color=info.get("color", "#888888"),
90
+ opacity=0.9, line=dict(width=1.5, color='white')),
91
+ name=info.get("short", crisis_key), showlegend=True,
92
+ hovertemplate=f"<b>{info.get('name', crisis_key)}</b><br>{info.get('key_signature', '')}<extra></extra>"
93
+ ))
94
+
95
+ if live_coords is not None:
96
+ fig.add_trace(go.Scatter(
97
+ x=[live_coords[0]], y=[live_coords[1]], mode='markers+text',
98
+ marker=dict(size=20, color=ACCENT_COLOR, symbol='star',
99
+ line=dict(width=2, color='white')),
100
+ text=["NOW"], textposition="top center",
101
+ textfont=dict(color=ACCENT_COLOR, size=12, family="Arial Black"),
102
+ name="Current Market", showlegend=True,
103
+ hovertemplate="<b>Current Market</b><extra></extra>"
104
+ ))
105
+
106
+ fig.update_layout(
107
+ paper_bgcolor=BACKGROUND_COLOR, plot_bgcolor=CARD_COLOR,
108
+ xaxis=dict(showticklabels=False, gridcolor=GRID_COLOR, zeroline=False,
109
+ title=dict(text="PC1 — Crisis Intensity →", font=dict(color="#888", size=11))),
110
+ yaxis=dict(showticklabels=False, gridcolor=GRID_COLOR, zeroline=False,
111
+ title=dict(text="PC2 — Crisis Character →", font=dict(color="#888", size=11))),
112
+ margin=dict(l=50, r=20, t=40, b=50), height=500,
113
+ title=dict(text="Crisis Fingerprint Embedding Space (PCA)", font=dict(color=TEXT_COLOR, size=13)),
114
+ legend=dict(font=dict(color=TEXT_COLOR, size=10), bgcolor=CARD_COLOR,
115
+ bordercolor=GRID_COLOR, borderwidth=1,
116
+ orientation="h", yanchor="top", y=-0.08, xanchor="center", x=0.5),
117
+ annotations=[
118
+ dict(text="Closer points = more structurally similar",
119
+ xref="paper", yref="paper", x=0.5, y=-0.22,
120
+ showarrow=False, font=dict(color="#666", size=9)),
121
+ ],
122
+ )
123
+ return fig
124
+
125
+
126
+ def make_indicator_heatmap(indicator_df: pd.DataFrame, available_indicators: list, days: int = 60) -> go.Figure:
127
+ """Heatmap of indicator z-scores over the last N days."""
128
+ recent = indicator_df[available_indicators].iloc[-days:].copy()
129
+ z_scored = (recent - recent.mean()) / (recent.std() + 1e-8)
130
+ z_scored = z_scored.clip(-3, 3)
131
+ short_names = [ind.replace('_', ' ')[:20] for ind in available_indicators]
132
+
133
+ fig = go.Figure(go.Heatmap(
134
+ z=z_scored.T.values, x=z_scored.index.strftime("%Y-%m-%d"), y=short_names,
135
+ colorscale=[[0.0, "#1a4f72"], [0.5, CARD_COLOR], [1.0, "#8B0000"]],
136
+ zmid=0,
137
+ colorbar=dict(title=dict(text="Z-Score", font=dict(color=TEXT_COLOR)), tickfont=dict(color=TEXT_COLOR)),
138
+ hoverongaps=False,
139
+ ))
140
+ fig.update_layout(
141
+ paper_bgcolor=BACKGROUND_COLOR, plot_bgcolor=BACKGROUND_COLOR,
142
+ xaxis=dict(tickangle=45, tickfont=dict(color=TEXT_COLOR, size=9),
143
+ title=dict(text="Date", font=dict(color=TEXT_COLOR))),
144
+ yaxis=dict(tickfont=dict(color=TEXT_COLOR, size=9),
145
+ title=dict(text="Market Indicator", font=dict(color=TEXT_COLOR))),
146
+ margin=dict(l=160, r=40, t=30, b=90), height=520,
147
+ title=dict(text=f"Indicator Stress Heatmap (Last {days} Days)",
148
+ font=dict(color=TEXT_COLOR, size=13)),
149
+ annotations=[dict(text="Blue = below average · Red = above average · Values are z-scored (std deviations from mean)",
150
+ xref="paper", yref="paper", x=0.5, y=-0.13,
151
+ showarrow=False, font=dict(color="#666", size=10))],
152
+ )
153
+ return fig
154
+
155
+
156
+ def make_dimension_timeseries(indicator_df: pd.DataFrame, available_indicators: list, days: int = 252) -> go.Figure:
157
+ """Line chart showing per-dimension composite stress over time."""
158
+ recent = indicator_df[available_indicators].iloc[-days:].copy()
159
+ dimension_colors = {
160
+ "Liquidity": "#E74C3C", "Volatility": "#E67E22", "Correlation": "#F1C40F",
161
+ "Credit": "#9B59B6", "Positioning": "#3498DB",
162
+ }
163
+
164
+ fig = go.Figure()
165
+ for dim_name, dim_indicators in INDICATOR_GROUPS.items():
166
+ dim_cols = [ind for ind in dim_indicators if ind in recent.columns]
167
+ if not dim_cols:
168
+ continue
169
+ dim_data = recent[dim_cols]
170
+ normalized = (dim_data - dim_data.mean()) / (dim_data.std() + 1e-8)
171
+ composite = normalized.abs().mean(axis=1)
172
+ composite_smooth = composite.rolling(5, min_periods=1).mean()
173
+ q99 = composite_smooth.quantile(0.95)
174
+ composite_scaled = (composite_smooth / max(q99, 0.01)) * 100
175
+ composite_scaled = composite_scaled.clip(0, 100)
176
+
177
+ fig.add_trace(go.Scatter(
178
+ x=recent.index, y=composite_scaled, name=dim_name,
179
+ line=dict(color=dimension_colors.get(dim_name, "#888888"), width=2),
180
+ mode='lines',
181
+ ))
182
+
183
+ fig.update_layout(
184
+ paper_bgcolor=BACKGROUND_COLOR, plot_bgcolor=CARD_COLOR,
185
+ xaxis=dict(gridcolor=GRID_COLOR, tickfont=dict(color=TEXT_COLOR),
186
+ title=dict(text="Date", font=dict(color=TEXT_COLOR))),
187
+ yaxis=dict(gridcolor=GRID_COLOR, tickfont=dict(color=TEXT_COLOR),
188
+ title=dict(text="Composite Stress Level (0–100)", font=dict(color=TEXT_COLOR)),
189
+ range=[0, 105], ticksuffix="%"),
190
+ legend=dict(font=dict(color=TEXT_COLOR, size=11), bgcolor=CARD_COLOR,
191
+ bordercolor=GRID_COLOR, borderwidth=1,
192
+ title=dict(text="Stress Dimensions", font=dict(color=TEXT_COLOR, size=11))),
193
+ margin=dict(l=60, r=20, t=30, b=60), height=370,
194
+ title=dict(text="Stress Dimension History", font=dict(color=TEXT_COLOR, size=13)),
195
+ annotations=[dict(text="Each line = average absolute z-score of indicators in that dimension, scaled 0–100",
196
+ xref="paper", yref="paper", x=0.5, y=-0.14,
197
+ showarrow=False, font=dict(color="#666", size=10))],
198
+ )
199
+ return fig
dashboard/state.py ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # dashboard/state.py
2
+
3
+ import streamlit as st
4
+ import pandas as pd
5
+ from datetime import datetime
6
+
7
+
8
+ def initialize_session_state():
9
+ """Initialize all session state variables if not already set."""
10
+ defaults = {
11
+ "data_loaded": False,
12
+ "raw_df": None,
13
+ "indicator_df": None,
14
+ "crisis_fingerprints": None,
15
+ "scaler": None,
16
+ "available_indicators": None,
17
+ "live_vector": None,
18
+ "similarity_results": None,
19
+ "dimension_scores": None,
20
+ "analyst_narrative": None,
21
+ "rewind_date": None,
22
+ "mode": "live",
23
+ "embedding_coords": None,
24
+ "live_embedding_coords": None,
25
+ }
26
+ for key, default_value in defaults.items():
27
+ if key not in st.session_state:
28
+ st.session_state[key] = default_value
data/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ # data package
data/crisis_library.py ADDED
@@ -0,0 +1,147 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # data/crisis_library.py
2
+
3
+ CRISIS_LIBRARY = {
4
+ "LTCM_1998": {
5
+ "name": "LTCM Collapse",
6
+ "short": "LTCM 1998",
7
+ "description": (
8
+ "Long-Term Capital Management, a highly leveraged hedge fund, collapsed after "
9
+ "Russian debt default. Required a Fed-orchestrated bailout. Key fingerprint: "
10
+ "basis blowout in theoretically related instruments, liquidity withdrawal from "
11
+ "obscure spreads before equity markets noticed."
12
+ ),
13
+ "stress_start": "1998-06-01",
14
+ "stress_end": "1998-09-01",
15
+ "peak_date": "1998-10-08",
16
+ "color": "#E74C3C",
17
+ "key_signature": "Basis blowouts + liquidity fragmentation"
18
+ },
19
+ "DOTCOM_2000": {
20
+ "name": "Dot-Com Bust",
21
+ "short": "Dot-Com 2000",
22
+ "description": (
23
+ "Technology bubble collapse. NASDAQ fell 78% peak to trough. Key fingerprint: "
24
+ "intra-equity sector divergence (tech vs value), VIX term structure flattening, "
25
+ "without the broad credit market stress seen in GFC."
26
+ ),
27
+ "stress_start": "2000-01-01",
28
+ "stress_end": "2000-03-10",
29
+ "peak_date": "2000-03-10",
30
+ "color": "#E67E22",
31
+ "key_signature": "Intra-equity sector divergence + vol term structure"
32
+ },
33
+ "GFC_2008": {
34
+ "name": "Global Financial Crisis",
35
+ "short": "GFC 2008",
36
+ "description": (
37
+ "Subprime mortgage collapse triggered global banking crisis. Key fingerprint: "
38
+ "TED spread explosion, commercial paper market seizure, everything correlated to 1, "
39
+ "funding markets frozen before equity indices reflected the stress."
40
+ ),
41
+ "stress_start": "2007-08-01",
42
+ "stress_end": "2008-09-15",
43
+ "peak_date": "2008-10-10",
44
+ "color": "#8E44AD",
45
+ "key_signature": "Funding market seizure + universal correlation spike"
46
+ },
47
+ "FLASH_CRASH_2010": {
48
+ "name": "Flash Crash",
49
+ "short": "Flash Crash 2010",
50
+ "description": (
51
+ "May 6, 2010: Dow Jones fell nearly 1000 points in minutes then recovered. "
52
+ "Key fingerprint: pure liquidity microstructure stress without credit signal, "
53
+ "very short duration. Demonstrates positioning-driven stress."
54
+ ),
55
+ "stress_start": "2010-04-15",
56
+ "stress_end": "2010-05-06",
57
+ "peak_date": "2010-05-06",
58
+ "color": "#27AE60",
59
+ "key_signature": "Microstructure-only liquidity collapse, no credit signal"
60
+ },
61
+ "EUROZONE_2011": {
62
+ "name": "Eurozone Debt Crisis",
63
+ "short": "Eurozone 2011",
64
+ "description": (
65
+ "Greek, Italian, Spanish sovereign debt crisis threatened euro breakup. "
66
+ "Key fingerprint: geographic contagion pattern in sovereign credit, EUR/USD stress, "
67
+ "financials leading equity drawdown."
68
+ ),
69
+ "stress_start": "2011-06-01",
70
+ "stress_end": "2011-08-08",
71
+ "peak_date": "2011-09-23",
72
+ "color": "#2980B9",
73
+ "key_signature": "Sovereign credit + financials leading + EUR stress"
74
+ },
75
+ "TAPER_TANTRUM_2013": {
76
+ "name": "Taper Tantrum",
77
+ "short": "Taper Tantrum 2013",
78
+ "description": (
79
+ "Fed signaled QE tapering; bond markets sold off violently. "
80
+ "Key fingerprint: rate-driven cross-asset stress, EM currency selloff, "
81
+ "bond-equity correlation flip, yield curve steepening velocity."
82
+ ),
83
+ "stress_start": "2013-05-01",
84
+ "stress_end": "2013-06-25",
85
+ "peak_date": "2013-06-25",
86
+ "color": "#16A085",
87
+ "key_signature": "Duration selloff + EM outflows + yield curve velocity"
88
+ },
89
+ "CHINA_OIL_2015": {
90
+ "name": "China/Oil Shock",
91
+ "short": "China/Oil 2015",
92
+ "description": (
93
+ "Chinese growth fears + oil collapse triggered global equity selloff. "
94
+ "Key fingerprint: commodity-financial system coupling, EM stress, "
95
+ "energy sector leading, oil-dollar relationship breaking down."
96
+ ),
97
+ "stress_start": "2015-06-01",
98
+ "stress_end": "2015-08-25",
99
+ "peak_date": "2015-08-25",
100
+ "color": "#D35400",
101
+ "key_signature": "Commodity-financial coupling + EM selloff"
102
+ },
103
+ "VOL_SHOCK_2018": {
104
+ "name": "Volmageddon",
105
+ "short": "Volmageddon 2018",
106
+ "description": (
107
+ "February 2018: Short volatility strategies (VIX ETPs) exploded, "
108
+ "triggering forced unwinds. Key fingerprint: VIX futures basis blowout, "
109
+ "positioning-driven without fundamental credit stress, very fast recovery."
110
+ ),
111
+ "stress_start": "2018-01-15",
112
+ "stress_end": "2018-02-05",
113
+ "peak_date": "2018-02-05",
114
+ "color": "#C0392B",
115
+ "key_signature": "VIX futures basis + short-vol unwind + no credit"
116
+ },
117
+ "COVID_2020": {
118
+ "name": "COVID Market Crash",
119
+ "short": "COVID 2020",
120
+ "description": (
121
+ "March 2020: Fastest 30% market crash in history. Key fingerprint: "
122
+ "everything selling simultaneously including gold (margin calls), "
123
+ "correlations going to 1 across ALL assets, dollar surging, "
124
+ "funding markets seizing within days."
125
+ ),
126
+ "stress_start": "2020-02-01",
127
+ "stress_end": "2020-03-23",
128
+ "peak_date": "2020-03-23",
129
+ "color": "#1ABC9C",
130
+ "key_signature": "Universal correlation spike + margin call fingerprint"
131
+ },
132
+ "SVB_2023": {
133
+ "name": "SVB Banking Crisis",
134
+ "short": "SVB 2023",
135
+ "description": (
136
+ "Silicon Valley Bank collapse triggered regional banking crisis. "
137
+ "Key fingerprint: regional bank CDS spike, financials sector stress, "
138
+ "rate sensitivity stress without broad credit market contagion. "
139
+ "Funding-specific rather than credit-market-wide."
140
+ ),
141
+ "stress_start": "2023-02-01",
142
+ "stress_end": "2023-03-10",
143
+ "peak_date": "2023-03-10",
144
+ "color": "#9B59B6",
145
+ "key_signature": "Financials-specific + rate sensitivity + contained credit"
146
+ }
147
+ }
data/indicators.py ADDED
@@ -0,0 +1,252 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # data/indicators.py
2
+
3
+ import pandas as pd
4
+ import numpy as np
5
+
6
+
7
+ def compute_velocity(series: pd.Series, window: int = 10) -> pd.Series:
8
+ """Rate of change over window days, as percentage."""
9
+ return series.pct_change(periods=window) * 100
10
+
11
+
12
+ def compute_zscore(series: pd.Series, window: int = 252) -> pd.Series:
13
+ """
14
+ Rolling z-score against a trailing window.
15
+ FIX #9: Clamp std to avoid near-zero division blowups.
16
+ """
17
+ mean = series.rolling(window=window, min_periods=window // 2).mean()
18
+ std = series.rolling(window=window, min_periods=window // 2).std()
19
+ # FIX #9: prevent near-zero std from producing huge z-scores
20
+ std = std.clip(lower=1e-8)
21
+ return (series - mean) / std
22
+
23
+
24
+ def compute_rolling_correlation(s1: pd.Series, s2: pd.Series, window: int = 21) -> pd.Series:
25
+ """Rolling correlation between two return series."""
26
+ r1 = s1.pct_change()
27
+ r2 = s2.pct_change()
28
+ return r1.rolling(window=window, min_periods=window // 2).corr(r2)
29
+
30
+
31
+ def compute_realized_vol(series: pd.Series, window: int = 21) -> pd.Series:
32
+ """Annualized realized volatility."""
33
+ log_returns = np.log(series / series.shift(1))
34
+ return log_returns.rolling(window=window, min_periods=window // 2).std() * np.sqrt(252) * 100
35
+
36
+
37
+ def compute_all_indicators(df: pd.DataFrame) -> pd.DataFrame:
38
+ """
39
+ Takes raw combined DataFrame from pipeline.py.
40
+ Returns a new DataFrame with all 40 indicator columns.
41
+ """
42
+ ind = pd.DataFrame(index=df.index)
43
+
44
+ # ── DIMENSION 1: LIQUIDITY ────────────────────────────────────────────────
45
+ if "ted_spread" in df.columns:
46
+ ind["ted_spread_level"] = df["ted_spread"]
47
+ ind["ted_spread_velocity"] = compute_velocity(df["ted_spread"], 10)
48
+ else:
49
+ ind["ted_spread_level"] = 0.0
50
+ ind["ted_spread_velocity"] = 0.0
51
+
52
+ if "hy_credit_spread" in df.columns and "ig_credit_spread" in df.columns:
53
+ ind["hy_ig_spread_ratio"] = df["hy_credit_spread"] / df["ig_credit_spread"].replace(0, np.nan)
54
+ ind["hy_spread_velocity"] = compute_velocity(df["hy_credit_spread"], 10)
55
+ ind["ig_spread_velocity"] = compute_velocity(df["ig_credit_spread"], 10)
56
+ ind["hy_spread_level"] = df["hy_credit_spread"]
57
+ ind["ig_spread_level"] = df["ig_credit_spread"]
58
+ ind["hy_spread_z_score"] = compute_zscore(df["hy_credit_spread"])
59
+ ind["ig_spread_z_score"] = compute_zscore(df["ig_credit_spread"])
60
+ else:
61
+ for col in ["hy_ig_spread_ratio", "hy_spread_velocity", "ig_spread_velocity",
62
+ "hy_spread_level", "ig_spread_level", "hy_spread_z_score", "ig_spread_z_score"]:
63
+ ind[col] = 0.0
64
+
65
+ if "yield_curve_10y2y" in df.columns:
66
+ ind["yield_curve_level"] = df["yield_curve_10y2y"]
67
+ ind["yield_curve_velocity"] = compute_velocity(df["yield_curve_10y2y"], 10)
68
+ else:
69
+ ind["yield_curve_level"] = 0.0
70
+ ind["yield_curve_velocity"] = 0.0
71
+
72
+ if "ief" in df.columns and "spy" in df.columns:
73
+ ind["bond_equity_correlation"] = compute_rolling_correlation(df["ief"], df["spy"], 21)
74
+ else:
75
+ ind["bond_equity_correlation"] = 0.0
76
+
77
+ # ── DIMENSION 2: VOLATILITY ───────────────────────────────────────────────
78
+ vix_col = "vix" if "vix" in df.columns else "vix_fred"
79
+ if vix_col in df.columns:
80
+ ind["vix_level"] = df[vix_col]
81
+ ind["vix_velocity"] = compute_velocity(df[vix_col], 5)
82
+ ind["vix_z_score"] = compute_zscore(df[vix_col])
83
+ ind["vol_of_vol"] = df[vix_col].diff().rolling(21).std()
84
+ else:
85
+ for col in ["vix_level", "vix_velocity", "vix_z_score", "vol_of_vol"]:
86
+ ind[col] = 0.0
87
+
88
+ if "vix_9d" in df.columns and "vix_3m" in df.columns:
89
+ ind["vix_term_structure"] = df["vix_9d"] / df["vix_3m"].replace(0, np.nan)
90
+ else:
91
+ ind["vix_term_structure"] = 1.0
92
+
93
+ if "spy" in df.columns:
94
+ ind["realized_vol_spy"] = compute_realized_vol(df["spy"], 21)
95
+ else:
96
+ ind["realized_vol_spy"] = 0.0
97
+
98
+ if "vix_level" in ind.columns and "realized_vol_spy" in ind.columns:
99
+ ind["implied_realized_vol_gap"] = ind["vix_level"] - ind["realized_vol_spy"]
100
+ else:
101
+ ind["implied_realized_vol_gap"] = 0.0
102
+
103
+ # FIX #5: cross_asset_vol_spike — use Series, not scalar fallback
104
+ vix_vel = ind["vix_velocity"] if "vix_velocity" in ind.columns else pd.Series(0.0, index=ind.index)
105
+ ind["cross_asset_vol_spike"] = np.abs(vix_vel)
106
+
107
+ # ── DIMENSION 3: CORRELATION ──────────────────────────────────────────────
108
+ if "spy" in df.columns and "ief" in df.columns:
109
+ ind["equity_bond_corr"] = compute_rolling_correlation(df["spy"], df["ief"], 21)
110
+ else:
111
+ ind["equity_bond_corr"] = 0.0
112
+
113
+ if "spy" in df.columns and "gld" in df.columns:
114
+ ind["equity_gold_corr"] = compute_rolling_correlation(df["spy"], df["gld"], 21)
115
+ else:
116
+ ind["equity_gold_corr"] = 0.0
117
+
118
+ if "spy" in df.columns and "hyg" in df.columns:
119
+ ind["equity_hy_corr"] = compute_rolling_correlation(df["spy"], df["hyg"], 21)
120
+ else:
121
+ ind["equity_hy_corr"] = 0.0
122
+
123
+ # Safe haven demand
124
+ safe_haven_components = []
125
+ for ticker in ["gld", "jpyusd", "chfusd"]:
126
+ if ticker in df.columns:
127
+ safe_haven_components.append(df[ticker].pct_change(21))
128
+ if safe_haven_components and "spy" in df.columns:
129
+ avg_safe = pd.concat(safe_haven_components, axis=1).mean(axis=1)
130
+ ind["safe_haven_demand"] = avg_safe - df["spy"].pct_change(21)
131
+ else:
132
+ ind["safe_haven_demand"] = 0.0
133
+
134
+ # Cross-sector correlation
135
+ sector_tickers = [t for t in ["xlf", "xle", "xlu", "qqq"] if t in df.columns]
136
+ if len(sector_tickers) >= 2:
137
+ sector_returns = df[sector_tickers].pct_change()
138
+ rolling_corrs = []
139
+ for i in range(len(sector_tickers)):
140
+ for j in range(i+1, len(sector_tickers)):
141
+ c = sector_returns[sector_tickers[i]].rolling(21).corr(sector_returns[sector_tickers[j]])
142
+ rolling_corrs.append(c)
143
+ ind["cross_sector_correlation"] = pd.concat(rolling_corrs, axis=1).mean(axis=1)
144
+ ind["correlation_breakdown_score"] = pd.concat(rolling_corrs, axis=1).std(axis=1)
145
+ else:
146
+ ind["cross_sector_correlation"] = 0.0
147
+ ind["correlation_breakdown_score"] = 0.0
148
+
149
+ if "spy" in df.columns and "eem" in df.columns:
150
+ ind["em_dm_divergence"] = df["spy"].pct_change(21) - df["eem"].pct_change(21)
151
+ else:
152
+ ind["em_dm_divergence"] = 0.0
153
+
154
+ if "uup" in df.columns:
155
+ ind["dollar_stress_signal"] = compute_velocity(df["uup"], 10)
156
+ else:
157
+ ind["dollar_stress_signal"] = 0.0
158
+
159
+ # ── DIMENSION 4: CREDIT ───────────────────────────────────────────────────
160
+ if "hy_credit_spread" in df.columns and "ig_credit_spread" in df.columns:
161
+ hy_ig_gap = df["hy_credit_spread"] - df["ig_credit_spread"]
162
+ ind["hy_ig_divergence_velocity"] = compute_velocity(hy_ig_gap, 10)
163
+ else:
164
+ ind["hy_ig_divergence_velocity"] = 0.0
165
+
166
+ if "lqd" in df.columns and "hyg" in df.columns:
167
+ ind["lqd_hyg_return_spread"] = df["lqd"].pct_change(21) - df["hyg"].pct_change(21)
168
+ else:
169
+ ind["lqd_hyg_return_spread"] = 0.0
170
+
171
+ if "xlf" in df.columns and "spy" in df.columns:
172
+ ind["xlf_spy_relative"] = df["xlf"].pct_change(21) - df["spy"].pct_change(21)
173
+ else:
174
+ ind["xlf_spy_relative"] = 0.0
175
+
176
+ if "hyg" in df.columns and "spy" in df.columns:
177
+ ind["credit_equity_dislocation"] = df["hyg"].pct_change(21) - df["spy"].pct_change(21)
178
+ else:
179
+ ind["credit_equity_dislocation"] = 0.0
180
+
181
+ # ── DIMENSION 5: POSITIONING / FLOWS ─────────────────────────────────────
182
+ for ticker, col_name in [("gld", "gold_flow_signal"), ("jpyusd", "jpy_flow_signal"),
183
+ ("chfusd", "chf_flow_signal")]:
184
+ if ticker in df.columns:
185
+ ind[col_name] = compute_velocity(df[ticker], 10)
186
+ else:
187
+ ind[col_name] = 0.0
188
+
189
+ if "xlu" in df.columns and "spy" in df.columns:
190
+ ind["utilities_relative"] = df["xlu"].pct_change(21) - df["spy"].pct_change(21)
191
+ else:
192
+ ind["utilities_relative"] = 0.0
193
+
194
+ if "xle" in df.columns and "spy" in df.columns:
195
+ ind["energy_relative"] = df["xle"].pct_change(21) - df["spy"].pct_change(21)
196
+ else:
197
+ ind["energy_relative"] = 0.0
198
+
199
+ if "uso" in df.columns:
200
+ ind["oil_velocity"] = compute_velocity(df["uso"], 10)
201
+ else:
202
+ ind["oil_velocity"] = 0.0
203
+
204
+ if "eem" in df.columns:
205
+ ind["em_outflow_signal"] = compute_velocity(df["eem"], 10) * -1
206
+ else:
207
+ ind["em_outflow_signal"] = 0.0
208
+
209
+ # Risk-off composite
210
+ risk_off_cols = [c for c in ["gold_flow_signal", "jpy_flow_signal",
211
+ "chf_flow_signal", "utilities_relative"] if c in ind.columns]
212
+ if risk_off_cols:
213
+ ind["risk_off_composite"] = ind[risk_off_cols].mean(axis=1)
214
+ else:
215
+ ind["risk_off_composite"] = 0.0
216
+
217
+ # ── FINAL CLEANUP ─────────────────────────────────────────────────────────
218
+ ind = ind.ffill().fillna(0.0)
219
+ ind = ind.replace([np.inf, -np.inf], 0.0)
220
+ return ind
221
+
222
+
223
+ # ── The 40 canonical indicator names, grouped by dimension ──────────────────
224
+ INDICATOR_GROUPS = {
225
+ "Liquidity": [
226
+ "ted_spread_level", "ted_spread_velocity", "hy_ig_spread_ratio",
227
+ "hy_spread_velocity", "ig_spread_velocity", "yield_curve_level",
228
+ "yield_curve_velocity", "bond_equity_correlation"
229
+ ],
230
+ "Volatility": [
231
+ "vix_level", "vix_velocity", "vix_term_structure", "vix_z_score",
232
+ "realized_vol_spy", "implied_realized_vol_gap", "vol_of_vol",
233
+ "cross_asset_vol_spike"
234
+ ],
235
+ "Correlation": [
236
+ "equity_bond_corr", "equity_gold_corr", "equity_hy_corr",
237
+ "safe_haven_demand", "cross_sector_correlation", "em_dm_divergence",
238
+ "dollar_stress_signal", "correlation_breakdown_score"
239
+ ],
240
+ "Credit": [
241
+ "hy_spread_level", "ig_spread_level", "hy_ig_divergence_velocity",
242
+ "lqd_hyg_return_spread", "xlf_spy_relative", "credit_equity_dislocation",
243
+ "ig_spread_z_score", "hy_spread_z_score"
244
+ ],
245
+ "Positioning": [
246
+ "gold_flow_signal", "jpy_flow_signal", "chf_flow_signal",
247
+ "utilities_relative", "energy_relative", "oil_velocity",
248
+ "em_outflow_signal", "risk_off_composite"
249
+ ]
250
+ }
251
+
252
+ ALL_INDICATORS = [ind for group in INDICATOR_GROUPS.values() for ind in group]
data/pipeline.py ADDED
@@ -0,0 +1,119 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # data/pipeline.py
2
+
3
+ import os
4
+ import pandas as pd
5
+ import numpy as np
6
+ import yfinance as yf
7
+ from fredapi import Fred
8
+ from dotenv import load_dotenv
9
+ from datetime import datetime, timedelta
10
+ import warnings
11
+ warnings.filterwarnings('ignore')
12
+
13
+ load_dotenv()
14
+
15
+ FRED_API_KEY = os.getenv("FRED_API_KEY")
16
+
17
+ FRED_SERIES = {
18
+ "ted_spread": "TEDRATE",
19
+ "yield_curve_10y2y": "T10Y2Y",
20
+ "yield_curve_10y3m": "T10Y3M",
21
+ "ig_credit_spread": "BAMLC0A0CM",
22
+ "hy_credit_spread": "BAMLH0A0HYM2",
23
+ "ig_hy_ratio": None,
24
+ "vix_fred": "VIXCLS",
25
+ }
26
+
27
+ YF_TICKERS = {
28
+ "spy": "SPY", "qqq": "QQQ", "ief": "IEF", "lqd": "LQD",
29
+ "hyg": "HYG", "gld": "GLD", "uso": "USO", "uup": "UUP",
30
+ "eem": "EEM", "xlf": "XLF", "xle": "XLE", "xlu": "XLU",
31
+ "vix": "^VIX", "vix_3m": "^VIX3M", "vix_9d": "^VIX9D",
32
+ "eurusd": "EURUSD=X", "jpyusd": "JPY=X", "chfusd": "CHF=X",
33
+ }
34
+
35
+
36
+ def fetch_fred_data(start_date: str, end_date: str) -> pd.DataFrame:
37
+ """Fetches all FRED series. Returns DataFrame with dates as index."""
38
+ fred = Fred(api_key=FRED_API_KEY)
39
+ frames = {}
40
+ for name, series_id in FRED_SERIES.items():
41
+ if series_id is None:
42
+ continue
43
+ try:
44
+ s = fred.get_series(series_id, observation_start=start_date, observation_end=end_date)
45
+ frames[name] = s
46
+ except Exception as e:
47
+ print(f"[FRED] Failed to fetch {name} ({series_id}): {e}")
48
+
49
+ df = pd.DataFrame(frames)
50
+ df.index = pd.to_datetime(df.index)
51
+ df = df.resample("B").last()
52
+ df = df.ffill().bfill()
53
+
54
+ if "ig_credit_spread" in df.columns and "hy_credit_spread" in df.columns:
55
+ df["ig_hy_ratio"] = df["hy_credit_spread"] / df["ig_credit_spread"].replace(0, np.nan)
56
+ return df
57
+
58
+
59
+ def fetch_yfinance_data(start_date: str, end_date: str) -> pd.DataFrame:
60
+ """
61
+ Fetches closing prices for all yfinance tickers.
62
+ FIX #4: Handles both MultiIndex and flat column formats.
63
+ FIX #7: Case-insensitive column matching.
64
+ """
65
+ tickers = list(YF_TICKERS.values())
66
+ raw = yf.download(tickers, start=start_date, end=end_date, progress=False)
67
+
68
+ # FIX #4: Defensive column extraction
69
+ if isinstance(raw.columns, pd.MultiIndex):
70
+ if "Close" in raw.columns.get_level_values(0):
71
+ close = raw["Close"].copy()
72
+ elif "Adj Close" in raw.columns.get_level_values(0):
73
+ close = raw["Adj Close"].copy()
74
+ else:
75
+ first_level = raw.columns.get_level_values(0).unique()[0]
76
+ close = raw[first_level].copy()
77
+ else:
78
+ close = raw.copy()
79
+
80
+ # FIX #7: Case-insensitive reverse map
81
+ reverse_map = {v: k for k, v in YF_TICKERS.items()}
82
+ reverse_map_lower = {v.lower(): k for k, v in YF_TICKERS.items()}
83
+ new_columns = []
84
+ for col in close.columns:
85
+ col_str = str(col)
86
+ if col_str in reverse_map:
87
+ new_columns.append(reverse_map[col_str])
88
+ elif col_str.lower() in reverse_map_lower:
89
+ new_columns.append(reverse_map_lower[col_str.lower()])
90
+ else:
91
+ new_columns.append(col_str)
92
+ close.columns = new_columns
93
+ close = close.ffill().bfill()
94
+ return close
95
+
96
+
97
+ def fetch_all_data(start_date: str | None = None, end_date: str | None = None) -> pd.DataFrame:
98
+ """Master fetch function. Returns a single combined DataFrame."""
99
+ if end_date is None:
100
+ end_date = datetime.today().strftime("%Y-%m-%d")
101
+ if start_date is None:
102
+ start_date = (datetime.today() - timedelta(days=5 * 365)).strftime("%Y-%m-%d")
103
+
104
+ print(f"[Pipeline] Fetching data from {start_date} to {end_date}...")
105
+ fred_df = fetch_fred_data(start_date, end_date)
106
+ yf_df = fetch_yfinance_data(start_date, end_date)
107
+
108
+ combined = pd.concat([fred_df, yf_df], axis=1)
109
+ combined = combined.resample("B").last().ffill().bfill()
110
+ print(f"[Pipeline] Fetched {len(combined)} rows, {len(combined.columns)} columns.")
111
+ return combined
112
+
113
+
114
+ def fetch_live_snapshot() -> pd.DataFrame:
115
+ """Fetches the most recent ~1 year of data for live fingerprinting."""
116
+ return fetch_all_data(
117
+ start_date=(datetime.today() - timedelta(days=400)).strftime("%Y-%m-%d"),
118
+ end_date=datetime.today().strftime("%Y-%m-%d")
119
+ )
details.md ADDED
@@ -0,0 +1,1924 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # AUTOPSY — Market Dislocation Fingerprint Intelligence System
2
+ ### Full Project Blueprint & Vibe-Coding Implementation Guide
3
+ **Hackathon: TechEx Intelligent Enterprise Solutions | Track 4: Data & Intelligence**
4
+ **Author: Arka Sarkar**
5
+
6
+ ---
7
+
8
+ ## 0. What You Are Building (Read This First)
9
+
10
+ AUTOPSY is a real-time market stress intelligence system. It answers one question:
11
+
12
+ > **"Which historical crisis does the current market structure most resemble — and what happened next?"**
13
+
14
+ It does this by:
15
+ 1. Pulling ~40 live market indicators across 5 structural dimensions (liquidity, correlation, volatility, credit, positioning)
16
+ 2. Computing a "fingerprint vector" that describes the current structural state of markets
17
+ 3. Comparing that vector against pre-computed fingerprints of 10 historical crises using embedding similarity
18
+ 4. Running an AI agent (Claude via Anthropic API) that reads the fingerprint match and writes a structured risk narrative
19
+ 5. Displaying everything on a live dashboard with radar charts, similarity scores, and a "rewind" feature that lets you replay any historical period
20
+
21
+ This is NOT a price prediction tool. It is a market structure stress detector. That distinction is important for your pitch.
22
+
23
+ ---
24
+
25
+ ## 1. Project Structure (Create Exactly This)
26
+
27
+ ```
28
+ autopsy/
29
+ ├── README.md
30
+ ├── requirements.txt
31
+ ├── .env.example
32
+ ├── .gitignore
33
+
34
+ ├── data/
35
+ │ ├── pipeline.py # Fetches all live + historical data
36
+ │ ├── indicators.py # Computes all 40 indicators from raw data
37
+ │ └── crisis_library.py # Defines the 10 historical crisis windows
38
+
39
+ ├── fingerprint/
40
+ │ ├── engine.py # Rolls features, normalizes, builds vectors
41
+ │ └── embedding.py # UMAP embedding + similarity search
42
+
43
+ ├── agent/
44
+ │ └── analyst.py # Claude API call, prompt, structured output
45
+
46
+ ├── dashboard/
47
+ │ ├── app.py # Main Streamlit app entry point
48
+ │ ├── charts.py # All Plotly chart functions
49
+ │ └── state.py # Session state management
50
+
51
+ └── tests/
52
+ └── test_pipeline.py # Basic smoke tests
53
+ ```
54
+
55
+ ---
56
+
57
+ ## 2. Environment Setup
58
+
59
+ ### 2.1 Python Version
60
+ Use Python 3.11+
61
+
62
+ ### 2.2 requirements.txt (Copy This Exactly)
63
+ ```
64
+ streamlit==1.35.0
65
+ plotly==5.22.0
66
+ pandas==2.2.2
67
+ numpy==1.26.4
68
+ scipy==1.13.0
69
+ scikit-learn==1.5.0
70
+ umap-learn==0.5.6
71
+ yfinance==0.2.40
72
+ fredapi==0.5.2
73
+ anthropic==0.28.0
74
+ python-dotenv==1.0.1
75
+ requests==2.32.3
76
+ ```
77
+
78
+ ### 2.3 .env.example (Copy This)
79
+ ```
80
+ FRED_API_KEY=your_fred_api_key_here
81
+ ANTHROPIC_API_KEY=your_anthropic_api_key_here
82
+ ```
83
+
84
+ ### 2.4 How to Get API Keys
85
+ - **FRED API Key**: Go to https://fred.stlouisfed.org/docs/api/api_key.html — free, instant
86
+ - **Anthropic API Key**: Go to https://console.anthropic.com — free tier available
87
+
88
+ ### 2.5 .gitignore
89
+ ```
90
+ .env
91
+ __pycache__/
92
+ *.pyc
93
+ .DS_Store
94
+ venv/
95
+ *.egg-info/
96
+ dist/
97
+ build/
98
+ ```
99
+
100
+ ---
101
+
102
+ ## 3. Data Layer — `data/pipeline.py`
103
+
104
+ ### 3.1 What This File Does
105
+ Fetches all raw market data from two sources:
106
+ - **FRED** (Federal Reserve Economic Data) — for credit spreads, funding markets, Treasury data
107
+ - **yfinance** — for equity prices, VIX term structure, FX rates, commodities
108
+
109
+ ### 3.2 FRED Series to Fetch
110
+ These are the exact FRED series IDs. Fetch daily data.
111
+
112
+ | Variable Name | FRED Series ID | What It Measures |
113
+ |---|---|---|
114
+ | ted_spread | TEDRATE | TED spread (3M LIBOR - 3M T-Bill) — funding stress |
115
+ | ois_spread | T10Y2Y | 10Y-2Y Treasury spread — yield curve shape |
116
+ | investment_grade_spread | BAMLC0A0CM | IG corporate credit spread |
117
+ | high_yield_spread | BAMLH0A0HYM2 | HY corporate credit spread |
118
+ | commercial_paper_spread | CPFF | Commercial paper funding spread |
119
+ | vix | VIXCLS | VIX (implied volatility) |
120
+ | move_index | (use yfinance proxy) | Bond market volatility |
121
+
122
+ ### 3.3 yfinance Tickers to Fetch
123
+ Fetch daily OHLCV data for all of these:
124
+
125
+ | Variable Name | Ticker | What It Measures |
126
+ |---|---|---|
127
+ | spy | SPY | S&P 500 equity |
128
+ | qqq | QQQ | Nasdaq 100 |
129
+ | ief | IEF | 7-10Y Treasury bonds |
130
+ | lqd | LQD | Investment grade bonds |
131
+ | hyg | HYG | High yield bonds |
132
+ | gld | GLD | Gold |
133
+ | uso | USO | Oil |
134
+ | uup | UUP | US Dollar index |
135
+ | eem | EEM | Emerging markets equity |
136
+ | xlf | XLF | Financials sector |
137
+ | xle | XLE | Energy sector |
138
+ | xlu | XLU | Utilities sector |
139
+ | vix_front | ^VIX | VIX (front month) |
140
+ | vix_3m | ^VIX3M | VIX 3-month |
141
+ | vix_9d | ^VIX9D | VIX 9-day |
142
+ | eur_usd | EURUSD=X | EUR/USD exchange rate |
143
+ | jpy_usd | JPY=X | Japanese Yen |
144
+ | chf_usd | CHF=X | Swiss Franc (safe haven) |
145
+
146
+ ### 3.4 pipeline.py Full Implementation
147
+
148
+ ```python
149
+ # data/pipeline.py
150
+
151
+ import os
152
+ import pandas as pd
153
+ import numpy as np
154
+ import yfinance as yf
155
+ from fredapi import Fred
156
+ from dotenv import load_dotenv
157
+ from datetime import datetime, timedelta
158
+ import warnings
159
+ warnings.filterwarnings('ignore')
160
+
161
+ load_dotenv()
162
+
163
+ FRED_API_KEY = os.getenv("FRED_API_KEY")
164
+
165
+ # ── FRED Series IDs ──────────────────────────────────────────────────────────
166
+ FRED_SERIES = {
167
+ "ted_spread": "TEDRATE",
168
+ "yield_curve_10y2y": "T10Y2Y",
169
+ "yield_curve_10y3m": "T10Y3M",
170
+ "ig_credit_spread": "BAMLC0A0CM",
171
+ "hy_credit_spread": "BAMLH0A0HYM2",
172
+ "ig_hy_ratio": None, # computed: hy / ig
173
+ "vix_fred": "VIXCLS",
174
+ }
175
+
176
+ # ── yfinance Tickers ──────────────────────────────────────────────────────────
177
+ YF_TICKERS = {
178
+ "spy": "SPY",
179
+ "qqq": "QQQ",
180
+ "ief": "IEF",
181
+ "lqd": "LQD",
182
+ "hyg": "HYG",
183
+ "gld": "GLD",
184
+ "uso": "USO",
185
+ "uup": "UUP",
186
+ "eem": "EEM",
187
+ "xlf": "XLF",
188
+ "xle": "XLE",
189
+ "xlu": "XLU",
190
+ "vix": "^VIX",
191
+ "vix_3m": "^VIX3M",
192
+ "vix_9d": "^VIX9D",
193
+ "eurusd": "EURUSD=X",
194
+ "jpyusd": "JPY=X",
195
+ "chfusd": "CHF=X",
196
+ }
197
+
198
+
199
+ def fetch_fred_data(start_date: str, end_date: str) -> pd.DataFrame:
200
+ """
201
+ Fetches all FRED series between start_date and end_date.
202
+ Returns a single DataFrame with dates as index, series names as columns.
203
+ Missing values forward-filled then backward-filled (FRED data has gaps on weekends).
204
+ """
205
+ fred = Fred(api_key=FRED_API_KEY)
206
+ frames = {}
207
+
208
+ for name, series_id in FRED_SERIES.items():
209
+ if series_id is None:
210
+ continue
211
+ try:
212
+ s = fred.get_series(series_id, observation_start=start_date, observation_end=end_date)
213
+ frames[name] = s
214
+ except Exception as e:
215
+ print(f"[FRED] Failed to fetch {name} ({series_id}): {e}")
216
+
217
+ df = pd.DataFrame(frames)
218
+ df.index = pd.to_datetime(df.index)
219
+ df = df.resample("B").last() # business days only
220
+ df = df.ffill().bfill()
221
+
222
+ # Compute derived series
223
+ if "ig_credit_spread" in df.columns and "hy_credit_spread" in df.columns:
224
+ df["ig_hy_ratio"] = df["hy_credit_spread"] / df["ig_credit_spread"].replace(0, np.nan)
225
+
226
+ return df
227
+
228
+
229
+ def fetch_yfinance_data(start_date: str, end_date: str) -> pd.DataFrame:
230
+ """
231
+ Fetches closing prices for all yfinance tickers.
232
+ Returns DataFrame: dates as index, ticker names as columns (using our friendly names).
233
+ """
234
+ tickers = list(YF_TICKERS.values())
235
+ raw = yf.download(tickers, start=start_date, end=end_date, progress=False, auto_adjust=True)
236
+
237
+ # Extract close prices only
238
+ if isinstance(raw.columns, pd.MultiIndex):
239
+ close = raw["Close"]
240
+ else:
241
+ close = raw[["Close"]]
242
+
243
+ # Rename columns to friendly names
244
+ reverse_map = {v: k for k, v in YF_TICKERS.items()}
245
+ close.columns = [reverse_map.get(col, col) for col in close.columns]
246
+ close = close.ffill().bfill()
247
+
248
+ return close
249
+
250
+
251
+ def fetch_all_data(start_date: str = None, end_date: str = None) -> pd.DataFrame:
252
+ """
253
+ Master fetch function. Returns a single combined DataFrame.
254
+ Default: last 5 years of data.
255
+ """
256
+ if end_date is None:
257
+ end_date = datetime.today().strftime("%Y-%m-%d")
258
+ if start_date is None:
259
+ start_date = (datetime.today() - timedelta(days=5 * 365)).strftime("%Y-%m-%d")
260
+
261
+ print(f"[Pipeline] Fetching data from {start_date} to {end_date}...")
262
+ fred_df = fetch_fred_data(start_date, end_date)
263
+ yf_df = fetch_yfinance_data(start_date, end_date)
264
+
265
+ # Align on business day index
266
+ combined = pd.concat([fred_df, yf_df], axis=1)
267
+ combined = combined.resample("B").last().ffill().bfill()
268
+
269
+ print(f"[Pipeline] Fetched {len(combined)} rows, {len(combined.columns)} columns.")
270
+ return combined
271
+
272
+
273
+ def fetch_live_snapshot() -> pd.DataFrame:
274
+ """
275
+ Fetches the most recent 252 trading days (1 year) of data.
276
+ Used by the dashboard for the live fingerprint.
277
+ Returns the same format as fetch_all_data().
278
+ """
279
+ return fetch_all_data(
280
+ start_date=(datetime.today() - timedelta(days=400)).strftime("%Y-%m-%d"),
281
+ end_date=datetime.today().strftime("%Y-%m-%d")
282
+ )
283
+ ```
284
+
285
+ ---
286
+
287
+ ## 4. Indicator Engine — `data/indicators.py`
288
+
289
+ ### 4.1 What This File Does
290
+ Takes the raw price/spread DataFrame from pipeline.py and computes the 40 structured indicators across 5 dimensions. Each indicator is a rolling statistic (velocity, z-score, or ratio) that captures stress in a specific dimension.
291
+
292
+ ### 4.2 The 5 Dimensions and Their Indicators
293
+
294
+ **DIMENSION 1: LIQUIDITY (8 indicators)**
295
+ - `ted_spread_level` — raw TED spread value
296
+ - `ted_spread_velocity` — 10-day rate of change of TED spread
297
+ - `hy_ig_spread_ratio` — HY spread / IG spread (credit quality bifurcation)
298
+ - `hy_spread_velocity` — 10-day rate of change of HY spread
299
+ - `ig_spread_velocity` — 10-day rate of change of IG spread
300
+ - `yield_curve_level` — 10Y-2Y spread level
301
+ - `yield_curve_velocity` — 10-day rate of change of yield curve
302
+ - `bond_equity_correlation` — 21-day rolling correlation between IEF and SPY returns
303
+
304
+ **DIMENSION 2: VOLATILITY (8 indicators)**
305
+ - `vix_level` — raw VIX value
306
+ - `vix_velocity` — 5-day rate of change of VIX
307
+ - `vix_term_structure` — VIX9D / VIX3M ratio (inversion = front-end stress)
308
+ - `vix_z_score` — VIX vs its own 252-day rolling mean/std
309
+ - `realized_vol_spy` — 21-day realized volatility of SPY
310
+ - `implied_realized_vol_gap` — VIX - realized_vol_spy (fear premium)
311
+ - `vol_of_vol` — 21-day rolling std of daily VIX changes
312
+ - `cross_asset_vol_spike` — max(VIX velocity, bond vol velocity) normalized
313
+
314
+ **DIMENSION 3: CORRELATION (8 indicators)**
315
+ - `equity_bond_corr` — 21-day rolling corr(SPY returns, IEF returns)
316
+ - `equity_gold_corr` — 21-day rolling corr(SPY returns, GLD returns)
317
+ - `equity_hy_corr` — 21-day rolling corr(SPY returns, HYG returns)
318
+ - `safe_haven_demand` — 21-day cumulative return of GLD + JPY + CHF vs SPY
319
+ - `cross_sector_correlation` — avg pairwise corr among XLF, XLE, XLU, QQQ over 21 days
320
+ - `em_dm_divergence` — 21-day return spread: SPY - EEM
321
+ - `dollar_stress_signal` — UUP 10-day momentum (dollar surges in crises)
322
+ - `correlation_breakdown_score` — std of pairwise sector correlations (breakdown = high std)
323
+
324
+ **DIMENSION 4: CREDIT (8 indicators)**
325
+ - `hy_spread_level` — raw HY OAS spread
326
+ - `ig_spread_level` — raw IG OAS spread
327
+ - `hy_ig_divergence_velocity` — 10-day change in (HY - IG) spread gap
328
+ - `lqd_hyg_return_spread` — 21-day return: LQD - HYG (flight to quality)
329
+ - `xlf_spy_relative` — XLF vs SPY 21-day relative performance (financials leading risk)
330
+ - `credit_equity_dislocation` — HYG 21-day return vs SPY 21-day return (should co-move)
331
+ - `ig_spread_z_score` — IG spread vs its 252-day rolling mean/std
332
+ - `hy_spread_z_score` — HY spread vs its 252-day rolling mean/std
333
+
334
+ **DIMENSION 5: POSITIONING / FLOWS (8 indicators)**
335
+ - `gold_flow_signal` — GLD 10-day momentum (safe haven buying)
336
+ - `jpy_flow_signal` — JPY 10-day momentum (yen strengthening = risk-off)
337
+ - `chf_flow_signal` — CHF 10-day momentum (CHF strengthening = risk-off)
338
+ - `utilities_relative` — XLU vs SPY 21-day relative (defensive rotation)
339
+ - `energy_relative` — XLE vs SPY 21-day relative (commodity stress proxy)
340
+ - `oil_velocity` — USO 10-day momentum
341
+ - `em_outflow_signal` — EEM 10-day momentum (EM outflows in crises)
342
+ - `risk_off_composite` — equal-weight composite of gold + yen + chf + utilities signals
343
+
344
+ ### 4.3 indicators.py Full Implementation
345
+
346
+ ```python
347
+ # data/indicators.py
348
+
349
+ import pandas as pd
350
+ import numpy as np
351
+
352
+
353
+ def compute_velocity(series: pd.Series, window: int = 10) -> pd.Series:
354
+ """Rate of change over window days, as percentage."""
355
+ return series.pct_change(periods=window) * 100
356
+
357
+
358
+ def compute_zscore(series: pd.Series, window: int = 252) -> pd.Series:
359
+ """Rolling z-score against a trailing window."""
360
+ mean = series.rolling(window=window, min_periods=window // 2).mean()
361
+ std = series.rolling(window=window, min_periods=window // 2).std()
362
+ return (series - mean) / std.replace(0, np.nan)
363
+
364
+
365
+ def compute_rolling_correlation(s1: pd.Series, s2: pd.Series, window: int = 21) -> pd.Series:
366
+ """Rolling correlation between two return series."""
367
+ r1 = s1.pct_change()
368
+ r2 = s2.pct_change()
369
+ return r1.rolling(window=window, min_periods=window // 2).corr(r2)
370
+
371
+
372
+ def compute_realized_vol(series: pd.Series, window: int = 21) -> pd.Series:
373
+ """Annualized realized volatility."""
374
+ log_returns = np.log(series / series.shift(1))
375
+ return log_returns.rolling(window=window, min_periods=window // 2).std() * np.sqrt(252) * 100
376
+
377
+
378
+ def compute_all_indicators(df: pd.DataFrame) -> pd.DataFrame:
379
+ """
380
+ Takes raw combined DataFrame from pipeline.py.
381
+ Returns a new DataFrame with all 40 indicator columns.
382
+ Index is the same as input (business days).
383
+ NaN-filled with forward fill, then 0 for any remaining gaps.
384
+ """
385
+ ind = pd.DataFrame(index=df.index)
386
+
387
+ # ── DIMENSION 1: LIQUIDITY ────────────────────────────────────────────────
388
+ if "ted_spread" in df.columns:
389
+ ind["ted_spread_level"] = df["ted_spread"]
390
+ ind["ted_spread_velocity"] = compute_velocity(df["ted_spread"], 10)
391
+ else:
392
+ ind["ted_spread_level"] = 0.0
393
+ ind["ted_spread_velocity"] = 0.0
394
+
395
+ if "hy_credit_spread" in df.columns and "ig_credit_spread" in df.columns:
396
+ ind["hy_ig_spread_ratio"] = df["hy_credit_spread"] / df["ig_credit_spread"].replace(0, np.nan)
397
+ ind["hy_spread_velocity"] = compute_velocity(df["hy_credit_spread"], 10)
398
+ ind["ig_spread_velocity"] = compute_velocity(df["ig_credit_spread"], 10)
399
+ ind["hy_spread_level"] = df["hy_credit_spread"]
400
+ ind["ig_spread_level"] = df["ig_credit_spread"]
401
+ ind["hy_spread_z_score"] = compute_zscore(df["hy_credit_spread"])
402
+ ind["ig_spread_z_score"] = compute_zscore(df["ig_credit_spread"])
403
+ else:
404
+ for col in ["hy_ig_spread_ratio", "hy_spread_velocity", "ig_spread_velocity",
405
+ "hy_spread_level", "ig_spread_level", "hy_spread_z_score", "ig_spread_z_score"]:
406
+ ind[col] = 0.0
407
+
408
+ if "yield_curve_10y2y" in df.columns:
409
+ ind["yield_curve_level"] = df["yield_curve_10y2y"]
410
+ ind["yield_curve_velocity"] = compute_velocity(df["yield_curve_10y2y"], 10)
411
+ else:
412
+ ind["yield_curve_level"] = 0.0
413
+ ind["yield_curve_velocity"] = 0.0
414
+
415
+ if "ief" in df.columns and "spy" in df.columns:
416
+ ind["bond_equity_correlation"] = compute_rolling_correlation(df["ief"], df["spy"], 21)
417
+ else:
418
+ ind["bond_equity_correlation"] = 0.0
419
+
420
+ # ── DIMENSION 2: VOLATILITY ───────────────────────────────────────────────
421
+ vix_col = "vix" if "vix" in df.columns else "vix_fred"
422
+ if vix_col in df.columns:
423
+ ind["vix_level"] = df[vix_col]
424
+ ind["vix_velocity"] = compute_velocity(df[vix_col], 5)
425
+ ind["vix_z_score"] = compute_zscore(df[vix_col])
426
+ ind["vol_of_vol"] = df[vix_col].diff().rolling(21).std()
427
+ else:
428
+ for col in ["vix_level", "vix_velocity", "vix_z_score", "vol_of_vol"]:
429
+ ind[col] = 0.0
430
+
431
+ if "vix_9d" in df.columns and "vix_3m" in df.columns:
432
+ ind["vix_term_structure"] = df["vix_9d"] / df["vix_3m"].replace(0, np.nan)
433
+ else:
434
+ ind["vix_term_structure"] = 1.0
435
+
436
+ if "spy" in df.columns:
437
+ ind["realized_vol_spy"] = compute_realized_vol(df["spy"], 21)
438
+ else:
439
+ ind["realized_vol_spy"] = 0.0
440
+
441
+ if "vix_level" in ind.columns and "realized_vol_spy" in ind.columns:
442
+ ind["implied_realized_vol_gap"] = ind["vix_level"] - ind["realized_vol_spy"]
443
+ else:
444
+ ind["implied_realized_vol_gap"] = 0.0
445
+
446
+ ind["cross_asset_vol_spike"] = np.abs(ind.get("vix_velocity", 0))
447
+
448
+ # ── DIMENSION 3: CORRELATION ──────────────────────────────────────────────
449
+ if "spy" in df.columns and "ief" in df.columns:
450
+ ind["equity_bond_corr"] = compute_rolling_correlation(df["spy"], df["ief"], 21)
451
+ else:
452
+ ind["equity_bond_corr"] = 0.0
453
+
454
+ if "spy" in df.columns and "gld" in df.columns:
455
+ ind["equity_gold_corr"] = compute_rolling_correlation(df["spy"], df["gld"], 21)
456
+ else:
457
+ ind["equity_gold_corr"] = 0.0
458
+
459
+ if "spy" in df.columns and "hyg" in df.columns:
460
+ ind["equity_hy_corr"] = compute_rolling_correlation(df["spy"], df["hyg"], 21)
461
+ else:
462
+ ind["equity_hy_corr"] = 0.0
463
+
464
+ # Safe haven demand: cumulative 21-day return of GLD + JPY + CHF vs SPY
465
+ safe_haven_components = []
466
+ for ticker in ["gld", "jpyusd", "chfusd"]:
467
+ if ticker in df.columns:
468
+ safe_haven_components.append(df[ticker].pct_change(21))
469
+ if safe_haven_components and "spy" in df.columns:
470
+ avg_safe = pd.concat(safe_haven_components, axis=1).mean(axis=1)
471
+ ind["safe_haven_demand"] = avg_safe - df["spy"].pct_change(21)
472
+ else:
473
+ ind["safe_haven_demand"] = 0.0
474
+
475
+ # Cross-sector correlation
476
+ sector_tickers = [t for t in ["xlf", "xle", "xlu", "qqq"] if t in df.columns]
477
+ if len(sector_tickers) >= 2:
478
+ sector_returns = df[sector_tickers].pct_change()
479
+ rolling_corrs = []
480
+ for i in range(len(sector_tickers)):
481
+ for j in range(i+1, len(sector_tickers)):
482
+ c = sector_returns[sector_tickers[i]].rolling(21).corr(sector_returns[sector_tickers[j]])
483
+ rolling_corrs.append(c)
484
+ ind["cross_sector_correlation"] = pd.concat(rolling_corrs, axis=1).mean(axis=1)
485
+ ind["correlation_breakdown_score"] = pd.concat(rolling_corrs, axis=1).std(axis=1)
486
+ else:
487
+ ind["cross_sector_correlation"] = 0.0
488
+ ind["correlation_breakdown_score"] = 0.0
489
+
490
+ if "spy" in df.columns and "eem" in df.columns:
491
+ ind["em_dm_divergence"] = df["spy"].pct_change(21) - df["eem"].pct_change(21)
492
+ else:
493
+ ind["em_dm_divergence"] = 0.0
494
+
495
+ if "uup" in df.columns:
496
+ ind["dollar_stress_signal"] = compute_velocity(df["uup"], 10)
497
+ else:
498
+ ind["dollar_stress_signal"] = 0.0
499
+
500
+ # ── DIMENSION 4: CREDIT ─────────────────────────���─────────────────────────
501
+ if "hy_credit_spread" in df.columns and "ig_credit_spread" in df.columns:
502
+ hy_ig_gap = df["hy_credit_spread"] - df["ig_credit_spread"]
503
+ ind["hy_ig_divergence_velocity"] = compute_velocity(hy_ig_gap, 10)
504
+ else:
505
+ ind["hy_ig_divergence_velocity"] = 0.0
506
+
507
+ if "lqd" in df.columns and "hyg" in df.columns:
508
+ ind["lqd_hyg_return_spread"] = df["lqd"].pct_change(21) - df["hyg"].pct_change(21)
509
+ else:
510
+ ind["lqd_hyg_return_spread"] = 0.0
511
+
512
+ if "xlf" in df.columns and "spy" in df.columns:
513
+ ind["xlf_spy_relative"] = df["xlf"].pct_change(21) - df["spy"].pct_change(21)
514
+ else:
515
+ ind["xlf_spy_relative"] = 0.0
516
+
517
+ if "hyg" in df.columns and "spy" in df.columns:
518
+ ind["credit_equity_dislocation"] = df["hyg"].pct_change(21) - df["spy"].pct_change(21)
519
+ else:
520
+ ind["credit_equity_dislocation"] = 0.0
521
+
522
+ # ── DIMENSION 5: POSITIONING / FLOWS ─────────────────────────────────────
523
+ flow_signals = {}
524
+ for ticker, col_name in [("gld", "gold_flow_signal"), ("jpyusd", "jpy_flow_signal"),
525
+ ("chfusd", "chf_flow_signal")]:
526
+ if ticker in df.columns:
527
+ ind[col_name] = compute_velocity(df[ticker], 10)
528
+ flow_signals[col_name] = ind[col_name]
529
+ else:
530
+ ind[col_name] = 0.0
531
+
532
+ if "xlu" in df.columns and "spy" in df.columns:
533
+ ind["utilities_relative"] = df["xlu"].pct_change(21) - df["spy"].pct_change(21)
534
+ else:
535
+ ind["utilities_relative"] = 0.0
536
+
537
+ if "xle" in df.columns and "spy" in df.columns:
538
+ ind["energy_relative"] = df["xle"].pct_change(21) - df["spy"].pct_change(21)
539
+ else:
540
+ ind["energy_relative"] = 0.0
541
+
542
+ if "uso" in df.columns:
543
+ ind["oil_velocity"] = compute_velocity(df["uso"], 10)
544
+ else:
545
+ ind["oil_velocity"] = 0.0
546
+
547
+ if "eem" in df.columns:
548
+ ind["em_outflow_signal"] = compute_velocity(df["eem"], 10) * -1 # invert: negative = outflow
549
+ else:
550
+ ind["em_outflow_signal"] = 0.0
551
+
552
+ # Risk-off composite
553
+ risk_off_cols = [c for c in ["gold_flow_signal", "jpy_flow_signal", "chf_flow_signal", "utilities_relative"]
554
+ if c in ind.columns]
555
+ if risk_off_cols:
556
+ ind["risk_off_composite"] = ind[risk_off_cols].mean(axis=1)
557
+ else:
558
+ ind["risk_off_composite"] = 0.0
559
+
560
+ # ── FINAL CLEANUP ─────────────────────────────────────────────────────────
561
+ ind = ind.ffill().fillna(0.0)
562
+ ind = ind.replace([np.inf, -np.inf], 0.0)
563
+
564
+ return ind
565
+
566
+
567
+ # ── The 40 canonical indicator names, grouped by dimension ──────────────────
568
+ INDICATOR_GROUPS = {
569
+ "Liquidity": [
570
+ "ted_spread_level", "ted_spread_velocity", "hy_ig_spread_ratio",
571
+ "hy_spread_velocity", "ig_spread_velocity", "yield_curve_level",
572
+ "yield_curve_velocity", "bond_equity_correlation"
573
+ ],
574
+ "Volatility": [
575
+ "vix_level", "vix_velocity", "vix_term_structure", "vix_z_score",
576
+ "realized_vol_spy", "implied_realized_vol_gap", "vol_of_vol",
577
+ "cross_asset_vol_spike"
578
+ ],
579
+ "Correlation": [
580
+ "equity_bond_corr", "equity_gold_corr", "equity_hy_corr",
581
+ "safe_haven_demand", "cross_sector_correlation", "em_dm_divergence",
582
+ "dollar_stress_signal", "correlation_breakdown_score"
583
+ ],
584
+ "Credit": [
585
+ "hy_spread_level", "ig_spread_level", "hy_ig_divergence_velocity",
586
+ "lqd_hyg_return_spread", "xlf_spy_relative", "credit_equity_dislocation",
587
+ "ig_spread_z_score", "hy_spread_z_score"
588
+ ],
589
+ "Positioning": [
590
+ "gold_flow_signal", "jpy_flow_signal", "chf_flow_signal",
591
+ "utilities_relative", "energy_relative", "oil_velocity",
592
+ "em_outflow_signal", "risk_off_composite"
593
+ ]
594
+ }
595
+
596
+ ALL_INDICATORS = [ind for group in INDICATOR_GROUPS.values() for ind in group]
597
+ ```
598
+
599
+ ---
600
+
601
+ ## 5. Crisis Library — `data/crisis_library.py`
602
+
603
+ ### 5.1 What This File Does
604
+ Defines the 10 historical crisis windows. Each crisis has:
605
+ - A name and short description
606
+ - A "stress window": the 60-day period BEFORE the main blowup (this is where the fingerprints form)
607
+ - A "peak date": the day the crisis peaked/was most visible
608
+
609
+ The fingerprint engine will extract indicator vectors from the stress window to build the crisis embedding library.
610
+
611
+ ```python
612
+ # data/crisis_library.py
613
+
614
+ CRISIS_LIBRARY = {
615
+ "LTCM_1998": {
616
+ "name": "LTCM Collapse",
617
+ "short": "LTCM 1998",
618
+ "description": (
619
+ "Long-Term Capital Management, a highly leveraged hedge fund, collapsed after "
620
+ "Russian debt default. Required a Fed-orchestrated bailout. Key fingerprint: "
621
+ "basis blowout in theoretically related instruments, liquidity withdrawal from "
622
+ "obscure spreads before equity markets noticed."
623
+ ),
624
+ "stress_start": "1998-06-01",
625
+ "stress_end": "1998-09-01",
626
+ "peak_date": "1998-10-08",
627
+ "color": "#E74C3C",
628
+ "key_signature": "Basis blowouts + liquidity fragmentation"
629
+ },
630
+ "DOTCOM_2000": {
631
+ "name": "Dot-Com Bust",
632
+ "short": "Dot-Com 2000",
633
+ "description": (
634
+ "Technology bubble collapse. NASDAQ fell 78% peak to trough. Key fingerprint: "
635
+ "intra-equity sector divergence (tech vs value), VIX term structure flattening, "
636
+ "without the broad credit market stress seen in GFC."
637
+ ),
638
+ "stress_start": "2000-01-01",
639
+ "stress_end": "2000-03-10",
640
+ "peak_date": "2000-03-10",
641
+ "color": "#E67E22",
642
+ "key_signature": "Intra-equity sector divergence + vol term structure"
643
+ },
644
+ "GFC_2008": {
645
+ "name": "Global Financial Crisis",
646
+ "short": "GFC 2008",
647
+ "description": (
648
+ "Subprime mortgage collapse triggered global banking crisis. Key fingerprint: "
649
+ "TED spread explosion, commercial paper market seizure, everything correlated to 1, "
650
+ "funding markets frozen before equity indices reflected the stress."
651
+ ),
652
+ "stress_start": "2007-08-01",
653
+ "stress_end": "2008-09-15",
654
+ "peak_date": "2008-10-10",
655
+ "color": "#8E44AD",
656
+ "key_signature": "Funding market seizure + universal correlation spike"
657
+ },
658
+ "FLASH_CRASH_2010": {
659
+ "name": "Flash Crash",
660
+ "short": "Flash Crash 2010",
661
+ "description": (
662
+ "May 6, 2010: Dow Jones fell nearly 1000 points in minutes then recovered. "
663
+ "Key fingerprint: pure liquidity microstructure stress without credit signal, "
664
+ "very short duration. Demonstrates positioning-driven stress."
665
+ ),
666
+ "stress_start": "2010-04-15",
667
+ "stress_end": "2010-05-06",
668
+ "peak_date": "2010-05-06",
669
+ "color": "#27AE60",
670
+ "key_signature": "Microstructure-only liquidity collapse, no credit signal"
671
+ },
672
+ "EUROZONE_2011": {
673
+ "name": "Eurozone Debt Crisis",
674
+ "short": "Eurozone 2011",
675
+ "description": (
676
+ "Greek, Italian, Spanish sovereign debt crisis threatened euro breakup. "
677
+ "Key fingerprint: geographic contagion pattern in sovereign credit, EUR/USD stress, "
678
+ "financials leading equity drawdown."
679
+ ),
680
+ "stress_start": "2011-06-01",
681
+ "stress_end": "2011-08-08",
682
+ "peak_date": "2011-09-23",
683
+ "color": "#2980B9",
684
+ "key_signature": "Sovereign credit + financials leading + EUR stress"
685
+ },
686
+ "TAPER_TANTRUM_2013": {
687
+ "name": "Taper Tantrum",
688
+ "short": "Taper Tantrum 2013",
689
+ "description": (
690
+ "Fed signaled QE tapering; bond markets sold off violently. "
691
+ "Key fingerprint: rate-driven cross-asset stress, EM currency selloff, "
692
+ "bond-equity correlation flip, yield curve steepening velocity."
693
+ ),
694
+ "stress_start": "2013-05-01",
695
+ "stress_end": "2013-06-25",
696
+ "peak_date": "2013-06-25",
697
+ "color": "#16A085",
698
+ "key_signature": "Duration selloff + EM outflows + yield curve velocity"
699
+ },
700
+ "CHINA_OIL_2015": {
701
+ "name": "China/Oil Shock",
702
+ "short": "China/Oil 2015",
703
+ "description": (
704
+ "Chinese growth fears + oil collapse triggered global equity selloff. "
705
+ "Key fingerprint: commodity-financial system coupling, EM stress, "
706
+ "energy sector leading, oil-dollar relationship breaking down."
707
+ ),
708
+ "stress_start": "2015-06-01",
709
+ "stress_end": "2015-08-25",
710
+ "peak_date": "2015-08-25",
711
+ "color": "#D35400",
712
+ "key_signature": "Commodity-financial coupling + EM selloff"
713
+ },
714
+ "VOL_SHOCK_2018": {
715
+ "name": "Volmageddon",
716
+ "short": "Volmageddon 2018",
717
+ "description": (
718
+ "February 2018: Short volatility strategies (VIX ETPs) exploded, "
719
+ "triggering forced unwinds. Key fingerprint: VIX futures basis blowout, "
720
+ "positioning-driven without fundamental credit stress, very fast recovery."
721
+ ),
722
+ "stress_start": "2018-01-15",
723
+ "stress_end": "2018-02-05",
724
+ "peak_date": "2018-02-05",
725
+ "color": "#C0392B",
726
+ "key_signature": "VIX futures basis + short-vol unwind + no credit"
727
+ },
728
+ "COVID_2020": {
729
+ "name": "COVID Market Crash",
730
+ "short": "COVID 2020",
731
+ "description": (
732
+ "March 2020: Fastest 30% market crash in history. Key fingerprint: "
733
+ "everything selling simultaneously including gold (margin calls), "
734
+ "correlations going to 1 across ALL assets, dollar surging, "
735
+ "funding markets seizing within days."
736
+ ),
737
+ "stress_start": "2020-02-01",
738
+ "stress_end": "2020-03-23",
739
+ "peak_date": "2020-03-23",
740
+ "color": "#1ABC9C",
741
+ "key_signature": "Universal correlation spike + margin call fingerprint"
742
+ },
743
+ "SVB_2023": {
744
+ "name": "SVB Banking Crisis",
745
+ "short": "SVB 2023",
746
+ "description": (
747
+ "Silicon Valley Bank collapse triggered regional banking crisis. "
748
+ "Key fingerprint: regional bank CDS spike, financials sector stress, "
749
+ "rate sensitivity stress without broad credit market contagion. "
750
+ "Funding-specific rather than credit-market-wide."
751
+ ),
752
+ "stress_start": "2023-02-01",
753
+ "stress_end": "2023-03-10",
754
+ "peak_date": "2023-03-10",
755
+ "color": "#9B59B6",
756
+ "key_signature": "Financials-specific + rate sensitivity + contained credit"
757
+ }
758
+ }
759
+ ```
760
+
761
+ ---
762
+
763
+ ## 6. Fingerprint Engine — `fingerprint/engine.py`
764
+
765
+ ### 6.1 What This File Does
766
+ For each crisis window, extracts a fingerprint vector (mean values of each indicator during the stress window, normalized). For the live market, extracts the most recent row as a fingerprint vector. All vectors are normalized to the same scale for comparison.
767
+
768
+ ```python
769
+ # fingerprint/engine.py
770
+
771
+ import pandas as pd
772
+ import numpy as np
773
+ from sklearn.preprocessing import RobustScaler
774
+ from data.indicators import ALL_INDICATORS, INDICATOR_GROUPS, compute_all_indicators
775
+ from data.crisis_library import CRISIS_LIBRARY
776
+ from data.pipeline import fetch_all_data
777
+
778
+
779
+ def extract_crisis_fingerprints(
780
+ indicator_df: pd.DataFrame,
781
+ scaler: RobustScaler = None
782
+ ) -> tuple[dict, RobustScaler]:
783
+ """
784
+ For each crisis in CRISIS_LIBRARY, extracts the mean indicator vector
785
+ during the stress window.
786
+
787
+ Returns:
788
+ crisis_fingerprints: dict mapping crisis_key -> np.array of shape (n_indicators,)
789
+ scaler: fitted RobustScaler (use this to normalize the live fingerprint too)
790
+ """
791
+ # Only use indicators that exist in our DataFrame
792
+ available_indicators = [ind for ind in ALL_INDICATORS if ind in indicator_df.columns]
793
+
794
+ # Build crisis fingerprint vectors
795
+ crisis_fingerprints = {}
796
+ for crisis_key, crisis_info in CRISIS_LIBRARY.items():
797
+ start = crisis_info["stress_start"]
798
+ end = crisis_info["stress_end"]
799
+ window = indicator_df.loc[start:end, available_indicators]
800
+ if len(window) == 0:
801
+ continue
802
+ # Use mean of the stress window as the fingerprint
803
+ crisis_fingerprints[crisis_key] = window.mean().values
804
+
805
+ if not crisis_fingerprints:
806
+ raise ValueError("No crisis fingerprints could be computed. Check data range.")
807
+
808
+ # Fit scaler on all crisis fingerprints combined
809
+ all_vectors = np.array(list(crisis_fingerprints.values()))
810
+ if scaler is None:
811
+ scaler = RobustScaler()
812
+ scaler.fit(all_vectors)
813
+
814
+ # Normalize all crisis fingerprints
815
+ for key in crisis_fingerprints:
816
+ crisis_fingerprints[key] = scaler.transform(
817
+ crisis_fingerprints[key].reshape(1, -1)
818
+ ).flatten()
819
+
820
+ return crisis_fingerprints, scaler, available_indicators
821
+
822
+
823
+ def extract_live_fingerprint(
824
+ indicator_df: pd.DataFrame,
825
+ scaler: RobustScaler,
826
+ available_indicators: list
827
+ ) -> np.ndarray:
828
+ """
829
+ Extracts the most recent row as the live fingerprint vector.
830
+ Uses the same scaler as crisis fingerprints.
831
+ """
832
+ latest = indicator_df[available_indicators].dropna().iloc[-1].values
833
+ return scaler.transform(latest.reshape(1, -1)).flatten()
834
+
835
+
836
+ def extract_historical_fingerprint(
837
+ indicator_df: pd.DataFrame,
838
+ scaler: RobustScaler,
839
+ available_indicators: list,
840
+ date: str
841
+ ) -> np.ndarray:
842
+ """
843
+ For the rewind feature: extract fingerprint as of a specific historical date.
844
+ Uses 21-day lookback window ending on the given date.
845
+ """
846
+ end_dt = pd.to_datetime(date)
847
+ start_dt = end_dt - pd.Timedelta(days=30)
848
+ window = indicator_df.loc[start_dt:end_dt, available_indicators]
849
+ if len(window) == 0:
850
+ return None
851
+ vec = window.mean().values
852
+ return scaler.transform(vec.reshape(1, -1)).flatten()
853
+
854
+
855
+ def compute_similarity_scores(
856
+ query_vector: np.ndarray,
857
+ crisis_fingerprints: dict
858
+ ) -> list[dict]:
859
+ """
860
+ Computes cosine similarity between the query vector and all crisis fingerprints.
861
+ Returns sorted list of dicts: [{ crisis_key, name, similarity, ... }, ...]
862
+ Similarity is in [0, 100] range.
863
+ """
864
+ from sklearn.metrics.pairwise import cosine_similarity
865
+
866
+ results = []
867
+ for crisis_key, crisis_vector in crisis_fingerprints.items():
868
+ crisis_info = CRISIS_LIBRARY[crisis_key]
869
+ sim = cosine_similarity(
870
+ query_vector.reshape(1, -1),
871
+ crisis_vector.reshape(1, -1)
872
+ )[0][0]
873
+ # Map from [-1, 1] to [0, 100]
874
+ similarity_pct = max(0, (sim + 1) / 2 * 100)
875
+ results.append({
876
+ "crisis_key": crisis_key,
877
+ "name": crisis_info["name"],
878
+ "short": crisis_info["short"],
879
+ "similarity": round(similarity_pct, 1),
880
+ "color": crisis_info["color"],
881
+ "key_signature": crisis_info["key_signature"],
882
+ "description": crisis_info["description"],
883
+ "peak_date": crisis_info["peak_date"],
884
+ })
885
+
886
+ results.sort(key=lambda x: x["similarity"], reverse=True)
887
+ return results
888
+
889
+
890
+ def compute_dimension_scores(
891
+ query_vector: np.ndarray,
892
+ available_indicators: list
893
+ ) -> dict:
894
+ """
895
+ Computes per-dimension stress scores from the normalized fingerprint vector.
896
+ Returns dict: dimension_name -> stress_score (0-100).
897
+ Higher = more stressed.
898
+ """
899
+ scores = {}
900
+ for dim_name, dim_indicators in INDICATOR_GROUPS.items():
901
+ # Find indices of this dimension's indicators in available_indicators
902
+ indices = [i for i, ind in enumerate(available_indicators) if ind in dim_indicators]
903
+ if not indices:
904
+ scores[dim_name] = 0.0
905
+ continue
906
+ dim_vec = query_vector[indices]
907
+ # Use absolute value and normalize to 0-100
908
+ score = np.abs(dim_vec).mean()
909
+ # Cap at reasonable max (robust scaler output rarely exceeds 3)
910
+ score = min(score / 3.0 * 100, 100)
911
+ scores[dim_name] = round(score, 1)
912
+ return scores
913
+ ```
914
+
915
+ ---
916
+
917
+ ## 7. Embedding Space — `fingerprint/embedding.py`
918
+
919
+ ### 7.1 What This File Does
920
+ Projects crisis fingerprints and the live fingerprint into 2D using UMAP for the scatter plot visualization. Also computes the dimension-level radar chart values.
921
+
922
+ ```python
923
+ # fingerprint/embedding.py
924
+
925
+ import numpy as np
926
+ import pandas as pd
927
+
928
+
929
+ def build_umap_embedding(crisis_fingerprints: dict, live_vector: np.ndarray = None):
930
+ """
931
+ Fits UMAP on crisis fingerprints and optionally projects the live vector.
932
+
933
+ Returns:
934
+ crisis_coords: dict mapping crisis_key -> (x, y) 2D coordinates
935
+ live_coords: (x, y) tuple for live market, or None
936
+ """
937
+ try:
938
+ import umap
939
+ except ImportError:
940
+ # Fallback to PCA if umap not available
941
+ from sklearn.decomposition import PCA
942
+ keys = list(crisis_fingerprints.keys())
943
+ vectors = np.array([crisis_fingerprints[k] for k in keys])
944
+ pca = PCA(n_components=2)
945
+ coords_2d = pca.fit_transform(vectors)
946
+ crisis_coords = {keys[i]: tuple(coords_2d[i]) for i in range(len(keys))}
947
+ live_coords = None
948
+ if live_vector is not None:
949
+ live_2d = pca.transform(live_vector.reshape(1, -1))[0]
950
+ live_coords = tuple(live_2d)
951
+ return crisis_coords, live_coords
952
+
953
+ keys = list(crisis_fingerprints.keys())
954
+ vectors = np.array([crisis_fingerprints[k] for k in keys])
955
+
956
+ all_vectors = vectors
957
+ if live_vector is not None:
958
+ all_vectors = np.vstack([vectors, live_vector.reshape(1, -1)])
959
+
960
+ reducer = umap.UMAP(n_components=2, n_neighbors=min(5, len(keys)-1), random_state=42)
961
+ coords_2d = reducer.fit_transform(all_vectors)
962
+
963
+ crisis_coords = {keys[i]: tuple(coords_2d[i]) for i in range(len(keys))}
964
+ live_coords = None
965
+ if live_vector is not None:
966
+ live_coords = tuple(coords_2d[-1])
967
+
968
+ return crisis_coords, live_coords
969
+ ```
970
+
971
+ ---
972
+
973
+ ## 8. AI Analyst Agent — `agent/analyst.py`
974
+
975
+ ### 8.1 What This File Does
976
+ Calls the Anthropic API with a structured prompt that includes the current fingerprint data and top crisis analogues. Returns a structured risk narrative.
977
+
978
+ ### 8.2 The Prompt Design
979
+ The prompt gives Claude:
980
+ - The top 3 crisis analogues and their similarity scores
981
+ - The per-dimension stress scores
982
+ - The top stressed indicators by name
983
+ - Instructions to produce exactly structured output
984
+
985
+ ```python
986
+ # agent/analyst.py
987
+
988
+ import os
989
+ import anthropic
990
+ from dotenv import load_dotenv
991
+
992
+ load_dotenv()
993
+
994
+ client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
995
+
996
+
997
+ def build_prompt(
998
+ similarity_results: list,
999
+ dimension_scores: dict,
1000
+ available_indicators: list,
1001
+ live_vector,
1002
+ query_date: str = "today"
1003
+ ) -> str:
1004
+ """
1005
+ Builds the structured prompt for the Claude analyst agent.
1006
+ """
1007
+ top_3 = similarity_results[:3]
1008
+
1009
+ # Find the top 5 most stressed indicators (highest absolute normalized values)
1010
+ import numpy as np
1011
+ indicator_stress = [(available_indicators[i], abs(live_vector[i]))
1012
+ for i in range(len(available_indicators))]
1013
+ indicator_stress.sort(key=lambda x: x[1], reverse=True)
1014
+ top_indicators = indicator_stress[:5]
1015
+
1016
+ top_analogue_text = "\n".join([
1017
+ f" {i+1}. {r['name']} ({r['short']}): {r['similarity']:.1f}% similarity\n"
1018
+ f" Key signature: {r['key_signature']}\n"
1019
+ f" Peak date: {r['peak_date']}"
1020
+ for i, r in enumerate(top_3)
1021
+ ])
1022
+
1023
+ dimension_text = "\n".join([
1024
+ f" {dim}: {score:.1f}/100 stress"
1025
+ for dim, score in sorted(dimension_scores.items(), key=lambda x: x[1], reverse=True)
1026
+ ])
1027
+
1028
+ indicator_text = "\n".join([
1029
+ f" {ind.replace('_', ' ')}: {val:.2f}σ deviation"
1030
+ for ind, val in top_indicators
1031
+ ])
1032
+
1033
+ prompt = f"""You are AUTOPSY, a quantitative market risk analyst system.
1034
+ Your job is to analyze current market structure and produce a concise, precise risk narrative.
1035
+
1036
+ ## Current Market Snapshot (as of {query_date})
1037
+
1038
+ ### Top Crisis Structural Analogues:
1039
+ {top_analogue_text}
1040
+
1041
+ ### Stress by Dimension (0-100 scale):
1042
+ {dimension_text}
1043
+
1044
+ ### Most Stressed Indicators:
1045
+ {indicator_text}
1046
+
1047
+ ## Your Task
1048
+
1049
+ Write a structured risk narrative with EXACTLY these four sections:
1050
+
1051
+ **STRUCTURAL ASSESSMENT** (2-3 sentences)
1052
+ Describe what the current market structure fingerprint reveals. Be specific about which dimensions are most elevated and why that matters structurally.
1053
+
1054
+ **HISTORICAL ANALOGUES** (3-4 sentences)
1055
+ Explain what the top 1-2 analogues share with the current fingerprint. Be specific about which dimensions match and which do NOT match — the differences are as important as the similarities. Avoid saying "this means a crash is coming."
1056
+
1057
+ **KEY DIVERGENCES** (2-3 sentences)
1058
+ What aspects of the current fingerprint explicitly differ from the top analogue? What does that tell us about the nature of the current stress?
1059
+
1060
+ **RISK POSTURE** (2-3 sentences)
1061
+ What should a risk-aware institutional investor monitor closely given this fingerprint? Frame as monitoring priorities, not as trading advice.
1062
+
1063
+ Keep the total response under 350 words. Be precise. Do not use hedging language like "it seems" or "perhaps". Write as a senior quant risk officer would brief a CIO."""
1064
+
1065
+ return prompt
1066
+
1067
+
1068
+ def run_analyst(
1069
+ similarity_results: list,
1070
+ dimension_scores: dict,
1071
+ available_indicators: list,
1072
+ live_vector,
1073
+ query_date: str = "today"
1074
+ ) -> str:
1075
+ """
1076
+ Calls Claude API and returns the structured narrative string.
1077
+ On API failure, returns a fallback message.
1078
+ """
1079
+ prompt = build_prompt(
1080
+ similarity_results, dimension_scores, available_indicators, live_vector, query_date
1081
+ )
1082
+
1083
+ try:
1084
+ message = client.messages.create(
1085
+ model="claude-sonnet-4-20250514",
1086
+ max_tokens=600,
1087
+ messages=[{"role": "user", "content": prompt}]
1088
+ )
1089
+ return message.content[0].text
1090
+ except Exception as e:
1091
+ return f"[Analyst unavailable: {str(e)}]\n\nTop analogue: {similarity_results[0]['name']} ({similarity_results[0]['similarity']:.1f}% similarity)"
1092
+ ```
1093
+
1094
+ ---
1095
+
1096
+ ## 9. Dashboard Charts — `dashboard/charts.py`
1097
+
1098
+ ### 9.1 What This File Does
1099
+ All Plotly chart functions. Each function takes data and returns a Plotly figure object. No Streamlit calls inside this file.
1100
+
1101
+ ```python
1102
+ # dashboard/charts.py
1103
+
1104
+ import plotly.graph_objects as go
1105
+ import plotly.express as px
1106
+ import pandas as pd
1107
+ import numpy as np
1108
+ from data.indicators import INDICATOR_GROUPS
1109
+ from data.crisis_library import CRISIS_LIBRARY
1110
+
1111
+
1112
+ BACKGROUND_COLOR = "#0E1117"
1113
+ CARD_COLOR = "#1A1D23"
1114
+ TEXT_COLOR = "#FAFAFA"
1115
+ ACCENT_COLOR = "#E84393"
1116
+ GRID_COLOR = "#2D3139"
1117
+
1118
+
1119
+ def make_radar_chart(dimension_scores: dict, title: str = "Current Market Stress") -> go.Figure:
1120
+ """
1121
+ Radar chart showing stress level across 5 dimensions.
1122
+ dimension_scores: dict of {dimension_name: score_0_to_100}
1123
+ """
1124
+ dims = list(dimension_scores.keys())
1125
+ scores = list(dimension_scores.values())
1126
+
1127
+ # Close the polygon
1128
+ dims_plot = dims + [dims[0]]
1129
+ scores_plot = scores + [scores[0]]
1130
+
1131
+ fig = go.Figure()
1132
+
1133
+ fig.add_trace(go.Scatterpolar(
1134
+ r=scores_plot,
1135
+ theta=dims_plot,
1136
+ fill='toself',
1137
+ fillcolor='rgba(232, 67, 147, 0.2)',
1138
+ line=dict(color=ACCENT_COLOR, width=2),
1139
+ name="Current"
1140
+ ))
1141
+
1142
+ fig.update_layout(
1143
+ polar=dict(
1144
+ bgcolor=CARD_COLOR,
1145
+ radialaxis=dict(
1146
+ visible=True,
1147
+ range=[0, 100],
1148
+ tickfont=dict(color=TEXT_COLOR, size=10),
1149
+ gridcolor=GRID_COLOR,
1150
+ ),
1151
+ angularaxis=dict(
1152
+ tickfont=dict(color=TEXT_COLOR, size=12),
1153
+ gridcolor=GRID_COLOR,
1154
+ ),
1155
+ ),
1156
+ showlegend=False,
1157
+ paper_bgcolor=BACKGROUND_COLOR,
1158
+ plot_bgcolor=BACKGROUND_COLOR,
1159
+ title=dict(text=title, font=dict(color=TEXT_COLOR, size=14)),
1160
+ margin=dict(l=60, r=60, t=60, b=60),
1161
+ height=380,
1162
+ )
1163
+ return fig
1164
+
1165
+
1166
+ def make_analogue_bar_chart(similarity_results: list) -> go.Figure:
1167
+ """
1168
+ Horizontal bar chart of crisis analogue similarity scores.
1169
+ """
1170
+ top_n = similarity_results[:10]
1171
+ names = [r["short"] for r in reversed(top_n)]
1172
+ scores = [r["similarity"] for r in reversed(top_n)]
1173
+ colors = [r["color"] for r in reversed(top_n)]
1174
+
1175
+ fig = go.Figure(go.Bar(
1176
+ x=scores,
1177
+ y=names,
1178
+ orientation='h',
1179
+ marker=dict(color=colors, opacity=0.85),
1180
+ text=[f"{s:.1f}%" for s in scores],
1181
+ textposition='outside',
1182
+ textfont=dict(color=TEXT_COLOR),
1183
+ ))
1184
+
1185
+ fig.update_layout(
1186
+ paper_bgcolor=BACKGROUND_COLOR,
1187
+ plot_bgcolor=BACKGROUND_COLOR,
1188
+ xaxis=dict(
1189
+ range=[0, 110],
1190
+ gridcolor=GRID_COLOR,
1191
+ tickfont=dict(color=TEXT_COLOR),
1192
+ title=dict(text="Structural Similarity (%)", font=dict(color=TEXT_COLOR)),
1193
+ ),
1194
+ yaxis=dict(tickfont=dict(color=TEXT_COLOR)),
1195
+ margin=dict(l=20, r=80, t=20, b=40),
1196
+ height=360,
1197
+ )
1198
+ return fig
1199
+
1200
+
1201
+ def make_embedding_scatter(crisis_coords: dict, live_coords: tuple = None) -> go.Figure:
1202
+ """
1203
+ 2D UMAP scatter plot of crisis fingerprints + current market.
1204
+ """
1205
+ fig = go.Figure()
1206
+
1207
+ # Plot each historical crisis
1208
+ for crisis_key, (x, y) in crisis_coords.items():
1209
+ info = CRISIS_LIBRARY.get(crisis_key, {})
1210
+ fig.add_trace(go.Scatter(
1211
+ x=[x], y=[y],
1212
+ mode='markers+text',
1213
+ marker=dict(
1214
+ size=16,
1215
+ color=info.get("color", "#888888"),
1216
+ opacity=0.85,
1217
+ line=dict(width=1, color='white')
1218
+ ),
1219
+ text=[info.get("short", crisis_key)],
1220
+ textposition="top center",
1221
+ textfont=dict(color=TEXT_COLOR, size=10),
1222
+ name=info.get("short", crisis_key),
1223
+ showlegend=False,
1224
+ hovertemplate=f"<b>{info.get('name', crisis_key)}</b><br>{info.get('key_signature', '')}<extra></extra>"
1225
+ ))
1226
+
1227
+ # Plot live market
1228
+ if live_coords is not None:
1229
+ fig.add_trace(go.Scatter(
1230
+ x=[live_coords[0]], y=[live_coords[1]],
1231
+ mode='markers+text',
1232
+ marker=dict(
1233
+ size=22,
1234
+ color=ACCENT_COLOR,
1235
+ symbol='star',
1236
+ line=dict(width=2, color='white')
1237
+ ),
1238
+ text=["NOW"],
1239
+ textposition="top center",
1240
+ textfont=dict(color=ACCENT_COLOR, size=12, family="Arial Black"),
1241
+ name="Current Market",
1242
+ showlegend=False,
1243
+ hovertemplate="<b>Current Market</b><extra></extra>"
1244
+ ))
1245
+
1246
+ fig.update_layout(
1247
+ paper_bgcolor=BACKGROUND_COLOR,
1248
+ plot_bgcolor=CARD_COLOR,
1249
+ xaxis=dict(showticklabels=False, gridcolor=GRID_COLOR, zeroline=False),
1250
+ yaxis=dict(showticklabels=False, gridcolor=GRID_COLOR, zeroline=False),
1251
+ margin=dict(l=20, r=20, t=20, b=20),
1252
+ height=400,
1253
+ title=dict(text="Crisis Fingerprint Embedding Space", font=dict(color=TEXT_COLOR, size=13)),
1254
+ )
1255
+ return fig
1256
+
1257
+
1258
+ def make_indicator_heatmap(indicator_df: pd.DataFrame, available_indicators: list, days: int = 60) -> go.Figure:
1259
+ """
1260
+ Heatmap of indicator z-scores over the last N days.
1261
+ Shows which indicators are most stressed and for how long.
1262
+ """
1263
+ recent = indicator_df[available_indicators].iloc[-days:].copy()
1264
+
1265
+ # Z-score each column over the display window for visualization
1266
+ z_scored = (recent - recent.mean()) / (recent.std() + 1e-8)
1267
+ z_scored = z_scored.clip(-3, 3)
1268
+
1269
+ # Shorten indicator names for display
1270
+ short_names = [ind.replace('_', ' ')[:20] for ind in available_indicators]
1271
+
1272
+ fig = go.Figure(go.Heatmap(
1273
+ z=z_scored.T.values,
1274
+ x=z_scored.index.strftime("%Y-%m-%d"),
1275
+ y=short_names,
1276
+ colorscale=[
1277
+ [0.0, "#1a4f72"], # low (negative z)
1278
+ [0.5, CARD_COLOR], # neutral
1279
+ [1.0, "#8B0000"], # high (positive z = stress)
1280
+ ],
1281
+ zmid=0,
1282
+ colorbar=dict(
1283
+ title="Z-Score",
1284
+ tickfont=dict(color=TEXT_COLOR),
1285
+ titlefont=dict(color=TEXT_COLOR),
1286
+ ),
1287
+ hoverongaps=False,
1288
+ ))
1289
+
1290
+ fig.update_layout(
1291
+ paper_bgcolor=BACKGROUND_COLOR,
1292
+ plot_bgcolor=BACKGROUND_COLOR,
1293
+ xaxis=dict(
1294
+ tickangle=45,
1295
+ tickfont=dict(color=TEXT_COLOR, size=9),
1296
+ title=dict(text="Date", font=dict(color=TEXT_COLOR)),
1297
+ ),
1298
+ yaxis=dict(tickfont=dict(color=TEXT_COLOR, size=9)),
1299
+ margin=dict(l=160, r=40, t=20, b=80),
1300
+ height=500,
1301
+ title=dict(text=f"Indicator Stress Heatmap (Last {days} Days)", font=dict(color=TEXT_COLOR, size=13)),
1302
+ )
1303
+ return fig
1304
+
1305
+
1306
+ def make_dimension_timeseries(indicator_df: pd.DataFrame, available_indicators: list, days: int = 252) -> go.Figure:
1307
+ """
1308
+ Line chart showing per-dimension composite stress over time.
1309
+ """
1310
+ recent = indicator_df[available_indicators].iloc[-days:].copy()
1311
+
1312
+ dimension_colors = {
1313
+ "Liquidity": "#E74C3C",
1314
+ "Volatility": "#E67E22",
1315
+ "Correlation": "#F1C40F",
1316
+ "Credit": "#9B59B6",
1317
+ "Positioning": "#3498DB",
1318
+ }
1319
+
1320
+ fig = go.Figure()
1321
+
1322
+ for dim_name, dim_indicators in INDICATOR_GROUPS.items():
1323
+ dim_cols = [ind for ind in dim_indicators if ind in recent.columns]
1324
+ if not dim_cols:
1325
+ continue
1326
+ dim_data = recent[dim_cols]
1327
+ # Normalize each column then take mean absolute value
1328
+ normalized = (dim_data - dim_data.mean()) / (dim_data.std() + 1e-8)
1329
+ composite = normalized.abs().mean(axis=1)
1330
+ # Smooth with 10-day rolling mean
1331
+ composite_smooth = composite.rolling(10, min_periods=1).mean()
1332
+ # Scale to 0-100
1333
+ composite_scaled = (composite_smooth / composite_smooth.quantile(0.99).clip(0.01)) * 100
1334
+ composite_scaled = composite_scaled.clip(0, 100)
1335
+
1336
+ fig.add_trace(go.Scatter(
1337
+ x=recent.index,
1338
+ y=composite_scaled,
1339
+ name=dim_name,
1340
+ line=dict(color=dimension_colors.get(dim_name, "#888888"), width=2),
1341
+ mode='lines',
1342
+ ))
1343
+
1344
+ fig.update_layout(
1345
+ paper_bgcolor=BACKGROUND_COLOR,
1346
+ plot_bgcolor=CARD_COLOR,
1347
+ xaxis=dict(
1348
+ gridcolor=GRID_COLOR,
1349
+ tickfont=dict(color=TEXT_COLOR),
1350
+ title=dict(text="Date", font=dict(color=TEXT_COLOR)),
1351
+ ),
1352
+ yaxis=dict(
1353
+ gridcolor=GRID_COLOR,
1354
+ tickfont=dict(color=TEXT_COLOR),
1355
+ title=dict(text="Stress (0-100)", font=dict(color=TEXT_COLOR)),
1356
+ range=[0, 105],
1357
+ ),
1358
+ legend=dict(
1359
+ font=dict(color=TEXT_COLOR),
1360
+ bgcolor=CARD_COLOR,
1361
+ ),
1362
+ margin=dict(l=60, r=20, t=20, b=40),
1363
+ height=350,
1364
+ title=dict(text="Stress Dimension History", font=dict(color=TEXT_COLOR, size=13)),
1365
+ )
1366
+ return fig
1367
+ ```
1368
+
1369
+ ---
1370
+
1371
+ ## 10. Dashboard State — `dashboard/state.py`
1372
+
1373
+ ```python
1374
+ # dashboard/state.py
1375
+
1376
+ import streamlit as st
1377
+ import pandas as pd
1378
+ from datetime import datetime
1379
+
1380
+
1381
+ def initialize_session_state():
1382
+ """Initialize all session state variables if not already set."""
1383
+ if "data_loaded" not in st.session_state:
1384
+ st.session_state.data_loaded = False
1385
+ if "raw_df" not in st.session_state:
1386
+ st.session_state.raw_df = None
1387
+ if "indicator_df" not in st.session_state:
1388
+ st.session_state.indicator_df = None
1389
+ if "crisis_fingerprints" not in st.session_state:
1390
+ st.session_state.crisis_fingerprints = None
1391
+ if "scaler" not in st.session_state:
1392
+ st.session_state.scaler = None
1393
+ if "available_indicators" not in st.session_state:
1394
+ st.session_state.available_indicators = None
1395
+ if "live_vector" not in st.session_state:
1396
+ st.session_state.live_vector = None
1397
+ if "similarity_results" not in st.session_state:
1398
+ st.session_state.similarity_results = None
1399
+ if "dimension_scores" not in st.session_state:
1400
+ st.session_state.dimension_scores = None
1401
+ if "analyst_narrative" not in st.session_state:
1402
+ st.session_state.analyst_narrative = None
1403
+ if "rewind_date" not in st.session_state:
1404
+ st.session_state.rewind_date = None
1405
+ if "mode" not in st.session_state:
1406
+ st.session_state.mode = "live" # "live" or "rewind"
1407
+ if "embedding_coords" not in st.session_state:
1408
+ st.session_state.embedding_coords = None
1409
+ if "live_embedding_coords" not in st.session_state:
1410
+ st.session_state.live_embedding_coords = None
1411
+ ```
1412
+
1413
+ ---
1414
+
1415
+ ## 11. Main Dashboard App — `dashboard/app.py`
1416
+
1417
+ ### 11.1 Structure
1418
+ The app has three views:
1419
+ 1. **Live Mode** — shows current market fingerprint, analogues, AI narrative
1420
+ 2. **Rewind Mode** — pick any historical date, see what AUTOPSY would have shown
1421
+ 3. **Heatmap Mode** — full indicator heatmap over last 60/90/252 days
1422
+
1423
+ ```python
1424
+ # dashboard/app.py
1425
+
1426
+ import streamlit as st
1427
+ import pandas as pd
1428
+ import numpy as np
1429
+ from datetime import datetime, timedelta
1430
+ import sys
1431
+ import os
1432
+
1433
+ # Ensure imports work from root
1434
+ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
1435
+
1436
+ from data.pipeline import fetch_all_data
1437
+ from data.indicators import compute_all_indicators
1438
+ from fingerprint.engine import (
1439
+ extract_crisis_fingerprints, extract_live_fingerprint,
1440
+ extract_historical_fingerprint, compute_similarity_scores, compute_dimension_scores
1441
+ )
1442
+ from fingerprint.embedding import build_umap_embedding
1443
+ from agent.analyst import run_analyst
1444
+ from dashboard.charts import (
1445
+ make_radar_chart, make_analogue_bar_chart, make_embedding_scatter,
1446
+ make_indicator_heatmap, make_dimension_timeseries
1447
+ )
1448
+ from dashboard.state import initialize_session_state
1449
+
1450
+ # ── Page Config ───────────────────────────────────────────────────────────────
1451
+ st.set_page_config(
1452
+ page_name="AUTOPSY",
1453
+ page_icon="🔬",
1454
+ layout="wide",
1455
+ initial_sidebar_state="expanded"
1456
+ )
1457
+
1458
+ # ── Custom CSS ────────────────────────────────────────────────────────────────
1459
+ st.markdown("""
1460
+ <style>
1461
+ .stApp { background-color: #0E1117; }
1462
+ .main-title {
1463
+ font-size: 2.8rem;
1464
+ font-weight: 900;
1465
+ color: #E84393;
1466
+ letter-spacing: -1px;
1467
+ margin-bottom: 0;
1468
+ }
1469
+ .subtitle {
1470
+ font-size: 1.0rem;
1471
+ color: #888;
1472
+ margin-top: -8px;
1473
+ margin-bottom: 24px;
1474
+ }
1475
+ .metric-card {
1476
+ background: #1A1D23;
1477
+ border-radius: 10px;
1478
+ padding: 16px 20px;
1479
+ border: 1px solid #2D3139;
1480
+ }
1481
+ .analogue-card {
1482
+ background: #1A1D23;
1483
+ border-radius: 8px;
1484
+ padding: 14px 16px;
1485
+ border-left: 4px solid #E84393;
1486
+ margin-bottom: 10px;
1487
+ }
1488
+ .narrative-box {
1489
+ background: #1A1D23;
1490
+ border-radius: 10px;
1491
+ padding: 20px 24px;
1492
+ border: 1px solid #2D3139;
1493
+ font-size: 0.92rem;
1494
+ line-height: 1.7;
1495
+ color: #D0D0D0;
1496
+ }
1497
+ .section-header {
1498
+ font-size: 1.1rem;
1499
+ font-weight: 700;
1500
+ color: #FAFAFA;
1501
+ margin: 20px 0 8px 0;
1502
+ border-bottom: 1px solid #2D3139;
1503
+ padding-bottom: 4px;
1504
+ }
1505
+ .stress-badge-high { color: #E74C3C; font-weight: bold; }
1506
+ .stress-badge-med { color: #E67E22; font-weight: bold; }
1507
+ .stress-badge-low { color: #27AE60; font-weight: bold; }
1508
+ </style>
1509
+ """, unsafe_allow_html=True)
1510
+
1511
+ initialize_session_state()
1512
+
1513
+
1514
+ # ── Sidebar ───────────────────────────────────────────────────────────────────
1515
+ with st.sidebar:
1516
+ st.markdown("## 🔬 AUTOPSY")
1517
+ st.markdown("*Market Dislocation Fingerprint Intelligence*")
1518
+ st.divider()
1519
+
1520
+ mode = st.radio(
1521
+ "Mode",
1522
+ options=["🔴 Live Market", "⏪ Rewind", "🗺 Heatmap"],
1523
+ index=0
1524
+ )
1525
+
1526
+ st.divider()
1527
+
1528
+ if mode == "⏪ Rewind":
1529
+ rewind_date = st.date_input(
1530
+ "Select Historical Date",
1531
+ value=pd.to_datetime("2020-03-01"),
1532
+ min_value=pd.to_datetime("2005-01-01"),
1533
+ max_value=pd.to_datetime("today")
1534
+ )
1535
+ st.caption("See what AUTOPSY would have shown on this date.")
1536
+
1537
+ if mode == "🗺 Heatmap":
1538
+ heatmap_days = st.selectbox("Lookback Window", [30, 60, 90, 252], index=1)
1539
+
1540
+ st.divider()
1541
+
1542
+ load_button = st.button("🔄 Load / Refresh Data", use_container_width=True)
1543
+
1544
+ if st.session_state.data_loaded:
1545
+ st.success("✅ Data loaded")
1546
+ if st.session_state.raw_df is not None:
1547
+ st.caption(f"Latest: {st.session_state.indicator_df.index[-1].strftime('%Y-%m-%d')}")
1548
+ else:
1549
+ st.warning("⚠️ Click to load data")
1550
+
1551
+ st.divider()
1552
+ st.caption("Data: FRED + Yahoo Finance\nAI: Claude (Anthropic)")
1553
+
1554
+
1555
+ # ── Data Loading ──────────────────────────────────────────────────────────────
1556
+ if load_button or not st.session_state.data_loaded:
1557
+ with st.spinner("Fetching market data (this takes ~30 seconds)..."):
1558
+ try:
1559
+ # Fetch data from 2000 to today to cover all crisis windows
1560
+ raw_df = fetch_all_data(start_date="2000-01-01")
1561
+ st.session_state.raw_df = raw_df
1562
+
1563
+ with st.spinner("Computing indicators..."):
1564
+ indicator_df = compute_all_indicators(raw_df)
1565
+ st.session_state.indicator_df = indicator_df
1566
+
1567
+ with st.spinner("Building crisis fingerprint library..."):
1568
+ crisis_fingerprints, scaler, available_indicators = extract_crisis_fingerprints(indicator_df)
1569
+ st.session_state.crisis_fingerprints = crisis_fingerprints
1570
+ st.session_state.scaler = scaler
1571
+ st.session_state.available_indicators = available_indicators
1572
+
1573
+ with st.spinner("Computing live fingerprint..."):
1574
+ live_vector = extract_live_fingerprint(indicator_df, scaler, available_indicators)
1575
+ st.session_state.live_vector = live_vector
1576
+ similarity_results = compute_similarity_scores(live_vector, crisis_fingerprints)
1577
+ st.session_state.similarity_results = similarity_results
1578
+ dimension_scores = compute_dimension_scores(live_vector, available_indicators)
1579
+ st.session_state.dimension_scores = dimension_scores
1580
+
1581
+ with st.spinner("Building embedding space..."):
1582
+ crisis_coords, live_coords = build_umap_embedding(crisis_fingerprints, live_vector)
1583
+ st.session_state.embedding_coords = crisis_coords
1584
+ st.session_state.live_embedding_coords = live_coords
1585
+
1586
+ with st.spinner("Running AI analyst..."):
1587
+ narrative = run_analyst(
1588
+ similarity_results, dimension_scores, available_indicators,
1589
+ live_vector, query_date=datetime.today().strftime("%B %d, %Y")
1590
+ )
1591
+ st.session_state.analyst_narrative = narrative
1592
+
1593
+ st.session_state.data_loaded = True
1594
+ st.rerun()
1595
+
1596
+ except Exception as e:
1597
+ st.error(f"Data loading failed: {str(e)}")
1598
+ st.exception(e)
1599
+
1600
+
1601
+ # ── Main Content ──────────────────────────────────────────────────────────────
1602
+ if not st.session_state.data_loaded:
1603
+ st.markdown('<div class="main-title">AUTOPSY</div>', unsafe_allow_html=True)
1604
+ st.markdown('<div class="subtitle">Market Dislocation Fingerprint Intelligence System</div>', unsafe_allow_html=True)
1605
+ st.info("👈 Click **Load / Refresh Data** in the sidebar to begin.")
1606
+
1607
+ st.markdown("### What is AUTOPSY?")
1608
+ st.markdown("""
1609
+ AUTOPSY analyzes the **structural fingerprint** of current market conditions across five dimensions:
1610
+ **Liquidity · Volatility · Correlation · Credit · Positioning**
1611
+
1612
+ It compares this fingerprint against pre-crisis signatures from 10 historical dislocations,
1613
+ identifies the closest structural analogues, and generates an AI-powered risk narrative.
1614
+
1615
+ This is not a price prediction tool. It is a **market structure stress detector**.
1616
+ """)
1617
+ st.stop()
1618
+
1619
+
1620
+ # ── Determine what vector/results to display ──────────────────────────────────
1621
+ if mode == "⏪ Rewind" and 'rewind_date' in dir():
1622
+ rewind_str = rewind_date.strftime("%Y-%m-%d")
1623
+ display_vector = extract_historical_fingerprint(
1624
+ st.session_state.indicator_df,
1625
+ st.session_state.scaler,
1626
+ st.session_state.available_indicators,
1627
+ rewind_str
1628
+ )
1629
+ if display_vector is None:
1630
+ st.error(f"No data available for {rewind_str}")
1631
+ st.stop()
1632
+ display_similarity = compute_similarity_scores(display_vector, st.session_state.crisis_fingerprints)
1633
+ display_dimension = compute_dimension_scores(display_vector, st.session_state.available_indicators)
1634
+ display_date_label = rewind_date.strftime("%B %d, %Y")
1635
+ rewind_narrative = run_analyst(
1636
+ display_similarity, display_dimension, st.session_state.available_indicators,
1637
+ display_vector, query_date=display_date_label
1638
+ )
1639
+ display_narrative = rewind_narrative
1640
+ rewind_coords, rewind_live_coords = build_umap_embedding(
1641
+ st.session_state.crisis_fingerprints, display_vector
1642
+ )
1643
+ display_embedding_coords = rewind_coords
1644
+ display_live_coords = rewind_live_coords
1645
+ is_rewind = True
1646
+ else:
1647
+ display_vector = st.session_state.live_vector
1648
+ display_similarity = st.session_state.similarity_results
1649
+ display_dimension = st.session_state.dimension_scores
1650
+ display_date_label = datetime.today().strftime("%B %d, %Y")
1651
+ display_narrative = st.session_state.analyst_narrative
1652
+ display_embedding_coords = st.session_state.embedding_coords
1653
+ display_live_coords = st.session_state.live_embedding_coords
1654
+ is_rewind = False
1655
+
1656
+
1657
+ # ── Header ────────────────────────────────────────────────────────────────────
1658
+ col_title, col_date = st.columns([3, 1])
1659
+ with col_title:
1660
+ label = f"⏪ REWIND: {display_date_label}" if is_rewind else "🔴 LIVE"
1661
+ st.markdown(f'<div class="main-title">AUTOPSY</div>', unsafe_allow_html=True)
1662
+ st.markdown(f'<div class="subtitle">Market Dislocation Fingerprint Intelligence · {label}</div>',
1663
+ unsafe_allow_html=True)
1664
+ with col_date:
1665
+ top_analogue = display_similarity[0]
1666
+ st.metric(
1667
+ label="Top Analogue",
1668
+ value=top_analogue["short"],
1669
+ delta=f"{top_analogue['similarity']:.1f}% match"
1670
+ )
1671
+
1672
+
1673
+ # ── Heatmap Mode ──────────────────────────────────────────────────────────────
1674
+ if mode == "🗺 Heatmap":
1675
+ st.markdown('<div class="section-header">Indicator Stress Heatmap</div>', unsafe_allow_html=True)
1676
+ fig_heatmap = make_indicator_heatmap(
1677
+ st.session_state.indicator_df,
1678
+ st.session_state.available_indicators,
1679
+ days=heatmap_days
1680
+ )
1681
+ st.plotly_chart(fig_heatmap, use_container_width=True)
1682
+
1683
+ st.markdown('<div class="section-header">Dimension Stress History</div>', unsafe_allow_html=True)
1684
+ fig_ts = make_dimension_timeseries(
1685
+ st.session_state.indicator_df,
1686
+ st.session_state.available_indicators,
1687
+ days=heatmap_days
1688
+ )
1689
+ st.plotly_chart(fig_ts, use_container_width=True)
1690
+ st.stop()
1691
+
1692
+
1693
+ # ── Main Dashboard Layout (Live + Rewind) ─────────────────────────────────────
1694
+ col_left, col_right = st.columns([1.1, 0.9])
1695
+
1696
+ with col_left:
1697
+
1698
+ # Radar chart
1699
+ st.markdown('<div class="section-header">Structural Stress Profile</div>', unsafe_allow_html=True)
1700
+ fig_radar = make_radar_chart(display_dimension)
1701
+ st.plotly_chart(fig_radar, use_container_width=True)
1702
+
1703
+ # Analogue bar chart
1704
+ st.markdown('<div class="section-header">Crisis Analogue Similarity</div>', unsafe_allow_html=True)
1705
+ fig_bar = make_analogue_bar_chart(display_similarity)
1706
+ st.plotly_chart(fig_bar, use_container_width=True)
1707
+
1708
+ with col_right:
1709
+
1710
+ # Top 3 analogue cards
1711
+ st.markdown('<div class="section-header">Top Structural Analogues</div>', unsafe_allow_html=True)
1712
+ for r in display_similarity[:3]:
1713
+ border_color = r["color"]
1714
+ st.markdown(f"""
1715
+ <div style="background:#1A1D23; border-radius:8px; padding:14px 16px;
1716
+ border-left:4px solid {border_color}; margin-bottom:10px;">
1717
+ <div style="font-weight:700; color:#FAFAFA; font-size:1.0rem;">
1718
+ {r['name']}
1719
+ <span style="float:right; color:{border_color}; font-size:1.1rem;">{r['similarity']:.1f}%</span>
1720
+ </div>
1721
+ <div style="color:#888; font-size:0.82rem; margin-top:4px;">{r['key_signature']}</div>
1722
+ <div style="color:#666; font-size:0.78rem; margin-top:2px;">Peak: {r['peak_date']}</div>
1723
+ </div>
1724
+ """, unsafe_allow_html=True)
1725
+
1726
+ # Embedding scatter
1727
+ st.markdown('<div class="section-header">Fingerprint Space</div>', unsafe_allow_html=True)
1728
+ fig_embed = make_embedding_scatter(display_embedding_coords, display_live_coords)
1729
+ st.plotly_chart(fig_embed, use_container_width=True)
1730
+
1731
+
1732
+ # ── AI Analyst Narrative ──────────────────────────────────────────────────────
1733
+ st.markdown('<div class="section-header">AI Risk Analyst</div>', unsafe_allow_html=True)
1734
+ if display_narrative:
1735
+ st.markdown(f'<div class="narrative-box">{display_narrative.replace(chr(10), "<br>")}</div>',
1736
+ unsafe_allow_html=True)
1737
+ else:
1738
+ st.info("AI narrative will appear after data is loaded.")
1739
+
1740
+
1741
+ # ── Dimension Timeseries ──────────────────────────────────────────────────────
1742
+ st.markdown('<div class="section-header">Stress Dimension History (1 Year)</div>', unsafe_allow_html=True)
1743
+ fig_ts = make_dimension_timeseries(
1744
+ st.session_state.indicator_df,
1745
+ st.session_state.available_indicators,
1746
+ days=252
1747
+ )
1748
+ st.plotly_chart(fig_ts, use_container_width=True)
1749
+
1750
+
1751
+ # ── Footer ────────────────────────────────────────────────────────────────────
1752
+ st.divider()
1753
+ st.caption(
1754
+ "AUTOPSY | Market Dislocation Fingerprint Intelligence | "
1755
+ "TechEx Intelligent Enterprise Solutions Hackathon 2026 | "
1756
+ "Data: FRED, Yahoo Finance | AI: Claude (Anthropic) | "
1757
+ "Not investment advice."
1758
+ )
1759
+ ```
1760
+
1761
+ ---
1762
+
1763
+ ## 12. Entry Point — Run From Root
1764
+
1765
+ Run the app from the `autopsy/` root directory:
1766
+
1767
+ ```bash
1768
+ streamlit run dashboard/app.py
1769
+ ```
1770
+
1771
+ ---
1772
+
1773
+ ## 13. Build Order (Follow This Exactly)
1774
+
1775
+ ### Step 1 — Setup
1776
+ ```bash
1777
+ mkdir autopsy && cd autopsy
1778
+ python -m venv venv
1779
+ source venv/bin/activate # Windows: venv\Scripts\activate
1780
+ pip install -r requirements.txt
1781
+ cp .env.example .env
1782
+ # Fill in FRED_API_KEY and ANTHROPIC_API_KEY in .env
1783
+ ```
1784
+
1785
+ ### Step 2 — Build Files In This Order
1786
+ Build each file completely before moving to the next. Do not skip steps.
1787
+
1788
+ 1. `data/crisis_library.py` — no dependencies, pure data
1789
+ 2. `data/pipeline.py` — data fetching
1790
+ 3. `data/indicators.py` — depends on nothing external
1791
+ 4. `fingerprint/engine.py` — depends on data layer
1792
+ 5. `fingerprint/embedding.py` — standalone
1793
+ 6. `agent/analyst.py` — standalone
1794
+ 7. `dashboard/state.py` — standalone
1795
+ 8. `dashboard/charts.py` — depends on crisis_library and indicator groups
1796
+ 9. `dashboard/app.py` — depends on everything
1797
+
1798
+ ### Step 3 — Test Data Pipeline First
1799
+ Before building the dashboard, verify data fetches:
1800
+ ```bash
1801
+ python -c "
1802
+ from data.pipeline import fetch_all_data
1803
+ from data.indicators import compute_all_indicators
1804
+ df = fetch_all_data(start_date='2019-01-01')
1805
+ print('Raw columns:', df.columns.tolist())
1806
+ ind = compute_all_indicators(df)
1807
+ print('Indicator columns:', ind.columns.tolist())
1808
+ print('Last row:', ind.iloc[-1])
1809
+ "
1810
+ ```
1811
+
1812
+ ### Step 4 — Test Fingerprint Engine
1813
+ ```bash
1814
+ python -c "
1815
+ from data.pipeline import fetch_all_data
1816
+ from data.indicators import compute_all_indicators
1817
+ from fingerprint.engine import extract_crisis_fingerprints, extract_live_fingerprint, compute_similarity_scores
1818
+ df = fetch_all_data(start_date='1998-01-01')
1819
+ ind = compute_all_indicators(df)
1820
+ cfp, scaler, avail = extract_crisis_fingerprints(ind)
1821
+ live = extract_live_fingerprint(ind, scaler, avail)
1822
+ results = compute_similarity_scores(live, cfp)
1823
+ for r in results[:3]:
1824
+ print(r['name'], r['similarity'])
1825
+ "
1826
+ ```
1827
+
1828
+ ### Step 5 — Test Agent
1829
+ ```bash
1830
+ python -c "
1831
+ from agent.analyst import run_analyst
1832
+ # Dummy data
1833
+ dummy_sim = [{'name': 'GFC 2008', 'short': 'GFC 2008', 'similarity': 67.0, 'key_signature': 'test', 'peak_date': '2008-10-10', 'description': 'test'}]
1834
+ dummy_dim = {'Liquidity': 72, 'Volatility': 45, 'Correlation': 60, 'Credit': 80, 'Positioning': 30}
1835
+ import numpy as np
1836
+ dummy_vec = np.zeros(40)
1837
+ result = run_analyst(dummy_sim, dummy_dim, [f'ind_{i}' for i in range(40)], dummy_vec, 'Test')
1838
+ print(result[:200])
1839
+ "
1840
+ ```
1841
+
1842
+ ### Step 6 — Run Dashboard
1843
+ ```bash
1844
+ streamlit run dashboard/app.py
1845
+ ```
1846
+
1847
+ ---
1848
+
1849
+ ## 14. Common Errors & Fixes
1850
+
1851
+ | Error | Cause | Fix |
1852
+ |---|---|---|
1853
+ | `KeyError` in FRED fetch | Series ID changed or unavailable | Check FRED website for current series ID |
1854
+ | `yfinance` returns empty | Ticker delisted or market closed | Use `period='5y'` fallback in yf.download |
1855
+ | `UMAP` import fails | umap-learn not installed | `pip install umap-learn` or it falls back to PCA automatically |
1856
+ | Anthropic `AuthenticationError` | Wrong API key | Check `.env` file, ensure no whitespace around key |
1857
+ | `No crisis fingerprints` error | Data range too short | Ensure `start_date='1998-01-01'` in fetch_all_data |
1858
+ | Dashboard blank on load | Data not loaded | Click "Load / Refresh Data" button |
1859
+ | Slow load (~60s) | Normal — fetching 25 years of FRED + 18 tickers | Add `@st.cache_data` decorator to heavy functions |
1860
+
1861
+ ### Adding `@st.cache_data` to Speed Up Reloads
1862
+ Wrap the heavy fetch functions in app.py:
1863
+
1864
+ ```python
1865
+ @st.cache_data(ttl=3600) # cache for 1 hour
1866
+ def load_all_data():
1867
+ raw_df = fetch_all_data(start_date="1998-01-01")
1868
+ indicator_df = compute_all_indicators(raw_df)
1869
+ return raw_df, indicator_df
1870
+ ```
1871
+
1872
+ ---
1873
+
1874
+ ## 15. Hackathon Pitch Notes
1875
+
1876
+ ### One-Line Pitch
1877
+ "AUTOPSY tells you which historical market crisis today's structure most resembles — before the price moves confirm it."
1878
+
1879
+ ### Track Fit
1880
+ Track 4: Data & Intelligence — multi-source financial data pipeline, AI-powered analytics agent, anomaly/structural detection, enterprise risk use case.
1881
+
1882
+ ### The Demo Flow (5 minutes)
1883
+ 1. **Open live dashboard** — show today's radar chart and top analogue (30 seconds)
1884
+ 2. **Rewind to Feb 2020** — show the fingerprint forming before COVID crash (60 seconds)
1885
+ 3. **Rewind to Aug 2007** — show GFC fingerprint building before Lehman (60 seconds)
1886
+ 4. **Read AI narrative** — show structured risk brief (30 seconds)
1887
+ 5. **Explain the research** — formal taxonomy, embedding space, what makes this original vs existing stress indices (60 seconds)
1888
+ 6. **Product roadmap** — arXiv preprint, institutional data subscription, real-time alerts (30 seconds)
1889
+
1890
+ ### Key Differentiator vs Existing Tools
1891
+ - **Not a stress index** (single number) — multi-dimensional fingerprint with structural decomposition
1892
+ - **Not scenario-based** — empirically derived from actual pre-crisis data
1893
+ - **Not price prediction** — structural pattern matching, avoids the "market timing" criticism
1894
+ - **Historical analogue mapping** — tells you not just "stress is high" but "which crisis this resembles and why"
1895
+
1896
+ ---
1897
+
1898
+ ## 16. arXiv Preprint Structure (Post-Hackathon)
1899
+
1900
+ ```
1901
+ Title: AUTOPSY: A Market Dislocation Fingerprint Framework for
1902
+ Real-Time Systemic Stress Detection
1903
+
1904
+ Abstract: We present AUTOPSY, a framework for detecting pre-dislocation
1905
+ structural fingerprints in financial markets...
1906
+
1907
+ 1. Introduction
1908
+ 2. Related Work (Stress indices, regime detection, systemic risk)
1909
+ 3. The Fingerprint Taxonomy (5 dimensions, 40 indicators)
1910
+ 4. Crisis Library Construction
1911
+ 5. Embedding Methodology (UMAP on normalized vectors)
1912
+ 6. Empirical Validation
1913
+ 6.1 In-sample: Do crisis fingerprints cluster meaningfully?
1914
+ 6.2 Lead time analysis: How early does each dimension activate?
1915
+ 6.3 Out-of-sample: Train on pre-2020, evaluate on COVID and SVB
1916
+ 7. The AUTOPSY System
1917
+ 8. Conclusion
1918
+
1919
+ Target: arXiv q-fin.RM or q-fin.ST
1920
+ ```
1921
+
1922
+ ---
1923
+
1924
+ *AUTOPSY | Arka Sarkar | TechEx Intelligent Enterprise Solutions Hackathon 2026*
fingerprint/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ # fingerprint package
fingerprint/embedding.py ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # fingerprint/embedding.py
2
+
3
+ import numpy as np
4
+ from sklearn.decomposition import PCA
5
+
6
+
7
+ def build_umap_embedding(crisis_fingerprints: dict, live_vector: np.ndarray | None = None):
8
+ """
9
+ Projects crisis fingerprints into 2D using PCA.
10
+ Deterministic — same result every run.
11
+ Name kept as build_umap_embedding for compatibility with existing imports.
12
+ """
13
+ keys = list(crisis_fingerprints.keys())
14
+ vectors = np.array([crisis_fingerprints[k] for k in keys])
15
+
16
+ all_vectors = vectors
17
+ if live_vector is not None:
18
+ all_vectors = np.vstack([vectors, live_vector.reshape(1, -1)])
19
+
20
+ pca = PCA(n_components=2)
21
+ coords_2d = pca.fit_transform(all_vectors)
22
+
23
+ crisis_coords = {keys[i]: tuple(coords_2d[i]) for i in range(len(keys))}
24
+ live_coords = None
25
+ if live_vector is not None:
26
+ live_coords = tuple(coords_2d[-1])
27
+
28
+ return crisis_coords, live_coords
fingerprint/engine.py ADDED
@@ -0,0 +1,134 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # fingerprint/engine.py
2
+
3
+ import pandas as pd
4
+ import numpy as np
5
+ from sklearn.preprocessing import RobustScaler
6
+ from data.indicators import ALL_INDICATORS, INDICATOR_GROUPS, compute_all_indicators
7
+ from data.crisis_library import CRISIS_LIBRARY
8
+ from data.pipeline import fetch_all_data
9
+
10
+
11
+ def extract_crisis_fingerprints(
12
+ indicator_df: pd.DataFrame,
13
+ scaler: RobustScaler | None = None
14
+ ) -> tuple[dict, RobustScaler, list]:
15
+ """
16
+ For each crisis in CRISIS_LIBRARY, extracts the mean indicator vector
17
+ during the stress window.
18
+
19
+ FIX #3: Return type corrected to tuple[dict, RobustScaler, list]
20
+ (blueprint said tuple[dict, RobustScaler] but returned 3 values).
21
+
22
+ Returns:
23
+ crisis_fingerprints: dict mapping crisis_key -> np.array of shape (n_indicators,)
24
+ scaler: fitted RobustScaler
25
+ available_indicators: list of indicator names actually present
26
+ """
27
+ available_indicators = [ind for ind in ALL_INDICATORS if ind in indicator_df.columns]
28
+
29
+ crisis_fingerprints = {}
30
+ for crisis_key, crisis_info in CRISIS_LIBRARY.items():
31
+ start = crisis_info["stress_start"]
32
+ end = crisis_info["stress_end"]
33
+ window = indicator_df.loc[start:end, available_indicators]
34
+ if len(window) == 0:
35
+ continue
36
+ crisis_fingerprints[crisis_key] = window.mean().values
37
+
38
+ if not crisis_fingerprints:
39
+ raise ValueError("No crisis fingerprints could be computed. Check data range.")
40
+
41
+ all_vectors = np.array(list(crisis_fingerprints.values()))
42
+ if scaler is None:
43
+ scaler = RobustScaler()
44
+ scaler.fit(all_vectors)
45
+
46
+ for key in crisis_fingerprints:
47
+ crisis_fingerprints[key] = scaler.transform(
48
+ crisis_fingerprints[key].reshape(1, -1)
49
+ ).flatten()
50
+
51
+ return crisis_fingerprints, scaler, available_indicators
52
+
53
+
54
+ def extract_live_fingerprint(
55
+ indicator_df: pd.DataFrame,
56
+ scaler: RobustScaler,
57
+ available_indicators: list
58
+ ) -> np.ndarray:
59
+ """Extracts the most recent row as the live fingerprint vector."""
60
+ latest = indicator_df[available_indicators].dropna().iloc[-1].values
61
+ return scaler.transform(latest.reshape(1, -1)).flatten()
62
+
63
+
64
+ def extract_historical_fingerprint(
65
+ indicator_df: pd.DataFrame,
66
+ scaler: RobustScaler,
67
+ available_indicators: list,
68
+ date: str
69
+ ) -> np.ndarray | None:
70
+ """
71
+ For the rewind feature: extract fingerprint as of a specific historical date.
72
+ Uses 21-day lookback window ending on the given date.
73
+ """
74
+ end_dt = pd.to_datetime(date)
75
+ start_dt = end_dt - pd.Timedelta(days=30)
76
+ window = indicator_df.loc[start_dt:end_dt, available_indicators]
77
+ if len(window) == 0:
78
+ return None
79
+ vec = window.mean().values
80
+ return scaler.transform(vec.reshape(1, -1)).flatten()
81
+
82
+
83
+ def compute_similarity_scores(
84
+ query_vector: np.ndarray,
85
+ crisis_fingerprints: dict
86
+ ) -> list[dict]:
87
+ """
88
+ Computes cosine similarity between the query vector and all crisis fingerprints.
89
+ Returns sorted list of dicts. Similarity is in [0, 100] range.
90
+ """
91
+ from sklearn.metrics.pairwise import cosine_similarity
92
+
93
+ results = []
94
+ for crisis_key, crisis_vector in crisis_fingerprints.items():
95
+ crisis_info = CRISIS_LIBRARY[crisis_key]
96
+ sim = cosine_similarity(
97
+ query_vector.reshape(1, -1),
98
+ crisis_vector.reshape(1, -1)
99
+ )[0][0]
100
+ similarity_pct = max(0, (sim + 1) / 2 * 100)
101
+ results.append({
102
+ "crisis_key": crisis_key,
103
+ "name": crisis_info["name"],
104
+ "short": crisis_info["short"],
105
+ "similarity": round(similarity_pct, 1),
106
+ "color": crisis_info["color"],
107
+ "key_signature": crisis_info["key_signature"],
108
+ "description": crisis_info["description"],
109
+ "peak_date": crisis_info["peak_date"],
110
+ })
111
+
112
+ results.sort(key=lambda x: x["similarity"], reverse=True)
113
+ return results
114
+
115
+
116
+ def compute_dimension_scores(
117
+ query_vector: np.ndarray,
118
+ available_indicators: list
119
+ ) -> dict:
120
+ """
121
+ Computes per-dimension stress scores from the normalized fingerprint vector.
122
+ Returns dict: dimension_name -> stress_score (0-100). Higher = more stressed.
123
+ """
124
+ scores = {}
125
+ for dim_name, dim_indicators in INDICATOR_GROUPS.items():
126
+ indices = [i for i, ind in enumerate(available_indicators) if ind in dim_indicators]
127
+ if not indices:
128
+ scores[dim_name] = 0.0
129
+ continue
130
+ dim_vec = query_vector[indices]
131
+ score = np.abs(dim_vec).mean()
132
+ score = min(score / 3.0 * 100, 100)
133
+ scores[dim_name] = round(score, 1)
134
+ return scores
pyrightconfig.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "venvPath": "/run/media/arka/New Volume/DEVELOPMENT/AUTOPSY",
3
+ "venv": ".venv",
4
+ "pythonVersion": "3.14",
5
+ "extraPaths": [
6
+ "./.venv/lib64/python3.14/site-packages"
7
+ ]
8
+ }
requirements.txt ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ streamlit>=1.57.0
2
+ plotly>=6.7.0
3
+ pandas>=3.0.3
4
+ numpy>=2.4.4
5
+ scipy>=1.17.1
6
+ scikit-learn>=1.5.0
7
+ umap-learn>=0.5.6
8
+ yfinance>=0.2.40
9
+ fredapi>=0.5.2
10
+ python-dotenv>=1.0.1
11
+ requests>=2.33.1
tests/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ # tests package
tests/test_pipeline.py ADDED
@@ -0,0 +1,162 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # tests/test_pipeline.py
2
+
3
+ """
4
+ Smoke tests for AUTOPSY.
5
+ These tests verify imports, data structures, and basic computation
6
+ without requiring API keys or network access.
7
+ """
8
+
9
+ import sys
10
+ import os
11
+ import numpy as np
12
+ import pandas as pd
13
+ import pytest
14
+
15
+ # Ensure imports work from project root
16
+ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
17
+
18
+
19
+ def test_crisis_library_structure():
20
+ """Verify crisis library has expected structure."""
21
+ from data.crisis_library import CRISIS_LIBRARY
22
+ assert len(CRISIS_LIBRARY) == 10
23
+ required_keys = {"name", "short", "description", "stress_start", "stress_end",
24
+ "peak_date", "color", "key_signature"}
25
+ for crisis_key, crisis_info in CRISIS_LIBRARY.items():
26
+ assert required_keys.issubset(crisis_info.keys()), f"{crisis_key} missing keys"
27
+ # Dates should be parseable
28
+ pd.to_datetime(crisis_info["stress_start"])
29
+ pd.to_datetime(crisis_info["stress_end"])
30
+ pd.to_datetime(crisis_info["peak_date"])
31
+
32
+
33
+ def test_indicator_groups_complete():
34
+ """Verify all 40 indicators are defined."""
35
+ from data.indicators import ALL_INDICATORS, INDICATOR_GROUPS
36
+ assert len(ALL_INDICATORS) == 40, f"Expected 40 indicators, got {len(ALL_INDICATORS)}"
37
+ assert len(INDICATOR_GROUPS) == 5, f"Expected 5 dimensions, got {len(INDICATOR_GROUPS)}"
38
+ for dim_name, indicators in INDICATOR_GROUPS.items():
39
+ assert len(indicators) == 8, f"{dim_name} should have 8 indicators, got {len(indicators)}"
40
+
41
+
42
+ def test_compute_indicators_with_synthetic_data():
43
+ """Test indicator computation with synthetic price data."""
44
+ from data.indicators import compute_all_indicators, ALL_INDICATORS
45
+
46
+ # Create 300 days of synthetic data
47
+ dates = pd.bdate_range("2023-01-01", periods=300)
48
+ np.random.seed(42)
49
+ df = pd.DataFrame({
50
+ "spy": 400 + np.cumsum(np.random.randn(300) * 2),
51
+ "ief": 100 + np.cumsum(np.random.randn(300) * 0.5),
52
+ "gld": 180 + np.cumsum(np.random.randn(300) * 1),
53
+ "hyg": 80 + np.cumsum(np.random.randn(300) * 0.3),
54
+ "lqd": 110 + np.cumsum(np.random.randn(300) * 0.2),
55
+ "qqq": 350 + np.cumsum(np.random.randn(300) * 3),
56
+ "uso": 70 + np.cumsum(np.random.randn(300) * 1.5),
57
+ "uup": 28 + np.cumsum(np.random.randn(300) * 0.1),
58
+ "eem": 40 + np.cumsum(np.random.randn(300) * 1),
59
+ "xlf": 35 + np.cumsum(np.random.randn(300) * 0.8),
60
+ "xle": 80 + np.cumsum(np.random.randn(300) * 1.2),
61
+ "xlu": 65 + np.cumsum(np.random.randn(300) * 0.4),
62
+ "vix": 20 + np.abs(np.random.randn(300) * 5),
63
+ "vix_3m": 22 + np.abs(np.random.randn(300) * 3),
64
+ "vix_9d": 18 + np.abs(np.random.randn(300) * 6),
65
+ "eurusd": 1.1 + np.cumsum(np.random.randn(300) * 0.005),
66
+ "jpyusd": 130 + np.cumsum(np.random.randn(300) * 0.5),
67
+ "chfusd": 0.9 + np.cumsum(np.random.randn(300) * 0.003),
68
+ "ted_spread": 0.3 + np.abs(np.random.randn(300) * 0.1),
69
+ "yield_curve_10y2y": 0.5 + np.random.randn(300) * 0.3,
70
+ "ig_credit_spread": 1.2 + np.abs(np.random.randn(300) * 0.2),
71
+ "hy_credit_spread": 4.0 + np.abs(np.random.randn(300) * 0.5),
72
+ }, index=dates)
73
+
74
+ ind = compute_all_indicators(df)
75
+
76
+ # Check all 40 indicators were produced
77
+ assert len(ind.columns) == 40, f"Expected 40 columns, got {len(ind.columns)}"
78
+ # Check no infinities
79
+ assert not np.isinf(ind.values).any(), "Found infinity values in indicators"
80
+ # Check shape preserved
81
+ assert len(ind) == 300
82
+
83
+
84
+ def test_fingerprint_engine_with_synthetic():
85
+ """Test fingerprint extraction and similarity scoring."""
86
+ from data.indicators import compute_all_indicators, ALL_INDICATORS
87
+ from fingerprint.engine import (
88
+ extract_crisis_fingerprints, extract_live_fingerprint,
89
+ compute_similarity_scores, compute_dimension_scores
90
+ )
91
+
92
+ # Build a long synthetic dataset covering crisis windows
93
+ dates = pd.bdate_range("1998-01-01", "2024-01-01")
94
+ np.random.seed(42)
95
+ n = len(dates)
96
+ df = pd.DataFrame({
97
+ "spy": 100 + np.cumsum(np.random.randn(n) * 0.5),
98
+ "ief": 50 + np.cumsum(np.random.randn(n) * 0.2),
99
+ "gld": 80 + np.cumsum(np.random.randn(n) * 0.3),
100
+ "hyg": 60 + np.cumsum(np.random.randn(n) * 0.15),
101
+ "lqd": 70 + np.cumsum(np.random.randn(n) * 0.1),
102
+ "qqq": 90 + np.cumsum(np.random.randn(n) * 0.8),
103
+ "uso": 40 + np.cumsum(np.random.randn(n) * 0.4),
104
+ "uup": 25 + np.cumsum(np.random.randn(n) * 0.05),
105
+ "eem": 35 + np.cumsum(np.random.randn(n) * 0.3),
106
+ "xlf": 30 + np.cumsum(np.random.randn(n) * 0.2),
107
+ "xle": 50 + np.cumsum(np.random.randn(n) * 0.3),
108
+ "xlu": 40 + np.cumsum(np.random.randn(n) * 0.1),
109
+ "vix": 20 + np.abs(np.random.randn(n) * 5),
110
+ "vix_3m": 22 + np.abs(np.random.randn(n) * 3),
111
+ "vix_9d": 18 + np.abs(np.random.randn(n) * 6),
112
+ "eurusd": 1.1 + np.cumsum(np.random.randn(n) * 0.002),
113
+ "jpyusd": 110 + np.cumsum(np.random.randn(n) * 0.2),
114
+ "chfusd": 0.95 + np.cumsum(np.random.randn(n) * 0.001),
115
+ "ted_spread": 0.3 + np.abs(np.random.randn(n) * 0.1),
116
+ "yield_curve_10y2y": 0.5 + np.random.randn(n) * 0.3,
117
+ "ig_credit_spread": 1.2 + np.abs(np.random.randn(n) * 0.2),
118
+ "hy_credit_spread": 4.0 + np.abs(np.random.randn(n) * 0.5),
119
+ }, index=dates)
120
+
121
+ ind = compute_all_indicators(df)
122
+
123
+ # Extract crisis fingerprints
124
+ crisis_fp, scaler, avail = extract_crisis_fingerprints(ind)
125
+ assert len(crisis_fp) > 0, "No crisis fingerprints extracted"
126
+ assert len(avail) == 40
127
+
128
+ # Extract live fingerprint
129
+ live_vec = extract_live_fingerprint(ind, scaler, avail)
130
+ assert live_vec.shape == (40,)
131
+
132
+ # Compute similarity
133
+ results = compute_similarity_scores(live_vec, crisis_fp)
134
+ assert len(results) == len(crisis_fp)
135
+ assert all(0 <= r["similarity"] <= 100 for r in results)
136
+
137
+ # Compute dimension scores
138
+ dim_scores = compute_dimension_scores(live_vec, avail)
139
+ assert len(dim_scores) == 5
140
+ assert all(0 <= s <= 100 for s in dim_scores.values())
141
+
142
+
143
+ def test_embedding():
144
+ """Test UMAP/PCA embedding."""
145
+ from fingerprint.embedding import build_umap_embedding
146
+
147
+ # Fake crisis fingerprints
148
+ np.random.seed(42)
149
+ crisis_fp = {f"crisis_{i}": np.random.randn(40) for i in range(5)}
150
+ live_vec = np.random.randn(40)
151
+
152
+ crisis_coords, live_coords = build_umap_embedding(crisis_fp, live_vec)
153
+ assert len(crisis_coords) == 5
154
+ assert live_coords is not None
155
+ assert len(live_coords) == 2
156
+ for key, (x, y) in crisis_coords.items():
157
+ assert isinstance(x, (float, np.floating))
158
+ assert isinstance(y, (float, np.floating))
159
+
160
+
161
+ if __name__ == "__main__":
162
+ pytest.main([__file__, "-v"])