Spaces:
Running
Running
Upload 13 files
Browse files- DEPLOYMENT_GUIDE.md +296 -0
- Dockerfile +48 -0
- Quasar_axrvi_ranker.py +0 -0
- README.md +628 -10
- entrypoint.sh +33 -0
- gitattributes +35 -0
- hub_dashboard.html +0 -0
- hub_dashboard_service.py +388 -0
- ranker_logging.py +56 -0
- ranker_logs_api.py +120 -0
- requirements.txt +34 -0
- supervisord.conf +41 -0
- websocket_hub.py +553 -0
DEPLOYMENT_GUIDE.md
ADDED
|
@@ -0,0 +1,296 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# QUASAR System β Deployment Guide
|
| 2 |
+
## v2.0 Architecture-Strict | 2026-03-25
|
| 3 |
+
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
## Overview
|
| 7 |
+
|
| 8 |
+
The refactored system enforces a **strict one-way data pipeline**:
|
| 9 |
+
|
| 10 |
+
```
|
| 11 |
+
Asset Spaces (V75, V100_1s, Crash500)
|
| 12 |
+
β WebSocket PUBLISH (send-only)
|
| 13 |
+
βΌ
|
| 14 |
+
Central WebSocket Hub βββ ingest, normalize, broadcast
|
| 15 |
+
β WebSocket SUBSCRIBE (read-only)
|
| 16 |
+
βΌ
|
| 17 |
+
Ranker Space (Quasar_axrvi_ranker.py)
|
| 18 |
+
β REST / Dashboard outputs
|
| 19 |
+
βΌ
|
| 20 |
+
Rankings β’ REST API β’ Dashboard
|
| 21 |
+
```
|
| 22 |
+
|
| 23 |
+
**No feedback loop exists.** The Ranker never writes back to the Hub or Asset Spaces.
|
| 24 |
+
|
| 25 |
+
---
|
| 26 |
+
|
| 27 |
+
## File Reference
|
| 28 |
+
|
| 29 |
+
| File | Role | Deploy As |
|
| 30 |
+
|------|------|-----------|
|
| 31 |
+
| `websocket_hub.py` | Central Hub β ingest, normalize, broadcast | Standalone FastAPI service |
|
| 32 |
+
| `websocket_client.py` | Publisher β Asset Space send-only client | Imported in each Asset Space |
|
| 33 |
+
| `Quasar_axrvi_ranker.py` | Ranker Space β subscriber + neural ranker + trading | Standalone process |
|
| 34 |
+
|
| 35 |
+
---
|
| 36 |
+
|
| 37 |
+
## 1. Requirements
|
| 38 |
+
|
| 39 |
+
```bash
|
| 40 |
+
pip install fastapi uvicorn websockets websocket-client torch numpy pydantic
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
For Hugging Face Spaces, add to `requirements.txt`:
|
| 44 |
+
```
|
| 45 |
+
fastapi
|
| 46 |
+
uvicorn[standard]
|
| 47 |
+
websockets
|
| 48 |
+
websocket-client
|
| 49 |
+
torch
|
| 50 |
+
numpy
|
| 51 |
+
pydantic
|
| 52 |
+
nest_asyncio
|
| 53 |
+
```
|
| 54 |
+
|
| 55 |
+
---
|
| 56 |
+
|
| 57 |
+
## 2. Deploy the Central Hub
|
| 58 |
+
|
| 59 |
+
```bash
|
| 60 |
+
# Local
|
| 61 |
+
python websocket_hub.py
|
| 62 |
+
|
| 63 |
+
# With explicit port
|
| 64 |
+
PORT=7860 python websocket_hub.py
|
| 65 |
+
|
| 66 |
+
# Production (Hugging Face Space)
|
| 67 |
+
# Set Space SDK to "Docker" or use app.py entry with:
|
| 68 |
+
uvicorn websocket_hub:app --host 0.0.0.0 --port 7860
|
| 69 |
+
```
|
| 70 |
+
|
| 71 |
+
**Hub endpoints:**
|
| 72 |
+
|
| 73 |
+
| Endpoint | Protocol | Role |
|
| 74 |
+
|----------|----------|------|
|
| 75 |
+
| `/ws/publish/{space_name}` | WebSocket | Publisher (Asset Spaces connect here) |
|
| 76 |
+
| `/ws/subscribe` | WebSocket | Subscriber (Ranker connects here) |
|
| 77 |
+
| `/rankings` | GET | Latest snapshots for all assets |
|
| 78 |
+
| `/metrics/{space_name}` | GET | Single-asset snapshot |
|
| 79 |
+
| `/health` | GET | Hub health and connection stats |
|
| 80 |
+
|
| 81 |
+
---
|
| 82 |
+
|
| 83 |
+
## 3. Integrate the Publisher into an Asset Space
|
| 84 |
+
|
| 85 |
+
Each asset space (V75, V100_1s, Crash500) imports `AssetSpacePublisher`:
|
| 86 |
+
|
| 87 |
+
```python
|
| 88 |
+
from websocket_client import AssetSpacePublisher, TrainingMetrics, VotingMetrics
|
| 89 |
+
|
| 90 |
+
# ββ Create and start (once, at startup) ββββββββββββββββββββββββββββββββββββββ
|
| 91 |
+
publisher = AssetSpacePublisher(
|
| 92 |
+
space_name = "V75",
|
| 93 |
+
hub_url = "ws://your-hub-host:7860/ws/publish/V75",
|
| 94 |
+
)
|
| 95 |
+
publisher.start()
|
| 96 |
+
|
| 97 |
+
# ββ In your training loop ββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 98 |
+
# After each training update:
|
| 99 |
+
publisher.publish_training(TrainingMetrics(
|
| 100 |
+
training_steps = step,
|
| 101 |
+
actor_loss = actor_loss,
|
| 102 |
+
critic_loss = critic_loss,
|
| 103 |
+
avn_loss = avn_loss,
|
| 104 |
+
avn_accuracy = avn_accuracy,
|
| 105 |
+
))
|
| 106 |
+
|
| 107 |
+
# After each agent vote:
|
| 108 |
+
publisher.publish_voting(VotingMetrics(
|
| 109 |
+
dominant_signal = "BUY", # "BUY" | "SELL" | "NEUTRAL"
|
| 110 |
+
buy_count = 7,
|
| 111 |
+
sell_count = 3,
|
| 112 |
+
))
|
| 113 |
+
|
| 114 |
+
# Or publish both together (preferred β fewer messages):
|
| 115 |
+
publisher.publish_combined(
|
| 116 |
+
training = TrainingMetrics(...),
|
| 117 |
+
voting = VotingMetrics(...),
|
| 118 |
+
)
|
| 119 |
+
```
|
| 120 |
+
|
| 121 |
+
**Reminder:** The publisher is send-only. Any unexpected message from the hub
|
| 122 |
+
is logged as a warning and discarded. No callbacks are invoked.
|
| 123 |
+
|
| 124 |
+
---
|
| 125 |
+
|
| 126 |
+
## 4. Deploy the Ranker Space
|
| 127 |
+
|
| 128 |
+
```bash
|
| 129 |
+
# Point at your hub's subscribe endpoint
|
| 130 |
+
python Quasar_axrvi_ranker.py \
|
| 131 |
+
--hub ws://your-hub-host:7860/ws/subscribe \
|
| 132 |
+
--assets V75 V100_1s CRASH1000 \
|
| 133 |
+
--bandit ucb \
|
| 134 |
+
--reward simple \
|
| 135 |
+
--model deriv_axrvi_model.pt
|
| 136 |
+
|
| 137 |
+
# Sync/thread mode (e.g., inside a Jupyter notebook or larger process)
|
| 138 |
+
python Quasar_axrvi_ranker.py --sync --hub ws://...
|
| 139 |
+
|
| 140 |
+
# Component tests (no network required)
|
| 141 |
+
python Quasar_axrvi_ranker.py --test
|
| 142 |
+
```
|
| 143 |
+
|
| 144 |
+
---
|
| 145 |
+
|
| 146 |
+
## 5. Ranking Formula
|
| 147 |
+
|
| 148 |
+
```
|
| 149 |
+
signal_confidence = max(buy_count, sell_count) / (buy_count + sell_count)
|
| 150 |
+
score = signal_confidence - avn_accuracy
|
| 151 |
+
```
|
| 152 |
+
|
| 153 |
+
| Scenario | Result |
|
| 154 |
+
|----------|--------|
|
| 155 |
+
| High confidence (0.9) + high accuracy (0.8) | score = +0.10 β good |
|
| 156 |
+
| High confidence (0.9) + low accuracy (0.3) | score = +0.60 β penalized (large gap) |
|
| 157 |
+
| Low confidence (0.5) + any accuracy | score β€ 0.0 β weak |
|
| 158 |
+
|
| 159 |
+
Assets are sorted by score in **ascending** order (smallest = most balanced = best).
|
| 160 |
+
|
| 161 |
+
---
|
| 162 |
+
|
| 163 |
+
## 6. Strict Data Schema
|
| 164 |
+
|
| 165 |
+
The hub enforces and broadcasts **only** these fields:
|
| 166 |
+
|
| 167 |
+
```
|
| 168 |
+
training:
|
| 169 |
+
training_steps (int)
|
| 170 |
+
actor_loss (float)
|
| 171 |
+
critic_loss (float)
|
| 172 |
+
avn_loss (float)
|
| 173 |
+
avn_accuracy (float, clamped [0,1])
|
| 174 |
+
|
| 175 |
+
voting:
|
| 176 |
+
dominant_signal ("BUY" | "SELL" | "NEUTRAL")
|
| 177 |
+
buy_count (int)
|
| 178 |
+
sell_count (int)
|
| 179 |
+
```
|
| 180 |
+
|
| 181 |
+
The following fields are **explicitly stripped** at the hub ingestion layer
|
| 182 |
+
and will never reach the Ranker:
|
| 183 |
+
|
| 184 |
+
- β rewards (matched, unmatched, duplicates, match_rate)
|
| 185 |
+
- β resource metrics (cpu_percent, memory_percent, memory_used_gb, quasar_memory_gb)
|
| 186 |
+
- β agent-level metrics (q_buy, q_sell, entropy, per-agent data)
|
| 187 |
+
- β buffer_size
|
| 188 |
+
- β any q-values or internal model outputs
|
| 189 |
+
|
| 190 |
+
---
|
| 191 |
+
|
| 192 |
+
## 7. Hugging Face Spaces Deployment
|
| 193 |
+
|
| 194 |
+
### Hub Space (`quasar-hub`)
|
| 195 |
+
|
| 196 |
+
`app.py`:
|
| 197 |
+
```python
|
| 198 |
+
from websocket_hub import app # FastAPI app, ready for uvicorn
|
| 199 |
+
```
|
| 200 |
+
|
| 201 |
+
`README.md` front-matter:
|
| 202 |
+
```yaml
|
| 203 |
+
---
|
| 204 |
+
sdk: docker
|
| 205 |
+
app_port: 7860
|
| 206 |
+
---
|
| 207 |
+
```
|
| 208 |
+
|
| 209 |
+
### Asset Space (e.g., `quasar-v75`)
|
| 210 |
+
|
| 211 |
+
In your existing Space's training entry point:
|
| 212 |
+
```python
|
| 213 |
+
from websocket_client import AssetSpacePublisher, TrainingMetrics, VotingMetrics
|
| 214 |
+
import os
|
| 215 |
+
|
| 216 |
+
HUB_URL = os.environ.get("HUB_WS_URL", "wss://your-hub-space.hf.space/ws/publish/V75")
|
| 217 |
+
publisher = AssetSpacePublisher("V75", HUB_URL)
|
| 218 |
+
publisher.start()
|
| 219 |
+
```
|
| 220 |
+
|
| 221 |
+
Set the environment variable `HUB_WS_URL` in each Space's settings.
|
| 222 |
+
|
| 223 |
+
### Ranker Space (`quasar-ranker`)
|
| 224 |
+
|
| 225 |
+
```python
|
| 226 |
+
# main.py (entry point)
|
| 227 |
+
import asyncio, os
|
| 228 |
+
from Quasar_axrvi_ranker import run_live_trading_system
|
| 229 |
+
|
| 230 |
+
HUB_SUB = os.environ.get("HUB_SUB_URL", "wss://your-hub-space.hf.space/ws/subscribe")
|
| 231 |
+
|
| 232 |
+
asyncio.run(run_live_trading_system(
|
| 233 |
+
asset_symbols = ["V75", "V100_1s", "CRASH1000"],
|
| 234 |
+
hub_ws_url = HUB_SUB,
|
| 235 |
+
))
|
| 236 |
+
```
|
| 237 |
+
|
| 238 |
+
---
|
| 239 |
+
|
| 240 |
+
## 8. Architecture Constraints (enforced in code)
|
| 241 |
+
|
| 242 |
+
| Constraint | Where enforced |
|
| 243 |
+
|------------|----------------|
|
| 244 |
+
| Publishers cannot receive data | `_on_message` in `AssetSpacePublisher` discards all inbound messages |
|
| 245 |
+
| Hub never writes to publishers | Publisher WebSocket endpoint is receive-only; no sends |
|
| 246 |
+
| Subscribers are read-only | Subscriber endpoint drains inbound messages without processing |
|
| 247 |
+
| No feedback loop | `HubSubscriber` has no send methods |
|
| 248 |
+
| Minimal schema | Hub `_validate_and_normalize()` strips all non-permitted fields |
|
| 249 |
+
| Thread-safe | All shared state protected by `asyncio.Lock` (hub) / `threading.Lock` (client, ranker) |
|
| 250 |
+
|
| 251 |
+
---
|
| 252 |
+
|
| 253 |
+
## 9. Environment Variables
|
| 254 |
+
|
| 255 |
+
| Variable | Used By | Description |
|
| 256 |
+
|----------|---------|-------------|
|
| 257 |
+
| `DERIV_API_KEY` | Ranker | Deriv API key for live trading |
|
| 258 |
+
| `PORT` | Hub | FastAPI server port (default 7860) |
|
| 259 |
+
| `HUB_WS_URL` | Asset Spaces | Publisher WebSocket URL |
|
| 260 |
+
| `HUB_SUB_URL` | Ranker | Subscriber WebSocket URL |
|
| 261 |
+
|
| 262 |
+
---
|
| 263 |
+
|
| 264 |
+
## 10. Quick Smoke Test
|
| 265 |
+
|
| 266 |
+
```bash
|
| 267 |
+
# Terminal 1 β start hub
|
| 268 |
+
python websocket_hub.py
|
| 269 |
+
|
| 270 |
+
# Terminal 2 β ranker tests (no network)
|
| 271 |
+
python Quasar_axrvi_ranker.py --test
|
| 272 |
+
|
| 273 |
+
# Terminal 3 β simulate a publisher
|
| 274 |
+
python - <<'EOF'
|
| 275 |
+
from websocket_client import AssetSpacePublisher, TrainingMetrics, VotingMetrics
|
| 276 |
+
import time
|
| 277 |
+
|
| 278 |
+
pub = AssetSpacePublisher("V75_TEST", "ws://localhost:7860/ws/publish/V75_TEST")
|
| 279 |
+
pub.start()
|
| 280 |
+
time.sleep(1)
|
| 281 |
+
for step in range(10):
|
| 282 |
+
pub.publish_combined(
|
| 283 |
+
TrainingMetrics(training_steps=step*100, avn_accuracy=0.5+step*0.04),
|
| 284 |
+
VotingMetrics(dominant_signal="BUY", buy_count=7, sell_count=3),
|
| 285 |
+
)
|
| 286 |
+
time.sleep(0.5)
|
| 287 |
+
print("Done. Check /rankings endpoint.")
|
| 288 |
+
EOF
|
| 289 |
+
|
| 290 |
+
# Terminal 4 β verify hub received data
|
| 291 |
+
curl http://localhost:7860/rankings | python -m json.tool
|
| 292 |
+
```
|
| 293 |
+
|
| 294 |
+
---
|
| 295 |
+
|
| 296 |
+
*End of deployment guide.*
|
Dockerfile
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Use official Python 3.11 slim image
|
| 2 |
+
FROM python:3.11-slim
|
| 3 |
+
|
| 4 |
+
WORKDIR /app
|
| 5 |
+
|
| 6 |
+
# Install system dependencies
|
| 7 |
+
RUN apt-get update && apt-get install -y --no-install-recommends \
|
| 8 |
+
build-essential \
|
| 9 |
+
git \
|
| 10 |
+
curl \
|
| 11 |
+
wget \
|
| 12 |
+
supervisor \
|
| 13 |
+
&& rm -rf /var/lib/apt/lists/*
|
| 14 |
+
|
| 15 |
+
# ββ Remove ALL default supervisor configs shipped by apt ββββββββββββββββββββββ
|
| 16 |
+
# The default /etc/supervisor/supervisord.conf includes inet_http_server on
|
| 17 |
+
# port 9001, which collides on restart. We replace it entirely with our own.
|
| 18 |
+
RUN rm -f /etc/supervisor/supervisord.conf \
|
| 19 |
+
/etc/supervisor/conf.d/*.conf
|
| 20 |
+
|
| 21 |
+
# Copy requirements first (for layer caching)
|
| 22 |
+
COPY requirements.txt .
|
| 23 |
+
|
| 24 |
+
# Install Python dependencies
|
| 25 |
+
RUN pip install --no-cache-dir --upgrade pip setuptools wheel && \
|
| 26 |
+
pip install --no-cache-dir -r requirements.txt
|
| 27 |
+
|
| 28 |
+
# Copy application code
|
| 29 |
+
COPY . .
|
| 30 |
+
|
| 31 |
+
# ββ Place our supervisor config at the root level so it is fully standalone ββ
|
| 32 |
+
COPY supervisord.conf /etc/supervisord.conf
|
| 33 |
+
RUN chmod 644 /etc/supervisord.conf
|
| 34 |
+
|
| 35 |
+
# ββ Copy and permission the entrypoint cleanup script ββββββββββββββββββββββββ
|
| 36 |
+
COPY entrypoint.sh /entrypoint.sh
|
| 37 |
+
RUN chmod +x /entrypoint.sh
|
| 38 |
+
|
| 39 |
+
# Logs directory
|
| 40 |
+
RUN mkdir -p /app/logs && chmod 777 /app/logs
|
| 41 |
+
|
| 42 |
+
# Expose only the ports our app actually uses
|
| 43 |
+
EXPOSE 7860
|
| 44 |
+
|
| 45 |
+
USER root
|
| 46 |
+
|
| 47 |
+
# Entrypoint cleans up stale state, then execs supervisord
|
| 48 |
+
CMD ["/entrypoint.sh"]
|
Quasar_axrvi_ranker.py
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
README.md
CHANGED
|
@@ -1,10 +1,628 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 4 |
+
β K1RL QUASAR β LOG METRICS READER (standalone) β
|
| 5 |
+
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
|
| 6 |
+
β β
|
| 7 |
+
β Drop this SINGLE FILE into any Asset Space. No other K1RL files needed. β
|
| 8 |
+
β β
|
| 9 |
+
β What it does: β
|
| 10 |
+
β β’ Subscribes to the V75:final_signals Redis channel (no log polling) β
|
| 11 |
+
β β’ Parses BUY/SELL signals via the same Redis ingestion flow as Rewards.py β
|
| 12 |
+
β β’ Publishes extracted voting metrics to the Quasar Hub over wss:// β
|
| 13 |
+
β β’ Runs entirely in a background daemon thread with its own asyncio loop β
|
| 14 |
+
β β’ Auto-reconnects on disconnection β
|
| 15 |
+
β β’ Auto-detects space name from HuggingFace SPACE_ID env var β
|
| 16 |
+
β β
|
| 17 |
+
β SIGNAL SOURCE: Redis (V75:final_signals) β logs are NOT consulted for signals β
|
| 18 |
+
β β
|
| 19 |
+
β Integration β 3 lines in app.py: β
|
| 20 |
+
β from log_metrics_reader import start_log_publisher β
|
| 21 |
+
β _publisher, _reader = start_log_publisher() β
|
| 22 |
+
β # done β signals stream from Redis to hub automatically β
|
| 23 |
+
β β
|
| 24 |
+
β pip dependencies: websocket-client>=1.6.0, redis, redis_config_v75, β
|
| 25 |
+
β redis_connection_manager β
|
| 26 |
+
β β
|
| 27 |
+
β VERSION: v2.0-redis-signals | 2026-03-28 | V75 β
|
| 28 |
+
β β
|
| 29 |
+
β CHANGE LOG: β
|
| 30 |
+
β v2.0 REMOVED log-based signal scraping entirely (_tail_log, _parse_lines, β
|
| 31 |
+
β extract_metrics_from_log, LogMetricsReader, _RawMetrics). β
|
| 32 |
+
β REPLACED with RedisSignalReader β identical Redis connection setup, β
|
| 33 |
+
β channel, and parse chain as Rewards.py v5.2.1-V75. β
|
| 34 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 35 |
+
"""
|
| 36 |
+
|
| 37 |
+
import asyncio
|
| 38 |
+
import json
|
| 39 |
+
import logging
|
| 40 |
+
import os
|
| 41 |
+
import sys
|
| 42 |
+
import threading
|
| 43 |
+
import time
|
| 44 |
+
from dataclasses import dataclass
|
| 45 |
+
from typing import Optional, Tuple
|
| 46 |
+
|
| 47 |
+
import websocket # pip: websocket-client>=1.6.0
|
| 48 |
+
|
| 49 |
+
# ββ Redis / V75 imports (same as Rewards.py) ββββββββββββββββββββββββββββββββββββββ
|
| 50 |
+
from redis_config_v75 import (
|
| 51 |
+
REDIS_URL,
|
| 52 |
+
REDIS_PASSWORD,
|
| 53 |
+
REDIS_DB_FEATURES,
|
| 54 |
+
prefixed_channel,
|
| 55 |
+
)
|
| 56 |
+
from redis_connection_manager import RedisAblyClient, RedisMessage
|
| 57 |
+
|
| 58 |
+
# ββ Namespaced signal channel (identical to Rewards.py) ββββββββββββββββββββββββββββββ
|
| 59 |
+
ABLY_SIGNAL_CHANNEL = prefixed_channel("final_signals") # β "V75:final_signals"
|
| 60 |
+
|
| 61 |
+
logging.basicConfig(
|
| 62 |
+
level=logging.INFO,
|
| 63 |
+
format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
|
| 64 |
+
datefmt="%Y-%m-%d %H:%M:%S",
|
| 65 |
+
stream=sys.stdout,
|
| 66 |
+
)
|
| 67 |
+
logger = logging.getLogger("LogMetricsReader")
|
| 68 |
+
|
| 69 |
+
# ββ Config β override via environment variables if needed βββββββββββββββββββββββββββββ
|
| 70 |
+
_DEFAULT_HUB_HOST = os.environ.get("QUASAR_HUB_HOST", "karlquant-quasar-executo.hf.space")
|
| 71 |
+
_DEFAULT_POLL_INTERVAL = float(os.environ.get("QUASAR_POLL_INTERVAL", "5.0")) # kept for factory compat
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 75 |
+
# SECTION 1 β STRICT METRIC CONTAINERS (unchanged)
|
| 76 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 77 |
+
|
| 78 |
+
@dataclass
|
| 79 |
+
class TrainingMetrics:
|
| 80 |
+
"""The only training fields permitted by the hub schema."""
|
| 81 |
+
training_steps: int = 0
|
| 82 |
+
actor_loss: float = 0.0
|
| 83 |
+
critic_loss: float = 0.0
|
| 84 |
+
avn_loss: float = 0.0
|
| 85 |
+
avn_accuracy: float = 0.0 # 0.0 β 1.0 float (NOT percent)
|
| 86 |
+
|
| 87 |
+
|
| 88 |
+
@dataclass
|
| 89 |
+
class VotingMetrics:
|
| 90 |
+
"""The only voting fields permitted by the hub schema."""
|
| 91 |
+
dominant_signal: str = "NEUTRAL" # "BUY" | "SELL" | "NEUTRAL"
|
| 92 |
+
buy_count: int = 0
|
| 93 |
+
sell_count: int = 0
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 97 |
+
# SECTION 2 β WEBSOCKET PUBLISHER (unchanged β embedded, no external K1RL imports)
|
| 98 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 99 |
+
|
| 100 |
+
class AssetSpacePublisher:
|
| 101 |
+
"""
|
| 102 |
+
Send-only WebSocket publisher. Runs in a background daemon thread.
|
| 103 |
+
Auto-reconnects with capped exponential back-off.
|
| 104 |
+
|
| 105 |
+
Never blocks the training loop or signal reader.
|
| 106 |
+
"""
|
| 107 |
+
|
| 108 |
+
_MAX_BACKOFF: int = 30
|
| 109 |
+
|
| 110 |
+
def __init__(
|
| 111 |
+
self,
|
| 112 |
+
space_name: str,
|
| 113 |
+
hub_url: str,
|
| 114 |
+
min_publish_interval: float = 0.5,
|
| 115 |
+
):
|
| 116 |
+
self.space_name = space_name
|
| 117 |
+
self.hub_url = hub_url
|
| 118 |
+
self.min_publish_interval = min_publish_interval
|
| 119 |
+
|
| 120 |
+
self._ws: Optional[websocket.WebSocketApp] = None
|
| 121 |
+
self._connected = False
|
| 122 |
+
self._running = False
|
| 123 |
+
self._thread: Optional[threading.Thread] = None
|
| 124 |
+
self._reconnect_count = 0
|
| 125 |
+
|
| 126 |
+
self._cache_lock = threading.Lock()
|
| 127 |
+
self._latest_training: Optional[TrainingMetrics] = None
|
| 128 |
+
self._latest_voting: Optional[VotingMetrics] = None
|
| 129 |
+
|
| 130 |
+
self._rate_lock = threading.Lock()
|
| 131 |
+
self._last_send_ts = {"training": 0.0, "voting": 0.0, "combined": 0.0}
|
| 132 |
+
|
| 133 |
+
self._stats = {
|
| 134 |
+
"messages_sent": 0,
|
| 135 |
+
"bytes_sent": 0,
|
| 136 |
+
"reconnect_count": 0,
|
| 137 |
+
"last_send_time": 0.0,
|
| 138 |
+
"dropped_rate": 0,
|
| 139 |
+
}
|
| 140 |
+
|
| 141 |
+
# ββ Lifecycle ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 142 |
+
|
| 143 |
+
def start(self) -> None:
|
| 144 |
+
if self._running:
|
| 145 |
+
return
|
| 146 |
+
self._running = True
|
| 147 |
+
self._thread = threading.Thread(
|
| 148 |
+
target=self._run_loop,
|
| 149 |
+
daemon=True,
|
| 150 |
+
name=f"Publisher-{self.space_name}",
|
| 151 |
+
)
|
| 152 |
+
self._thread.start()
|
| 153 |
+
logger.info(f"[{self.space_name}] Publisher started β {self.hub_url}")
|
| 154 |
+
|
| 155 |
+
def stop(self) -> None:
|
| 156 |
+
self._running = False
|
| 157 |
+
if self._ws:
|
| 158 |
+
try:
|
| 159 |
+
self._ws.close()
|
| 160 |
+
except Exception:
|
| 161 |
+
pass
|
| 162 |
+
if self._thread:
|
| 163 |
+
self._thread.join(timeout=3)
|
| 164 |
+
logger.info(f"[{self.space_name}] Publisher stopped")
|
| 165 |
+
|
| 166 |
+
@property
|
| 167 |
+
def is_connected(self) -> bool:
|
| 168 |
+
return self._connected
|
| 169 |
+
|
| 170 |
+
def get_stats(self) -> dict:
|
| 171 |
+
return {**self._stats, "connected": self._connected, "running": self._running}
|
| 172 |
+
|
| 173 |
+
# ββ WebSocket loop βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 174 |
+
|
| 175 |
+
def _run_loop(self) -> None:
|
| 176 |
+
while self._running:
|
| 177 |
+
try:
|
| 178 |
+
self._connect_and_run()
|
| 179 |
+
except Exception as e:
|
| 180 |
+
logger.error(f"[{self.space_name}] Connection error: {e}")
|
| 181 |
+
if not self._running:
|
| 182 |
+
break
|
| 183 |
+
backoff = min(self._MAX_BACKOFF, 2 ** min(self._reconnect_count, 4))
|
| 184 |
+
logger.info(
|
| 185 |
+
f"[{self.space_name}] Reconnecting in {backoff}s⦠"
|
| 186 |
+
f"(attempt #{self._reconnect_count + 1})"
|
| 187 |
+
)
|
| 188 |
+
time.sleep(backoff)
|
| 189 |
+
self._reconnect_count += 1
|
| 190 |
+
self._stats["reconnect_count"] = self._reconnect_count
|
| 191 |
+
|
| 192 |
+
def _connect_and_run(self) -> None:
|
| 193 |
+
self._ws = websocket.WebSocketApp(
|
| 194 |
+
self.hub_url,
|
| 195 |
+
on_open = self._on_open,
|
| 196 |
+
on_message = self._on_message,
|
| 197 |
+
on_error = self._on_error,
|
| 198 |
+
on_close = self._on_close,
|
| 199 |
+
)
|
| 200 |
+
self._ws.run_forever(
|
| 201 |
+
ping_interval = 30,
|
| 202 |
+
ping_timeout = 10,
|
| 203 |
+
sslopt = {"check_hostname": True},
|
| 204 |
+
)
|
| 205 |
+
|
| 206 |
+
# ββ Callbacks ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 207 |
+
|
| 208 |
+
def _on_open(self, ws) -> None:
|
| 209 |
+
self._connected = True
|
| 210 |
+
self._reconnect_count = 0
|
| 211 |
+
self._stats["reconnect_count"] = 0
|
| 212 |
+
logger.info(f"[{self.space_name}] β
Connected to hub")
|
| 213 |
+
self._send_raw({"type": "identify", "space": self.space_name})
|
| 214 |
+
with self._cache_lock:
|
| 215 |
+
t, v = self._latest_training, self._latest_voting
|
| 216 |
+
if t:
|
| 217 |
+
self._send_training_payload(t)
|
| 218 |
+
if v:
|
| 219 |
+
self._send_voting_payload(v)
|
| 220 |
+
|
| 221 |
+
def _on_message(self, ws, message: str) -> None:
|
| 222 |
+
logger.warning(
|
| 223 |
+
f"[{self.space_name}] β οΈ Unexpected hub message β discarded: {message[:80]}"
|
| 224 |
+
)
|
| 225 |
+
|
| 226 |
+
def _on_error(self, ws, error) -> None:
|
| 227 |
+
logger.error(f"[{self.space_name}] WebSocket error: {error}")
|
| 228 |
+
self._connected = False
|
| 229 |
+
|
| 230 |
+
def _on_close(self, ws, code, msg) -> None:
|
| 231 |
+
self._connected = False
|
| 232 |
+
logger.info(f"[{self.space_name}] Connection closed (code={code})")
|
| 233 |
+
|
| 234 |
+
# ββ Rate limiter ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 235 |
+
|
| 236 |
+
def _rate_ok(self, key: str) -> bool:
|
| 237 |
+
if self.min_publish_interval <= 0:
|
| 238 |
+
return True
|
| 239 |
+
now = time.time()
|
| 240 |
+
with self._rate_lock:
|
| 241 |
+
if now - self._last_send_ts[key] >= self.min_publish_interval:
|
| 242 |
+
self._last_send_ts[key] = now
|
| 243 |
+
return True
|
| 244 |
+
self._stats["dropped_rate"] += 1
|
| 245 |
+
return False
|
| 246 |
+
|
| 247 |
+
# ββ Send primitives βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 248 |
+
|
| 249 |
+
def _send_raw(self, payload: dict) -> bool:
|
| 250 |
+
if not (self._ws and self._connected):
|
| 251 |
+
return False
|
| 252 |
+
try:
|
| 253 |
+
text = json.dumps(payload)
|
| 254 |
+
self._ws.send(text)
|
| 255 |
+
self._stats["messages_sent"] += 1
|
| 256 |
+
self._stats["bytes_sent"] += len(text)
|
| 257 |
+
self._stats["last_send_time"] = time.time()
|
| 258 |
+
return True
|
| 259 |
+
except Exception as e:
|
| 260 |
+
logger.error(f"[{self.space_name}] Send error: {e}")
|
| 261 |
+
self._connected = False
|
| 262 |
+
return False
|
| 263 |
+
|
| 264 |
+
def _send_training_payload(self, m: TrainingMetrics) -> bool:
|
| 265 |
+
return self._send_raw({
|
| 266 |
+
"type": "training",
|
| 267 |
+
"data": {
|
| 268 |
+
"training_steps": m.training_steps,
|
| 269 |
+
"actor_loss": m.actor_loss,
|
| 270 |
+
"critic_loss": m.critic_loss,
|
| 271 |
+
"avn_loss": m.avn_loss,
|
| 272 |
+
"avn_accuracy": m.avn_accuracy,
|
| 273 |
+
},
|
| 274 |
+
})
|
| 275 |
+
|
| 276 |
+
def _send_voting_payload(self, m: VotingMetrics) -> bool:
|
| 277 |
+
return self._send_raw({
|
| 278 |
+
"type": "voting",
|
| 279 |
+
"data": {
|
| 280 |
+
"dominant_signal": m.dominant_signal,
|
| 281 |
+
"buy_count": m.buy_count,
|
| 282 |
+
"sell_count": m.sell_count,
|
| 283 |
+
},
|
| 284 |
+
})
|
| 285 |
+
|
| 286 |
+
# ββ Public API ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 287 |
+
|
| 288 |
+
def publish_training(self, metrics: TrainingMetrics) -> bool:
|
| 289 |
+
with self._cache_lock:
|
| 290 |
+
self._latest_training = metrics
|
| 291 |
+
if not self._rate_ok("training"):
|
| 292 |
+
return False
|
| 293 |
+
return self._send_training_payload(metrics)
|
| 294 |
+
|
| 295 |
+
def publish_voting(self, metrics: VotingMetrics) -> bool:
|
| 296 |
+
with self._cache_lock:
|
| 297 |
+
self._latest_voting = metrics
|
| 298 |
+
if not self._rate_ok("voting"):
|
| 299 |
+
return False
|
| 300 |
+
return self._send_voting_payload(metrics)
|
| 301 |
+
|
| 302 |
+
def publish_combined(self, training: TrainingMetrics, voting: VotingMetrics) -> bool:
|
| 303 |
+
with self._cache_lock:
|
| 304 |
+
self._latest_training = training
|
| 305 |
+
self._latest_voting = voting
|
| 306 |
+
if not self._rate_ok("combined"):
|
| 307 |
+
return False
|
| 308 |
+
return self._send_raw({
|
| 309 |
+
"type": "metrics",
|
| 310 |
+
"training": {
|
| 311 |
+
"training_steps": training.training_steps,
|
| 312 |
+
"actor_loss": training.actor_loss,
|
| 313 |
+
"critic_loss": training.critic_loss,
|
| 314 |
+
"avn_loss": training.avn_loss,
|
| 315 |
+
"avn_accuracy": training.avn_accuracy,
|
| 316 |
+
},
|
| 317 |
+
"voting": {
|
| 318 |
+
"dominant_signal": voting.dominant_signal,
|
| 319 |
+
"buy_count": voting.buy_count,
|
| 320 |
+
"sell_count": voting.sell_count,
|
| 321 |
+
},
|
| 322 |
+
})
|
| 323 |
+
|
| 324 |
+
def publish_heartbeat(self) -> bool:
|
| 325 |
+
return self._send_raw({"type": "heartbeat"})
|
| 326 |
+
|
| 327 |
+
|
| 328 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 329 |
+
# SECTION 3 β REDIS SIGNAL READER
|
| 330 |
+
#
|
| 331 |
+
# β
REPLACES: LogMetricsReader (log polling), _tail_log, _parse_lines,
|
| 332 |
+
# extract_metrics_from_log, _RawMetrics β all removed.
|
| 333 |
+
#
|
| 334 |
+
# Signal source is now exclusively the V75:final_signals Redis channel.
|
| 335 |
+
# Connection setup, channel name, and full parse chain are transplanted
|
| 336 |
+
# verbatim from Rewards.py v5.2.1-V75._on_signal / initialize().
|
| 337 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 338 |
+
|
| 339 |
+
class RedisSignalReader:
|
| 340 |
+
"""
|
| 341 |
+
Subscribes to the namespaced V75:final_signals Redis channel and feeds
|
| 342 |
+
parsed BUY/SELL signals directly into an AssetSpacePublisher as VotingMetrics.
|
| 343 |
+
|
| 344 |
+
Redis is the ONLY source of truth for signals β no log file is consulted.
|
| 345 |
+
|
| 346 |
+
Parse chain (identical to Rewards.py._on_signal):
|
| 347 |
+
1. Unwrap RedisMessage β .data attribute
|
| 348 |
+
2. Unwrap nested envelope β {"data": {...}}
|
| 349 |
+
3. Decode JSON strings
|
| 350 |
+
4. Extract final_action with fallback to action
|
| 351 |
+
5. Extract signal_keys (normalised to list)
|
| 352 |
+
6. Extract price
|
| 353 |
+
7. Validate: action must be BUY|SELL, price must be non-zero
|
| 354 |
+
8. Silently ignore any malformed payload (no exception raised)
|
| 355 |
+
|
| 356 |
+
Threading model:
|
| 357 |
+
Runs in a daemon thread that owns its own asyncio event loop, mirroring
|
| 358 |
+
how Rewards.py bridges the blocking RedisAblyClient listener thread into
|
| 359 |
+
async coroutines without touching the caller's event loop.
|
| 360 |
+
"""
|
| 361 |
+
|
| 362 |
+
def __init__(self, publisher: AssetSpacePublisher):
|
| 363 |
+
self.publisher = publisher
|
| 364 |
+
|
| 365 |
+
self._running = False
|
| 366 |
+
self._thread: Optional[threading.Thread] = None
|
| 367 |
+
self._loop: Optional[asyncio.AbstractEventLoop] = None
|
| 368 |
+
|
| 369 |
+
# Rolling buy/sell counters β VotingMetrics
|
| 370 |
+
self._buy_count = 0
|
| 371 |
+
self._sell_count = 0
|
| 372 |
+
self._count_lock = threading.Lock()
|
| 373 |
+
|
| 374 |
+
self._stats = {
|
| 375 |
+
"signals_received": 0,
|
| 376 |
+
"signals_parsed": 0,
|
| 377 |
+
"signals_dropped": 0,
|
| 378 |
+
}
|
| 379 |
+
|
| 380 |
+
# ββ Lifecycle ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 381 |
+
|
| 382 |
+
def start(self) -> None:
|
| 383 |
+
if self._running:
|
| 384 |
+
return
|
| 385 |
+
self._running = True
|
| 386 |
+
self._thread = threading.Thread(
|
| 387 |
+
target=self._run_loop,
|
| 388 |
+
daemon=True,
|
| 389 |
+
name=f"RedisSignalReader-{self.publisher.space_name}",
|
| 390 |
+
)
|
| 391 |
+
self._thread.start()
|
| 392 |
+
logger.info(
|
| 393 |
+
f"[{self.publisher.space_name}] RedisSignalReader started "
|
| 394 |
+
f"β channel={ABLY_SIGNAL_CHANNEL}"
|
| 395 |
+
)
|
| 396 |
+
|
| 397 |
+
def stop(self) -> None:
|
| 398 |
+
self._running = False
|
| 399 |
+
if self._loop and self._loop.is_running():
|
| 400 |
+
self._loop.call_soon_threadsafe(self._loop.stop)
|
| 401 |
+
if self._thread:
|
| 402 |
+
self._thread.join(timeout=5)
|
| 403 |
+
logger.info(f"[{self.publisher.space_name}] RedisSignalReader stopped")
|
| 404 |
+
|
| 405 |
+
def get_stats(self) -> dict:
|
| 406 |
+
return {**self._stats, "running": self._running}
|
| 407 |
+
|
| 408 |
+
# ββ Asyncio thread entry point ββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 409 |
+
|
| 410 |
+
def _run_loop(self) -> None:
|
| 411 |
+
"""
|
| 412 |
+
Runs in a daemon thread. Creates a fresh asyncio event loop so this
|
| 413 |
+
reader never interferes with any event loop the host process may have.
|
| 414 |
+
"""
|
| 415 |
+
self._loop = asyncio.new_event_loop()
|
| 416 |
+
asyncio.set_event_loop(self._loop)
|
| 417 |
+
try:
|
| 418 |
+
self._loop.run_until_complete(self._async_main())
|
| 419 |
+
except Exception as e:
|
| 420 |
+
logger.error(
|
| 421 |
+
f"[{self.publisher.space_name}] RedisSignalReader loop error: {e}"
|
| 422 |
+
)
|
| 423 |
+
finally:
|
| 424 |
+
self._loop.close()
|
| 425 |
+
|
| 426 |
+
# ββ Core async subscription βββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 427 |
+
|
| 428 |
+
async def _async_main(self) -> None:
|
| 429 |
+
"""
|
| 430 |
+
Initialise the RedisAblyClient exactly as Rewards.py does, subscribe to
|
| 431 |
+
ABLY_SIGNAL_CHANNEL, and keep the loop alive until stop() is called.
|
| 432 |
+
|
| 433 |
+
Connection parameters mirror Rewards.py.RewardsEngine.initialize():
|
| 434 |
+
β’ redis_url = REDIS_URL (from redis_config_v75)
|
| 435 |
+
β’ password = REDIS_PASSWORD
|
| 436 |
+
β’ use_streams = True
|
| 437 |
+
β’ database = REDIS_DB_FEATURES (V75: DB 0)
|
| 438 |
+
"""
|
| 439 |
+
ably = RedisAblyClient(
|
| 440 |
+
redis_url=REDIS_URL,
|
| 441 |
+
password=REDIS_PASSWORD,
|
| 442 |
+
use_streams=True,
|
| 443 |
+
database=REDIS_DB_FEATURES,
|
| 444 |
+
)
|
| 445 |
+
|
| 446 |
+
channel = ably.channels.get(ABLY_SIGNAL_CHANNEL)
|
| 447 |
+
|
| 448 |
+
# _on_signal is called by the blocking RedisAblyClient listener thread.
|
| 449 |
+
# Identical callback signature and guard logic to Rewards.py._on_signal.
|
| 450 |
+
def _on_signal(message) -> None:
|
| 451 |
+
self._stats["signals_received"] += 1
|
| 452 |
+
|
| 453 |
+
parsed = self._parse_signal_message(message)
|
| 454 |
+
if parsed is None:
|
| 455 |
+
self._stats["signals_dropped"] += 1
|
| 456 |
+
return
|
| 457 |
+
|
| 458 |
+
self._stats["signals_parsed"] += 1
|
| 459 |
+
action = parsed["action"]
|
| 460 |
+
|
| 461 |
+
with self._count_lock:
|
| 462 |
+
if action == "BUY":
|
| 463 |
+
self._buy_count += 1
|
| 464 |
+
else:
|
| 465 |
+
self._sell_count += 1
|
| 466 |
+
buy = self._buy_count
|
| 467 |
+
sell = self._sell_count
|
| 468 |
+
|
| 469 |
+
dominant = "BUY" if buy >= sell else "SELL"
|
| 470 |
+
vm = VotingMetrics(
|
| 471 |
+
dominant_signal=dominant,
|
| 472 |
+
buy_count=buy,
|
| 473 |
+
sell_count=sell,
|
| 474 |
+
)
|
| 475 |
+
self.publisher.publish_voting(vm)
|
| 476 |
+
|
| 477 |
+
logger.info(
|
| 478 |
+
f"[{self.publisher.space_name}] π Signal {action} "
|
| 479 |
+
f"@ {parsed['entry_price']:.5f} | "
|
| 480 |
+
f"keys={len(parsed['signal_keys'])} | "
|
| 481 |
+
f"buy={buy} sell={sell} dominant={dominant}"
|
| 482 |
+
)
|
| 483 |
+
|
| 484 |
+
await channel.subscribe("message", _on_signal)
|
| 485 |
+
logger.info(
|
| 486 |
+
f"[{self.publisher.space_name}] β
Subscribed to {ABLY_SIGNAL_CHANNEL} "
|
| 487 |
+
f"(V75 namespace, DB={REDIS_DB_FEATURES})"
|
| 488 |
+
)
|
| 489 |
+
|
| 490 |
+
# Idle loop β RedisAblyClient delivers messages via its own listener thread.
|
| 491 |
+
while self._running:
|
| 492 |
+
await asyncio.sleep(1.0)
|
| 493 |
+
|
| 494 |
+
ably.close()
|
| 495 |
+
|
| 496 |
+
# ββ Signal parser (transplanted verbatim from Rewards.py._on_signal) βββββββββββββ
|
| 497 |
+
|
| 498 |
+
def _parse_signal_message(self, message) -> Optional[dict]:
|
| 499 |
+
"""
|
| 500 |
+
Full parse chain from Rewards.py, extracted into a standalone helper.
|
| 501 |
+
|
| 502 |
+
Steps:
|
| 503 |
+
1. Unwrap RedisMessage β .data
|
| 504 |
+
2. Unwrap nested {"data": ...} envelope
|
| 505 |
+
3. Parse JSON strings
|
| 506 |
+
4. Extract final_action with fallback to action, uppercase
|
| 507 |
+
5. Extract signal_keys
|
| 508 |
+
6. Extract price
|
| 509 |
+
7. Validate action β {BUY, SELL}
|
| 510 |
+
8. Validate price is non-zero
|
| 511 |
+
9. Normalise signal_keys to a list (cap at 8, matching Rewards.py)
|
| 512 |
+
10. Return None silently for any malformed payload β never raises
|
| 513 |
+
|
| 514 |
+
Returns a dict with keys: action, signal_keys, entry_price, payload.
|
| 515 |
+
Returns None if the payload is malformed or the signal should be ignored.
|
| 516 |
+
"""
|
| 517 |
+
try:
|
| 518 |
+
# Step 1 β unwrap RedisMessage
|
| 519 |
+
data = message.data if isinstance(message, RedisMessage) else message
|
| 520 |
+
|
| 521 |
+
# Step 2 β unwrap nested data envelope
|
| 522 |
+
if isinstance(data, dict) and "data" in data:
|
| 523 |
+
data = data["data"]
|
| 524 |
+
|
| 525 |
+
# Step 3 β decode JSON strings
|
| 526 |
+
if isinstance(data, str):
|
| 527 |
+
data = json.loads(data)
|
| 528 |
+
|
| 529 |
+
# Step 4 β extract action (final_action with fallback to action)
|
| 530 |
+
action = data.get("final_action", data.get("action", "")).upper()
|
| 531 |
+
|
| 532 |
+
# Step 5 β extract signal_keys
|
| 533 |
+
signal_keys = data.get("signal_keys", [])
|
| 534 |
+
|
| 535 |
+
# Step 6 β extract price
|
| 536 |
+
entry_price = data.get("price", 0.0)
|
| 537 |
+
|
| 538 |
+
# Step 7 β validate action
|
| 539 |
+
if action not in ("BUY", "SELL"):
|
| 540 |
+
return None
|
| 541 |
+
|
| 542 |
+
# Step 8 β validate price
|
| 543 |
+
if not entry_price or entry_price == 0.0:
|
| 544 |
+
return None
|
| 545 |
+
|
| 546 |
+
# Step 9 β normalise signal_keys to list, cap at 8
|
| 547 |
+
if not isinstance(signal_keys, list):
|
| 548 |
+
signal_keys = [str(signal_keys)]
|
| 549 |
+
|
| 550 |
+
return {
|
| 551 |
+
"action": action,
|
| 552 |
+
"signal_keys": signal_keys[:8],
|
| 553 |
+
"entry_price": float(entry_price),
|
| 554 |
+
"payload": data,
|
| 555 |
+
}
|
| 556 |
+
|
| 557 |
+
except Exception:
|
| 558 |
+
# Step 10 β silently discard malformed payloads
|
| 559 |
+
return None
|
| 560 |
+
|
| 561 |
+
|
| 562 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 563 |
+
# SECTION 4 β ONE-LINE FACTORY (signature unchanged β drop-in replacement)
|
| 564 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 565 |
+
|
| 566 |
+
def start_log_publisher(
|
| 567 |
+
poll_interval: float = _DEFAULT_POLL_INTERVAL,
|
| 568 |
+
hub_host: str = _DEFAULT_HUB_HOST,
|
| 569 |
+
space_name: Optional[str] = None,
|
| 570 |
+
# Legacy parameters accepted but unused β kept for call-site compatibility.
|
| 571 |
+
log_path: Optional[str] = None,
|
| 572 |
+
tail_lines: Optional[int] = None,
|
| 573 |
+
) -> Tuple[AssetSpacePublisher, RedisSignalReader]:
|
| 574 |
+
"""
|
| 575 |
+
One-line drop-in for each asset space.
|
| 576 |
+
|
| 577 |
+
Creates the WebSocket publisher AND the Redis signal reader, starts both,
|
| 578 |
+
and returns them so they stay alive for the process lifetime.
|
| 579 |
+
|
| 580 |
+
Signal source: Redis channel V75:final_signals (NOT the log file)
|
| 581 |
+
Auto-detects space name from HuggingFace SPACE_ID env var.
|
| 582 |
+
|
| 583 |
+
Usage in app.py
|
| 584 |
+
---------------
|
| 585 |
+
from log_metrics_reader import start_log_publisher
|
| 586 |
+
_publisher, _reader = start_log_publisher()
|
| 587 |
+
|
| 588 |
+
Override defaults
|
| 589 |
+
-----------------
|
| 590 |
+
_publisher, _reader = start_log_publisher(
|
| 591 |
+
poll_interval = 10.0, # rate-limit floor for WebSocket publishes
|
| 592 |
+
)
|
| 593 |
+
|
| 594 |
+
Environment variable overrides (no code change needed per space)
|
| 595 |
+
----------------------------------------------------------------
|
| 596 |
+
QUASAR_HUB_HOST = "karlquant-quasar-executo.hf.space"
|
| 597 |
+
QUASAR_POLL_INTERVAL = "5.0"
|
| 598 |
+
|
| 599 |
+
NOTE: log_path and tail_lines parameters are accepted for call-site
|
| 600 |
+
compatibility but are silently ignored β signal scraping from logs has
|
| 601 |
+
been removed. Signals are sourced exclusively from Redis.
|
| 602 |
+
"""
|
| 603 |
+
if log_path is not None or tail_lines is not None:
|
| 604 |
+
logger.warning(
|
| 605 |
+
"start_log_publisher: log_path / tail_lines arguments are ignored. "
|
| 606 |
+
"Signal acquisition from log files has been removed. "
|
| 607 |
+
"Signals are sourced exclusively from Redis (V75:final_signals)."
|
| 608 |
+
)
|
| 609 |
+
|
| 610 |
+
# Resolve space name β HF injects SPACE_ID="Owner/SpaceName" at runtime
|
| 611 |
+
if space_name is None:
|
| 612 |
+
raw = os.environ.get("SPACE_ID", "")
|
| 613 |
+
space_name = raw.split("/", 1)[-1] if "/" in raw else (raw or "UnknownSpace")
|
| 614 |
+
|
| 615 |
+
# Always wss://, never explicit port β HF proxies through 443
|
| 616 |
+
hub_url = f"wss://{hub_host}/ws/publish/{space_name}"
|
| 617 |
+
|
| 618 |
+
publisher = AssetSpacePublisher(
|
| 619 |
+
space_name = space_name,
|
| 620 |
+
hub_url = hub_url,
|
| 621 |
+
min_publish_interval = max(poll_interval * 0.9, 0.5),
|
| 622 |
+
)
|
| 623 |
+
publisher.start()
|
| 624 |
+
|
| 625 |
+
reader = RedisSignalReader(publisher=publisher)
|
| 626 |
+
reader.start()
|
| 627 |
+
|
| 628 |
+
return publisher, reader
|
entrypoint.sh
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 3 |
+
# K1RL QUASAR β Entrypoint
|
| 4 |
+
# Cleans up any stale processes/ports before launching supervisord
|
| 5 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 6 |
+
set -e
|
| 7 |
+
|
| 8 |
+
echo "=== QUASAR entrypoint: cleaning up stale state ==="
|
| 9 |
+
|
| 10 |
+
# Kill any lingering supervisord from a previous run
|
| 11 |
+
pkill -f supervisord 2>/dev/null || true
|
| 12 |
+
|
| 13 |
+
# Kill any lingering hub/ranker Python processes
|
| 14 |
+
pkill -f websocket_hub.py 2>/dev/null || true
|
| 15 |
+
pkill -f Quasar_axrvi_ranker.py 2>/dev/null || true
|
| 16 |
+
|
| 17 |
+
# Give OS a moment to release sockets
|
| 18 |
+
sleep 1
|
| 19 |
+
|
| 20 |
+
# Remove stale PID files
|
| 21 |
+
rm -f /tmp/supervisord.pid
|
| 22 |
+
|
| 23 |
+
# Confirm our app files exist (catches missing-file issues early)
|
| 24 |
+
for f in /app/websocket_hub.py /app/Quasar_axrvi_ranker.py; do
|
| 25 |
+
if [ ! -f "$f" ]; then
|
| 26 |
+
echo "ERROR: Required file missing: $f"
|
| 27 |
+
echo "Make sure it is committed to your HF Space repository."
|
| 28 |
+
exit 1
|
| 29 |
+
fi
|
| 30 |
+
done
|
| 31 |
+
|
| 32 |
+
echo "=== Starting supervisord ==="
|
| 33 |
+
exec /usr/bin/supervisord -c /etc/supervisord.conf
|
gitattributes
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
hub_dashboard.html
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
hub_dashboard_service.py
ADDED
|
@@ -0,0 +1,388 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 4 |
+
β K1RL QUASAR β HUB DASHBOARD SERVICE (WITH RANKER LOGS INTEGRATION) β
|
| 5 |
+
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
|
| 6 |
+
β Architecture role: READ-ONLY subscriber β serves dashboard UI β
|
| 7 |
+
β VERSION: v1.1 (UPDATED) | 2026-03-26 β
|
| 8 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββοΏ½οΏ½οΏ½βββββββββββββββββββββββββββββββββββββββ
|
| 9 |
+
"""
|
| 10 |
+
|
| 11 |
+
import json
|
| 12 |
+
import logging
|
| 13 |
+
import os
|
| 14 |
+
import sys
|
| 15 |
+
import threading
|
| 16 |
+
import time
|
| 17 |
+
from collections import deque
|
| 18 |
+
from datetime import datetime
|
| 19 |
+
from pathlib import Path
|
| 20 |
+
from typing import Dict, List, Optional
|
| 21 |
+
|
| 22 |
+
import websocket
|
| 23 |
+
from flask import Flask, Response, jsonify, send_from_directory
|
| 24 |
+
from flask_cors import CORS
|
| 25 |
+
|
| 26 |
+
from ranker_logging import RankerLogger, EventCategory, LogLevel
|
| 27 |
+
from ranker_logs_api import ranker_logs_bp, init_ranker_logs_api
|
| 28 |
+
|
| 29 |
+
# ββ Logging βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 30 |
+
logging.basicConfig(
|
| 31 |
+
level=logging.INFO,
|
| 32 |
+
format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
|
| 33 |
+
datefmt="%Y-%m-%d %H:%M:%S",
|
| 34 |
+
stream=sys.stdout,
|
| 35 |
+
)
|
| 36 |
+
logger = logging.getLogger("HubDashboardService")
|
| 37 |
+
|
| 38 |
+
# ββ Config ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 39 |
+
_HUB_HOST = os.environ.get("QUASAR_HUB_HOST", "karlquant-quasar-executo.hf.space")
|
| 40 |
+
_DASHBOARD_PORT = int(os.environ.get("DASHBOARD_PORT", "8051"))
|
| 41 |
+
_HTML_PATH = os.environ.get(
|
| 42 |
+
"DASHBOARD_HTML",
|
| 43 |
+
str(Path(__file__).parent / "hub_dashboard.html"),
|
| 44 |
+
)
|
| 45 |
+
|
| 46 |
+
_METRIC_HISTORY_LEN = int(os.environ.get("QUASAR_METRIC_HISTORY", "200"))
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 50 |
+
# SECTION 1 β IN-MEMORY STATE STORE
|
| 51 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 52 |
+
|
| 53 |
+
class DashboardState:
|
| 54 |
+
"""
|
| 55 |
+
Thread-safe cache of everything the dashboard needs.
|
| 56 |
+
"""
|
| 57 |
+
|
| 58 |
+
def __init__(self, history_len: int = _METRIC_HISTORY_LEN):
|
| 59 |
+
self._lock = threading.RLock()
|
| 60 |
+
self._history_len = history_len
|
| 61 |
+
|
| 62 |
+
self.snapshots: Dict[str, dict] = {}
|
| 63 |
+
self.metric_history: Dict[str, deque] = {}
|
| 64 |
+
|
| 65 |
+
self.hub_connected = False
|
| 66 |
+
self.start_time = time.time()
|
| 67 |
+
self.messages_rx = 0
|
| 68 |
+
self.last_update_ts = 0.0
|
| 69 |
+
self.reconnect_count = 0
|
| 70 |
+
|
| 71 |
+
self._ranked: List[dict] = []
|
| 72 |
+
|
| 73 |
+
def ingest_snapshot(self, space_name: str, snapshot: dict) -> None:
|
| 74 |
+
"""Accept a full snapshot dict."""
|
| 75 |
+
with self._lock:
|
| 76 |
+
self.snapshots[space_name] = snapshot
|
| 77 |
+
self.messages_rx += 1
|
| 78 |
+
self.last_update_ts = time.time()
|
| 79 |
+
self._append_metric_history(space_name, snapshot)
|
| 80 |
+
self._recompute_rankings()
|
| 81 |
+
|
| 82 |
+
def ingest_bulk(self, snapshots: dict) -> None:
|
| 83 |
+
"""Accept bulk payload."""
|
| 84 |
+
with self._lock:
|
| 85 |
+
for name, snap in snapshots.items():
|
| 86 |
+
self.snapshots[name] = snap
|
| 87 |
+
self._append_metric_history(name, snap)
|
| 88 |
+
self.last_update_ts = time.time()
|
| 89 |
+
self._recompute_rankings()
|
| 90 |
+
|
| 91 |
+
def _append_metric_history(self, space_name: str, snap: dict) -> None:
|
| 92 |
+
"""Push latest training numbers onto the per-asset deque."""
|
| 93 |
+
if space_name not in self.metric_history:
|
| 94 |
+
self.metric_history[space_name] = deque(maxlen=self._history_len)
|
| 95 |
+
training = snap.get("training", {})
|
| 96 |
+
if training:
|
| 97 |
+
self.metric_history[space_name].append({
|
| 98 |
+
"ts": snap.get("last_updated", time.time()),
|
| 99 |
+
"training_steps": training.get("training_steps", 0),
|
| 100 |
+
"actor_loss": training.get("actor_loss", 0.0),
|
| 101 |
+
"critic_loss": training.get("critic_loss", 0.0),
|
| 102 |
+
"avn_loss": training.get("avn_loss", 0.0),
|
| 103 |
+
"avn_accuracy": training.get("avn_accuracy", 0.0),
|
| 104 |
+
})
|
| 105 |
+
|
| 106 |
+
def _recompute_rankings(self) -> None:
|
| 107 |
+
"""Replicate ranker formula: score = signal_confidence - avn_accuracy"""
|
| 108 |
+
ranked = []
|
| 109 |
+
for name, snap in self.snapshots.items():
|
| 110 |
+
training = snap.get("training", {})
|
| 111 |
+
voting = snap.get("voting", {})
|
| 112 |
+
buy = voting.get("buy_count", 0)
|
| 113 |
+
sell = voting.get("sell_count", 0)
|
| 114 |
+
total = buy + sell
|
| 115 |
+
sig_conf = (max(buy, sell) / total) if total > 0 else 0.0
|
| 116 |
+
avn_acc = training.get("avn_accuracy", 0.0)
|
| 117 |
+
score = sig_conf - avn_acc
|
| 118 |
+
ranked.append({
|
| 119 |
+
"rank": 0,
|
| 120 |
+
"space_name": name,
|
| 121 |
+
"score": round(score, 4),
|
| 122 |
+
"signal_confidence": round(sig_conf, 4),
|
| 123 |
+
"avn_accuracy": round(avn_acc, 4),
|
| 124 |
+
"dominant_signal": voting.get("dominant_signal", "NEUTRAL"),
|
| 125 |
+
"buy_count": buy,
|
| 126 |
+
"sell_count": sell,
|
| 127 |
+
"training_steps": training.get("training_steps", 0),
|
| 128 |
+
"actor_loss": training.get("actor_loss", 0.0),
|
| 129 |
+
"critic_loss": training.get("critic_loss", 0.0),
|
| 130 |
+
"avn_loss": training.get("avn_loss", 0.0),
|
| 131 |
+
"last_updated": snap.get("last_updated", 0.0),
|
| 132 |
+
})
|
| 133 |
+
ranked.sort(key=lambda r: r["score"], reverse=True)
|
| 134 |
+
for i, r in enumerate(ranked):
|
| 135 |
+
r["rank"] = i + 1
|
| 136 |
+
self._ranked = ranked
|
| 137 |
+
|
| 138 |
+
def get_state(self) -> dict:
|
| 139 |
+
with self._lock:
|
| 140 |
+
return {
|
| 141 |
+
"rankings": list(self._ranked),
|
| 142 |
+
"metric_history": {
|
| 143 |
+
name: list(h)
|
| 144 |
+
for name, h in self.metric_history.items()
|
| 145 |
+
},
|
| 146 |
+
"health": self._health_snapshot(),
|
| 147 |
+
"timestamp": datetime.utcnow().isoformat() + "Z",
|
| 148 |
+
}
|
| 149 |
+
|
| 150 |
+
def get_rankings(self) -> List[dict]:
|
| 151 |
+
with self._lock:
|
| 152 |
+
return list(self._ranked)
|
| 153 |
+
|
| 154 |
+
def get_metric_history(self, limit: int = 100) -> dict:
|
| 155 |
+
with self._lock:
|
| 156 |
+
return {
|
| 157 |
+
name: list(h)[-limit:]
|
| 158 |
+
for name, h in self.metric_history.items()
|
| 159 |
+
}
|
| 160 |
+
|
| 161 |
+
def _health_snapshot(self) -> dict:
|
| 162 |
+
return {
|
| 163 |
+
"hub_connected": self.hub_connected,
|
| 164 |
+
"spaces_connected": len(self.snapshots),
|
| 165 |
+
"messages_rx": self.messages_rx,
|
| 166 |
+
"last_update_ts": self.last_update_ts,
|
| 167 |
+
"last_update_ago": round(time.time() - self.last_update_ts, 1)
|
| 168 |
+
if self.last_update_ts else None,
|
| 169 |
+
"uptime_seconds": round(time.time() - self.start_time, 0),
|
| 170 |
+
"reconnect_count": self.reconnect_count,
|
| 171 |
+
}
|
| 172 |
+
|
| 173 |
+
def get_health(self) -> dict:
|
| 174 |
+
with self._lock:
|
| 175 |
+
return self._health_snapshot()
|
| 176 |
+
|
| 177 |
+
|
| 178 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 179 |
+
# SECTION 2 β WEBSOCKET SUBSCRIBER
|
| 180 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 181 |
+
|
| 182 |
+
class HubSubscriber:
|
| 183 |
+
"""Connects to hub and maintains per-asset snapshots."""
|
| 184 |
+
|
| 185 |
+
_MAX_BACKOFF = 30
|
| 186 |
+
|
| 187 |
+
def __init__(
|
| 188 |
+
self,
|
| 189 |
+
state: DashboardState,
|
| 190 |
+
hub_host: str = _HUB_HOST,
|
| 191 |
+
ranker_logger: Optional[RankerLogger] = None,
|
| 192 |
+
):
|
| 193 |
+
self.state = state
|
| 194 |
+
self.hub_url = f"wss://{hub_host}/ws/subscribe"
|
| 195 |
+
self.ranker_logger = ranker_logger
|
| 196 |
+
self._ws: Optional[websocket.WebSocketApp] = None
|
| 197 |
+
self._running = False
|
| 198 |
+
self._thread: Optional[threading.Thread] = None
|
| 199 |
+
self._reconnect_count = 0
|
| 200 |
+
|
| 201 |
+
def start(self) -> None:
|
| 202 |
+
if self._running:
|
| 203 |
+
return
|
| 204 |
+
self._running = True
|
| 205 |
+
self._thread = threading.Thread(
|
| 206 |
+
target=self._run_loop, daemon=True, name="HubSubscriber",
|
| 207 |
+
)
|
| 208 |
+
self._thread.start()
|
| 209 |
+
logger.info(f"HubSubscriber started β {self.hub_url}")
|
| 210 |
+
|
| 211 |
+
def stop(self) -> None:
|
| 212 |
+
self._running = False
|
| 213 |
+
if self._ws:
|
| 214 |
+
try:
|
| 215 |
+
self._ws.close()
|
| 216 |
+
except Exception:
|
| 217 |
+
pass
|
| 218 |
+
if self._thread:
|
| 219 |
+
self._thread.join(timeout=5)
|
| 220 |
+
|
| 221 |
+
def _run_loop(self) -> None:
|
| 222 |
+
while self._running:
|
| 223 |
+
try:
|
| 224 |
+
self._connect()
|
| 225 |
+
except Exception as e:
|
| 226 |
+
logger.error(f"[HubSubscriber] Error: {e}")
|
| 227 |
+
if not self._running:
|
| 228 |
+
break
|
| 229 |
+
self.state.hub_connected = False
|
| 230 |
+
self.state.reconnect_count = self._reconnect_count
|
| 231 |
+
backoff = min(self._MAX_BACKOFF, 2 ** min(self._reconnect_count, 4))
|
| 232 |
+
logger.info(f"[HubSubscriber] Reconnecting in {backoff}sβ¦")
|
| 233 |
+
time.sleep(backoff)
|
| 234 |
+
self._reconnect_count += 1
|
| 235 |
+
|
| 236 |
+
def _connect(self) -> None:
|
| 237 |
+
self._ws = websocket.WebSocketApp(
|
| 238 |
+
self.hub_url,
|
| 239 |
+
on_open = self._on_open,
|
| 240 |
+
on_message = self._on_message,
|
| 241 |
+
on_error = self._on_error,
|
| 242 |
+
on_close = self._on_close,
|
| 243 |
+
)
|
| 244 |
+
self._ws.run_forever(
|
| 245 |
+
ping_interval = 25,
|
| 246 |
+
ping_timeout = 10,
|
| 247 |
+
sslopt = {"check_hostname": True},
|
| 248 |
+
)
|
| 249 |
+
|
| 250 |
+
def _on_open(self, ws) -> None:
|
| 251 |
+
self.state.hub_connected = True
|
| 252 |
+
self._reconnect_count = 0
|
| 253 |
+
if self.ranker_logger:
|
| 254 |
+
self.ranker_logger.connection_event("Hub WebSocket", "connected")
|
| 255 |
+
logger.info("[HubSubscriber] β
Connected to hub")
|
| 256 |
+
|
| 257 |
+
def _on_message(self, ws, raw: str) -> None:
|
| 258 |
+
try:
|
| 259 |
+
msg = json.loads(raw)
|
| 260 |
+
mtype = msg.get("type", "")
|
| 261 |
+
|
| 262 |
+
if mtype == "initial_state":
|
| 263 |
+
snaps = msg.get("snapshots", {})
|
| 264 |
+
self.state.ingest_bulk(snaps)
|
| 265 |
+
logger.info(f"[HubSubscriber] Initial state: {len(snaps)} spaces")
|
| 266 |
+
|
| 267 |
+
elif mtype == "metrics_update":
|
| 268 |
+
space = msg.get("space_name")
|
| 269 |
+
snap = msg.get("snapshot", {})
|
| 270 |
+
if space and snap:
|
| 271 |
+
self.state.ingest_snapshot(space, snap)
|
| 272 |
+
if self.ranker_logger:
|
| 273 |
+
training = snap.get("training", {})
|
| 274 |
+
voting = snap.get("voting", {})
|
| 275 |
+
self.ranker_logger.hub_update(
|
| 276 |
+
asset=space,
|
| 277 |
+
avn_accuracy=training.get("avn_accuracy", 0.0),
|
| 278 |
+
signal_confidence=(
|
| 279 |
+
max(voting.get("buy_count", 0), voting.get("sell_count", 0)) /
|
| 280 |
+
(voting.get("buy_count", 0) + voting.get("sell_count", 0) or 1)
|
| 281 |
+
)
|
| 282 |
+
)
|
| 283 |
+
|
| 284 |
+
except json.JSONDecodeError:
|
| 285 |
+
logger.warning("[HubSubscriber] Malformed JSON")
|
| 286 |
+
except Exception as e:
|
| 287 |
+
logger.error(f"[HubSubscriber] Message error: {e}")
|
| 288 |
+
|
| 289 |
+
def _on_error(self, ws, error) -> None:
|
| 290 |
+
logger.error(f"[HubSubscriber] WS error: {error}")
|
| 291 |
+
self.state.hub_connected = False
|
| 292 |
+
if self.ranker_logger:
|
| 293 |
+
self.ranker_logger.connection_event("Hub WebSocket", "error", str(error))
|
| 294 |
+
|
| 295 |
+
def _on_close(self, ws, code, msg) -> None:
|
| 296 |
+
self.state.hub_connected = False
|
| 297 |
+
logger.info(f"[HubSubscriber] Connection closed (code={code})")
|
| 298 |
+
if self.ranker_logger:
|
| 299 |
+
self.ranker_logger.connection_event("Hub WebSocket", "disconnected")
|
| 300 |
+
|
| 301 |
+
|
| 302 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 303 |
+
# SECTION 3 β FLASK APP
|
| 304 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 305 |
+
|
| 306 |
+
_state = DashboardState()
|
| 307 |
+
_ranker_logger = RankerLogger(
|
| 308 |
+
buffer_size = 1000,
|
| 309 |
+
log_dir = "./ranker_logs",
|
| 310 |
+
on_event = None, # Optional: push to WebSocket here
|
| 311 |
+
)
|
| 312 |
+
|
| 313 |
+
app = Flask(__name__)
|
| 314 |
+
CORS(app)
|
| 315 |
+
|
| 316 |
+
# Register ranker logs API
|
| 317 |
+
init_ranker_logs_api(_ranker_logger)
|
| 318 |
+
app.register_blueprint(ranker_logs_bp)
|
| 319 |
+
|
| 320 |
+
|
| 321 |
+
@app.route("/")
|
| 322 |
+
def index():
|
| 323 |
+
html_path = Path(_HTML_PATH)
|
| 324 |
+
if html_path.exists():
|
| 325 |
+
return send_from_directory(str(html_path.parent), html_path.name)
|
| 326 |
+
return (
|
| 327 |
+
"<h1>hub_dashboard.html not found</h1>"
|
| 328 |
+
f"<p>Expected: <code>{_HTML_PATH}</code></p>",
|
| 329 |
+
404,
|
| 330 |
+
)
|
| 331 |
+
|
| 332 |
+
|
| 333 |
+
@app.route("/api/state")
|
| 334 |
+
def api_state():
|
| 335 |
+
"""Full dashboard state β polled by hub_dashboard.html every 2 s."""
|
| 336 |
+
return jsonify(_state.get_state())
|
| 337 |
+
|
| 338 |
+
|
| 339 |
+
@app.route("/api/rankings")
|
| 340 |
+
def api_rankings():
|
| 341 |
+
return jsonify({"rankings": _state.get_rankings()})
|
| 342 |
+
|
| 343 |
+
|
| 344 |
+
@app.route("/api/metrics/history")
|
| 345 |
+
def api_metrics_history():
|
| 346 |
+
limit = int(request.args.get("limit", 100))
|
| 347 |
+
return jsonify(_state.get_metric_history(limit=limit))
|
| 348 |
+
|
| 349 |
+
|
| 350 |
+
@app.route("/api/health")
|
| 351 |
+
def api_health():
|
| 352 |
+
return jsonify({
|
| 353 |
+
"service": "hub_dashboard_service",
|
| 354 |
+
"version": "v1.1",
|
| 355 |
+
"timestamp": datetime.utcnow().isoformat() + "Z",
|
| 356 |
+
**_state.get_health(),
|
| 357 |
+
})
|
| 358 |
+
|
| 359 |
+
|
| 360 |
+
def _start_background_services() -> None:
|
| 361 |
+
sub = HubSubscriber(_state, hub_host=_HUB_HOST, ranker_logger=_ranker_logger)
|
| 362 |
+
sub.start()
|
| 363 |
+
|
| 364 |
+
global _subscriber
|
| 365 |
+
_subscriber = sub
|
| 366 |
+
|
| 367 |
+
|
| 368 |
+
_subscriber = None
|
| 369 |
+
|
| 370 |
+
_start_background_services()
|
| 371 |
+
|
| 372 |
+
logger.info("=" * 70)
|
| 373 |
+
logger.info(f"K1RL QUASAR β Hub Dashboard Service (v1.1 with Ranker Logs)")
|
| 374 |
+
logger.info(f"Hub WebSocket : wss://{_HUB_HOST}/ws/subscribe")
|
| 375 |
+
logger.info(f"Dashboard HTML : {_HTML_PATH}")
|
| 376 |
+
logger.info(f"Service port : {_DASHBOARD_PORT}")
|
| 377 |
+
logger.info(f"Logs API : /api/ranker/logs/*")
|
| 378 |
+
logger.info(f"Log directory : ./ranker_logs")
|
| 379 |
+
logger.info("=" * 70)
|
| 380 |
+
|
| 381 |
+
|
| 382 |
+
if __name__ == "__main__":
|
| 383 |
+
app.run(
|
| 384 |
+
host = "0.0.0.0",
|
| 385 |
+
port = _DASHBOARD_PORT,
|
| 386 |
+
debug = False,
|
| 387 |
+
threaded = True,
|
| 388 |
+
)
|
ranker_logging.py
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import logging
|
| 2 |
+
import sys
|
| 3 |
+
from enum import Enum, auto
|
| 4 |
+
|
| 5 |
+
class LogLevel(Enum):
|
| 6 |
+
DEBUG = auto()
|
| 7 |
+
INFO = auto()
|
| 8 |
+
WARNING = auto()
|
| 9 |
+
ERROR = auto()
|
| 10 |
+
CRITICAL = auto()
|
| 11 |
+
|
| 12 |
+
class EventCategory(Enum):
|
| 13 |
+
INITIALIZATION = 'INITIALIZATION'
|
| 14 |
+
PROCESSING = 'PROCESSING'
|
| 15 |
+
TERMINATION = 'TERMINATION'
|
| 16 |
+
ERROR_OCCURRED = 'ERROR_OCCURRED'
|
| 17 |
+
|
| 18 |
+
class LogEntry:
|
| 19 |
+
def __init__(self, level: LogLevel, category: EventCategory, message: str):
|
| 20 |
+
self.level = level
|
| 21 |
+
self.category = category
|
| 22 |
+
self.message = message
|
| 23 |
+
self.timestamp = logging.Formatter.formatTime(logging.LogRecord('', 0, '', 0, message, None, None))
|
| 24 |
+
|
| 25 |
+
def __str__(self):
|
| 26 |
+
return f"[{self.timestamp}] [{self.level.name}] [{self.category.value}] {self.message}"
|
| 27 |
+
|
| 28 |
+
class RankerLogger:
|
| 29 |
+
def __init__(self, name: str):
|
| 30 |
+
self.logger = logging.getLogger(name)
|
| 31 |
+
self.logger.setLevel(logging.DEBUG)
|
| 32 |
+
ch = logging.StreamHandler(sys.stdout)
|
| 33 |
+
ch.setLevel(logging.DEBUG)
|
| 34 |
+
formatter = logging.Formatter('%(message)s')
|
| 35 |
+
ch.setFormatter(formatter)
|
| 36 |
+
self.logger.addHandler(ch)
|
| 37 |
+
|
| 38 |
+
def log(self, level: LogLevel, category: EventCategory, message: str):
|
| 39 |
+
entry = LogEntry(level, category, message)
|
| 40 |
+
if level == LogLevel.DEBUG:
|
| 41 |
+
self.logger.debug(str(entry))
|
| 42 |
+
elif level == LogLevel.INFO:
|
| 43 |
+
self.logger.info(str(entry))
|
| 44 |
+
elif level == LogLevel.WARNING:
|
| 45 |
+
self.logger.warning(str(entry))
|
| 46 |
+
elif level == LogLevel.ERROR:
|
| 47 |
+
self.logger.error(str(entry))
|
| 48 |
+
elif level == LogLevel.CRITICAL:
|
| 49 |
+
self.logger.critical(str(entry))
|
| 50 |
+
|
| 51 |
+
class RankerLogBridge:
|
| 52 |
+
def __init__(self, ranker_logger: RankerLogger):
|
| 53 |
+
self.ranker_logger = ranker_logger
|
| 54 |
+
|
| 55 |
+
def log_event(self, level: LogLevel, category: EventCategory, message: str):
|
| 56 |
+
self.ranker_logger.log(level, category, message)
|
ranker_logs_api.py
ADDED
|
@@ -0,0 +1,120 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 4 |
+
β QUASAR RANKER β LOGS REST API β
|
| 5 |
+
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
|
| 6 |
+
β Flask blueprint for exposing ranker logs via REST endpoints. β
|
| 7 |
+
β VERSION: v1.0 | 2026-03-26 β
|
| 8 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 9 |
+
"""
|
| 10 |
+
|
| 11 |
+
from flask import Blueprint, jsonify, request, send_file
|
| 12 |
+
from typing import Optional
|
| 13 |
+
from pathlib import Path
|
| 14 |
+
|
| 15 |
+
ranker_logs_bp = Blueprint("ranker_logs", __name__, url_prefix="/api/ranker/logs")
|
| 16 |
+
|
| 17 |
+
_logger: Optional[object] = None
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def init_ranker_logs_api(ranker_logger):
|
| 21 |
+
"""Call this from hub_dashboard_service.py during initialization."""
|
| 22 |
+
global _logger
|
| 23 |
+
_logger = ranker_logger
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
@ranker_logs_bp.route("/recent", methods=["GET"])
|
| 27 |
+
def get_recent_logs():
|
| 28 |
+
"""GET /api/ranker/logs/recent?limit=50&category=signal"""
|
| 29 |
+
if not _logger:
|
| 30 |
+
return jsonify({"error": "Logger not initialized"}), 500
|
| 31 |
+
|
| 32 |
+
limit = int(request.args.get("limit", 50))
|
| 33 |
+
category = request.args.get("category")
|
| 34 |
+
|
| 35 |
+
entries = _logger.get_recent(n=limit, category=category)
|
| 36 |
+
return jsonify({
|
| 37 |
+
"logs": entries,
|
| 38 |
+
"count": len(entries),
|
| 39 |
+
"stats": _logger.get_stats(),
|
| 40 |
+
})
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
@ranker_logs_bp.route("/asset/<asset>", methods=["GET"])
|
| 44 |
+
def get_asset_logs(asset: str):
|
| 45 |
+
"""GET /api/ranker/logs/asset/V75?limit=30"""
|
| 46 |
+
if not _logger:
|
| 47 |
+
return jsonify({"error": "Logger not initialized"}), 500
|
| 48 |
+
|
| 49 |
+
limit = int(request.args.get("limit", 30))
|
| 50 |
+
entries = _logger.get_by_asset(asset, n=limit)
|
| 51 |
+
|
| 52 |
+
return jsonify({
|
| 53 |
+
"asset": asset,
|
| 54 |
+
"logs": entries,
|
| 55 |
+
"count": len(entries),
|
| 56 |
+
})
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
@ranker_logs_bp.route("/level/<level>", methods=["GET"])
|
| 60 |
+
def get_level_logs(level: str):
|
| 61 |
+
"""GET /api/ranker/logs/level/ERROR?limit=50"""
|
| 62 |
+
if not _logger:
|
| 63 |
+
return jsonify({"error": "Logger not initialized"}), 500
|
| 64 |
+
|
| 65 |
+
limit = int(request.args.get("limit", 50))
|
| 66 |
+
entries = _logger.get_by_level(level, n=limit)
|
| 67 |
+
|
| 68 |
+
return jsonify({
|
| 69 |
+
"level": level.upper(),
|
| 70 |
+
"logs": entries,
|
| 71 |
+
"count": len(entries),
|
| 72 |
+
})
|
| 73 |
+
|
| 74 |
+
|
| 75 |
+
@ranker_logs_bp.route("/stats", methods=["GET"])
|
| 76 |
+
def get_log_stats():
|
| 77 |
+
"""GET /api/ranker/logs/stats"""
|
| 78 |
+
if not _logger:
|
| 79 |
+
return jsonify({"error": "Logger not initialized"}), 500
|
| 80 |
+
|
| 81 |
+
return jsonify(_logger.get_stats())
|
| 82 |
+
|
| 83 |
+
|
| 84 |
+
@ranker_logs_bp.route("/export", methods=["GET"])
|
| 85 |
+
def export_logs():
|
| 86 |
+
"""GET /api/ranker/logs/export?limit=500 β download JSON"""
|
| 87 |
+
if not _logger:
|
| 88 |
+
return jsonify({"error": "Logger not initialized"}), 500
|
| 89 |
+
|
| 90 |
+
limit = int(request.args.get("limit", 500))
|
| 91 |
+
export_path = Path("/tmp/ranker_logs_export.json")
|
| 92 |
+
|
| 93 |
+
try:
|
| 94 |
+
_logger.export_json(str(export_path), n=limit)
|
| 95 |
+
return send_file(
|
| 96 |
+
export_path,
|
| 97 |
+
mimetype="application/json",
|
| 98 |
+
as_attachment=True,
|
| 99 |
+
download_name="ranker_logs_export.json"
|
| 100 |
+
)
|
| 101 |
+
except Exception as e:
|
| 102 |
+
return jsonify({"error": str(e)}), 500
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
@ranker_logs_bp.route("/clear", methods=["POST"])
|
| 106 |
+
def clear_logs():
|
| 107 |
+
"""POST /api/ranker/logs/clear β Clear in-memory buffer"""
|
| 108 |
+
if not _logger:
|
| 109 |
+
return jsonify({"error": "Logger not initialized"}), 500
|
| 110 |
+
|
| 111 |
+
try:
|
| 112 |
+
_logger.clear_buffer()
|
| 113 |
+
return jsonify({"status": "cleared"})
|
| 114 |
+
except Exception as e:
|
| 115 |
+
return jsonify({"error": str(e)}), 500
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
@ranker_logs_bp.errorhandler(404)
|
| 119 |
+
def not_found(error):
|
| 120 |
+
return jsonify({"error": "Endpoint not found"}), 404
|
requirements.txt
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 2 |
+
# K1RL QUASAR β MINIMAL REQUIREMENTS
|
| 3 |
+
# Only packages actually used in the codebase
|
| 4 |
+
# VERSION: 2.2 | 2026-03-29
|
| 5 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 6 |
+
|
| 7 |
+
# Core ML/Neural Network
|
| 8 |
+
torch==2.1.0
|
| 9 |
+
numpy==1.24.3
|
| 10 |
+
|
| 11 |
+
# Web Framework & WebSocket
|
| 12 |
+
fastapi==0.104.0
|
| 13 |
+
uvicorn==0.24.0
|
| 14 |
+
websockets==14.0
|
| 15 |
+
websocket-client==1.6.0
|
| 16 |
+
|
| 17 |
+
# Dashboard UI
|
| 18 |
+
flask==3.0.0
|
| 19 |
+
flask-cors==4.0.0
|
| 20 |
+
|
| 21 |
+
# Process Management
|
| 22 |
+
supervisor==4.2.4
|
| 23 |
+
|
| 24 |
+
# Async Utilities
|
| 25 |
+
nest_asyncio==1.5.6
|
| 26 |
+
|
| 27 |
+
# Data Serialization
|
| 28 |
+
orjson==3.9.0
|
| 29 |
+
|
| 30 |
+
# Environment Management
|
| 31 |
+
python-dotenv==1.0.0
|
| 32 |
+
|
| 33 |
+
# HuggingFace β persistent checkpoint storage
|
| 34 |
+
huggingface_hub==0.20.3
|
supervisord.conf
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[supervisord]
|
| 2 |
+
nodaemon=true
|
| 3 |
+
user=root
|
| 4 |
+
logfile=/dev/null
|
| 5 |
+
logfile_maxbytes=0
|
| 6 |
+
pidfile=/tmp/supervisord.pid
|
| 7 |
+
loglevel=info
|
| 8 |
+
|
| 9 |
+
[program:app]
|
| 10 |
+
command=python /app/websocket_hub.py
|
| 11 |
+
directory=/app
|
| 12 |
+
autostart=true
|
| 13 |
+
autorestart=true
|
| 14 |
+
startsecs=5
|
| 15 |
+
startretries=10
|
| 16 |
+
stopasgroup=true
|
| 17 |
+
killasgroup=true
|
| 18 |
+
stdout_logfile=/dev/stdout
|
| 19 |
+
stdout_logfile_maxbytes=0
|
| 20 |
+
stderr_logfile=/dev/stderr
|
| 21 |
+
stderr_logfile_maxbytes=0
|
| 22 |
+
environment=PORT=7860,DASHBOARD_HTML=/app/hub_dashboard.html,PYTHONUNBUFFERED=1
|
| 23 |
+
|
| 24 |
+
[program:worker1]
|
| 25 |
+
command=python /app/Quasar_axrvi_ranker.py --hub ws://localhost:7860/ws/subscribe --sync
|
| 26 |
+
directory=/app
|
| 27 |
+
autostart=true
|
| 28 |
+
autorestart=true
|
| 29 |
+
startsecs=5
|
| 30 |
+
startretries=10
|
| 31 |
+
stopasgroup=true
|
| 32 |
+
killasgroup=true
|
| 33 |
+
stdout_logfile=/dev/stdout
|
| 34 |
+
stdout_logfile_maxbytes=0
|
| 35 |
+
stderr_logfile=/dev/stderr
|
| 36 |
+
stderr_logfile_maxbytes=0
|
| 37 |
+
environment=PYTHONUNBUFFERED=1
|
| 38 |
+
|
| 39 |
+
[group:space]
|
| 40 |
+
programs=app,worker1
|
| 41 |
+
priority=999
|
websocket_hub.py
ADDED
|
@@ -0,0 +1,553 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 4 |
+
β K1RL QUASAR β CENTRAL WEBSOCKET HUB β
|
| 5 |
+
β ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
|
| 6 |
+
β β
|
| 7 |
+
β Architecture role: INGEST β NORMALIZE β BROADCAST β
|
| 8 |
+
β β
|
| 9 |
+
β β’ Accepts publisher connections from Asset Spaces (/ws/publish/{space_name}) β
|
| 10 |
+
β β’ Accepts subscriber connections from Ranker Space (/ws/subscribe) β
|
| 11 |
+
β β’ ONE-WAY: Publisher β Hub β Subscriber β
|
| 12 |
+
β β’ Hub NEVER writes back to publishers β
|
| 13 |
+
β β’ Hub stores latest snapshot per asset (NO history) β
|
| 14 |
+
β β
|
| 15 |
+
β STRICT DATA MODEL (training + voting fields ONLY): β
|
| 16 |
+
β training: training_steps, actor_loss, critic_loss, avn_loss, avn_accuracy β
|
| 17 |
+
β voting: dominant_signal, buy_count, sell_count β
|
| 18 |
+
β β
|
| 19 |
+
β VERSION: v2.0-arch-strict | 2026-03-25 β
|
| 20 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 21 |
+
"""
|
| 22 |
+
|
| 23 |
+
import asyncio
|
| 24 |
+
import copy
|
| 25 |
+
import json
|
| 26 |
+
import logging
|
| 27 |
+
import os
|
| 28 |
+
import time
|
| 29 |
+
from datetime import datetime
|
| 30 |
+
from pathlib import Path
|
| 31 |
+
from typing import Dict, List, Optional, Set
|
| 32 |
+
|
| 33 |
+
import uvicorn
|
| 34 |
+
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
|
| 35 |
+
from fastapi.middleware.cors import CORSMiddleware
|
| 36 |
+
from fastapi.responses import FileResponse, JSONResponse
|
| 37 |
+
|
| 38 |
+
# βββ Logging ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 39 |
+
logging.basicConfig(
|
| 40 |
+
level=logging.INFO,
|
| 41 |
+
format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
|
| 42 |
+
datefmt="%Y-%m-%d %H:%M:%S",
|
| 43 |
+
)
|
| 44 |
+
logger = logging.getLogger("QuasarHub")
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 48 |
+
# SECTION 1 β STRICT DATA MODEL
|
| 49 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 50 |
+
|
| 51 |
+
# Fields that are permitted through the hub. All others are silently dropped.
|
| 52 |
+
_ALLOWED_TRAINING_FIELDS: frozenset = frozenset({
|
| 53 |
+
"training_steps",
|
| 54 |
+
"actor_loss",
|
| 55 |
+
"critic_loss",
|
| 56 |
+
"avn_loss",
|
| 57 |
+
"avn_accuracy",
|
| 58 |
+
})
|
| 59 |
+
|
| 60 |
+
_ALLOWED_VOTING_FIELDS: frozenset = frozenset({
|
| 61 |
+
"dominant_signal",
|
| 62 |
+
"buy_count",
|
| 63 |
+
"sell_count",
|
| 64 |
+
"last_price", # FIX: was silently dropped β needed by subscribers
|
| 65 |
+
"signal_source", # FIX: was silently dropped β needed by subscribers
|
| 66 |
+
})
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
def _empty_snapshot(space_name: str) -> dict:
|
| 70 |
+
"""Return a clean, zeroed snapshot for a space."""
|
| 71 |
+
return {
|
| 72 |
+
"space_name": space_name,
|
| 73 |
+
"last_updated": 0.0,
|
| 74 |
+
"training": {
|
| 75 |
+
"training_steps": 0,
|
| 76 |
+
"actor_loss": 0.0,
|
| 77 |
+
"critic_loss": 0.0,
|
| 78 |
+
"avn_loss": 0.0,
|
| 79 |
+
"avn_accuracy": 0.0,
|
| 80 |
+
},
|
| 81 |
+
"voting": {
|
| 82 |
+
"dominant_signal": "NEUTRAL",
|
| 83 |
+
"buy_count": 0,
|
| 84 |
+
"sell_count": 0,
|
| 85 |
+
"last_price": 0.0, # FIX: added to match allowed fields
|
| 86 |
+
"signal_source": "LOG", # FIX: added to match allowed fields
|
| 87 |
+
},
|
| 88 |
+
}
|
| 89 |
+
|
| 90 |
+
|
| 91 |
+
def _validate_and_normalize(space_name: str, raw: dict) -> Optional[dict]:
|
| 92 |
+
"""
|
| 93 |
+
Validate incoming payload. Return a clean normalized dict or None if invalid.
|
| 94 |
+
|
| 95 |
+
Strict rules:
|
| 96 |
+
β’ Must contain at least one of 'training' or 'voting' keys.
|
| 97 |
+
β’ Unknown top-level keys are dropped.
|
| 98 |
+
β’ Unknown sub-keys inside training/voting are dropped.
|
| 99 |
+
β’ Values are coerced to the correct Python types; malformed values are zeroed.
|
| 100 |
+
"""
|
| 101 |
+
training_raw = raw.get("training", {})
|
| 102 |
+
voting_raw = raw.get("voting", {})
|
| 103 |
+
|
| 104 |
+
if not isinstance(training_raw, dict):
|
| 105 |
+
training_raw = {}
|
| 106 |
+
if not isinstance(voting_raw, dict):
|
| 107 |
+
voting_raw = {}
|
| 108 |
+
|
| 109 |
+
if not training_raw and not voting_raw:
|
| 110 |
+
return None # Nothing useful
|
| 111 |
+
|
| 112 |
+
# --- training ---
|
| 113 |
+
def _float(v, default: float = 0.0) -> float:
|
| 114 |
+
try:
|
| 115 |
+
return float(v)
|
| 116 |
+
except (TypeError, ValueError):
|
| 117 |
+
return default
|
| 118 |
+
|
| 119 |
+
def _int(v, default: int = 0) -> int:
|
| 120 |
+
try:
|
| 121 |
+
return int(v)
|
| 122 |
+
except (TypeError, ValueError):
|
| 123 |
+
return default
|
| 124 |
+
|
| 125 |
+
training: dict = {}
|
| 126 |
+
if training_raw:
|
| 127 |
+
training = {
|
| 128 |
+
"training_steps": _int(training_raw.get("training_steps", 0)),
|
| 129 |
+
"actor_loss": _float(training_raw.get("actor_loss", 0.0)),
|
| 130 |
+
"critic_loss": _float(training_raw.get("critic_loss", 0.0)),
|
| 131 |
+
"avn_loss": _float(training_raw.get("avn_loss", 0.0)),
|
| 132 |
+
"avn_accuracy": max(0.0, min(1.0, _float(training_raw.get("avn_accuracy", 0.0)))),
|
| 133 |
+
}
|
| 134 |
+
|
| 135 |
+
# --- voting ---
|
| 136 |
+
voting: dict = {}
|
| 137 |
+
if voting_raw:
|
| 138 |
+
raw_signal = voting_raw.get("dominant_signal", "NEUTRAL")
|
| 139 |
+
if not isinstance(raw_signal, str):
|
| 140 |
+
raw_signal = "NEUTRAL"
|
| 141 |
+
raw_source = voting_raw.get("signal_source", "LOG")
|
| 142 |
+
if not isinstance(raw_source, str):
|
| 143 |
+
raw_source = "LOG"
|
| 144 |
+
voting = {
|
| 145 |
+
"dominant_signal": raw_signal.upper() if raw_signal.upper() in {"BUY", "SELL", "NEUTRAL"} else "NEUTRAL",
|
| 146 |
+
"buy_count": _int(voting_raw.get("buy_count", 0)),
|
| 147 |
+
"sell_count": _int(voting_raw.get("sell_count", 0)),
|
| 148 |
+
"last_price": _float(voting_raw.get("last_price", 0.0)), # FIX: now passed through
|
| 149 |
+
"signal_source": raw_source, # FIX: now passed through
|
| 150 |
+
}
|
| 151 |
+
|
| 152 |
+
return {
|
| 153 |
+
"space_name": space_name,
|
| 154 |
+
"training": training,
|
| 155 |
+
"voting": voting,
|
| 156 |
+
}
|
| 157 |
+
|
| 158 |
+
|
| 159 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 160 |
+
# SECTION 2 β CONNECTION MANAGER
|
| 161 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 162 |
+
|
| 163 |
+
class ConnectionManager:
|
| 164 |
+
"""
|
| 165 |
+
Manages publisher (Asset Space) and subscriber (Ranker) WebSocket connections.
|
| 166 |
+
|
| 167 |
+
Design:
|
| 168 |
+
β’ Publishers β write-only lane (/ws/publish/{space_name})
|
| 169 |
+
β’ Subscribers β read-only lane (/ws/subscribe)
|
| 170 |
+
β’ No cross-talk: subscribers never send to publishers
|
| 171 |
+
"""
|
| 172 |
+
|
| 173 |
+
def __init__(self):
|
| 174 |
+
# Keyed by space_name β WebSocket
|
| 175 |
+
self._publishers: Dict[str, WebSocket] = {}
|
| 176 |
+
# Set of subscriber sockets
|
| 177 |
+
self._subscribers: Set[WebSocket] = set()
|
| 178 |
+
# Latest normalized snapshot per space
|
| 179 |
+
self._snapshots: Dict[str, dict] = {}
|
| 180 |
+
# Asyncio lock for thread-safe mutation
|
| 181 |
+
self._lock = asyncio.Lock()
|
| 182 |
+
# Message counter (read by /api/state)
|
| 183 |
+
self._total_ingested: int = 0
|
| 184 |
+
|
| 185 |
+
# ββ Publisher lifecycle ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 186 |
+
|
| 187 |
+
async def register_publisher(self, space_name: str, ws: WebSocket) -> None:
|
| 188 |
+
await ws.accept()
|
| 189 |
+
async with self._lock:
|
| 190 |
+
self._publishers[space_name] = ws
|
| 191 |
+
if space_name not in self._snapshots:
|
| 192 |
+
self._snapshots[space_name] = _empty_snapshot(space_name)
|
| 193 |
+
logger.info(f"π‘ Publisher connected: {space_name} "
|
| 194 |
+
f"(total={len(self._publishers)})")
|
| 195 |
+
|
| 196 |
+
async def unregister_publisher(self, space_name: str) -> None:
|
| 197 |
+
async with self._lock:
|
| 198 |
+
self._publishers.pop(space_name, None)
|
| 199 |
+
logger.info(f"π‘ Publisher disconnected: {space_name}")
|
| 200 |
+
|
| 201 |
+
# ββ Subscriber lifecycle βββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 202 |
+
|
| 203 |
+
async def register_subscriber(self, ws: WebSocket) -> None:
|
| 204 |
+
await ws.accept()
|
| 205 |
+
async with self._lock:
|
| 206 |
+
self._subscribers.add(ws)
|
| 207 |
+
logger.info(f"π Subscriber connected (total={len(self._subscribers)})")
|
| 208 |
+
|
| 209 |
+
async def unregister_subscriber(self, ws: WebSocket) -> None:
|
| 210 |
+
async with self._lock:
|
| 211 |
+
self._subscribers.discard(ws)
|
| 212 |
+
logger.info(f"π Subscriber disconnected (total={len(self._subscribers)})")
|
| 213 |
+
|
| 214 |
+
# ββ Ingestion pipeline βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 215 |
+
|
| 216 |
+
async def ingest(self, space_name: str, raw_payload: dict) -> None:
|
| 217 |
+
"""
|
| 218 |
+
Validate β Normalize β Store β Broadcast pipeline.
|
| 219 |
+
Called for every inbound message from a publisher.
|
| 220 |
+
"""
|
| 221 |
+
normalized = _validate_and_normalize(space_name, raw_payload)
|
| 222 |
+
if normalized is None:
|
| 223 |
+
logger.debug(f"[{space_name}] Payload dropped (no valid fields)")
|
| 224 |
+
return
|
| 225 |
+
|
| 226 |
+
# β
FIX #6: deep-copy the snapshot *inside* the lock before releasing.
|
| 227 |
+
# Without this, snap is a live reference. Another publisher coroutine
|
| 228 |
+
# can acquire the lock and mutate snap["training"] / snap["voting"]
|
| 229 |
+
# in-place while json.dumps(snapshot) is still serializing it during
|
| 230 |
+
# broadcast, producing torn / mixed data sent to subscribers.
|
| 231 |
+
async with self._lock:
|
| 232 |
+
snap = self._snapshots.setdefault(space_name, _empty_snapshot(space_name))
|
| 233 |
+
snap["last_updated"] = time.time()
|
| 234 |
+
|
| 235 |
+
if normalized["training"]:
|
| 236 |
+
snap["training"].update(normalized["training"])
|
| 237 |
+
if normalized["voting"]:
|
| 238 |
+
snap["voting"].update(normalized["voting"])
|
| 239 |
+
|
| 240 |
+
self._total_ingested += 1
|
| 241 |
+
snap_copy = copy.deepcopy(snap) # frozen snapshot for broadcast
|
| 242 |
+
|
| 243 |
+
# Broadcast outside the lock using the immutable copy
|
| 244 |
+
await self._broadcast_update(space_name, snap_copy)
|
| 245 |
+
|
| 246 |
+
# ββ Broadcast engine βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 247 |
+
|
| 248 |
+
async def _broadcast_update(self, space_name: str, snapshot: dict) -> None:
|
| 249 |
+
"""Fan-out the updated snapshot to all subscribers (event-driven)."""
|
| 250 |
+
if not self._subscribers:
|
| 251 |
+
return
|
| 252 |
+
|
| 253 |
+
message = json.dumps({
|
| 254 |
+
"type": "metrics_update",
|
| 255 |
+
"space_name": space_name,
|
| 256 |
+
"snapshot": snapshot,
|
| 257 |
+
"hub_timestamp": time.time(),
|
| 258 |
+
})
|
| 259 |
+
|
| 260 |
+
dead: list = []
|
| 261 |
+
for ws in list(self._subscribers):
|
| 262 |
+
try:
|
| 263 |
+
await ws.send_text(message)
|
| 264 |
+
except Exception:
|
| 265 |
+
dead.append(ws)
|
| 266 |
+
|
| 267 |
+
if dead:
|
| 268 |
+
async with self._lock:
|
| 269 |
+
for ws in dead:
|
| 270 |
+
self._subscribers.discard(ws)
|
| 271 |
+
|
| 272 |
+
async def send_initial_state(self, ws: WebSocket) -> None:
|
| 273 |
+
"""Send full current state to a newly connected subscriber."""
|
| 274 |
+
async with self._lock:
|
| 275 |
+
snapshots_copy = dict(self._snapshots)
|
| 276 |
+
|
| 277 |
+
message = json.dumps({
|
| 278 |
+
"type": "initial_state",
|
| 279 |
+
"snapshots": snapshots_copy,
|
| 280 |
+
"hub_timestamp": time.time(),
|
| 281 |
+
})
|
| 282 |
+
await ws.send_text(message)
|
| 283 |
+
|
| 284 |
+
# ββ Read-only accessors (REST API) βββββββββββββββββββββββββββββββββββββββββββββββ
|
| 285 |
+
|
| 286 |
+
def get_snapshot(self, space_name: str) -> Optional[dict]:
|
| 287 |
+
return self._snapshots.get(space_name)
|
| 288 |
+
|
| 289 |
+
def get_all_snapshots(self) -> dict:
|
| 290 |
+
return dict(self._snapshots)
|
| 291 |
+
|
| 292 |
+
def get_health(self) -> dict:
|
| 293 |
+
now = time.time()
|
| 294 |
+
return {
|
| 295 |
+
"publishers": {
|
| 296 |
+
name: {
|
| 297 |
+
"last_updated": self._snapshots.get(name, {}).get("last_updated", 0),
|
| 298 |
+
"stale_seconds": round(now - self._snapshots.get(name, {}).get("last_updated", now), 1),
|
| 299 |
+
}
|
| 300 |
+
for name in self._publishers
|
| 301 |
+
},
|
| 302 |
+
"subscriber_count": len(self._subscribers),
|
| 303 |
+
}
|
| 304 |
+
|
| 305 |
+
|
| 306 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 307 |
+
# SECTION 3 β FASTAPI APPLICATION
|
| 308 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 309 |
+
|
| 310 |
+
app = FastAPI(
|
| 311 |
+
title="K1RL QUASAR Hub",
|
| 312 |
+
description="Central WebSocket hub β ingest, normalize, broadcast (one-way)",
|
| 313 |
+
version="2.0.0",
|
| 314 |
+
)
|
| 315 |
+
app.add_middleware(
|
| 316 |
+
CORSMiddleware,
|
| 317 |
+
allow_origins=["*"],
|
| 318 |
+
allow_methods=["*"],
|
| 319 |
+
allow_headers=["*"],
|
| 320 |
+
)
|
| 321 |
+
|
| 322 |
+
manager = ConnectionManager()
|
| 323 |
+
|
| 324 |
+
|
| 325 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 326 |
+
# SECTION 4 β WEBSOCKET ENDPOINTS
|
| 327 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 328 |
+
|
| 329 |
+
@app.websocket("/ws/publish/{space_name}")
|
| 330 |
+
async def ws_publisher_endpoint(websocket: WebSocket, space_name: str):
|
| 331 |
+
"""
|
| 332 |
+
Publisher endpoint β Asset Spaces connect here.
|
| 333 |
+
SEND ONLY: hub never writes back to this socket.
|
| 334 |
+
|
| 335 |
+
Accepted message types:
|
| 336 |
+
{"type": "metrics", "training": {...}, "voting": {...}}
|
| 337 |
+
{"type": "training", "data": {...}}
|
| 338 |
+
{"type": "voting", "data": {...}}
|
| 339 |
+
{"type": "heartbeat"}
|
| 340 |
+
"""
|
| 341 |
+
await manager.register_publisher(space_name, websocket)
|
| 342 |
+
try:
|
| 343 |
+
while True:
|
| 344 |
+
raw_text = await websocket.receive_text()
|
| 345 |
+
try:
|
| 346 |
+
data = json.loads(raw_text)
|
| 347 |
+
except json.JSONDecodeError:
|
| 348 |
+
logger.warning(f"[{space_name}] Malformed JSON β skipped")
|
| 349 |
+
continue
|
| 350 |
+
|
| 351 |
+
msg_type = data.get("type", "")
|
| 352 |
+
|
| 353 |
+
if msg_type == "metrics":
|
| 354 |
+
# Full combined payload
|
| 355 |
+
await manager.ingest(space_name, {
|
| 356 |
+
"training": data.get("training", {}),
|
| 357 |
+
"voting": data.get("voting", {}),
|
| 358 |
+
})
|
| 359 |
+
|
| 360 |
+
elif msg_type == "training":
|
| 361 |
+
await manager.ingest(space_name, {
|
| 362 |
+
"training": data.get("data", {}),
|
| 363 |
+
"voting": {},
|
| 364 |
+
})
|
| 365 |
+
|
| 366 |
+
elif msg_type == "voting":
|
| 367 |
+
await manager.ingest(space_name, {
|
| 368 |
+
"training": {},
|
| 369 |
+
"voting": data.get("data", {}),
|
| 370 |
+
})
|
| 371 |
+
|
| 372 |
+
elif msg_type in ("heartbeat", "identify", "ping"):
|
| 373 |
+
# Silently acknowledged β no reply sent back
|
| 374 |
+
pass
|
| 375 |
+
|
| 376 |
+
else:
|
| 377 |
+
logger.debug(f"[{space_name}] Unrecognised type '{msg_type}' β dropped")
|
| 378 |
+
|
| 379 |
+
except WebSocketDisconnect:
|
| 380 |
+
pass
|
| 381 |
+
except Exception as e:
|
| 382 |
+
logger.error(f"[{space_name}] Publisher error: {e}")
|
| 383 |
+
finally:
|
| 384 |
+
await manager.unregister_publisher(space_name)
|
| 385 |
+
|
| 386 |
+
|
| 387 |
+
@app.websocket("/ws/subscribe")
|
| 388 |
+
async def ws_subscriber_endpoint(websocket: WebSocket):
|
| 389 |
+
"""
|
| 390 |
+
Subscriber endpoint β Ranker Space connects here.
|
| 391 |
+
READ ONLY: subscribers must not send data; any inbound messages are discarded.
|
| 392 |
+
|
| 393 |
+
Messages received by subscriber:
|
| 394 |
+
{"type": "initial_state", "snapshots": {...}, "hub_timestamp": ...}
|
| 395 |
+
{"type": "metrics_update", "space_name": "...", "snapshot": {...}, "hub_timestamp": ...}
|
| 396 |
+
"""
|
| 397 |
+
await manager.register_subscriber(websocket)
|
| 398 |
+
await manager.send_initial_state(websocket)
|
| 399 |
+
try:
|
| 400 |
+
while True:
|
| 401 |
+
# Drain any inbound messages without processing them (read-only contract)
|
| 402 |
+
await websocket.receive_text()
|
| 403 |
+
except WebSocketDisconnect:
|
| 404 |
+
pass
|
| 405 |
+
except Exception as e:
|
| 406 |
+
logger.error(f"Subscriber error: {e}")
|
| 407 |
+
finally:
|
| 408 |
+
await manager.unregister_subscriber(websocket)
|
| 409 |
+
|
| 410 |
+
|
| 411 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 412 |
+
# SECTION 5 β REST API (READ-ONLY)
|
| 413 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 414 |
+
|
| 415 |
+
@app.get("/rankings")
|
| 416 |
+
async def get_rankings():
|
| 417 |
+
"""
|
| 418 |
+
Return latest snapshot for all assets.
|
| 419 |
+
Ranker may also poll this endpoint as a fallback.
|
| 420 |
+
"""
|
| 421 |
+
return {
|
| 422 |
+
"snapshots": manager.get_all_snapshots(),
|
| 423 |
+
"timestamp": datetime.utcnow().isoformat() + "Z",
|
| 424 |
+
}
|
| 425 |
+
|
| 426 |
+
|
| 427 |
+
@app.get("/metrics/{space_name}")
|
| 428 |
+
async def get_space_metrics(space_name: str):
|
| 429 |
+
"""Return latest snapshot for a single asset space."""
|
| 430 |
+
snap = manager.get_snapshot(space_name)
|
| 431 |
+
if snap is None:
|
| 432 |
+
return {"error": f"Unknown space: {space_name}"}
|
| 433 |
+
return snap
|
| 434 |
+
|
| 435 |
+
|
| 436 |
+
@app.get("/health")
|
| 437 |
+
async def get_health():
|
| 438 |
+
return {
|
| 439 |
+
"status": "ok",
|
| 440 |
+
"timestamp": datetime.utcnow().isoformat() + "Z",
|
| 441 |
+
**manager.get_health(),
|
| 442 |
+
}
|
| 443 |
+
|
| 444 |
+
|
| 445 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 446 |
+
# SECTION 6 β DASHBOARD UI ROUTES
|
| 447 |
+
# (serves hub_dashboard.html at / and provides /api/state for the polling frontend)
|
| 448 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 449 |
+
|
| 450 |
+
_HTML_PATH = Path(os.environ.get(
|
| 451 |
+
"DASHBOARD_HTML",
|
| 452 |
+
Path(__file__).parent / "hub_dashboard.html",
|
| 453 |
+
))
|
| 454 |
+
|
| 455 |
+
|
| 456 |
+
def _compute_rankings() -> List[dict]:
|
| 457 |
+
"""
|
| 458 |
+
Derive ranked list from manager snapshots using the AXRVI formula:
|
| 459 |
+
signal_confidence = max(buy, sell) / (buy + sell) if total > 0 else 0
|
| 460 |
+
score = signal_confidence - avn_accuracy
|
| 461 |
+
Sorted descending by score.
|
| 462 |
+
"""
|
| 463 |
+
ranked: List[dict] = []
|
| 464 |
+
for name, snap in manager.get_all_snapshots().items():
|
| 465 |
+
training = snap.get("training", {})
|
| 466 |
+
voting = snap.get("voting", {})
|
| 467 |
+
buy = voting.get("buy_count", 0)
|
| 468 |
+
sell = voting.get("sell_count", 0)
|
| 469 |
+
total = buy + sell
|
| 470 |
+
sig_conf = (max(buy, sell) / total) if total > 0 else 0.0
|
| 471 |
+
avn_acc = training.get("avn_accuracy", 0.0)
|
| 472 |
+
score = round(sig_conf - avn_acc, 6)
|
| 473 |
+
ranked.append({
|
| 474 |
+
"rank": 0,
|
| 475 |
+
"space_name": name,
|
| 476 |
+
"score": score,
|
| 477 |
+
"signal_confidence": round(sig_conf, 6),
|
| 478 |
+
"avn_accuracy": round(avn_acc, 6),
|
| 479 |
+
"dominant_signal": voting.get("dominant_signal", "NEUTRAL"),
|
| 480 |
+
"buy_count": buy,
|
| 481 |
+
"sell_count": sell,
|
| 482 |
+
"training_steps": training.get("training_steps", 0),
|
| 483 |
+
"actor_loss": training.get("actor_loss", 0.0),
|
| 484 |
+
"critic_loss": training.get("critic_loss", 0.0),
|
| 485 |
+
"avn_loss": training.get("avn_loss", 0.0),
|
| 486 |
+
"last_updated": snap.get("last_updated", 0.0),
|
| 487 |
+
})
|
| 488 |
+
ranked.sort(key=lambda r: r["score"], reverse=True)
|
| 489 |
+
for i, r in enumerate(ranked):
|
| 490 |
+
r["rank"] = i + 1
|
| 491 |
+
return ranked
|
| 492 |
+
|
| 493 |
+
|
| 494 |
+
@app.get("/")
|
| 495 |
+
async def serve_dashboard():
|
| 496 |
+
"""Serve the dashboard HTML. Returns a helpful message if the file is missing."""
|
| 497 |
+
if _HTML_PATH.exists():
|
| 498 |
+
return FileResponse(str(_HTML_PATH), media_type="text/html")
|
| 499 |
+
return JSONResponse(
|
| 500 |
+
status_code=200,
|
| 501 |
+
content={
|
| 502 |
+
"service": "K1RL QUASAR Hub",
|
| 503 |
+
"status": "running",
|
| 504 |
+
"note": "hub_dashboard.html not found β upload it to the Space",
|
| 505 |
+
"expected": str(_HTML_PATH),
|
| 506 |
+
"endpoints": ["/rankings", "/health", "/api/state", "/ws/publish/{space}", "/ws/subscribe"],
|
| 507 |
+
},
|
| 508 |
+
)
|
| 509 |
+
|
| 510 |
+
|
| 511 |
+
@app.get("/api/state")
|
| 512 |
+
async def api_state():
|
| 513 |
+
"""
|
| 514 |
+
Full dashboard state polled by hub_dashboard.html every 2 s.
|
| 515 |
+
Returns rankings + metric history stub + health.
|
| 516 |
+
"""
|
| 517 |
+
h = manager.get_health()
|
| 518 |
+
rankings = _compute_rankings()
|
| 519 |
+
return JSONResponse({
|
| 520 |
+
"rankings": rankings,
|
| 521 |
+
"metric_history": {}, # history kept client-side via snapshot deltas
|
| 522 |
+
"health": {
|
| 523 |
+
"hub_connected": True, # we ARE the hub
|
| 524 |
+
"spaces_connected": len(manager.get_all_snapshots()),
|
| 525 |
+
"messages_rx": manager._total_ingested,
|
| 526 |
+
"last_update_ts": max(
|
| 527 |
+
(s.get("last_updated", 0) for s in manager.get_all_snapshots().values()),
|
| 528 |
+
default=0.0,
|
| 529 |
+
),
|
| 530 |
+
"last_update_ago": round(
|
| 531 |
+
time.time() - max(
|
| 532 |
+
(s.get("last_updated", 0) for s in manager.get_all_snapshots().values()),
|
| 533 |
+
default=time.time(),
|
| 534 |
+
), 1
|
| 535 |
+
),
|
| 536 |
+
"uptime_seconds": round(time.time() - _START_TIME, 0),
|
| 537 |
+
"reconnect_count": 0,
|
| 538 |
+
},
|
| 539 |
+
"timestamp": datetime.utcnow().isoformat() + "Z",
|
| 540 |
+
})
|
| 541 |
+
|
| 542 |
+
|
| 543 |
+
_START_TIME = time.time()
|
| 544 |
+
|
| 545 |
+
|
| 546 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 547 |
+
# SECTION 7 β ENTRY POINT
|
| 548 |
+
# ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 549 |
+
|
| 550 |
+
if __name__ == "__main__":
|
| 551 |
+
port = int(os.environ.get("PORT", 7860))
|
| 552 |
+
logger.info(f"π QUASAR Hub starting on port {port}")
|
| 553 |
+
uvicorn.run(app, host="0.0.0.0", port=port, log_level="info")
|