docs/DEVELOPER_GUIDE.md

# Hermes Futures Desk — Complete Developer Guide

## 1. Purpose

Hermes Futures Desk is an authenticated Futures analysis and risk-management layer installed into the existing Hermes Agent runtime. It discovers markets, normalizes real market data, produces deterministic `LONG`, `SHORT`, or `NO_TRADE` outcomes, calculates a bounded trade plan, applies server-side risk controls, and optionally routes a fully revalidated plan to the existing Paper execution path.

The system is intentionally conservative:

- Datasource 4 is authoritative for Futures verification and safety.
- Binance public data may fill missing or unusable market fields but cannot override DS4 safety.
- Datasource 2 provides complementary context only.
- External AI provides advisory explanation only.
- The browser is never trusted to authorize execution.
- No new web server, FastAPI application, port, or trading engine is created.

## 2. Runtime summary

```text
Hugging Face Space / Docker container
└── /opt/hermes                         Upstream Hermes Agent source/runtime
    ├── dashboard on 0.0.0.0:7860      Existing Hermes web application
    ├── tools/futures_dashboard_api.py  Installed repository overlay
    ├── tools/templates/...html         Installed Luxury dashboard template
    ├── trading/*                       Installed Futures modules
    └── .hermes_futures_overlay_manifest.json

/opt/data
├── scripts/                            Entrypoint, persistence, runtime audit
├── hermes_overlay/                     Restored/persisted copy; not preferred over image overlay
├── futures_symbols_cache.json          Optional symbol cache
├── telegram_state.json                 Telegram owner/watchlist/alert state
└── persistent Hermes data

/opt/hermesface_overlay                 Immutable overlay copied from the current image
```

The Docker image clones Hermes Agent into `/opt/hermes`, installs Python/Node dependencies, copies repository scripts into `/opt/data/scripts`, and copies the current overlay into both `/opt/data/hermes_overlay` and `/opt/hermesface_overlay`. At startup, `scripts/sync_hf.py` installs the overlay into `/opt/hermes`, writes a SHA-256 manifest, patches the existing Hermes dashboard to include the routers, and starts the dashboard on port `7860`.

## 3. Main modules

| Module | Responsibility |
|---|---|
| `scripts/entrypoint.sh` | Runtime directory creation, dashboard auth configuration, and handoff to `sync_hf.py`. |
| `scripts/sync_hf.py` | Persistence restore/sync, overlay installation, manifest generation, router mounting, Telegram polling isolation, and process startup. |
| `hermes_overlay/tools/futures_dashboard_api.py` | Authenticated Futures HTTP routes, runtime file diagnostics, symbol catalog, market endpoint, analysis route, and Paper revalidation route. |
| `hermes_overlay/tools/templates/hermes_futures_desk_luxury.html` | Luxury Obsidian & Gold single-page dashboard. |
| `hermes_overlay/trading/dual_datasource_client.py` | DS4 → Binance → DS2 acquisition, normalization, provenance, health metadata, and `noTradeGuard` aggregation. |
| `hermes_overlay/trading/binance_public_client.py` | Unauthenticated Binance Futures fallback and explicit regional-restriction reporting. |
| `hermes_overlay/trading/trade_cycle.py` | Deterministic signal scoring, SL/TP construction, risk sizing, plan creation, and optional Paper orchestration. |
| `hermes_overlay/trading/risk.py` | Risk profiles, leverage caps/haircut, quantity sizing, and hard risk gates. |
| `hermes_overlay/trading/futures_execution.py` | Existing Paper account/position book and execution validation. |
| `hermes_overlay/trading/state.py` | Bounded in-memory dashboard state; no decision authority. |
| `hermes_overlay/trading/symbols.py` | Symbol normalization for DS4 and CCXT formats. |
| `hermes_overlay/external_ai/advisory.py` | Optional Hermes auxiliary advisory via task `futures_advisory` (see `advisory_schema.py`). |
| `hermes_overlay/plugins/futures_trading/` | Hermes plugin: read-only tool registration + auxiliary task wiring. |
| `hermes_overlay/skills/futures/market-context-advisory/` | Skill + parser for structured advisory responses. |
| `hermes_overlay/tools/telegram_bot.py` | Webhook-only, analysis-only Telegram adapter and owner bootstrap. |
| `scripts/verify_futures_runtime.py` | Read-only deployed runtime audit; never calls Paper Execute. |

## 4. End-to-end request flow

### 4.1 Market display

1. Browser requests `GET /api/futures/market` with symbol, interval, and limit.
2. Router normalizes the symbol and calls `get_market_context()`.
3. Datasource client requests DS4 with bounded KuCoin-compatible millisecond `from`/`to` parameters.
4. Missing, unusable, or stale fields are requested from Binance public fallback.
5. Datasource 2 is queried for complementary context and may fill only still-missing legitimate fields.
6. Each normalized field receives source, timestamp, freshness, validity, and fallback metadata.
7. The endpoint returns only real normalized values or an explicit `partial`, `stale`, or `unavailable` state.
8. The browser renders charts and diagnostics without modifying server decisions.

### 4.2 Deterministic analysis

1. Browser sends `POST /api/futures/analyze`.
2. Server runs `run_futures_cycle(..., execute=False)`.
3. DS4 safety state, Futures verification, required fields, and freshness gates are checked first.
4. A deterministic score is calculated only from normalized real inputs.
5. A directional plan is created only when score and confirmation thresholds pass.
6. SL/TP are derived from ATR and configured reward-to-risk rules.
7. Risk sizing calculates quantity from equity loss at Stop Loss.
8. Slippage is estimated from the real order book.
9. The result is stored as a bounded server-side plan and returned with a `planId`.

### 4.3 Paper execution

Paper execution is not a continuation of browser state. The server performs all checks again:

- requested symbol is a verified Futures contract;
- `planId` matches the latest server plan;
- symbol and risk profile have not changed;
- plan was not already executed;
- plan has not expired;
- decision is `LONG` or `SHORT`;
- DS4 verification and trading readiness remain valid;
- `noTradeGuard` is false;
- plan is marked executable and risk-approved;
- runtime trading mode is `paper`;
- a fresh analysis-only cycle still authorizes the plan.

Only after these checks does the server invoke the existing Paper execution path.

## 5. Datasource authority

```text
Datasource 4 → Binance public fallback → Datasource 2
```

Datasource 4 owns contract verification and all Futures safety semantics. The critical fields are:

```text
contract
ticker
orderbook
funding
openInterest
```

OHLCV, indicators, sentiment, and ATR are also normalized and attributed. Missing or non-fresh critical fields activate `noTradeGuard` and set `tradingReadiness=blocked`.

Binance public data is unauthenticated and field-level only. HTTP 451 is represented as `Regionally restricted`; it is never reported as healthy. Datasource 2 cannot verify Futures, clear `noTradeGuard`, or override DS4 data that is present and usable.

## 6. Health model

The code deliberately separates:

- `transportStatus`: whether the HTTP request succeeded;
- `dataUsability`: whether parsed data is suitable for use;
- `freshness`: whether provider timestamp or authoritative DS4 state proves freshness;
- `completeness`: whether expected fields were supplied;
- `mergeStatus`: whether the combined context is complete;
- `tradingReadiness`: whether deterministic safety gates allow a plan.

A successful HTTP response does not make market data fresh. Fallback data without a provider timestamp remains `unknown` and cannot pass a Futures freshness gate.

## 7. Analysis and plan states

### Analysis states

```text
NOT_ANALYZED
ANALYZING
LONG
SHORT
NO_TRADE
ANALYSIS_FAILED
STALE
API_UNAVAILABLE
```

### Plan types

- `directional_plan`: a valid directional plan before final execution checks.
- `non_executable_plan`: directional values exist but one or more safety/risk gates block execution.
- rejected/no-direction analysis: `NO_TRADE` with no executable plan geometry.

### Market endpoint states

- `available`: real candles and required display fields are fresh and usable.
- `partial`: values exist but freshness or completeness is not fully proven.
- `stale`: required display data is stale or invalid.
- `unavailable`: real candles or a current price could not be obtained.

## 8. Deterministic scoring and risk rules

Default analysis thresholds are environment-overridable:

| Setting | Default |
|---|---:|
| Minimum absolute signal score | `0.55` |
| Minimum signal components | `3` |
| Minimum direction confirmations | `2` |
| Stop ATR multiplier | `1.2` |
| Take Profit reward-to-risk | `1.8` |
| Minimum stop distance | `20` bps |
| Plan maximum age | `20` seconds |
| Requested leverage | `5x` |

Risk profiles:

| Profile | Equity risk | Maximum leverage |
|---|---:|---:|
| Conservative | 1% | 5x |
| Moderate | 3% | 10x |
| Aggressive | 5% | 15x |

Sizing is based on loss at Stop Loss:

```text
risk_amount   = account_equity × risk_percent
stop_distance = abs(entry_price - stop_loss)
quantity      = risk_amount / stop_distance
```

When ATR is at least 3% of price, effective leverage is reduced by 50% and never increased beyond the risk-profile cap.

## 9. Frontend behavior

The dashboard is a single packaged HTML template with inline CSS and JavaScript. It uses the existing authenticated FastAPI origin and `fetch(..., credentials='same-origin', cache='no-store')`.

Major features:

- symbol search and verified/market-only catalog counts;
- local watchlist and recent markets;
- real candle/line chart, four intervals, three candle limits, volume, crosshair, and tooltip;
- market source, freshness, funding, Open Interest, best bid/ask/spread, and readiness;
- display-only diagnostics from returned candles;
- per-field provenance;
- deterministic analysis and Paper Execute controls;
- plan geometry and execution checklist;
- datasource detail cards and sanitized technical diagnostics;
- Paper account and positions;
- local activity/history, JSON export, copy summary, density/theme preferences;
- manual and automatic refresh controls.

Browser storage never grants server permission. Selecting a historical symbol or changing risk invalidates the current browser plan and requires a new server analysis.

## 10. Authentication

The upstream Hermes dashboard is protected by its existing authentication middleware. The Futures router also supports local HTTP Basic enforcement when `HERMES_ADMIN_PASSWORD` is set:

```text
username: HERMES_DASHBOARD_BASIC_AUTH_USERNAME (default: admin)
password: HERMES_ADMIN_PASSWORD
```

`entrypoint.sh` writes a hashed credential into Hermes `config.yaml` before the server binds publicly. Secrets must be configured as Hugging Face Space secrets or injected environment variables, never committed.

## 11. Telegram model

Telegram is webhook-only and analysis-only:

- `POST /api/telegram/webhook` validates `X-Telegram-Bot-Api-Secret-Token`.
- Owner bootstrap uses a one-time private-chat `/claim <secret>` command.
- Authorized users come from configured IDs or the persisted owner.
- Commands call `run_futures_cycle(..., execute=False)` only.
- Direct delivery may use a proxy; proactive delivery may use an HMAC relay.
- Polling must remain disabled.

No Telegram command can execute a Futures position.

## 12. Runtime integrity

During overlay installation, `sync_hf.py` copies the current image overlay into `/opt/hermes` and writes `.hermes_futures_overlay_manifest.json`. The status endpoint compares repository/overlay/runtime/template/router hashes when those paths are available.

Runtime status semantics:

- `verified`: evidence exists and all expected hashes match;
- `mismatch`: evidence exists and one or more hashes differ;
- `unknown`: required evidence is unavailable.

Missing files must never be reported as verified.

## 13. Development workflow

1. Start from the current repository files under `asset-space/hermes_overlay`.
2. Do not copy old loose reference files over the repository.
3. Keep changes focused and additive to the API contract.
4. Preserve the single server and port architecture.
5. Add tests for normalization, provenance, state transitions, and server-side execution checks.
6. Run static checks and focused Futures tests.
7. Review secrets and generated files before commit.
8. Deploy through the existing Space workflow.
9. Verify the installed hashes and authenticated routes.
10. Inspect browser Console and Network.
11. Never click Paper Execute during deployment verification.

## 14. Read-only runtime audit

```bash
export HERMES_ADMIN_PASSWORD='...'
export HERMES_DASHBOARD_BASIC_AUTH_USERNAME='admin'
export HF_TOKEN='...' # required when the target Space is private
python scripts/verify_futures_runtime.py \
  --base-url https://really-amin-simplechatbot.hf.space \
  --symbol BTCUSDT \
  --analyze \
  --report .runtime_audit/futures_runtime_audit.json
```

The utility checks `/futures`, status, symbols, positions, all market intervals, and optionally one analysis-only request. It never calls `/api/futures/paper/execute`.

## 15. Known deployment limitations

- Binance public Futures endpoints may return HTTP 451 in the current Hugging Face region.
- Real DS4 payload names and provider timestamps must be verified against live deployed responses.
- The current package has static validation results but not a completed authenticated production verification cycle.
- The UI uses a single large HTML template; future refactoring must preserve runtime template installation and avoid introducing a second frontend server.

## 16. Definition of done

A change is complete only when:

- the correct template and router are installed and hash-verified;
- authenticated routes return expected structured responses;
- real market data renders for 1m, 5m, 15m, and 1h or returns an explicit unavailable state;
- Console has no critical error and Network requests are authenticated;
- datasource health and attribution are truthful;
- deterministic safety logic is unchanged;
- Telegram remains webhook-only;
- no secret is exposed;
- no trade is executed during verification.