docs/TESTING_AND_VERIFICATION.md

# Testing and Verification

## Test locations

```text
hermes_overlay/tests/
```

Existing focused areas include:

- Binance public fallback;
- nested DS4 merge behavior;
- external advisory boundary;
- Futures dashboard/state;
- Futures integration;
- Luxury template markers;
- optional MCP runtime isolation;
- Telegram webhook;
- trade-cycle field paths.

## Recommended validation layers

### 1. Static validation

```bash
python -m compileall hermes_overlay scripts
node --check /tmp/hermes_dashboard_script.js
ruff check hermes_overlay scripts
```

Also check:

- duplicate DOM IDs;
- missing JavaScript DOM references;
- undefined CSS custom properties;
- `git diff --check`;
- absence of secrets/generated files.

### 2. Focused unit tests

Required focus:

- seconds-to-milliseconds conversion;
- already-millisecond timestamps;
- ordered bounded KuCoin ranges;
- ticker/funding/Open Interest aliases;
- malformed and ambiguous provider shapes;
- per-field source/timestamp/freshness attribution;
- transport health versus usability/readiness;
- datasource-specific error attribution;
- market endpoint canonical shape;
- no fabricated values;
- safe rejection when required fields are missing or non-fresh;
- initial/failure UI states;
- Execute-disabled reasons and server safety gates.

### 3. Futures regression suite

Run the existing Futures tests once after focused tests pass. Avoid repeatedly running unrelated broad suites while iterating on a narrow failure.

### 4. Read-only deployed audit

```bash
export HERMES_ADMIN_PASSWORD='...'
python scripts/verify_futures_runtime.py \
  --base-url https://really-amin-asset.hf.space \
  --symbol BTCUSDT \
  --analyze \
  --report .runtime_audit/futures_runtime_audit.json
```

The audit never calls Paper Execute.

### 5. Browser verification

Authenticated desktop and mobile verification must cover:

- Console free of critical errors;
- valid Network status and JSON payloads;
- chart rendering for all intervals;
- Candles/Line, volume, tooltip, crosshair;
- watchlist/recent/history/export/density/theme/auto-refresh controls;
- field provenance and datasource cards;
- state machine and score semantics;
- visible Execute-disabled reason;
- no Paper Execute click.

## Acceptance matrix

| Area | Required result |
|---|---|
| Runtime files | `verified`, or documented investigation for `unknown`; never unexplained mismatch. |
| Status API | 200 authenticated, structured source/runtime fields. |
| Symbols API | Accurate total/verified/market-only counts. |
| Positions API | Deliberate empty state or formatted real Paper positions. |
| Market API | Real canonical candles or explicit structured unavailable state. |
| Analysis API | Deterministic result; `NO_TRADE` is allowed and expected when unsafe. |
| Paper Execute | Not called during verification. |
| Binance 451 | Clearly reported as regional restriction. |
| Telegram | Webhook-only; no polling adapter. |
| Secrets | None in source, logs, reports, screenshots, or package. |

## Testing safety

Tests must not:

- place Paper/Testnet/Live orders;
- require real exchange credentials;
- fabricate production responses in deployed paths;
- weaken guards to make assertions pass;
- treat an unavailable score as zero;
- report missing runtime evidence as verified.