Spaces:
Paused
Paused
File size: 5,319 Bytes
35743bd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | # 🛡️ Resilience Guide — OmniRoute
> How OmniRoute keeps your AI coding workflow running when providers fail.
## Overview
OmniRoute implements a multi-layered resilience system that ensures zero downtime:
```
Client Request
→ Rate Limit Check (per-IP, per-connection)
→ Combo Routing (13 strategies)
→ Connection Selection (P2C, round-robin, etc.)
→ Request Queue & Pacing
→ Execute (provider-specific executor)
→ On Failure:
→ Connection Cooldown (exponential backoff)
→ Circuit Breaker (provider-level)
→ Wait For Cooldown (auto-retry)
→ Next Combo Target (fallback chain)
→ Response
```
---
## Request Queue & Pacing
Per-connection request buckets smooth bursts before they hit upstream rate caps.
Configure in `Dashboard → Settings → Resilience`:
| Setting | Default | Description |
| --------------- | ------- | ------------------------------------ |
| Queue Size | `10` | Max queued requests per connection |
| Pacing Interval | `0ms` | Minimum gap between requests |
| Max Concurrent | `5` | Simultaneous requests per connection |
---
## Connection Cooldown
A single connection cools down after retryable failures. Features:
- **Exponential Backoff** — progressively longer cooldowns after each failure
- **`Retry-After` Header Support** — respects upstream hints
- **Configurable Base/Max** — tune cooldown duration per use case
- **Auto-Recovery** — connection automatically becomes available after cooldown expires
---
## Circuit Breaker
Provider-level protection against cascading failures:
1. **Connection-scoped `429` rate limits** stay in Connection Cooldown (don't trip the breaker)
2. **Provider-wide transient errors** (5xx, network timeouts) increment the failure counter
3. **Breaker trips** only after fallback is exhausted AND the provider still fails
4. **Recovery** — breaker automatically moves to half-open state after timeout, tests with probe request
Configure thresholds in `Dashboard → Settings → Resilience`.
---
## Wait For Cooldown
Instead of immediately failing when all connections are in cooldown, OmniRoute can wait for the earliest connection to expire and retry:
- **Automatic** — server waits for the earliest cooldown to expire
- **Transparent** — client sees a slightly delayed response instead of an error
- **Configurable** — enable/disable per combo or globally
---
## Anti-Thundering Herd
When multiple concurrent requests hit a failing provider simultaneously:
- **Mutex Protection** — only one retry attempt at a time per connection
- **Semaphore** — limits concurrent retry storms across connections
- **Deduplication** — identical requests within 5s window are deduplicated
---
## Combo Fallback Chains
The primary resilience mechanism. Configure in `Dashboard → Combos`:
```txt
Combo: "always-on"
1. cc/claude-opus-4-7 ← Primary (subscription)
2. cx/gpt-5.2-codex ← Secondary (subscription)
3. glm/glm-4.7 ← Cheap backup ($0.5/1M)
4. if/kimi-k2-thinking ← Free fallback (unlimited)
```
When provider #1 fails (quota, rate, or health), OmniRoute automatically routes to #2, then #3, then #4 — with zero manual intervention.
### 13 Routing Strategies
| Strategy | Description |
| ------------------- | ---------------------------------- |
| `priority` | First available in order |
| `weighted` | Weighted distribution |
| `fill-first` | Fill primary before moving |
| `round-robin` | Rotate through all targets |
| `p2c` | Power-of-two choices (quota-aware) |
| `random` | Random selection |
| `least-used` | Least recently used |
| `cost-optimized` | Cheapest available |
| `strict-random` | True random (no tracking) |
| `auto` | OmniRoute selects based on context |
| `lkgp` | Last Known Good Provider |
| `context-optimized` | Best for current context window |
| `context-relay` | Session handoff during rotation |
---
## TLS Fingerprint Spoofing
OmniRoute makes proxied traffic look like legitimate browser/CLI requests:
- **Browser-like TLS** via `wreq-js` — prevents bot detection
- **CLI Fingerprint Matching** — reorders headers and body fields to match native CLI binary signatures (Claude Code, Codex, etc.)
- **Proxy IP Preservation** — stealth features work on top of proxy IP masking
---
## Health Dashboard
Monitor all resilience components in real-time at `Dashboard → Health`:
- **Uptime** — server uptime and last restart
- **Provider Breaker States** — open/closed/half-open per provider
- **Connection Cooldowns** — active cooldowns with expiry times
- **Cache Stats** — signature + semantic cache hit rates
- **Lockouts** — API key lockouts and IP bans
- **Latency** — p50/p95/p99 percentiles
---
## See Also
- [Architecture Guide](ARCHITECTURE.md) — System architecture and internals
- [User Guide](USER_GUIDE.md) — Providers, combos, CLI integration
- [Auto-Combo Engine](AUTO-COMBO.md) — 6-factor scoring, mode packs
|