| ## v6.1 — Anti-Detection & Browser Impersonation Fix |
|
|
| ### Critical Fix: Cloudflare/Bot Detection Bypass |
|
|
| The engine was being blocked by Cloudflare and other anti-bot systems because: |
|
|
| 1. **User-Agent was `"BexEngine/6.0"`** — immediately flagged as bot |
| 2. **Missing browser headers** — no `Sec-CH-UA`, `Sec-Fetch-*`, `Accept-Encoding`, etc. |
| 3. **No HTTP/2** — real browsers always negotiate H2 |
| 4. **No cookie persistence** — CF challenges set cookies that must be sent on subsequent requests |
| 5. **No compression** — browsers always send `Accept-Encoding: gzip, deflate, br` |
|
|
| ### Changes (in `crates/bex-core/`): |
|
|
| **`config.rs`**: Default user_agent changed from `"BexEngine/6.0"` to a real Chrome 137 UA string. Timeouts increased from 15s to 30s (CF challenges can be slow). |
| |
| **`http_service.rs`**: Complete rewrite of the HTTP layer: |
| - Real Chrome 137 default headers (Sec-CH-UA, Sec-Fetch-*, Accept-Encoding, etc.) |
| - HTTP/2 support enabled (Chrome always uses it) |
| - gzip/brotli/deflate decompression (browsers always send these) |
| - Cookie store (`cookie_store(true)`) for CF session persistence |
| - Auto-generates Referer from URL origin when not provided by plugin |
| - Plugin headers override defaults (so plugins can customize) |
|
|
| **`Cargo.toml`**: Added reqwest features: `gzip`, `brotli`, `deflate`, `cookies`, `http2` |
|
|
| ### Testing Results: |
|
|
| | Site | Search | Episodes | Stream Resolution | |
| |------|--------|----------|-------------------| |
| | AnimeKai (anikai.to) | ✅ Works | ✅ Works | ⚠️ CF JS Challenge | |
| | enc-dec.app API | ✅ Works | N/A | N/A | |
|
|
| The stream resolution endpoint (`/ajax/links/view`) is behind a Cloudflare JS challenge that requires actual JavaScript execution. This needs the `bex-js` QuickJS engine to solve the CF challenge first. The `CloudflareKiller` approach from cloudstream uses a WebView for this. |
|
|
| ### Remaining Work: CF Challenge Solver |
|
|
| For sites that use Cloudflare JS challenges on their API endpoints, we need: |
| 1. A CF challenge solver using `bex-js` (QuickJS) that executes the challenge JS |
| 2. Cookie persistence across the session so solved challenges don't need re-solving |
| 3. The cookie store fix (already implemented) handles step 2 automatically |
|
|
|
|
