--- title: Selfapi-v2 emoji: 📊 colorFrom: gray colorTo: blue sdk: docker app_port: 3001 pinned: false --- # Selfapi-v2 — Headless Chrome + Unpacked Extensions A [Hugging Face Space](https://huggingface.co/spaces/akash1313/selfapi-v2) that runs a persistent, headless **Chrome for Testing** browser inside Docker with three custom unpacked extensions auto-loaded, plus a live web monitor to watch the browser in real time. The browser stays logged in across restarts — on Hugging Face the profile is snapshotted to a **Postgres database** (see [Persistence](#persistence-staying-logged-in) below); on a self-hosted Docker setup it lives in a named volume. The bundled extensions sync cookies / automate workflows for ChatGPT, Google Gemini, and Google Flow. --- ## Bundled extensions | Extension | Source dir | Loaded at | What it does | |-----------|-----------|-----------|--------------| | **Gpt Agent Api** | `free-Chatgpt-api/gpt-extension` | `/opt/gpt-extension` | Auto-syncs ChatGPT cookies to a local ChatGPT Free API server | | **Gemini Api Agent** | `free-gemini-api/gemini-extension` | `/opt/gemini-extension` | Auto-syncs Google Gemini cookies to a local Free Gemini API server | Both are Manifest V3 extensions with background service workers. Each subfolder also contains the matching backend server. --- ## Backend servers Each extension is the browser half of a standalone project. The extension uses your authenticated browser session and forwards requests to a **local** API server (which you run yourself — see each subproject's README for setup): | Project | Server | Talks to | Purpose | |---------|--------|----------|---------| | `free-Chatgpt-api` | Go | `127.0.0.1:9225` | ChatGPT backend API from your logged-in session | | `free-gemini-api` | Go reverse-proxy | local server | Gemini text / Imagen 3 images / Gemini video | | `flow-agent` | Python FastAPI + CLI | `127.0.0.1:8100` | Google Flow T2V / V2V / I2V video + T2I / I2I images | > ✅ The HF Space now runs **all three servers in the same container** as Chrome, and the > monitor doubles as a **single API gateway** that fronts them on one public port — see > [API gateway](#api-gateway). (The extensions dial their servers on hardcoded > `127.0.0.1:9225/9226/9227`, so the servers *must* share Chrome's localhost — a separate > Space can't reach them.) `docker-compose.yml` does the same wiring for self-hosting. --- ## How it works ``` HF Space (port 3001) │ ▼ ┌─────────────────────────────────────────────────────────┐ │ Docker container (Debian 12) │ │ │ │ profilesync ──► restores profile from Postgres on boot │ │ (Go) and backs it up every 5 min / on exit │ │ │ │ │ monitor (3-tab) ─┤ serves the live UI + /chrome.log │ │ (Go, :3001) │ │ │ ▼ │ │ Chrome for Testing 145 (--headless=new) │ │ --load-extension=/opt/gpt,/opt/gemini,/opt/flow │ │ --remote-debugging-port=9222 (CDP) │ │ --user-data-dir=/home/chrome/data (persistent) │ │ ▲ │ │ socat :9223 ─────┘ forwards external CDP → 127.0.0.1 │ └─────────────────────────────────────────────────────────┘ ``` `start_hf.sh` is the boot sequence: 0. **Restore profile** — if `DATABASE_URL` is set, `profilesync restore` pulls the last Chrome profile snapshot from Postgres so logins survive an HF restart/rebuild. 1. **Profile repair & cleanup** — fixes a nested `data/data` layout and removes cache/junk to keep the persistent profile small. 2. **Lock removal** — clears `Singleton*` / `LOCK` files left by an unclean shutdown. 3. **Preferences injection** — enables `extensions.ui.developer_mode` and sets a clean `exit_type` so the "restore pages?" popup never appears. 4. **Monitor server** — starts the Go 3-tab monitor on `:3001` and symlinks `chrome.log`. 5. **CDP proxy** — `socat` exposes the DevTools port (9223 → 127.0.0.1:9222). 6. **Auto-backup** — if `DATABASE_URL` is set, `profilesync backup` runs every 5 min (plus a final backup on `SIGTERM`/`SIGINT` shutdown). 7. **Launch Chrome** — Chrome for Testing in `--headless=new` with all three extensions. --- ## Persistence (staying logged in) Hugging Face Spaces on the **free `cpu-basic` tier have no persistent disk** — the container filesystem (including `/home/chrome/data`) is **wiped on every restart, rebuild, or factory reboot**. To keep logins, the profile is snapshotted to an **external Postgres database**. How it works (the `profilesync` Go tool, built into the image): - **On boot:** restores the profile from Postgres (`profilesync restore`). - **Every 5 min + on shutdown:** backs the profile up (`profilesync backup`). - Stores it in one row: `chrome_profile(id, updated_at, data bytea)` — table auto-created. - **All of this is gated on the `DATABASE_URL` env var.** If it is not set, restore/backup are silently skipped and logins are lost on restart. ### Setup — one-time 1. **Create a free Postgres** at [neon.tech](https://neon.tech) → new project → copy the connection string. It looks like: ``` postgresql://USER:PASSWORD@HOST.neon.tech/DBNAME?sslmode=require ``` 2. **Add it as a Space secret:** HF Space → **Settings** → **Variables and secrets** → **New secret** - **Name:** `DATABASE_URL` - **Value:** the connection string from step 1 3. Saving the secret **auto-restarts** the Space. Because nothing was backed up yet, the first boot has nothing to restore — so **log in once** after the secret is set. Within 5 minutes that login is snapshotted to Postgres and will survive all future restarts. > 💡 Verify a backup landed: connect to the DB and run > `SELECT id, updated_at, octet_length(data) FROM chrome_profile;` — a row with a non-zero > size means the profile is persisted. > 🔒 The connection string is a credential — keep it only in the HF secret. A local copy in > `neon-db` is **gitignored** so it never reaches this public repo. --- ## Why Chrome for Testing (not regular Chrome) Branded **Google Chrome ≥ 128 silently ignores `--load-extension`** (removed by the [March 2025 Chrome RFC](https://developer.chrome.com/blog/remote-debugging-port) to curb abuse). On a branded build the extensions never load and `chrome://extensions/` shows an empty list with **no error in the log**. The fix is to run **[Chrome for Testing](https://googlechromelabs.github.io/chrome-for-testing/)** — the unbranded build of the same Chrome version where `--load-extension` still works. The Dockerfile downloads CfT `145.0.7632.117` to `/opt/chrome-linux64` and `start_hf.sh` launches that binary. Everything else (monitor, CDP proxy, flags) is unchanged. > Alternatives if you ever upgrade: use the CDP `Extensions.loadUnpacked` method > (requires `--remote-debugging-pipe` + `--enable-unsafe-extension-debugging`), or plain > Chromium. Both also keep `--load-extension`-style loading working. --- ## User-Agent must match the OS The Space runs on **Linux**, so Chrome (and the backend HTTP clients) advertise a **Linux** User-Agent: ``` Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36 ``` A **macOS UA on a Linux box is an inconsistency** that Cloudflare Turnstile flags, causing the ChatGPT "verify you are human" check to loop forever (the tick never clears). The browser UA lives in `start_hf.sh`; the ChatGPT backend (`free-Chatgpt-api/cookies.go`) uses the **same UA and Chrome version (145)** so the `cf_clearance` cookie — which Cloudflare binds to the exact UA — stays valid across the browser and the backend. --- ## Endpoints | URL | Description | |-----|-------------| | `https://akash1313-selfapi-v2.hf.space/` | Live Chrome monitor UI — **key required** (open `/?key=YOUR_API_KEY` once; it sets a cookie) | | `https://akash1313-selfapi-v2.hf.space/chrome.log` | Streaming Chrome log — **key required** | | `https://akash1313-selfapi-v2.hf.space/gpt/…` | ChatGPT API (gateway → `127.0.0.1:9225`) | | `https://akash1313-selfapi-v2.hf.space/gemini/…` | Gemini API (gateway → `127.0.0.1:8000`) | | `https://akash1313-selfapi-v2.hf.space/flow/…` | Flow API (gateway → `127.0.0.1:8101`) | | `127.0.0.1:9222` (in-container) | Chrome DevTools Protocol | | `:9223` | CDP proxied for external tools (via socat) | --- ## API gateway The monitor (`monitor/main.go`) is also a reverse-proxy gateway: HF exposes only one public port (`3001`), so all three backend APIs are reached under one base URL by path prefix. The gateway strips the prefix and forwards to the server on localhost: | Prefix | Backend | Example endpoints | |--------|---------|-------------------| | `/gpt/` | free-Chatgpt-api `:9225` | `/gpt/api/chat`, `/gpt/v1/chat/completions`, `/gpt/health` | | `/gemini/` | free-gemini-api `:8000` | `/gemini/chat`, `/gemini/status`, `/gemini/output/*` | | `/flow/` | flow-agent `:8101` | `/flow/generate/video`, `/flow/generate/image`, `/flow/health` | ### Auth — required **Every** route — the API prefixes, the monitor UI (`/`), `/api/*`, `/chrome.log`, `/logs/*` — is gated by an **`API_KEY`** Space secret. Send it as a Bearer token, `?key=`, or (for the browser UI) open `/?key=YOUR_API_KEY` once to set an `apikey` cookie. **Fail-closed:** if `API_KEY` is unset every route returns `503`, so the logged-in accounts are never exposed. This Space is also **private** on Hugging Face, so the URL itself requires HF auth — defense in depth. 1. Set the Space secret **`API_KEY`** (Settings → Variables and secrets) to a strong random string. 2. Call it: ```bash curl -X POST https://akash1313-selfapi-v2.hf.space/gpt/api/chat \ -H "Authorization: Bearer $API_KEY" \ -H "Content-Type: application/json" \ -d '{"message":"hello"}' ``` No/!wrong key → `401`. Wrong path → forwarded to the backend (may `404` there). --- ## Project layout ``` chrome-space/ ├── Dockerfile # builds on akashyadav758/chrome, installs CfT, copies extensions ├── docker-compose.yml # full self-host stack: browser + 3 API servers on one loopback ├── start_hf.sh # container entrypoint / Chrome boot sequence ├── profilesync/ # Go tool: Chrome profile <-> Postgres snapshot ├── monitor/ # Go 3-tab live monitor (GPT / Gemini / Flow) ├── free-Chatgpt-api/ # ChatGPT Free API server + gpt-extension ├── free-gemini-api/ # Free Gemini API server + gemini-extension └── flow-agent/ # Flow automation CLI + Flow-extension ``` --- ## Deploy This repo **is** the HF Space — pushing to `main` triggers an automatic rebuild: ```bash git push origin main ``` First-time / persistence checklist: 1. Push the code (above). 2. Set the `DATABASE_URL` secret — see [Persistence](#persistence-staying-logged-in). 3. Log in once in the monitor UI; the profile auto-backs up within 5 min. > `hf-token` and `neon-db` are gitignored — keep your HF token and DB credentials out of > commits. --- ## Self-hosting (full stack) To run the browser **and** all three backend API servers locally, use `docker-compose.yml`. The server containers join the chrome container's network namespace (`network_mode: "service:chrome"`), so they all share one `127.0.0.1` — exactly what the extensions expect. The Chrome profile persists in a named Docker volume (no Postgres needed). ```bash docker compose up --build -d # browser + chatgpt + gemini + flow docker compose down # profile survives in the chrome-profile volume ``` Or just the browser image: ```bash # Build docker build --platform linux/amd64 -t chrome-space . # Run docker run -d --name chrome-space -p 3001:3001 --platform linux/amd64 chrome-space # Verify the 3 extensions loaded (look for service_worker targets) docker exec chrome-space curl -s http://127.0.0.1:9222/json \ | python3 -c 'import json,sys;[print(t["type"],t["url"]) for t in json.load(sys.stdin)]' ``` Open to watch the browser. --- ## Troubleshooting | Symptom | Cause / fix | |---------|-------------| | Login lost after restart/rebuild | `DATABASE_URL` not set (or DB unreachable) → set the Postgres secret, see [Persistence](#persistence-staying-logged-in) | | ChatGPT Cloudflare "verify you are human" loops, tick never passes | UA/OS mismatch (macOS UA on Linux) — ensure the **Linux** UA in `start_hf.sh`. If it still loops, it's the **datacenter IP** (HF = AWS); route through a residential proxy via `--proxy-server=http://user:pass@host:port` | | `chrome://extensions/` empty, no error in log | Branded Chrome ignoring `--load-extension` → use Chrome for Testing (see above) | | `COPY failed: ... not found` at build | Extension source dir name in the Dockerfile doesn't match disk | | Extension listed but service worker idle | MV3 headless quirk — open the extension's toggle or hit **Update** on the extensions page | | `Failed to connect to the bus` spam in log | Harmless D-Bus warnings in a container with no system bus | | "Restore pages?" popup | Handled by the `exit_type=Normal` Preferences injection in `start_hf.sh` |