Hugging Face's logo

Spaces:
akash1313
/
selfapi-v2
Sleeping

App Files Community
selfapi-v2 / README.md
akashyadav758
Remove Flow agent from stack (extension, server, gateway, monitor tab)
23a7459
preview code
|
Raw
History Blame Contribute Delete
13.8 kB
metadata 
title: Selfapi-v2
emoji: 📊
colorFrom: gray
colorTo: blue
sdk: docker
app_port: 3001
pinned: false

Selfapi-v2 — Headless Chrome + Unpacked Extensions

A Hugging Face Space that runs a persistent, headless Chrome for Testing browser inside Docker with three custom unpacked extensions auto-loaded, plus a live web monitor to watch the browser in real time.

The browser stays logged in across restarts — on Hugging Face the profile is snapshotted to a Postgres database (see Persistence below); on a self-hosted Docker setup it lives in a named volume. The bundled extensions sync cookies / automate workflows for ChatGPT, Google Gemini, and Google Flow.

Bundled extensions

Extension Source dir Loaded at What it does
Gpt Agent Api free-Chatgpt-api/gpt-extension /opt/gpt-extension Auto-syncs ChatGPT cookies to a local ChatGPT Free API server
Gemini Api Agent free-gemini-api/gemini-extension /opt/gemini-extension Auto-syncs Google Gemini cookies to a local Free Gemini API server

Both are Manifest V3 extensions with background service workers. Each subfolder also contains the matching backend server.

Backend servers

Each extension is the browser half of a standalone project. The extension uses your authenticated browser session and forwards requests to a local API server (which you run yourself — see each subproject's README for setup):

Project Server Talks to Purpose
free-Chatgpt-api Go 127.0.0.1:9225 ChatGPT backend API from your logged-in session
free-gemini-api Go reverse-proxy local server Gemini text / Imagen 3 images / Gemini video
flow-agent Python FastAPI + CLI 127.0.0.1:8100 Google Flow T2V / V2V / I2V video + T2I / I2I images

✅ The HF Space now runs all three servers in the same container as Chrome, and the monitor doubles as a single API gateway that fronts them on one public port — see API gateway. (The extensions dial their servers on hardcoded 127.0.0.1:9225/9226/9227, so the servers must share Chrome's localhost — a separate Space can't reach them.) docker-compose.yml does the same wiring for self-hosting.

How it works 

HF Space (port 3001)
        │
        ▼
┌─────────────────────────────────────────────────────────┐
│ Docker container (Debian 12)                             │
│                                                          │
│  profilesync ──► restores profile from Postgres on boot  │
│   (Go)          and backs it up every 5 min / on exit    │
│                   │                                      │
│  monitor (3-tab) ─┤  serves the live UI + /chrome.log    │
│   (Go, :3001)     │                                      │
│                   ▼                                      │
│  Chrome for Testing 145 (--headless=new)                 │
│    --load-extension=/opt/gpt,/opt/gemini,/opt/flow       │
│    --remote-debugging-port=9222  (CDP)                   │
│    --user-data-dir=/home/chrome/data  (persistent)       │
│                   ▲                                      │
│  socat :9223 ─────┘  forwards external CDP → 127.0.0.1   │
└─────────────────────────────────────────────────────────┘

start_hf.sh is the boot sequence:

  1. Restore profile — if DATABASE_URL is set, profilesync restore pulls the last Chrome profile snapshot from Postgres so logins survive an HF restart/rebuild.
  2. Profile repair & cleanup — fixes a nested data/data layout and removes cache/junk to keep the persistent profile small.
  3. Lock removal — clears Singleton* / LOCK files left by an unclean shutdown.
  4. Preferences injection — enables extensions.ui.developer_mode and sets a clean exit_type so the "restore pages?" popup never appears.
  5. Monitor server — starts the Go 3-tab monitor on :3001 and symlinks chrome.log.
  6. CDP proxy — socat exposes the DevTools port (9223 → 127.0.0.1:9222).
  7. Auto-backup — if DATABASE_URL is set, profilesync backup runs every 5 min (plus a final backup on SIGTERM/SIGINT shutdown).
  8. Launch Chrome — Chrome for Testing in --headless=new with all three extensions.

Persistence (staying logged in)

Hugging Face Spaces on the free cpu-basic tier have no persistent disk — the container filesystem (including /home/chrome/data) is wiped on every restart, rebuild, or factory reboot. To keep logins, the profile is snapshotted to an external Postgres database.

How it works (the profilesync Go tool, built into the image):

  • On boot: restores the profile from Postgres (profilesync restore).
  • Every 5 min + on shutdown: backs the profile up (profilesync backup).
  • Stores it in one row: chrome_profile(id, updated_at, data bytea) — table auto-created.
  • All of this is gated on the DATABASE_URL env var. If it is not set, restore/backup are silently skipped and logins are lost on restart.

Setup — one-time

  1. Create a free Postgres at neon.tech → new project → copy the connection string. It looks like:
    postgresql://USER:PASSWORD@HOST.neon.tech/DBNAME?sslmode=require
  2. Add it as a Space secret: HF Space → Settings → Variables and secrets → New secret
    • Name: DATABASE_URL
    • Value: the connection string from step 1
  3. Saving the secret auto-restarts the Space. Because nothing was backed up yet, the first boot has nothing to restore — so log in once after the secret is set. Within 5 minutes that login is snapshotted to Postgres and will survive all future restarts.

💡 Verify a backup landed: connect to the DB and run SELECT id, updated_at, octet_length(data) FROM chrome_profile; — a row with a non-zero size means the profile is persisted.

🔒 The connection string is a credential — keep it only in the HF secret. A local copy in neon-db is gitignored so it never reaches this public repo.

Why Chrome for Testing (not regular Chrome)

Branded Google Chrome ≥ 128 silently ignores --load-extension (removed by the March 2025 Chrome RFC to curb abuse). On a branded build the extensions never load and chrome://extensions/ shows an empty list with no error in the log.

The fix is to run Chrome for Testing — the unbranded build of the same Chrome version where --load-extension still works. The Dockerfile downloads CfT 145.0.7632.117 to /opt/chrome-linux64 and start_hf.sh launches that binary. Everything else (monitor, CDP proxy, flags) is unchanged.

Alternatives if you ever upgrade: use the CDP Extensions.loadUnpacked method (requires --remote-debugging-pipe + --enable-unsafe-extension-debugging), or plain Chromium. Both also keep --load-extension-style loading working.

User-Agent must match the OS

The Space runs on Linux, so Chrome (and the backend HTTP clients) advertise a Linux User-Agent:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36

A macOS UA on a Linux box is an inconsistency that Cloudflare Turnstile flags, causing the ChatGPT "verify you are human" check to loop forever (the tick never clears). The browser UA lives in start_hf.sh; the ChatGPT backend (free-Chatgpt-api/cookies.go) uses the same UA and Chrome version (145) so the cf_clearance cookie — which Cloudflare binds to the exact UA — stays valid across the browser and the backend.

Endpoints

URL Description
https://akash1313-selfapi-v2.hf.space/ Live Chrome monitor UI — key required (open /?key=YOUR_API_KEY once; it sets a cookie)
https://akash1313-selfapi-v2.hf.space/chrome.log Streaming Chrome log — key required
https://akash1313-selfapi-v2.hf.space/gpt/… ChatGPT API (gateway → 127.0.0.1:9225)
https://akash1313-selfapi-v2.hf.space/gemini/… Gemini API (gateway → 127.0.0.1:8000)
https://akash1313-selfapi-v2.hf.space/flow/… Flow API (gateway → 127.0.0.1:8101)
127.0.0.1:9222 (in-container) Chrome DevTools Protocol
:9223 CDP proxied for external tools (via socat)

API gateway

The monitor (monitor/main.go) is also a reverse-proxy gateway: HF exposes only one public port (3001), so all three backend APIs are reached under one base URL by path prefix. The gateway strips the prefix and forwards to the server on localhost:

Prefix Backend Example endpoints
/gpt/ free-Chatgpt-api :9225 /gpt/api/chat, /gpt/v1/chat/completions, /gpt/health
/gemini/ free-gemini-api :8000 /gemini/chat, /gemini/status, /gemini/output/*
/flow/ flow-agent :8101 /flow/generate/video, /flow/generate/image, /flow/health

Auth — required

Every route — the API prefixes, the monitor UI (/), /api/*, /chrome.log, /logs/* — is gated by an API_KEY Space secret. Send it as a Bearer token, ?key=, or (for the browser UI) open /?key=YOUR_API_KEY once to set an apikey cookie. Fail-closed: if API_KEY is unset every route returns 503, so the logged-in accounts are never exposed. This Space is also private on Hugging Face, so the URL itself requires HF auth — defense in depth.

  1. Set the Space secret API_KEY (Settings → Variables and secrets) to a strong random string.
  2. Call it:
    curl -X POST https://akash1313-selfapi-v2.hf.space/gpt/api/chat \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"message":"hello"}'
    No/!wrong key → 401. Wrong path → forwarded to the backend (may 404 there).

Project layout 

chrome-space/
├── Dockerfile              # builds on akashyadav758/chrome, installs CfT, copies extensions
├── docker-compose.yml      # full self-host stack: browser + 3 API servers on one loopback
├── start_hf.sh             # container entrypoint / Chrome boot sequence
├── profilesync/            # Go tool: Chrome profile <-> Postgres snapshot
├── monitor/                # Go 3-tab live monitor (GPT / Gemini / Flow)
├── free-Chatgpt-api/       # ChatGPT Free API server + gpt-extension
├── free-gemini-api/        # Free Gemini API server + gemini-extension
└── flow-agent/             # Flow automation CLI + Flow-extension

Deploy

This repo is the HF Space — pushing to main triggers an automatic rebuild:

git push origin main

First-time / persistence checklist:

  1. Push the code (above).
  2. Set the DATABASE_URL secret — see Persistence.
  3. Log in once in the monitor UI; the profile auto-backs up within 5 min.

hf-token and neon-db are gitignored — keep your HF token and DB credentials out of commits.

Self-hosting (full stack)

To run the browser and all three backend API servers locally, use docker-compose.yml. The server containers join the chrome container's network namespace (network_mode: "service:chrome"), so they all share one 127.0.0.1 — exactly what the extensions expect. The Chrome profile persists in a named Docker volume (no Postgres needed).

docker compose up --build -d      # browser + chatgpt + gemini + flow
docker compose down               # profile survives in the chrome-profile volume

Or just the browser image:

# Build
docker build --platform linux/amd64 -t chrome-space .

# Run
docker run -d --name chrome-space -p 3001:3001 --platform linux/amd64 chrome-space

# Verify the 3 extensions loaded (look for service_worker targets)
docker exec chrome-space curl -s http://127.0.0.1:9222/json \
  | python3 -c 'import json,sys;[print(t["type"],t["url"]) for t in json.load(sys.stdin)]'

Open http://localhost:3001 to watch the browser.

Troubleshooting

Symptom Cause / fix
Login lost after restart/rebuild DATABASE_URL not set (or DB unreachable) → set the Postgres secret, see Persistence
ChatGPT Cloudflare "verify you are human" loops, tick never passes UA/OS mismatch (macOS UA on Linux) — ensure the Linux UA in start_hf.sh. If it still loops, it's the datacenter IP (HF = AWS); route through a residential proxy via --proxy-server=http://user:pass@host:port
chrome://extensions/ empty, no error in log Branded Chrome ignoring --load-extension → use Chrome for Testing (see above)
COPY failed: ... not found at build Extension source dir name in the Dockerfile doesn't match disk
Extension listed but service worker idle MV3 headless quirk — open the extension's toggle or hit Update on the extensions page
Failed to connect to the bus spam in log Harmless D-Bus warnings in a container with no system bus
"Restore pages?" popup Handled by the exit_type=Normal Preferences injection in start_hf.sh