# PixelStream 🖥️➡️🌐

A mini "TeamViewer for a browser". You open a web UI, hit **Start Browser**, and a
**headless Chromium running inside a Docker container** streams its screen back to
your browser in real time. You can **click, scroll, type, and navigate** — and it
all happens inside that remote headless browser...........


---

## How it works

```
┌──────────────┐   click / scroll / type (WebSocket)   ┌───────────────┐   CDP    ┌──────────────────┐
│  Web UI      │  ───────────────────────────────────► │  Node backend │ ───────► │ Headless Chromium │
│ (Next.js)    │                                        │  (ws + CDP)   │          │   (Puppeteer)     │
│  <canvas>    │  ◄─────────────────────────────────── │               │ ◄─────── │                  │
└──────────────┘     live JPEG frames (WebSocket)       └───────────────┘ screencast └──────────────────┘
```

- **Frontend** (`/frontend`) — Next.js app. A `<canvas>` paints the live frames;
  mouse/keyboard/scroll events are captured, converted to real browser
  coordinates, and sent over a WebSocket.
- **Backend** (`/backend`) — Node.js + `ws` + Puppeteer. Launches Chromium, uses the
  **Chrome DevTools Protocol (CDP)** `Page.startScreencast` to get a continuous
  stream of JPEG frames, and injects input via `Input.dispatchMouseEvent` /
  `Input.dispatchKeyEvent`.
- **Docker** — both services run in containers via `docker compose`. The backend
  uses the official Puppeteer image (Chromium + system deps preinstalled).

### Why CDP screencast (not a screenshot loop)?

`Page.startScreencast` only emits a frame when the page actually changes, and it's
much smoother and lower-latency than taking `page.screenshot()` on a timer.

---

## Run it

### Option A — Docker (recommended)

```bash
docker compose up --build
```

Then open **http://localhost:3000** and click **Start Browser**.

### Option B — Local (no Docker, for fast iteration)

```bash
# terminal 1
cd backend && npm install && npm start      # ws server on :8080

# terminal 2
cd frontend && npm install && npm run dev    # UI on :3000
```

---

## Deploy (Frontend → Vercel, Backend → Render)

> ⚠️ The browser page is served over **HTTPS** (Vercel), so it can only open a
> **secure** WebSocket (`wss://`). Render gives you an `https://…onrender.com`
> URL, whose WebSocket endpoint is `wss://…onrender.com`. Plain `ws://` will be
> blocked as mixed content.

### 1. Backend on Render

- New **Web Service** → connect this repo.
- Runtime: **Docker**, Dockerfile path: `backend/Dockerfile`, context: `backend`
  (or just use the included `render.yaml` Blueprint).
- Deploy. Note the URL, e.g. `https://pixelstream-backend.onrender.com`.
- Render injects `PORT` automatically — the server already reads it.

### 2. Frontend on Vercel

- New Project → import this repo → set **Root Directory = `frontend`**.
- Add an environment variable:

  | Key | Value |
  |-----|-------|
  | `NEXT_PUBLIC_WS_URL` | `wss://pixelstream-backend.onrender.com` |

- Deploy. Open the Vercel URL → **Start Browser**.


---

## What works ✅

- Start a headless Chromium in a Docker container from the web UI
- Live screen streaming via CDP screencast (JPEG frames over WebSocket)
- Mouse: move, left/right click, scroll
- Keyboard: printable characters + common special keys (Enter, Backspace, Tab,
  arrows, Esc, Delete…)
- URL bar to navigate the remote browser
- Accurate click mapping (canvas coords → real viewport coords)

## Known limitations / where it gets hard ⚠️

- **Keyboard coverage** — only common special keys are mapped; modifier
  combos (Ctrl+C, Shift+selection) aren't fully wired yet.
- **Single session** — one browser per server; no multi-user / multi-tab.
- **Streaming** — JPEG-over-WebSocket is simple but bandwidth-heavy; WebRTC or
  VP8/H.264 encoding would be smoother at scale.
- **Chromium-in-Docker** — needs `--no-sandbox`, `--disable-dev-shm-usage`, and
  `shm_size: 1gb` or Chromium crashes. (These are already set.)

## Next steps 🚀

1. Full keyboard model with modifiers + IME/composition events
2. Multiple isolated sessions (one container per user, spawned on demand)
3. WebRTC streaming for lower latency / adaptive quality
4. Reconnect handling + session timeouts + resource cleanup

---

## Project structure

```
Pixel/
├── docker-compose.yml
├── backend/
│   ├── Dockerfile
│   ├── package.json
│   └── server.js          # ws server + Puppeteer + CDP screencast/input
└── frontend/
    ├── Dockerfile
    ├── package.json
    ├── next.config.js
    └── app/
        ├── layout.js
        └── page.js        # UI + canvas + input capture
```