scripts/record_demo.md

# Proof-tier recording — shot list

A ~90-second screen recording that shows the real streaming pipeline end to end,
then the live demo site. Keep it tight; recruiters skim.

## Before you hit record

- Free the RAM: close other apps. Have Docker Desktop running.
- Reset for a clean run so counts start at zero. **Use `--profile stream` on the
  `down`** — the Spark checkpoint volume belongs to the profiled `spark-job`
  service, and a plain `docker compose down -v` leaves it behind. A stale
  checkpoint makes the next run fail with
  `offset was changed from N to M, some data may have been missed`:
  ```bash
  docker compose --profile stream down -v   # wipe ALL volumes incl. spark_checkpoints
  docker compose up -d                       # redpanda + cassandra + qdrant
  ```
  Wait until `docker compose ps` shows cassandra **healthy** (~60 s). Then
  produce **once** and run the Spark job **once** (below) — because the job is
  checkpointed, re-running it without a fresh reset processes zero new rows.
- Have four terminals (or panes) ready in the repo with the **venv active** —
  the Python commands below need the project deps (`confluent-kafka`, etc.):
  ```powershell
  .venv\Scripts\Activate.ps1        # Windows PowerShell
  # source .venv/bin/activate       # macOS/Linux
  ```
  If activation is blocked by execution policy, either run
  `Set-ExecutionPolicy -Scope Process RemoteSigned` first, or skip activation
  and call the interpreter directly: `.venv\Scripts\python.exe <script> ...`.
- Optional: increase terminal font size for legibility.

## Recording tools

- Windows: **Xbox Game Bar** (Win+G) or **OBS Studio** (free). OBS gives the
  cleanest output and lets you crop to the terminal + browser.
- Export as MP4, 1080p, < 30 MB if you plan to commit it; otherwise upload
  unlisted to YouTube and embed the link (see below).

> Command blocks below are written for **Windows PowerShell** (the recording
> shell); bash/macOS equivalents are noted where they differ. Python commands
> assume the **venv is active** (see prep) — otherwise call `.venv\Scripts\python.exe`.

## The shots

1. **Architecture (5 s).** Show the README architecture diagram or the app's
   "Architecture" tab. One sentence of voiceover/caption:
   "Documents stream through Kafka, Spark embeds them, they land in Cassandra
   and Qdrant."

2. **Start streaming (20 s).** Split screen:
   - Terminal A — produce into Kafka (venv active — see prep):
     ```powershell
     python producer/produce.py --limit 300 --rate 100
     ```
   - Terminal B — run the Spark job; let the log line land on camera. The
     `--build` is needed the first time (and after any code change):
     ```powershell
     docker compose --profile stream up --build spark-job
     ```
     Point at: `INFO Batch N: wrote 300 docs to Cassandra + Qdrant`, then
     `INFO Stream finished.` and a clean `exited with code 0`.

3. **Prove both stores populated (15 s).**
   ```powershell
   docker exec streamsearch-cassandra cqlsh -e "SELECT count(*) FROM streamsearch.documents;"
   (Invoke-RestMethod http://localhost:6533/collections/documents).result.points_count
   ```
   Show 300 in each. (bash/macOS: `curl -s http://localhost:6533/collections/documents | jq .result.points_count`
   — needs `jq`; in PowerShell use `curl.exe`, since bare `curl` is an alias for
   `Invoke-WebRequest`.)

4. **Query the streamed index with the same search stack (20 s).**
   ```powershell
   python scripts/query_proof.py "support vector machine kernels"
   ```
   Point at a result whose `dense_rank` and `bm25_rank` differ (e.g. dense 10 /
   bm25 2) to show hybrid fusion doing real work.

5. **The live demo site (25 s).** Open the HF Space
   (https://huggingface.co/spaces/sharmaaryan/streamsearch):
   - run a search, show ranked results with scores;
   - paste a document into the **Ingest** box, submit, then search a phrase from
     it and show it appear in results within seconds.

6. **End card (5 s).** Show the live URL on screen.

## After recording

- If committing the file: put it at `docs/demo.mp4` and reference it in the
  README (`![demo](docs/demo.mp4)` renders as a link on GitHub; GitHub also
  auto-embeds an uploaded MP4 in a release/issue you can link to).
- If hosting (YouTube/Loom): set `VIDEO_URL` and the Streamlit UI shows it in
  the Architecture tab; put the same link at the top of the README.