scripts/record_demo.md

Proof-tier recording — shot list

A ~90-second screen recording that shows the real streaming pipeline end to end, then the live demo site. Keep it tight; recruiters skim.

Before you hit record

• Free the RAM: close other apps. Have Docker Desktop running.
• Reset for a clean run so counts start at zero. Use --profile stream on the down — the Spark checkpoint volume belongs to the profiled spark-job service, and a plain docker compose down -v leaves it behind. A stale checkpoint makes the next run fail with offset was changed from N to M, some data may have been missed:

  docker compose --profile stream down -v   # wipe ALL volumes incl. spark_checkpoints
  docker compose up -d                       # redpanda + cassandra + qdrant
  

  Wait until docker compose ps shows cassandra healthy (~60 s). Then produce once and run the Spark job once (below) — because the job is checkpointed, re-running it without a fresh reset processes zero new rows.
• Have four terminals (or panes) ready in the repo with the venv active — the Python commands below need the project deps (confluent-kafka, etc.):

  .venv\Scripts\Activate.ps1        # Windows PowerShell
  # source .venv/bin/activate       # macOS/Linux
  

  If activation is blocked by execution policy, either run Set-ExecutionPolicy -Scope Process RemoteSigned first, or skip activation and call the interpreter directly: .venv\Scripts\python.exe <script> ....
• Optional: increase terminal font size for legibility.

Recording tools

• Windows: Xbox Game Bar (Win+G) or OBS Studio (free). OBS gives the cleanest output and lets you crop to the terminal + browser.
• Export as MP4, 1080p, < 30 MB if you plan to commit it; otherwise upload unlisted to YouTube and embed the link (see below).

Command blocks below are written for Windows PowerShell (the recording shell); bash/macOS equivalents are noted where they differ. Python commands assume the venv is active (see prep) — otherwise call .venv\Scripts\python.exe.

The shots

1. Architecture (5 s). Show the README architecture diagram or the app's "Architecture" tab. One sentence of voiceover/caption: "Documents stream through Kafka, Spark embeds them, they land in Cassandra and Qdrant."

2. Start streaming (20 s). Split screen:

   • Terminal A — produce into Kafka (venv active — see prep):

     python producer/produce.py --limit 300 --rate 100
     

   • Terminal B — run the Spark job; let the log line land on camera. The --build is needed the first time (and after any code change):

     docker compose --profile stream up --build spark-job
     

     Point at: INFO Batch N: wrote 300 docs to Cassandra + Qdrant, then INFO Stream finished. and a clean exited with code 0.

3. Prove both stores populated (15 s).

   docker exec streamsearch-cassandra cqlsh -e "SELECT count(*) FROM streamsearch.documents;"
   (Invoke-RestMethod http://localhost:6533/collections/documents).result.points_count
   

   Show 300 in each. (bash/macOS: curl -s http://localhost:6533/collections/documents | jq .result.points_count — needs jq; in PowerShell use curl.exe, since bare curl is an alias for Invoke-WebRequest.)

4. Query the streamed index with the same search stack (20 s).

   python scripts/query_proof.py "support vector machine kernels"
   

   Point at a result whose dense_rank and bm25_rank differ (e.g. dense 10 / bm25 2) to show hybrid fusion doing real work.

5. The live demo site (25 s). Open the HF Space (https://huggingface.co/spaces/sharmaaryan/streamsearch):

   • run a search, show ranked results with scores;
   • paste a document into the Ingest box, submit, then search a phrase from it and show it appear in results within seconds.

6. End card (5 s). Show the live URL on screen.

After recording

• If committing the file: put it at docs/demo.mp4 and reference it in the README (![demo](docs/demo.mp4) renders as a link on GitHub; GitHub also auto-embeds an uploaded MP4 in a release/issue you can link to).
• If hosting (YouTube/Loom): set VIDEO_URL and the Streamlit UI shows it in the Architecture tab; put the same link at the top of the README.