Ship prebuilt embeddings; stop embedding at Docker build time (fixes build timeout)

The Dockerfile ran 'python -m canlex.embed' at image build, embedding the whole
corpus on the build host. As the corpus grew to ~20k chunks that step reached
~33 minutes and began tripping Hugging Face Spaces' job timeout -- the build
finished and pushed the image, but the job was marked failed and the container
torn down, taking the server down.

Fix: commit data/processed/embeddings.npz as a git-LFS artifact (it is
deterministic and ~34 MB) and COPY it into the image; the build now only
pre-fetches the bge query model and the reranker (a few minutes) instead of
re-embedding. embeddings.npz un-gitignored, LFS-tracked, and removed from
.dockerignore. Verified the committed vectors' ids match the committed corpus
exactly (20,381).

Workflow change: after any corpus edit, run 'python -m canlex.embed' and commit
the regenerated embeddings.npz (it no longer rebuilds itself in the image).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

.dockerignore

@@ -6,12 +6,9 @@ __pycache__/
 **/__pycache__/
 *.pyc
 
-# Raw downloads are not needed -- only data/processed/*.json is.
+# Raw downloads are not needed -- only data/processed/ is.
 data/raw/
 
-# Regenerated inside the image by 'python -m canlex.embed'.
-data/processed/embeddings.npz
-
 # Local-only: the CanLII key (injected as a runtime secret) and disk caches.
 canlii_key.txt
 data/citator_dbmap.json

.gitattributes

@@ -1 +1,2 @@
 data/processed/caselaw.json filter=lfs diff=lfs merge=lfs -text
+data/processed/embeddings.npz filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -3,6 +3,5 @@ __pycache__/
 .venv/
 venv/
 data/raw/
-data/processed/embeddings.npz
 canlii_key.txt
 data/citator_*.json

Dockerfile

@@ -19,9 +19,15 @@ WORKDIR /app
 COPY requirements.txt .
 RUN pip install --no-cache-dir -r requirements.txt
 
-# Application code and the processed corpus (section-chunk JSON).
+# Application code, the processed corpus (section-chunk JSON), and the prebuilt
+# semantic embeddings (embeddings.npz, committed via git-LFS). Shipping the
+# embeddings as an artifact keeps the build fast: embedding ~20k chunks on the
+# build host took ~33 min and was tripping Hugging Face's job timeout as the
+# corpus grew. Regenerate with 'python -m canlex.embed' after any corpus change
+# and commit the result.
 COPY --chown=app:app canlex/ ./canlex/
 COPY --chown=app:app data/processed/*.json ./data/processed/
+COPY --chown=app:app data/processed/embeddings.npz ./data/processed/
 
 USER app
 ENV HOME=/app \
@@ -30,9 +36,11 @@ ENV HOME=/app \
     PORT=7860 \
     PYTHONUNBUFFERED=1
 
-# Build the semantic embeddings and pre-fetch the cross-encoder model, so the
-# model cache is baked into the image and the first request needs no network.
-RUN python -m canlex.embed \
+# Pre-fetch the embedder and cross-encoder models so the cache is baked into the
+# image and the first request needs no network. The corpus is NOT re-embedded
+# here -- the committed embeddings.npz is used as-is; only the bge query model
+# (for embedding live queries) and the reranker are downloaded.
+RUN python -c "from canlex.embed import Embedder; Embedder()" \
  && python -c "from canlex.rerank import Reranker; Reranker()"
 
 # From here on, model files are served from the baked cache, never fetched.

data/processed/embeddings.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1e6554d83a8a3a37d77a0f0e032e311ceb3187d2bb742170c05549e2c4d65f7
+size 36126270