rohitsar567 Claude Opus 4.7 (1M context) commited on
Commit
0690bd6
·
1 Parent(s): 251f011

fix(deploy): KI-119 — disable persistent /data/vectors symlink

Browse files

ROOT CAUSE of HF Space empty-retrieval bug found.

Pre-fix entrypoint.sh:
rm -rf /app/rag/vectors # delete fresh snapshot
ln -sf /data/vectors /app/rag/vectors # symlink to persistent disk

The Dockerfile snapshot_downloads a fresh Chroma from the HF Dataset
at build time. The entrypoint then DELETED that fresh copy and pointed
to /data/vectors, which on second+ builds held STALE Chroma from a
prior run — including the corrupted profile_anonymous row that broke
every collection.query() call. Our dataset upload couldn't help because
the symlink overrode it. /api/coverage reported 7356 chunks (from the
stale persistent copy) but collection.query() returned 0 on every QA
turn — KI-111's wrap caught the exception and degraded to empty.

Post-fix:
- /data/vectors symlink REMOVED. Vectors stay at /app/rag/vectors,
sourced fresh from the dataset on every build.
- Stale /app/rag/vectors symlink unlinked at boot if found, so the
fresh snapshot_download directory underneath becomes visible.
- DuckDB persistence kept (no corruption surface, benefits from
cross-rebuild metadata cache).

To purge the stale /data on the Space: Settings → Factory Reset.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. entrypoint.sh +26 -7
entrypoint.sh CHANGED
@@ -14,20 +14,39 @@
14
 
15
  set -e
16
 
17
- # Use HF Spaces persistent disk if mounted at /data, else fall back to /app
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  if [ -d "/data" ] && [ -w "/data" ]; then
19
- export VECTOR_DIR="/data/vectors"
20
  export DUCKDB_PATH="/data/policies.duckdb"
21
- # Symlink so the app code (which reads from rag/vectors and
22
- # rag/policies.duckdb) finds them on the persistent disk
23
- mkdir -p /data/vectors
24
- rm -rf /app/rag/vectors
25
- ln -sf /data/vectors /app/rag/vectors
26
  if [ ! -f /data/policies.duckdb ] && [ -f /app/rag/policies.duckdb ]; then
27
  cp /app/rag/policies.duckdb /data/policies.duckdb
28
  fi
29
  rm -f /app/rag/policies.duckdb
30
  ln -sf /data/policies.duckdb /app/rag/policies.duckdb
 
 
 
 
 
 
 
31
  fi
32
 
33
  # Validate Chroma is readable + populated; rebuild if not.
 
14
 
15
  set -e
16
 
17
+ # KI-119 (2026-05-15) DISABLE persistent-disk symlink for vectors.
18
+ #
19
+ # Pre-fix, this block unconditionally rm -rf'd the /app/rag/vectors
20
+ # directory (freshly snapshot_downloaded from the HF dataset at build
21
+ # time) and replaced it with a symlink to /data/vectors. On second+
22
+ # builds, /data/vectors held STALE Chroma from a prior run — including
23
+ # the corrupted profile_anonymous row from the KI-102 deploy that broke
24
+ # every collection.query() call. The dataset upload couldn't help because
25
+ # the entrypoint's symlink overrode it.
26
+ #
27
+ # We don't need persistent vectors. The whole point of pushing rag/vectors
28
+ # to the companion HF Dataset is that EVERY Space rebuild pulls a fresh
29
+ # copy. /data persistence is now disabled for vectors; the app reads
30
+ # directly from /app/rag/vectors which contains the just-downloaded fresh
31
+ # snapshot. DuckDB persistence kept (it's only used for cached metadata
32
+ # and benefits from cross-rebuild persistence with no corruption surface).
33
+ #
34
+ # If you NEED to test with a clean /data, manually `rm -rf /data/vectors`
35
+ # on the Space's persistent disk via Settings → Reset.
36
  if [ -d "/data" ] && [ -w "/data" ]; then
 
37
  export DUCKDB_PATH="/data/policies.duckdb"
 
 
 
 
 
38
  if [ ! -f /data/policies.duckdb ] && [ -f /app/rag/policies.duckdb ]; then
39
  cp /app/rag/policies.duckdb /data/policies.duckdb
40
  fi
41
  rm -f /app/rag/policies.duckdb
42
  ln -sf /data/policies.duckdb /app/rag/policies.duckdb
43
+ # Vectors stay at /app/rag/vectors — read from the fresh dataset
44
+ # snapshot. The previous /data/vectors symlink is intentionally removed.
45
+ if [ -L "/app/rag/vectors" ]; then
46
+ # Pre-existing symlink from older deploys — unlink it so /app reads
47
+ # the fresh snapshot_download'd directory underneath.
48
+ rm /app/rag/vectors 2>/dev/null || true
49
+ fi
50
  fi
51
 
52
  # Validate Chroma is readable + populated; rebuild if not.