Spaces:
Sleeping
fix(deploy): KI-119 — disable persistent /data/vectors symlink
Browse filesROOT CAUSE of HF Space empty-retrieval bug found.
Pre-fix entrypoint.sh:
rm -rf /app/rag/vectors # delete fresh snapshot
ln -sf /data/vectors /app/rag/vectors # symlink to persistent disk
The Dockerfile snapshot_downloads a fresh Chroma from the HF Dataset
at build time. The entrypoint then DELETED that fresh copy and pointed
to /data/vectors, which on second+ builds held STALE Chroma from a
prior run — including the corrupted profile_anonymous row that broke
every collection.query() call. Our dataset upload couldn't help because
the symlink overrode it. /api/coverage reported 7356 chunks (from the
stale persistent copy) but collection.query() returned 0 on every QA
turn — KI-111's wrap caught the exception and degraded to empty.
Post-fix:
- /data/vectors symlink REMOVED. Vectors stay at /app/rag/vectors,
sourced fresh from the dataset on every build.
- Stale /app/rag/vectors symlink unlinked at boot if found, so the
fresh snapshot_download directory underneath becomes visible.
- DuckDB persistence kept (no corruption surface, benefits from
cross-rebuild metadata cache).
To purge the stale /data on the Space: Settings → Factory Reset.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- entrypoint.sh +26 -7
|
@@ -14,20 +14,39 @@
|
|
| 14 |
|
| 15 |
set -e
|
| 16 |
|
| 17 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
if [ -d "/data" ] && [ -w "/data" ]; then
|
| 19 |
-
export VECTOR_DIR="/data/vectors"
|
| 20 |
export DUCKDB_PATH="/data/policies.duckdb"
|
| 21 |
-
# Symlink so the app code (which reads from rag/vectors and
|
| 22 |
-
# rag/policies.duckdb) finds them on the persistent disk
|
| 23 |
-
mkdir -p /data/vectors
|
| 24 |
-
rm -rf /app/rag/vectors
|
| 25 |
-
ln -sf /data/vectors /app/rag/vectors
|
| 26 |
if [ ! -f /data/policies.duckdb ] && [ -f /app/rag/policies.duckdb ]; then
|
| 27 |
cp /app/rag/policies.duckdb /data/policies.duckdb
|
| 28 |
fi
|
| 29 |
rm -f /app/rag/policies.duckdb
|
| 30 |
ln -sf /data/policies.duckdb /app/rag/policies.duckdb
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
fi
|
| 32 |
|
| 33 |
# Validate Chroma is readable + populated; rebuild if not.
|
|
|
|
| 14 |
|
| 15 |
set -e
|
| 16 |
|
| 17 |
+
# KI-119 (2026-05-15) — DISABLE persistent-disk symlink for vectors.
|
| 18 |
+
#
|
| 19 |
+
# Pre-fix, this block unconditionally rm -rf'd the /app/rag/vectors
|
| 20 |
+
# directory (freshly snapshot_downloaded from the HF dataset at build
|
| 21 |
+
# time) and replaced it with a symlink to /data/vectors. On second+
|
| 22 |
+
# builds, /data/vectors held STALE Chroma from a prior run — including
|
| 23 |
+
# the corrupted profile_anonymous row from the KI-102 deploy that broke
|
| 24 |
+
# every collection.query() call. The dataset upload couldn't help because
|
| 25 |
+
# the entrypoint's symlink overrode it.
|
| 26 |
+
#
|
| 27 |
+
# We don't need persistent vectors. The whole point of pushing rag/vectors
|
| 28 |
+
# to the companion HF Dataset is that EVERY Space rebuild pulls a fresh
|
| 29 |
+
# copy. /data persistence is now disabled for vectors; the app reads
|
| 30 |
+
# directly from /app/rag/vectors which contains the just-downloaded fresh
|
| 31 |
+
# snapshot. DuckDB persistence kept (it's only used for cached metadata
|
| 32 |
+
# and benefits from cross-rebuild persistence with no corruption surface).
|
| 33 |
+
#
|
| 34 |
+
# If you NEED to test with a clean /data, manually `rm -rf /data/vectors`
|
| 35 |
+
# on the Space's persistent disk via Settings → Reset.
|
| 36 |
if [ -d "/data" ] && [ -w "/data" ]; then
|
|
|
|
| 37 |
export DUCKDB_PATH="/data/policies.duckdb"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
if [ ! -f /data/policies.duckdb ] && [ -f /app/rag/policies.duckdb ]; then
|
| 39 |
cp /app/rag/policies.duckdb /data/policies.duckdb
|
| 40 |
fi
|
| 41 |
rm -f /app/rag/policies.duckdb
|
| 42 |
ln -sf /data/policies.duckdb /app/rag/policies.duckdb
|
| 43 |
+
# Vectors stay at /app/rag/vectors — read from the fresh dataset
|
| 44 |
+
# snapshot. The previous /data/vectors symlink is intentionally removed.
|
| 45 |
+
if [ -L "/app/rag/vectors" ]; then
|
| 46 |
+
# Pre-existing symlink from older deploys — unlink it so /app reads
|
| 47 |
+
# the fresh snapshot_download'd directory underneath.
|
| 48 |
+
rm /app/rag/vectors 2>/dev/null || true
|
| 49 |
+
fi
|
| 50 |
fi
|
| 51 |
|
| 52 |
# Validate Chroma is readable + populated; rebuild if not.
|