lvwerra HF Staff Claude Fable 5 commited on
Commit
bddccfa
·
1 Parent(s): 0cdd247

Codex home moves to local disk; rollouts stay on the bucket

Browse files

The per-file sqlite symlink was whack-a-mole: codex put the mmap'd
-shm sibling next to the symlink (on the bucket) and grew two more
databases (goals_1, memories_1) there — sqlite on FUSE keeps
corrupting. Codex now gets the OpenClaw treatment: CODEX_HOME on local
disk, the append-only rollouts staying durable on the bucket via one
sessions/ symlink (pinned rollout paths keep working), small state
restored at boot and synced back every 60s, sqlite purely local and
rebuilt from rollouts when the disk resets. Existing bucket sqlite
remnants are quarantined into db-backups/am-quarantine.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Files changed (1) hide show
  1. entrypoint.sh +29 -14
entrypoint.sh CHANGED
@@ -14,10 +14,10 @@ export DATA_DIR
14
  # restarts (gemini ~/.gemini, opencode ~/.local/share, hermes ~/.hermes, etc.).
15
  export HOME="$DATA_DIR/home"
16
  mkdir -p "$HOME"
17
- # Claude/Codex keep their established dirs (so existing logins keep working).
 
18
  export CLAUDE_CONFIG_DIR="$DATA_DIR/state/claude"
19
- export CODEX_HOME="$DATA_DIR/state/codex"
20
- mkdir -p "$CLAUDE_CONFIG_DIR" "$CODEX_HOME"
21
 
22
  # NOTE: every var exported below must be listed in NON_SECRET (server/src/
23
  # index.js) — this script runs after the build-time env snapshot, so anything
@@ -30,17 +30,32 @@ export AM_LOCAL="/home/node/local"
30
  if ! mkdir -p "$AM_LOCAL/bin" 2>/dev/null; then AM_LOCAL="$DATA_DIR/.local-cache"; mkdir -p "$AM_LOCAL/bin"; fi
31
  export UV_CACHE_DIR="$AM_LOCAL/uv-cache"
32
 
33
- # Codex keeps a local SQLite database (logs_2.sqlite) in CODEX_HOME. SQLite on
34
- # the FUSE bucket corrupts (object storage can't lock/mmap properly) codex
35
- # then reports "file is not a database" on startup. The DB is a rebuildable
36
- # cache derived from the rollouts, so it needs speed, not durability: point it
37
- # at local disk via symlink. When the local file vanishes on restart, codex
38
- # simply rebuilds it.
39
- mkdir -p "$AM_LOCAL/codex-db"
40
- if [ ! -L "$CODEX_HOME/logs_2.sqlite" ]; then
41
- rm -f "$CODEX_HOME/logs_2.sqlite" "$CODEX_HOME/logs_2.sqlite-wal" "$CODEX_HOME/logs_2.sqlite-shm" 2>/dev/null || true
42
- ln -s "$AM_LOCAL/codex-db/logs_2.sqlite" "$CODEX_HOME/logs_2.sqlite"
43
- fi
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  export PIP_CACHE_DIR="$AM_LOCAL/pip-cache"
45
  export PYTHONPYCACHEPREFIX="$AM_LOCAL/pycache"
46
  export PYTHONUSERBASE="$AM_LOCAL/py" # pip install --user → local, fast
 
14
  # restarts (gemini ~/.gemini, opencode ~/.local/share, hermes ~/.hermes, etc.).
15
  export HOME="$DATA_DIR/home"
16
  mkdir -p "$HOME"
17
+ # Claude keeps its established dir (so existing logins keep working). Codex's
18
+ # home moves to local disk below (its SQLite databases corrupt on the bucket).
19
  export CLAUDE_CONFIG_DIR="$DATA_DIR/state/claude"
20
+ mkdir -p "$CLAUDE_CONFIG_DIR"
 
21
 
22
  # NOTE: every var exported below must be listed in NON_SECRET (server/src/
23
  # index.js) — this script runs after the build-time env snapshot, so anything
 
30
  if ! mkdir -p "$AM_LOCAL/bin" 2>/dev/null; then AM_LOCAL="$DATA_DIR/.local-cache"; mkdir -p "$AM_LOCAL/bin"; fi
31
  export UV_CACHE_DIR="$AM_LOCAL/uv-cache"
32
 
33
+ # Codex keeps a growing family of SQLite databases (logs_2, goals_1, memories_1
34
+ # plus mmap'd -shm siblings) that corrupt on the FUSE bucket: SQLite needs real
35
+ # locking/mmap. Same cure as OpenClaw codex's HOME lives on LOCAL disk. The
36
+ # heavyweight append-only rollouts stay on the bucket via one symlink (plain
37
+ # files never corrupted there, and pinned rollout paths keep working); the
38
+ # small durable state (auth, config, history) restores at boot and syncs back
39
+ # every 60s; the SQLite caches are purely local, rebuilt from rollouts when
40
+ # the disk resets.
41
+ CODEX_DURABLE="$DATA_DIR/state/codex"
42
+ export CODEX_HOME="$AM_LOCAL/codex-home"
43
+ mkdir -p "$CODEX_HOME" "$CODEX_DURABLE/sessions" "$CODEX_DURABLE/db-backups"
44
+ # quarantine sqlite remnants on the bucket (incl. the earlier symlink attempt)
45
+ mkdir -p "$CODEX_DURABLE/db-backups/am-quarantine"
46
+ for f in "$CODEX_DURABLE"/logs_2.sqlite* "$CODEX_DURABLE"/goals_1.sqlite* "$CODEX_DURABLE"/memories_1.sqlite*; do
47
+ [ -e "$f" ] || [ -L "$f" ] && mv "$f" "$CODEX_DURABLE/db-backups/am-quarantine/" 2>/dev/null || true
48
+ done
49
+ rsync -a --exclude 'sessions' --exclude '*.sqlite*' --exclude 'db-backups' \
50
+ --exclude 'cache' --exclude '.tmp' --exclude 'mcp-oauth-locks' \
51
+ "$CODEX_DURABLE/" "$CODEX_HOME/" 2>/dev/null || true
52
+ ln -sfn "$CODEX_DURABLE/sessions" "$CODEX_HOME/sessions"
53
+ ( while :; do
54
+ sleep 60
55
+ rsync -a --exclude 'sessions' --exclude '*.sqlite*' --exclude 'db-backups' \
56
+ --exclude 'cache' --exclude '.tmp' --exclude 'mcp-oauth-locks' \
57
+ "$CODEX_HOME/" "$CODEX_DURABLE/" 2>/dev/null || true
58
+ done ) &
59
  export PIP_CACHE_DIR="$AM_LOCAL/pip-cache"
60
  export PYTHONPYCACHEPREFIX="$AM_LOCAL/pycache"
61
  export PYTHONUSERBASE="$AM_LOCAL/py" # pip install --user → local, fast