Farhan Beg commited on
Commit
bf86338
·
1 Parent(s): 8b639d8

fix(restore): exempt .db files from size cap — state.db was 52MB > 50MB limit

Browse files

The real root cause of session loss: state.db (the SQLite database
holding all session data) is 52 MB, which exceeds the 50 MB
MAX_FILE_SIZE_BYTES cap. should_exclude() returned True for state.db →
it was silently skipped on BOTH backup and restore.

The 50 MB cap was meant for model caches and other large non-essential
artifacts, not for essential SQLite databases. As the user's session
history grew past 50 MB, state.db started being excluded — sessions
appeared in the sessions/*.json transcripts (which are small files,
backed up fine) but the main index database was dropped.

Fix:
- Bump MAX_FILE_SIZE_BYTES default from 50 MB to 200 MB (configurable
via SYNC_MAX_FILE_BYTES env var).
- Exempt .db files from the size cap entirely — they're always
essential regardless of size. The cap is for caches, not databases.

This restores state.db (and any other .db file like kanban.db,
response_store.db, lcm.db) to the backup/restore cycle.

Files changed (1) hide show
  1. hermes-sync.py +7 -1
hermes-sync.py CHANGED
@@ -61,7 +61,7 @@ HF_USERNAME = os.environ.get("HF_USERNAME", "").strip()
61
  SPACE_AUTHOR_NAME = os.environ.get("SPACE_AUTHOR_NAME", "").strip()
62
  BACKUP_DATASET_NAME = os.environ.get("BACKUP_DATASET_NAME", "huggingmes-backup").strip()
63
  INCLUDE_ENV = os.environ.get("SYNC_INCLUDE_ENV", "").strip().lower() in {"1", "true", "yes"}
64
- MAX_FILE_SIZE_BYTES = int(os.environ.get("SYNC_MAX_FILE_BYTES", str(50 * 1024 * 1024)))
65
 
66
  EXCLUDED_DIRS = {
67
  ".cache",
@@ -239,6 +239,12 @@ def should_exclude(rel_posix: str, path: Path) -> bool:
239
  name_lower = path.name.lower()
240
  if name_lower.endswith(EXCLUDED_SUFFIXES):
241
  return True
 
 
 
 
 
 
242
  try:
243
  return path.stat().st_size > MAX_FILE_SIZE_BYTES
244
  except OSError:
 
61
  SPACE_AUTHOR_NAME = os.environ.get("SPACE_AUTHOR_NAME", "").strip()
62
  BACKUP_DATASET_NAME = os.environ.get("BACKUP_DATASET_NAME", "huggingmes-backup").strip()
63
  INCLUDE_ENV = os.environ.get("SYNC_INCLUDE_ENV", "").strip().lower() in {"1", "true", "yes"}
64
+ MAX_FILE_SIZE_BYTES = int(os.environ.get("SYNC_MAX_FILE_BYTES", str(200 * 1024 * 1024)))
65
 
66
  EXCLUDED_DIRS = {
67
  ".cache",
 
239
  name_lower = path.name.lower()
240
  if name_lower.endswith(EXCLUDED_SUFFIXES):
241
  return True
242
+ # SQLite .db files are always essential (they hold sessions, kanban,
243
+ # response store, etc.) — never exclude them by size, regardless of
244
+ # how large they grow. The size cap is for model caches and other
245
+ # large non-essential artifacts.
246
+ if name_lower.endswith(".db"):
247
+ return False
248
  try:
249
  return path.stat().st_size > MAX_FILE_SIZE_BYTES
250
  except OSError: