crawler: retry queue + 429 backoff, robots.txt Disallow, config-driven docs paths

track eval_history.json in repo for all DBs

fix: merge operational config fields on HF restart + agentfactory scope fix

fix: remove UTF-8 BOM from DB config files

security: fix P1 issues - XSS, dockerignore, sessionStorage, tenant isolation

Fix: remove 6-char password min, no-cache headers, path-scoped crawl

Fix: MALI IDK gate, identity audit, gap KB wrong DB, crawl interval, autofill

set per-DB admin_password to DB name for all 7 DBs

fix: commit DB config files so bots work even when GitHub sync fails