Spaces:
Build error
fix: chdir into app-data dir at desktop startup, fixing 30+ writes at once (P-1)
Browse filesRoot cause: grepped every bare (non-__file__-anchored) CWD-relative
path in the codebase. Found DATA_RAW_PATH/DATA_PROCESSED_PATH/
DATA_SAMPLES_PATH in config/settings.py were dead code exactly like
LOGS_PATH was before its own fix -- zero real consumers reference
them. Instead, 30+ real call sites hardcode their own copy of the
same path directly: all 19 scrapers' data/samples/*.json sample-
fallback output, processing/pipeline.py (data/processed + data/samples),
ai/multi_investigator.py, ai/risk_scorer.py,
ai/self_learning/schema_learner.py, processing/alias_graph.py, and
graph/loader.py's own glob() read of pipeline output.
Trigger condition: on a real Windows install, a double-clicked app's
CWD is unpredictable, and the install directory itself is typically
read-only for a non-admin user. Every one of these 30+ writes would
fail outright under that condition -- not just lose regeneratable
cache data, but error out entirely (scrapers couldn't run at all).
Why this fix looks different from this branch's sibling commits
(case_memory.json, logs, dossier output): those needed explicit
per-file redirection because their paths were __file__-anchored
(computed from the file's own location on disk, independent of CWD).
Every one of these 30+ paths, by contrast, is genuinely bare
CWD-relative with no anchoring at all -- confirmed via grep, not
assumed. That means a single os.chdir() before any other import fixes
all 30+ at once, with zero changes needed to any individual file,
rather than repeating 30 individual redirections.
The one real risk this could have introduced: python-dotenv's
load_dotenv() is called with no arguments in 10 different files
across the codebase, and naively I'd have guessed that's CWD-
dependent -- if it were, chdir()-ing before those calls run would
have silently broken .env/API-key loading in desktop mode. Verified
directly, empirically, before writing this fix (not assumed): built
a minimal reproduction proving load_dotenv() resolves relative to the
*calling file's own frame*, not CWD -- confirmed a chdir() between a
script's own location and the load_dotenv() call inside it does NOT
break discovery of a .env file that sits next to that script. This
codebase's own load_dotenv() call sites are all genuine files with
real __file__ values (not exec()'d strings), so they're unaffected.
Also verified no other CWD-relative assumption exists anywhere else
in the codebase (frontend serving, dataset CSV loading, static file
directories) that this chdir() could break -- grepped for all of
them, zero hits beyond the 30+ already accounted for.
Verified: executed packaging/desktop_entry.py's actual top-level
init code via runpy.run_path() (the same file-path-based execution
PyInstaller's own Analysis() uses, and how a developer runs this
script directly -- NOT via , which
collides with the real PyPI library also installed in
any standard Python environment; a real, pre-existing naming
collision worth knowing about but which does not affect either actual
invocation path, both of which are file-path-based, not import-based).
Confirmed CWD actually changes to the app-data directory, confirmed a
real bare-relative write (exactly matching every scraper's own
pattern) lands there, confirmed the real api.main app still imports
correctly afterward, and confirmed the project's own source directory
is never touched. Full suite: 116 passed, 0 failed.
- config/settings.py +14 -0
- packaging/desktop_entry.py +33 -0
|
@@ -182,6 +182,20 @@ MAX_RETRIES = 3
|
|
| 182 |
REQUEST_TIMEOUT = 30
|
| 183 |
|
| 184 |
# Data paths
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 185 |
DATA_RAW_PATH = "data/raw/"
|
| 186 |
DATA_PROCESSED_PATH = "data/processed/"
|
| 187 |
DATA_SAMPLES_PATH = "data/samples/"
|
|
|
|
| 182 |
REQUEST_TIMEOUT = 30
|
| 183 |
|
| 184 |
# Data paths
|
| 185 |
+
# NOTE (P-1 finding, same shape as LOGS_PATH's own history before its fix):
|
| 186 |
+
# these three constants are dead code -- grepped every consumer, zero hits
|
| 187 |
+
# anywhere in the tree. All 30+ real writers/readers (19 scrapers' sample
|
| 188 |
+
# fallback output, processing/pipeline.py, ai/multi_investigator.py,
|
| 189 |
+
# ai/risk_scorer.py, ai/self_learning/schema_learner.py,
|
| 190 |
+
# processing/alias_graph.py, graph/loader.py's glob read) hardcode their
|
| 191 |
+
# own bare "data/processed/..." or "data/samples/..." string directly
|
| 192 |
+
# instead. Unlike LOGS_PATH, these were NOT individually wired up to this
|
| 193 |
+
# constant, because every one of them is genuinely CWD-relative (no
|
| 194 |
+
# __file__-anchoring anywhere) -- so the actual desktop-mode fix is a
|
| 195 |
+
# single os.chdir() into get_app_data_dir() in packaging/desktop_entry.py,
|
| 196 |
+
# which redirects all 30+ of them at once with no per-file changes needed.
|
| 197 |
+
# Left here as an accurate description of the intended folder layout, not
|
| 198 |
+
# as something any code currently reads.
|
| 199 |
DATA_RAW_PATH = "data/raw/"
|
| 200 |
DATA_PROCESSED_PATH = "data/processed/"
|
| 201 |
DATA_SAMPLES_PATH = "data/samples/"
|
|
@@ -19,6 +19,30 @@ whenever the app is started this way, and so config/settings.py's local
|
|
| 19 |
config-file loading is the primary configuration path a desktop user
|
| 20 |
actually has, rather than a code path that only matters at packaging time.
|
| 21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
Binds to 127.0.0.1 only, matching the target architecture ("bound to
|
| 23 |
127.0.0.1 only") -- this process is meant to be reached only by the Tauri
|
| 24 |
shell's own webview (Phase D4), never by the network.
|
|
@@ -41,6 +65,15 @@ else:
|
|
| 41 |
_app_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
| 42 |
sys.path.insert(0, _app_root)
|
| 43 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
import uvicorn
|
| 45 |
from api.main import app
|
| 46 |
|
|
|
|
| 19 |
config-file loading is the primary configuration path a desktop user
|
| 20 |
actually has, rather than a code path that only matters at packaging time.
|
| 21 |
|
| 22 |
+
P-1 fix: also chdir()s into the OS-appropriate app-data directory before
|
| 23 |
+
any other import. Grepped every bare (non-__file__-anchored) CWD-relative
|
| 24 |
+
path in the codebase: 30+ real call sites across all 19 scrapers'
|
| 25 |
+
data/samples/*.json sample-fallback output, processing/pipeline.py,
|
| 26 |
+
ai/multi_investigator.py, ai/risk_scorer.py, ai/self_learning/
|
| 27 |
+
schema_learner.py, processing/alias_graph.py, and graph/loader.py's own
|
| 28 |
+
glob() read of pipeline output -- none of which reference the (equally
|
| 29 |
+
dead) DATA_RAW_PATH/DATA_PROCESSED_PATH/DATA_SAMPLES_PATH constants in
|
| 30 |
+
config/settings.py; each just hardcodes its own "data/processed/..." or
|
| 31 |
+
"data/samples/..." string directly. On a real Windows install, a
|
| 32 |
+
double-clicked app's CWD defaults to (and the install directory itself is
|
| 33 |
+
typically read-only for) a non-admin user -- meaning every one of those
|
| 34 |
+
30+ writes would fail outright, not just lose regeneratable cache data.
|
| 35 |
+
Verified every one of them is genuinely CWD-relative (confirmed via
|
| 36 |
+
direct testing that python-dotenv's own load_dotenv() calls elsewhere in
|
| 37 |
+
the codebase are NOT affected by this, since dotenv resolves relative to
|
| 38 |
+
the calling file's own frame, not CWD) -- so a single chdir() here fixes
|
| 39 |
+
all 30+ call sites identically, with zero changes needed to any of them,
|
| 40 |
+
rather than repeating the explicit per-file redirection this branch's
|
| 41 |
+
sibling commits used for case_memory.json/logs/dossier output (which
|
| 42 |
+
were __file__-anchored, not CWD-relative, and so could NOT have been
|
| 43 |
+
fixed this way). Web-deployment behavior is unchanged -- this file is
|
| 44 |
+
never imported by the Docker/app.py path.
|
| 45 |
+
|
| 46 |
Binds to 127.0.0.1 only, matching the target architecture ("bound to
|
| 47 |
127.0.0.1 only") -- this process is meant to be reached only by the Tauri
|
| 48 |
shell's own webview (Phase D4), never by the network.
|
|
|
|
| 65 |
_app_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
| 66 |
sys.path.insert(0, _app_root)
|
| 67 |
|
| 68 |
+
# P-1 fix: chdir into a guaranteed-writable directory before any other
|
| 69 |
+
# import that might (transitively, later, at request time) write a bare
|
| 70 |
+
# CWD-relative path. Must come after sys.path is set up (needs to import
|
| 71 |
+
# config.settings) but before uvicorn/api.main are imported.
|
| 72 |
+
from config.settings import get_app_data_dir
|
| 73 |
+
_workdir = get_app_data_dir()
|
| 74 |
+
os.makedirs(_workdir, exist_ok=True)
|
| 75 |
+
os.chdir(_workdir)
|
| 76 |
+
|
| 77 |
import uvicorn
|
| 78 |
from api.main import app
|
| 79 |
|