File size: 3,709 Bytes
c9d432b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 | """scope_cache.py — stale-while-refresh for the scope-shaped caches (the owner cache rule).
THE RULE, ported from the Streamlit host's cache architecture ([[ri-os-cache-architecture]]):
once a copy exists, NO request ever blocks on a rebuild. A fresh hit returns. A STALE hit
returns the stale copy immediately and refreshes ONCE in a background thread (per-key
single-flight). Only a true miss — no copy at all — builds synchronously, and concurrent
missers wait on the builder's lock instead of stampeding Odoo with N identical pulls.
WHY THIS EXISTS. The wave-2 caches were expire-and-block: at TTL+1s the next visitor paid the
full Odoo pool build (10–30s) in the request path, every 15 minutes, per scope — measured live
as the "extremely long loads". Serving 15-minute-old figures for the seconds a refresh takes is
the documented, owner-chosen trade; blocking a person on an Odoo pull is not.
WHAT THIS DOES NOT CHANGE. The cache MAP and its entry shape stay exactly as the routes had
them — `rt.pool_cache[key] = (ts, value)` — because `verify_api.py` asserts that map holds only
scope-keyed raw values (the cross-user-leak gate). This module only decides WHEN `build()` runs
and on which thread.
⚠ `build` runs with no request context on the refresh path — it must be scope-shaped only
(team_id/agent/granularity), never session-shaped. That is already the routes' own leak rule.
"""
import threading
import time
#: (id(cache), key) -> Lock. Keyed by the cache object's identity so two tenants' runtimes can
#: never share a flight; bounded by the caches' own eviction (a lock per live cache key).
_FLIGHTS = {}
_FLIGHTS_GUARD = threading.Lock()
def _flight(cache, key):
fk = (id(cache), key)
with _FLIGHTS_GUARD:
lk = _FLIGHTS.get(fk)
if lk is None:
lk = _FLIGHTS[fk] = threading.Lock()
# The map only grows with distinct live keys; prune anything not currently held once it
# gets silly. Cheap and rare — this is hygiene, not a hot path.
if len(_FLIGHTS) > 512:
for k in [k for k, v in _FLIGHTS.items() if not v.locked()][:256]:
_FLIGHTS.pop(k, None)
return lk
def get(cache, key, ttl, build, evict=None):
"""The value for `key`, per the rule above. `evict` (optional callable) runs after a write
to apply the caller's own bound — bounds differ per cache and stay owned by the caller."""
now = time.time()
hit = cache.get(key)
if hit and now - hit[0] < ttl:
return hit[1]
lk = _flight(cache, key)
if hit:
# STALE: serve it now; refresh once in the background. If a refresh is already in
# flight, this request just rides the stale copy — that is the whole point.
if lk.acquire(blocking=False):
def _refresh():
try:
cache[key] = (time.time(), build())
if evict:
evict()
except Exception:
pass # stale stays servable; the next expiry retries
finally:
lk.release()
threading.Thread(target=_refresh, daemon=True,
name=f"scope-refresh:{key!r}").start()
return hit[1]
# MISS: nothing to serve — build in the request path, single-flight. Late arrivals block
# on the lock and then reuse the winner's entry instead of rebuilding.
with lk:
hit = cache.get(key)
if hit and time.time() - hit[0] < ttl:
return hit[1]
value = build()
cache[key] = (time.time(), value)
if evict:
evict()
return value
|