api/scope_cache.py

"""scope_cache.py — stale-while-refresh for the scope-shaped caches (the owner cache rule).

THE RULE, ported from the Streamlit host's cache architecture ([[ri-os-cache-architecture]]):
once a copy exists, NO request ever blocks on a rebuild. A fresh hit returns. A STALE hit
returns the stale copy immediately and refreshes ONCE in a background thread (per-key
single-flight). Only a true miss — no copy at all — builds synchronously, and concurrent
missers wait on the builder's lock instead of stampeding Odoo with N identical pulls.

WHY THIS EXISTS. The wave-2 caches were expire-and-block: at TTL+1s the next visitor paid the
full Odoo pool build (10–30s) in the request path, every 15 minutes, per scope — measured live
as the "extremely long loads". Serving 15-minute-old figures for the seconds a refresh takes is
the documented, owner-chosen trade; blocking a person on an Odoo pull is not.

WHAT THIS DOES NOT CHANGE. The cache MAP and its entry shape stay exactly as the routes had
them — `rt.pool_cache[key] = (ts, value)` — because `verify_api.py` asserts that map holds only
scope-keyed raw values (the cross-user-leak gate). This module only decides WHEN `build()` runs
and on which thread.

⚠ `build` runs with no request context on the refresh path — it must be scope-shaped only
(team_id/agent/granularity), never session-shaped. That is already the routes' own leak rule.
"""
import threading
import time

#: (id(cache), key) -> Lock. Keyed by the cache object's identity so two tenants' runtimes can
#: never share a flight; bounded by the caches' own eviction (a lock per live cache key).
_FLIGHTS = {}
_FLIGHTS_GUARD = threading.Lock()


def _flight(cache, key):
    fk = (id(cache), key)
    with _FLIGHTS_GUARD:
        lk = _FLIGHTS.get(fk)
        if lk is None:
            lk = _FLIGHTS[fk] = threading.Lock()
        # The map only grows with distinct live keys; prune anything not currently held once it
        # gets silly. Cheap and rare — this is hygiene, not a hot path.
        if len(_FLIGHTS) > 512:
            for k in [k for k, v in _FLIGHTS.items() if not v.locked()][:256]:
                _FLIGHTS.pop(k, None)
        return lk


def get(cache, key, ttl, build, evict=None):
    """The value for `key`, per the rule above. `evict` (optional callable) runs after a write
    to apply the caller's own bound — bounds differ per cache and stay owned by the caller."""
    now = time.time()
    hit = cache.get(key)
    if hit and now - hit[0] < ttl:
        return hit[1]

    lk = _flight(cache, key)

    if hit:
        # STALE: serve it now; refresh once in the background. If a refresh is already in
        # flight, this request just rides the stale copy — that is the whole point.
        if lk.acquire(blocking=False):
            def _refresh():
                try:
                    cache[key] = (time.time(), build())
                    if evict:
                        evict()
                except Exception:
                    pass                      # stale stays servable; the next expiry retries
                finally:
                    lk.release()
            threading.Thread(target=_refresh, daemon=True,
                             name=f"scope-refresh:{key!r}").start()
        return hit[1]

    # MISS: nothing to serve — build in the request path, single-flight. Late arrivals block
    # on the lock and then reuse the winner's entry instead of rebuilding.
    with lk:
        hit = cache.get(key)
        if hit and time.time() - hit[0] < ttl:
            return hit[1]
        value = build()
        cache[key] = (time.time(), value)
        if evict:
            evict()
        return value