Reliability: bake ESM-2 weights + bound in-memory jobs/rate-limit

Three audit quick-wins:
- Dockerfile: HF_HOME moved to a BAKED /opt/hf-cache and the default ESM-2 (35M)
weights are prefetched at build time, so a cold start no longer re-downloads
them (HF_HOME used to live under ephemeral /tmp). Best-effort prefetch (|| echo)
so a build-time hiccup can't break the image — it just lazy-loads at runtime.
- server.py: cap _JOBS at 2000 and evict the oldest (with their edit locks) so a
long-lived worker can't grow unbounded.
- server.py: the rate-limit table now evicts the STALEST buckets when it gets
large, instead of clear()-ing the whole thing (which reset every honest user's
window and could be abused by a unique-IP flood).

Verified: 324 tests pass; _JOBS cap evicts oldest + prunes locks; rate-limit
drops stalest buckets (recent kept), no global wipe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Dockerfile

@@ -35,15 +35,24 @@ RUN pip install --no-cache-dir -e .
 # non-root user a writable /tmp but not /app/.dee).
 ENV HOME=/tmp \
     XDG_CACHE_HOME=/tmp/.cache \
-    HF_HOME=/tmp/.cache/huggingface \
+    HF_HOME=/opt/hf-cache \
+    HF_HUB_DISABLE_TELEMETRY=1 \
     HOST=0.0.0.0 \
     PORT=7860 \
     PYTHONUNBUFFERED=1
-RUN mkdir -p /tmp/.cache/huggingface /tmp/.dee/output /tmp/.dee/state \
-    && chown -R app:app /tmp/.dee /tmp/.cache
+# HF_HOME now points at a BAKED, image-resident dir (not ephemeral /tmp) so the
+# default model's weights persist across Space restarts instead of re-downloading.
+RUN mkdir -p /tmp/.cache /tmp/.dee/output /tmp/.dee/state /opt/hf-cache \
+    && chown -R app:app /tmp/.dee /tmp/.cache /opt/hf-cache
 
 USER app
 
+# Bake the default ESM-2 (35M) weights into the image so a cold start doesn't pay
+# the re-download cost. Best-effort: if the prefetch fails at build time, the
+# build still succeeds and the app simply lazy-loads the model at runtime.
+RUN python -c "from transformers import AutoTokenizer, AutoModelForMaskedLM; m='facebook/esm2_t12_35M_UR50D'; AutoTokenizer.from_pretrained(m); AutoModelForMaskedLM.from_pretrained(m)" \
+    || echo 'ESM-2 prefetch skipped — model will lazy-load at runtime'
+
 EXPOSE 7860
 
 # Production WSGI server. ONE gthread worker (state lives in-process), threads

dee/server.py

@@ -268,9 +268,19 @@ def _get_job(job_id: str) -> Optional[JobState]:
         return _JOBS.get(job_id)
 
 
+_JOBS_MAX = 2000   # cap retained jobs so a long-lived process can't grow forever
+
+
 def _put_job(job: JobState) -> None:
     with _JOBS_LOCK:
         _JOBS[job.job_id] = job
+        # Bound memory: previously every job + its edit lock was kept for the
+        # process lifetime. Dicts preserve insertion order, so evict the oldest
+        # (long-finished) jobs once we're over the cap.
+        while len(_JOBS) > _JOBS_MAX:
+            old_id = next(iter(_JOBS))
+            _JOBS.pop(old_id, None)
+            _EDIT_LOCKS.pop(old_id, None)
 
 
 # ----------------------------------------------------------------- preview cache
@@ -812,9 +822,18 @@ def _check_rate_limit(path: str, key: str):
     now = time.monotonic()
     try:
         with _RL_LOCK:
-            # Cheap protection against an IP-rotating flood bloating the table.
+            # Protect against an IP-rotating flood bloating the table — but evict
+            # the STALEST buckets (oldest last-hit), not the whole table. A blunt
+            # clear() would reset every honest user's window, and a unique-IP
+            # flood could exploit that to wipe limits on demand.
             if len(_RL_HITS) > _RL_MAX_KEYS:
-                _RL_HITS.clear()
+                try:
+                    victims = sorted(_RL_HITS.items(),
+                                     key=lambda kv: (kv[1][-1] if kv[1] else 0.0))
+                    for vk, _dq in victims[: max(1, len(_RL_HITS) // 5)]:
+                        _RL_HITS.pop(vk, None)
+                except Exception:  # noqa: BLE001
+                    _RL_HITS.clear()  # last-ditch fallback
             dq = _RL_HITS.get(bucket)
             if dq is None:
                 dq = deque()