Spaces:
Running
Running
| """Error scrubber——把 raw exception 收斂成 opaque category label。 | |
| 【為什麼存在】 | |
| ml-service 的 sentry_sdk.capture_exception(exc) 會把 exc.args 原樣寫進 | |
| Sentry event 的 exception.values[].value 欄位: | |
| - huggingface_hub.HfHubHTTPError 的 401 message 含 token=hf_… | |
| - psycopg.OperationalError / asyncpg.* 的連線失敗 message 含 Neon DSN | |
| - httpx.ConnectError 含完整 hostname + port | |
| Sentry 整合 token 一旦外流(讀權限 → exception detail),attacker 直接拿 | |
| 到 plaintext credentials,就算 breadcrumbs 全清也擋不住。 | |
| 【設計】 | |
| 鏡像 packages/error-scrubber/src/index.ts 的 scrubErrorForAudit: | |
| 1. class name 比對先(HfHubHTTPError、TimeoutError、StripeXxx、 | |
| PostgresError、DatabaseError、OperationalError、ReplyError) | |
| 2. 再 substring fallback(postgres://、rediss://、stripe、timeout、 | |
| 401+(huggingface|hf_)) | |
| 3. fall-through 一律 "unknown error" | |
| 純函數,不 raise,weird __str__ 用 try/except 收掉。Operator 從 | |
| stdout/file logger 撈 unredacted exc 對應 label 做 forensic correlation | |
| (同 trace context / request id),Sentry 只見 label。 | |
| 【caller】 | |
| - apps/ml-service/src/main.py:_on_preload_done (lifespan 預載 task) | |
| - apps/ml-service/src/lib/background.py:_log_unhandled_exception | |
| (fire-and-forget) | |
| 【Sibling rule】 | |
| TypeScript 端對應:`packages/error-scrubber/scrubErrorForAudit`。 | |
| Memory: feedback_python_sentry_capture_needs_scrub.md + | |
| feedback_sentry_breadcrumb_bypasses_message_scrub.md。 | |
| """ | |
| from __future__ import annotations | |
| # 不要在這裡 import sentry_sdk / db / huggingface_hub——helper 只看 | |
| # class name (str) + message (str),不需要任何第三方型別。隔離 import 也 | |
| # 讓 helper 在最早期 (pre-init_sentry) 也能安全使用。 | |
| def scrub_exc_for_audit(exc: BaseException | None) -> str: | |
| """收斂任意 exception 為 opaque category label,永不 leak raw 字串。 | |
| 【優先序】 | |
| class-name 檢查在前,substring 檢查在後——HfHubHTTPError 含 timeout | |
| 字眼仍判 hf auth;OperationalError 帶 timeout 仍判 db。class 才反映 | |
| 故障性質,substring 只是 fallback for libraries that wrap exceptions | |
| without preserving class name. | |
| 【return 值】 | |
| "hf auth error" | "timeout" | "stripe api error" | "db error" | |
| | "redis error" | "network error" | "unknown error" | |
| """ | |
| # ── 0. None / 非 exception → unknown ──────────────────────────────── | |
| if exc is None or not isinstance(exc, BaseException): | |
| return "unknown error" | |
| name = type(exc).__name__ | |
| # exc.__str__ 可能 raise(自訂例外的怪 __str__)—收成空字串走 class-only path | |
| try: | |
| message = str(exc) | |
| except Exception: | |
| message = "" | |
| lower = message.lower() | |
| # ── 1. HF auth class(F3 新 bucket)───────────────────────────────── | |
| # HfHubHTTPError class name 一見即判,不用看 message——huggingface_hub | |
| # 401 / 403 / token expired 全部走這個 class。 | |
| # | |
| # c-series wave-2 security-review:huggingface_hub.errors 下 4 個常見 | |
| # 子類別 class name 不以 "HfHub" 開頭,原 startswith 抓不到 → | |
| # message(含 token / repo URL)落到 unknown error 後配合 | |
| # LoggingIntegration breadcrumb 進 Sentry。明列 catch。 | |
| if name == "HfHubHTTPError" or name.startswith("HfHub") or name in { | |
| "RepositoryNotFoundError", | |
| "EntryNotFoundError", | |
| "LocalEntryNotFoundError", | |
| "BadRequestError", | |
| }: | |
| return "hf auth error" | |
| # ── 2. Timeout class——builtins.TimeoutError、asyncio.TimeoutError | |
| # (Python 3.11+ 兩者已統一)、httpx.TimeoutException 等。 | |
| if name in {"TimeoutError", "TimeoutException", "AsyncTimeoutError"}: | |
| return "timeout" | |
| # ── 3. Stripe class——stripe-python SDK 例外多以 Stripe 為前綴 | |
| # (StripeError / CardError / APIConnectionError 屬 stripe.error 模組 | |
| # 但 class name 都 Stripe-prefix 或在 stripe namespace)。 | |
| if name.startswith("Stripe"): | |
| return "stripe api error" | |
| # ── 4. Postgres / DB class——psycopg、asyncpg、SQLAlchemy 都用以下名稱 | |
| # | |
| # c-series wave-2 security-review:asyncpg exception 名稱跟 psycopg 不同; | |
| # ConnectionFailureError / PoolConnectionError / ConnectionDoesNotExistError | |
| # 內常嵌 'could not connect to server "ep-xxx.neon.tech"' 形式,原 | |
| # postgres:// substring fallback 抓不到 → 落 unknown error 後 LoggingIntegration | |
| # breadcrumb 還是把 raw hostname 送進 Sentry。明列 catch。 | |
| # PoolTimeout / PoolClosedError 也加進來涵蓋 psycopg.pool / asyncpg.pool。 | |
| if name in { | |
| "OperationalError", | |
| "PostgresError", | |
| "DatabaseError", | |
| "InterfaceError", | |
| "DataError", | |
| "IntegrityError", | |
| "InternalError", | |
| "ProgrammingError", | |
| "ConnectionFailureError", # asyncpg | |
| "PoolConnectionError", # asyncpg | |
| "ConnectionDoesNotExistError", # asyncpg | |
| "PoolTimeout", # psycopg.pool | |
| "PoolClosedError", # asyncpg / psycopg | |
| }: | |
| return "db error" | |
| # ── 5. Redis class——redis-py 的 ConnectionError 跟 builtins 撞名 | |
| # 不能單看 class,要靠 substring;ReplyError 比較明確。 | |
| if name == "ReplyError": | |
| return "redis error" | |
| # ── 6. Substring:postgres URL(順序在 stripe substring 前—— | |
| # postgres error message 偶爾會帶 stripe webhook metadata 字串) | |
| if "postgres://" in lower or "postgresql://" in lower: | |
| return "db error" | |
| # ── 7. Substring:HF 401 信號——token 字串或 huggingface domain | |
| # 需要兩個訊號("401" + ("huggingface" or "hf_") )才判,避免任意 | |
| # 401 都誤判 HF。 | |
| if "401" in lower and ("huggingface" in lower or "hf_" in lower): | |
| return "hf auth error" | |
| # ── 8. Substring:stripe(third-party retry wrapper 包過 Stripe 例外 | |
| # 後 class name 會 lost) | |
| if "stripe" in lower: | |
| return "stripe api error" | |
| # ── 9. Substring:redis URL | |
| if "redis://" in lower or "rediss://" in lower: | |
| return "redis error" | |
| # ── 10. Substring:timeout 關鍵字(class check 沒抓到的 wrapped timeout) | |
| if "timeout" in lower or "timed out" in lower: | |
| return "timeout" | |
| # ── 11. Substring:fetch / network 失敗信號(httpx ConnectError、 | |
| # OSError ECONNREFUSED 等的訊息常見字眼)。需配 class 才升級—— | |
| # 避免 user-controlled string 含 "ECONNREFUSED" 誤判。 | |
| if name in {"ConnectError", "NetworkError", "ConnectionError"} and ( | |
| "econnrefused" in lower | |
| or "enotfound" in lower | |
| or "etimedout" in lower | |
| or "name or service not known" in lower | |
| ): | |
| return "network error" | |
| # ── 12. fall-through ─────────────────────────────────────────────── | |
| return "unknown error" | |