feat(jobs): server-driven bulk job runner — survives browser tab closure
Browse filesPOST /api/v1/jobs accepts the whole list, validates the JWT + job ownership,
and finalizes the run server-side: per-chunk verifier_checks upserts,
processed_count/valid/risky/bad heartbeats on verifier_jobs, completed/error
finalization and risky+unprocessed refunds via the service-role ledger.
This removes the 'submitting client is gone' failure class (10 of the last 15
failed CSV runs on 2026-10-20/21): the browser no longer orchestrates chunking,
so a closed/throttled tab can no longer orphan a run holding thousands of
credits. classify_engine_result mirrors src/lib/verifier.ts classify() exactly
(NXDOMAIN -> Invalid/bad, dns_timeout/a_fallback_only -> Unknown/risky).
Frontend counterpart: runBulk v2 posts /jobs and polls the DB row only, with a
legacy chunk-loop fallback for engines predating this route (404/405).
Tests: tests/test_verifier_jobs.py (12 cases: classification edges, auth,
ownership, lifecycle finalization, chunk-failure refund path).
- app/api/jobs.py +346 -0
- app/db/repo.py +67 -0
- app/main.py +4 -0
- tests/test_verifier_jobs.py +278 -0
|
@@ -0,0 +1,346 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Server-driven bulk job runner — the fix for the "submitting client is gone"
|
| 2 |
+
failure class (10 of the last 15 failed CSV runs on 2026-10-20/21).
|
| 3 |
+
|
| 4 |
+
Before this module the BROWSER orchestrated every bulk run: it chunked the
|
| 5 |
+
file, submitted each chunk to /validate/bulk, polled the engine, mirrored rows
|
| 6 |
+
into verifier_checks and finalized the job. If the tab closed, throttled or
|
| 7 |
+
slept mid-run, the job orphaned in 'processing' until the reaper closed it and
|
| 8 |
+
refunded the unprocessed remainder. The engine itself was healthy — its chunk
|
| 9 |
+
task even completed 500/500 while the browser row froze at 0/3000.
|
| 10 |
+
|
| 11 |
+
Contract (v2):
|
| 12 |
+
POST /api/v1/jobs {job_id, emails, deep?, final?}
|
| 13 |
+
- JWT required (productFetch always attaches the Supabase session token).
|
| 14 |
+
- verifier_jobs row must exist, belong to the JWT subject and still be
|
| 15 |
+
open (queued/processing).
|
| 16 |
+
- A background task validates the list in chunks (bulk_sync_chunk_size),
|
| 17 |
+
upserts verifier_checks rows after EVERY chunk, heartbeats verifier_jobs
|
| 18 |
+
(processed_count + counts + updated_at) so the dashboard and the reaper
|
| 19 |
+
both see live progress, and — on the final batch — flips the job to
|
| 20 |
+
'completed' and refunds risky rows server-side.
|
| 21 |
+
- Any chunk-level failure finalizes the job as 'error' and refunds BOTH
|
| 22 |
+
the risky rows and the unprocessed remainder (same ledger mechanics as
|
| 23 |
+
the client finalize and the pg_cron reaper; idempotent via
|
| 24 |
+
refunded_credits).
|
| 25 |
+
The client polls verifier_jobs directly after accepting 202 — it no longer
|
| 26 |
+
needs to stay alive for the run to finish.
|
| 27 |
+
"""
|
| 28 |
+
import asyncio
|
| 29 |
+
import json
|
| 30 |
+
import logging
|
| 31 |
+
import time
|
| 32 |
+
from typing import List, Optional
|
| 33 |
+
|
| 34 |
+
from fastapi import APIRouter, HTTPException, Request
|
| 35 |
+
from pydantic import BaseModel
|
| 36 |
+
|
| 37 |
+
from ..auth import verify_supabase_jwt
|
| 38 |
+
from ..config import settings
|
| 39 |
+
from ..db import repo
|
| 40 |
+
from ..models import ValidationResult
|
| 41 |
+
from ..validator.engine import engine
|
| 42 |
+
|
| 43 |
+
logger = logging.getLogger("emailvalidator.jobs")
|
| 44 |
+
|
| 45 |
+
router = APIRouter()
|
| 46 |
+
|
| 47 |
+
# Strong references so the event loop cannot GC an in-flight runner, and a
|
| 48 |
+
# semaphore so concurrent job submissions cannot spawn unbounded SMTP work.
|
| 49 |
+
_job_tasks: set[asyncio.Task] = set()
|
| 50 |
+
_job_semaphore = asyncio.Semaphore(settings.bulk_concurrency)
|
| 51 |
+
|
| 52 |
+
GROUPS = ("good", "risky", "bad")
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
class JobRequest(BaseModel):
|
| 56 |
+
job_id: str
|
| 57 |
+
emails: List[str]
|
| 58 |
+
deep: bool = True
|
| 59 |
+
final: bool = True
|
| 60 |
+
user_id: Optional[str] = None # informational; ownership comes from the JWT
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
# ---- classification (mirror of src/lib/verifier.ts classify()) ------------
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
def classify_engine_result(r: ValidationResult) -> dict:
|
| 67 |
+
"""Map an engine ValidationResult to the verifier_checks row shape the
|
| 68 |
+
dashboard renders. Mirrors the frontend classify() branch order so a row
|
| 69 |
+
never disagrees with a client-side re-classification:
|
| 70 |
+
disposable → bad | dead domain → bad | dns inconclusive → risky |
|
| 71 |
+
valid+smtp → good | engine invalid → bad | catch-all → risky | else risky.
|
| 72 |
+
"""
|
| 73 |
+
disposable = bool(getattr(r.disposable, "is_disposable", False))
|
| 74 |
+
catch_all = bool(getattr(r.catch_all, "is_catch_all", False))
|
| 75 |
+
dns = r.dns
|
| 76 |
+
status = (r.status.value if hasattr(r.status, "value") else str(r.status)) or "unknown"
|
| 77 |
+
|
| 78 |
+
checks_run = r.checks_run or []
|
| 79 |
+
dns_error = (dns.error or "") if dns else ""
|
| 80 |
+
dns_failure = dns_error == "dns_timeout" or "timeout" in dns_error.lower()
|
| 81 |
+
a_fallback_only = bool(dns and dns.domain_exists and not dns.has_mx_records)
|
| 82 |
+
smtp_verified = bool(r.smtp and r.smtp.verified is True)
|
| 83 |
+
reason = r.reason_code
|
| 84 |
+
|
| 85 |
+
if disposable:
|
| 86 |
+
out_status, result, grp = "disposable", "Disposable", "bad"
|
| 87 |
+
reason = reason or "disposable"
|
| 88 |
+
elif dns is not None and dns.domain_exists is False:
|
| 89 |
+
# NXDOMAIN is definitive (the stale-build regression that showed a
|
| 90 |
+
# 0% Bad column on a 3000-row list must never resurrect server-side).
|
| 91 |
+
out_status, result, grp = "invalid", "Invalid", "bad"
|
| 92 |
+
reason = reason or "nxdomain"
|
| 93 |
+
elif dns_failure or a_fallback_only:
|
| 94 |
+
out_status, result, grp = "unknown", "Unknown", "risky"
|
| 95 |
+
reason = reason or ("dns_timeout" if dns_failure else "a_fallback_only")
|
| 96 |
+
elif status == "valid" and smtp_verified:
|
| 97 |
+
out_status, result, grp = "valid", "Ok", "good"
|
| 98 |
+
elif status == "invalid":
|
| 99 |
+
out_status, result, grp = "invalid", "Invalid", "bad"
|
| 100 |
+
elif catch_all:
|
| 101 |
+
out_status, result, grp = "catch_all", "Catch-all", "risky"
|
| 102 |
+
reason = reason or "catch_all_detected"
|
| 103 |
+
else:
|
| 104 |
+
out_status, result, grp = "unknown", "Unknown", "risky"
|
| 105 |
+
reason = reason or "smtp_unavailable"
|
| 106 |
+
|
| 107 |
+
score = r.scoring.score if r.scoring is not None else None
|
| 108 |
+
return {
|
| 109 |
+
"email": r.email,
|
| 110 |
+
"status": out_status,
|
| 111 |
+
"result": result,
|
| 112 |
+
"grp": grp,
|
| 113 |
+
"score": score,
|
| 114 |
+
"reason": reason,
|
| 115 |
+
"pass_through": None,
|
| 116 |
+
"checks_run": checks_run,
|
| 117 |
+
"verified_at": r.validated_at,
|
| 118 |
+
"run_id": r.run_id,
|
| 119 |
+
"validator_version": r.validator_version,
|
| 120 |
+
}
|
| 121 |
+
|
| 122 |
+
|
| 123 |
+
# ---- persistence helpers (service-role writes, best-effort like repo) ------
|
| 124 |
+
|
| 125 |
+
|
| 126 |
+
async def _write_checks(rows: List[dict]) -> None:
|
| 127 |
+
if not rows:
|
| 128 |
+
return
|
| 129 |
+
try:
|
| 130 |
+
await asyncio.to_thread(repo.write_verifier_checks, rows)
|
| 131 |
+
except Exception as exc: # never lose a run to one write blip
|
| 132 |
+
logger.warning("job_checks_write_failed: %s", exc)
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
async def _update_job(job_id: str, **fields) -> None:
|
| 136 |
+
try:
|
| 137 |
+
await asyncio.to_thread(repo.update_verifier_job, job_id, fields)
|
| 138 |
+
except Exception as exc:
|
| 139 |
+
logger.warning("job_update_failed job=%s: %s", job_id, exc)
|
| 140 |
+
|
| 141 |
+
|
| 142 |
+
async def _finalize_failed(job_id: str, error: str, counts: dict, processed: int) -> None:
|
| 143 |
+
"""Close a failed job and refund risky + unprocessed server-side.
|
| 144 |
+
|
| 145 |
+
Mirrors the ledger mechanics of public.refund_job_risky() (migration
|
| 146 |
+
20261020000000) minus its auth.uid() guard — the engine writes with the
|
| 147 |
+
service-role key, under which auth.uid() is null and the SQL function
|
| 148 |
+
would raise. Idempotency comes from verifier_jobs.refunded_credits.
|
| 149 |
+
"""
|
| 150 |
+
await _update_job(
|
| 151 |
+
job_id,
|
| 152 |
+
status="error",
|
| 153 |
+
error=error[:500],
|
| 154 |
+
processed_count=processed,
|
| 155 |
+
valid_count=counts.get("good", 0),
|
| 156 |
+
risky_count=counts.get("risky", 0),
|
| 157 |
+
bad_count=counts.get("bad", 0),
|
| 158 |
+
)
|
| 159 |
+
try:
|
| 160 |
+
job = await asyncio.to_thread(repo.get_verifier_job, job_id)
|
| 161 |
+
if not job:
|
| 162 |
+
return
|
| 163 |
+
total = int(job.get("total_count") or 0)
|
| 164 |
+
charged = int(job.get("credits_charged") or 0)
|
| 165 |
+
refunded = int(job.get("refunded_credits") or 0)
|
| 166 |
+
risky = int(job.get("risky_count") or 0)
|
| 167 |
+
unprocessed = max(total - int(job.get("processed_count") or 0), 0)
|
| 168 |
+
due = min(risky + unprocessed, charged) - refunded
|
| 169 |
+
if due > 0:
|
| 170 |
+
await asyncio.to_thread(repo.refund_credits, job["user_id"], due,
|
| 171 |
+
"Auto-refund — risky and unprocessed results are not billed")
|
| 172 |
+
await _update_job(job_id, refunded_credits=refunded + due)
|
| 173 |
+
except Exception as exc:
|
| 174 |
+
logger.warning("job_refund_failed job=%s: %s", job_id, exc)
|
| 175 |
+
|
| 176 |
+
|
| 177 |
+
# ---- the runner ------------------------------------------------------------
|
| 178 |
+
|
| 179 |
+
|
| 180 |
+
async def _run_verifier_job(job_id: str, user_id: str, emails: List[str], deep: bool,
|
| 181 |
+
final: bool) -> None:
|
| 182 |
+
started = time.monotonic()
|
| 183 |
+
counts = {g: 0 for g in GROUPS}
|
| 184 |
+
processed = 0
|
| 185 |
+
# Defensive dedupe (the frontend already deduped; a client bug must not
|
| 186 |
+
# double-write rows — (job_id, email) upsert would silently dedupe them
|
| 187 |
+
# while counts drift).
|
| 188 |
+
seen: set[str] = set()
|
| 189 |
+
ordered: List[str] = []
|
| 190 |
+
for e in emails:
|
| 191 |
+
e = (e or "").strip().lower()
|
| 192 |
+
if e and "@" in e and e not in seen:
|
| 193 |
+
seen.add(e)
|
| 194 |
+
ordered.append(e)
|
| 195 |
+
emails = ordered
|
| 196 |
+
total = len(emails)
|
| 197 |
+
if total == 0:
|
| 198 |
+
await _finalize_failed(job_id, "No valid addresses reached the engine.", counts, 0)
|
| 199 |
+
return
|
| 200 |
+
|
| 201 |
+
chunk_size = max(int(settings.bulk_sync_chunk_size or 50), 10)
|
| 202 |
+
async with _job_semaphore:
|
| 203 |
+
try:
|
| 204 |
+
await _update_job(job_id, status="processing")
|
| 205 |
+
for i in range(0, total, chunk_size):
|
| 206 |
+
chunk = emails[i:i + chunk_size]
|
| 207 |
+
try:
|
| 208 |
+
results = await engine.validate_batch(chunk, deep=deep)
|
| 209 |
+
except Exception as exc: # one bad stage must not kill the job
|
| 210 |
+
logger.warning("job_chunk_failed job=%s: %s", job_id, exc)
|
| 211 |
+
await _finalize_failed(
|
| 212 |
+
job_id,
|
| 213 |
+
f"Engine chunk failed after {processed}/{total} addresses: {exc}",
|
| 214 |
+
counts, processed,
|
| 215 |
+
)
|
| 216 |
+
return
|
| 217 |
+
if len(results) != len(chunk):
|
| 218 |
+
await _finalize_failed(
|
| 219 |
+
job_id,
|
| 220 |
+
f"Engine returned {len(results)} of {len(chunk)} results for one chunk",
|
| 221 |
+
counts, processed,
|
| 222 |
+
)
|
| 223 |
+
return
|
| 224 |
+
rows = [classify_engine_result(r) for r in results]
|
| 225 |
+
await _write_checks(rows)
|
| 226 |
+
for row in rows:
|
| 227 |
+
counts[row["grp"]] = counts.get(row["grp"], 0) + 1
|
| 228 |
+
processed += len(chunk)
|
| 229 |
+
# Heartbeat + live progress in one write; the dashboard polls
|
| 230 |
+
# this row instead of the engine, and the reaper reads it.
|
| 231 |
+
await _update_job(
|
| 232 |
+
job_id,
|
| 233 |
+
processed_count=processed,
|
| 234 |
+
valid_count=counts["good"],
|
| 235 |
+
risky_count=counts["risky"],
|
| 236 |
+
bad_count=counts["bad"],
|
| 237 |
+
)
|
| 238 |
+
if final:
|
| 239 |
+
await _update_job(
|
| 240 |
+
job_id,
|
| 241 |
+
status="completed",
|
| 242 |
+
error=None,
|
| 243 |
+
processed_count=processed,
|
| 244 |
+
valid_count=counts["good"],
|
| 245 |
+
risky_count=counts["risky"],
|
| 246 |
+
bad_count=counts["bad"],
|
| 247 |
+
)
|
| 248 |
+
# Risky rows are never billed; the completed-job refund covers
|
| 249 |
+
# exactly the risky_count (unprocessed term is zero by design).
|
| 250 |
+
try:
|
| 251 |
+
job = await asyncio.to_thread(repo.get_verifier_job, job_id)
|
| 252 |
+
if job:
|
| 253 |
+
charged = int(job.get("credits_charged") or 0)
|
| 254 |
+
refunded = int(job.get("refunded_credits") or 0)
|
| 255 |
+
due = min(counts["risky"], charged) - refunded
|
| 256 |
+
if due > 0:
|
| 257 |
+
await asyncio.to_thread(
|
| 258 |
+
repo.refund_credits, job["user_id"], due,
|
| 259 |
+
"Auto-refund — risky results are not billed")
|
| 260 |
+
await _update_job(job_id, refunded_credits=refunded + due)
|
| 261 |
+
except Exception as exc:
|
| 262 |
+
logger.warning("job_risky_refund_failed job=%s: %s", job_id, exc)
|
| 263 |
+
logger.info("job_batch_done job=%s processed=%s/%s final=%s secs=%.1f",
|
| 264 |
+
job_id, processed, total, final, time.monotonic() - started)
|
| 265 |
+
except Exception as exc: # absolute safety net
|
| 266 |
+
logger.exception("job_runner_crashed job=%s", job_id)
|
| 267 |
+
await _finalize_failed(job_id, f"Runner crashed: {exc}", counts, processed)
|
| 268 |
+
|
| 269 |
+
|
| 270 |
+
# ---- routes ----------------------------------------------------------------
|
| 271 |
+
|
| 272 |
+
|
| 273 |
+
@router.post("/jobs", tags=["Bulk Jobs"])
|
| 274 |
+
async def create_server_job(request: Request, payload: JobRequest):
|
| 275 |
+
"""Accept a bulk job for server-side processing (202 + DB polling)."""
|
| 276 |
+
token = request.headers.get("authorization", "")
|
| 277 |
+
token = token[7:].strip() if token.lower().startswith("bearer ") else ""
|
| 278 |
+
if not token:
|
| 279 |
+
raise HTTPException(401, "Missing bearer token — sign in first.")
|
| 280 |
+
try:
|
| 281 |
+
claims = await verify_supabase_jwt(token)
|
| 282 |
+
except Exception as exc:
|
| 283 |
+
raise HTTPException(401, f"Invalid or expired token: {exc}")
|
| 284 |
+
user_id = claims.get("sub", "")
|
| 285 |
+
if not user_id:
|
| 286 |
+
raise HTTPException(401, "Token has no subject claim.")
|
| 287 |
+
if not payload.emails:
|
| 288 |
+
raise HTTPException(400, "emails must not be empty.")
|
| 289 |
+
if not settings.supabase_configured:
|
| 290 |
+
raise HTTPException(503, "Server persistence is not configured (SUPABASE_URL/SERVICE key).")
|
| 291 |
+
|
| 292 |
+
job = await asyncio.to_thread(repo.get_verifier_job, payload.job_id)
|
| 293 |
+
if not job:
|
| 294 |
+
raise HTTPException(404, f"Job {payload.job_id} not found — create it first.")
|
| 295 |
+
if str(job.get("user_id")) != user_id:
|
| 296 |
+
# Ownership mismatch: identical response for missing/foreign jobs.
|
| 297 |
+
raise HTTPException(404, f"Job {payload.job_id} not found for this account.")
|
| 298 |
+
if job.get("status") not in ("queued", "processing"):
|
| 299 |
+
raise HTTPException(409, f"Job {payload.job_id} is already {job.get('status')}.")
|
| 300 |
+
if not payload.emails:
|
| 301 |
+
raise HTTPException(400, "emails must not be empty.")
|
| 302 |
+
|
| 303 |
+
# Cap one submission so a giant file cannot pin the runner forever: the
|
| 304 |
+
# client splits into final-flagged batches (20k ≈ 600 KB JSON).
|
| 305 |
+
MAX_PER_CALL = 20000
|
| 306 |
+
emails = payload.emails[:MAX_PER_CALL]
|
| 307 |
+
|
| 308 |
+
task = asyncio.create_task(
|
| 309 |
+
_run_verifier_job(payload.job_id, user_id, emails, payload.deep, payload.final)
|
| 310 |
+
)
|
| 311 |
+
_job_tasks.add(task)
|
| 312 |
+
task.add_done_callback(_job_tasks.discard)
|
| 313 |
+
|
| 314 |
+
return {
|
| 315 |
+
"accepted": len(emails),
|
| 316 |
+
"job_id": payload.job_id,
|
| 317 |
+
"status": "processing",
|
| 318 |
+
"final": payload.final,
|
| 319 |
+
"message": "Processing server-side — poll verifier_jobs for progress.",
|
| 320 |
+
}
|
| 321 |
+
|
| 322 |
+
|
| 323 |
+
@router.get("/jobs/{job_id}", tags=["Bulk Jobs"])
|
| 324 |
+
async def job_status(job_id: str, request: Request):
|
| 325 |
+
"""Lightweight engine-side echo of the job row (debug aid; the dashboard
|
| 326 |
+
polls the DB directly and does not depend on this)."""
|
| 327 |
+
token = request.headers.get("authorization", "")
|
| 328 |
+
token = token[7:].strip() if token.lower().startswith("bearer ") else ""
|
| 329 |
+
if not token:
|
| 330 |
+
raise HTTPException(401, "Missing bearer token — sign in first.")
|
| 331 |
+
try:
|
| 332 |
+
claims = await verify_supabase_jwt(token)
|
| 333 |
+
except Exception as exc:
|
| 334 |
+
raise HTTPException(401, f"Invalid or expired token: {exc}")
|
| 335 |
+
job = await asyncio.to_thread(repo.get_verifier_job, job_id)
|
| 336 |
+
if not job or str(job.get("user_id")) != claims.get("sub", ""):
|
| 337 |
+
raise HTTPException(404, f"Job {job_id} not found for this account.")
|
| 338 |
+
return {
|
| 339 |
+
"job_id": job_id,
|
| 340 |
+
"status": job.get("status"),
|
| 341 |
+
"total": job.get("total_count"),
|
| 342 |
+
"processed": job.get("processed_count"),
|
| 343 |
+
"valid": job.get("valid_count"),
|
| 344 |
+
"risky": job.get("risky_count"),
|
| 345 |
+
"bad": job.get("bad_count"),
|
| 346 |
+
}
|
|
@@ -220,3 +220,70 @@ def save_domain_record(domain: str, dns_result: Any) -> Optional[Dict[str, Any]]
|
|
| 220 |
"has_dkim": bool(dns_result and dns_result.has_dkim),
|
| 221 |
}
|
| 222 |
return _upsert("domain_records", payload, on_conflict="domain")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 220 |
"has_dkim": bool(dns_result and dns_result.has_dkim),
|
| 221 |
}
|
| 222 |
return _upsert("domain_records", payload, on_conflict="domain")
|
| 223 |
+
|
| 224 |
+
|
| 225 |
+
# ---------------------------------------------------------------------------
|
| 226 |
+
# Verifier jobs — server-driven bulk runs (app/api/jobs.py)
|
| 227 |
+
# The engine writes the same tables the browser used to write, so a job now
|
| 228 |
+
# finishes even when the submitting tab is gone.
|
| 229 |
+
# ---------------------------------------------------------------------------
|
| 230 |
+
def get_verifier_job(job_id: str) -> Optional[Dict[str, Any]]:
|
| 231 |
+
rows = _select("verifier_jobs", "*", limit=1, eq={"id": job_id})
|
| 232 |
+
return rows[0] if rows else None
|
| 233 |
+
|
| 234 |
+
|
| 235 |
+
def update_verifier_job(job_id: str, fields: Dict[str, Any]) -> None:
|
| 236 |
+
"""Patch the job row. Callers pass business fields only; updated_at is
|
| 237 |
+
always touched so the stale-job reaper sees live heartbeats."""
|
| 238 |
+
payload = {**fields, "updated_at": "now()"}
|
| 239 |
+
client = _client()
|
| 240 |
+
if client is None:
|
| 241 |
+
return
|
| 242 |
+
try:
|
| 243 |
+
client.table("verifier_jobs").update(payload).eq("id", job_id).execute()
|
| 244 |
+
except Exception as exc:
|
| 245 |
+
logger.warning("supabase_verifier_job_update_failed", job_id=job_id, error=str(exc))
|
| 246 |
+
|
| 247 |
+
|
| 248 |
+
def write_verifier_checks(rows: List[Dict[str, Any]]) -> None:
|
| 249 |
+
"""Upsert verifier_checks rows, idempotent on (job_id, email)."""
|
| 250 |
+
if not rows:
|
| 251 |
+
return
|
| 252 |
+
client = _client()
|
| 253 |
+
if client is None:
|
| 254 |
+
return
|
| 255 |
+
try:
|
| 256 |
+
client.table("verifier_checks").upsert(rows, on_conflict="job_id,email").execute()
|
| 257 |
+
except Exception as exc:
|
| 258 |
+
logger.warning("supabase_verifier_checks_upsert_failed", count=len(rows), error=str(exc))
|
| 259 |
+
|
| 260 |
+
|
| 261 |
+
def refund_credits(user_id: str, amount: int, description: str) -> None:
|
| 262 |
+
"""Credit balance + ledger row (service-role twin of the SQL refund path).
|
| 263 |
+
Idempotency is the caller's duty via verifier_jobs.refunded_credits."""
|
| 264 |
+
if amount <= 0 or not user_id:
|
| 265 |
+
return
|
| 266 |
+
client = _client()
|
| 267 |
+
if client is None:
|
| 268 |
+
return
|
| 269 |
+
try:
|
| 270 |
+
# Atomic increment via an RPC is unavailable server-side without a
|
| 271 |
+
# dedicated function; PostgREST PATCH with a computed value races, so
|
| 272 |
+
# read-modify-write under a single-writer assumption (one runner task
|
| 273 |
+
# per job). The refunded_credits ledger column keeps each refund
|
| 274 |
+
# one-shot even if a retry replays the same due amount.
|
| 275 |
+
resp = client.table("user_credits").select("balance").eq("user_id", user_id).limit(1).execute()
|
| 276 |
+
rows = resp.data or []
|
| 277 |
+
balance = int(rows[0]["balance"]) if rows else 0
|
| 278 |
+
client.table("user_credits").update({
|
| 279 |
+
"balance": balance + int(amount),
|
| 280 |
+
"last_updated": "now()",
|
| 281 |
+
}).eq("user_id", user_id).execute()
|
| 282 |
+
client.table("credit_transactions").insert({
|
| 283 |
+
"user_id": user_id,
|
| 284 |
+
"type": "refund",
|
| 285 |
+
"amount": int(amount),
|
| 286 |
+
"description": description,
|
| 287 |
+
}).execute()
|
| 288 |
+
except Exception as exc:
|
| 289 |
+
logger.warning("supabase_refund_credits_failed", user_id=user_id, error=str(exc))
|
|
@@ -10,6 +10,7 @@ import time
|
|
| 10 |
|
| 11 |
from .config import settings
|
| 12 |
from .api.routes import router
|
|
|
|
| 13 |
from .auth import license_middleware
|
| 14 |
from .utils.logger import setup_logging
|
| 15 |
from .db import repo
|
|
@@ -176,6 +177,9 @@ async def global_exception_handler(request: Request, exc: Exception):
|
|
| 176 |
|
| 177 |
# ---- Include Routes -------------------------------------------------
|
| 178 |
app.include_router(router, prefix="/api/v1")
|
|
|
|
|
|
|
|
|
|
| 179 |
|
| 180 |
|
| 181 |
# ---- Health alias for the website productApi probe -------------------
|
|
|
|
| 10 |
|
| 11 |
from .config import settings
|
| 12 |
from .api.routes import router
|
| 13 |
+
from .api.jobs import router as jobs_router
|
| 14 |
from .auth import license_middleware
|
| 15 |
from .utils.logger import setup_logging
|
| 16 |
from .db import repo
|
|
|
|
| 177 |
|
| 178 |
# ---- Include Routes -------------------------------------------------
|
| 179 |
app.include_router(router, prefix="/api/v1")
|
| 180 |
+
# Server-driven bulk jobs (POST /api/v1/jobs): the engine finalizes the run
|
| 181 |
+
# with DB heartbeats so a closed browser tab can no longer orphan it.
|
| 182 |
+
app.include_router(jobs_router, prefix="/api/v1")
|
| 183 |
|
| 184 |
|
| 185 |
# ---- Health alias for the website productApi probe -------------------
|
|
@@ -0,0 +1,278 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Server-driven bulk job runner (app/api/jobs.py).
|
| 2 |
+
|
| 3 |
+
Regression class this locks down (2026-10-20/21 incidents, 10 of the last 15
|
| 4 |
+
failed CSV runs): the browser used to orchestrate chunking, so a closed tab
|
| 5 |
+
orphaned verifier_jobs rows in 'processing' holding thousands of credits. The
|
| 6 |
+
engine now finalizes the run server-side with per-chunk DB writes and
|
| 7 |
+
heartbeats, and refunds risky + unprocessed rows on failure.
|
| 8 |
+
"""
|
| 9 |
+
import asyncio
|
| 10 |
+
import unittest
|
| 11 |
+
from unittest.mock import AsyncMock, MagicMock, patch
|
| 12 |
+
|
| 13 |
+
from fastapi.testclient import TestClient
|
| 14 |
+
|
| 15 |
+
from app.api import jobs as jobs_mod
|
| 16 |
+
from app.api.jobs import classify_engine_result
|
| 17 |
+
from app.main import app
|
| 18 |
+
from app.models import (
|
| 19 |
+
CatchAllResult,
|
| 20 |
+
DNSResult,
|
| 21 |
+
DisposableResult,
|
| 22 |
+
ValidationResult,
|
| 23 |
+
ScoringResult,
|
| 24 |
+
SMTPResult,
|
| 25 |
+
SyntaxResult,
|
| 26 |
+
ValidationStatus,
|
| 27 |
+
)
|
| 28 |
+
|
| 29 |
+
client = TestClient(app)
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def _result(email: str, *, status=ValidationStatus.UNKNOWN, domain_exists=True,
|
| 33 |
+
has_mx=True, dns_error=None, disposable=False, catch_all=False,
|
| 34 |
+
smtp_verified=None, smtp_code=None, score=70, reason_code=None) -> ValidationResult:
|
| 35 |
+
dns = DNSResult(
|
| 36 |
+
domain_exists=domain_exists,
|
| 37 |
+
has_mx_records=has_mx,
|
| 38 |
+
error=dns_error,
|
| 39 |
+
error_code=dns_error,
|
| 40 |
+
)
|
| 41 |
+
syntax = SyntaxResult(passed=True, normalized_email=email, domain=email.split("@")[-1])
|
| 42 |
+
return ValidationResult(
|
| 43 |
+
email=email,
|
| 44 |
+
status=status,
|
| 45 |
+
syntax=syntax,
|
| 46 |
+
dns=dns,
|
| 47 |
+
disposable=DisposableResult(is_disposable=disposable, is_free_provider=False,
|
| 48 |
+
is_role_based=False, confidence=0.9) if disposable else None,
|
| 49 |
+
catch_all=CatchAllResult(is_catch_all=catch_all, confidence=0.9) if catch_all else None,
|
| 50 |
+
smtp=SMTPResult(verified=smtp_verified, smtp_code=smtp_code) if smtp_code or smtp_verified is not None else None,
|
| 51 |
+
scoring=ScoringResult(score=score, risk_level="medium", breakdown={}, warnings=[]),
|
| 52 |
+
processing_time_ms=1.0,
|
| 53 |
+
validated_at="2026-10-21T00:00:00+00:00",
|
| 54 |
+
checks_run=["syntax", "dns"],
|
| 55 |
+
reason_code=reason_code,
|
| 56 |
+
run_id="testrun",
|
| 57 |
+
validator_version="3.1.0",
|
| 58 |
+
)
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
class ClassifyTests(unittest.TestCase):
|
| 62 |
+
"""classify_engine_result must mirror src/lib/verifier.ts classify()."""
|
| 63 |
+
|
| 64 |
+
def test_nxdomain_is_invalid_bad(self):
|
| 65 |
+
# 185/3000 rows in the 2026-10-21 run were dead domains mis-classified
|
| 66 |
+
# as Unknown/risky by the stale frontend build. Server-side rows must
|
| 67 |
+
# be Invalid/bad — a domain that does not exist cannot accept mail.
|
| 68 |
+
row = classify_engine_result(
|
| 69 |
+
_result("a@dead.test", status=ValidationStatus.INVALID,
|
| 70 |
+
domain_exists=False, has_mx=False, dns_error="nxdomain",
|
| 71 |
+
reason_code="nxdomain")
|
| 72 |
+
)
|
| 73 |
+
self.assertEqual(row["grp"], "bad")
|
| 74 |
+
self.assertEqual(row["status"], "invalid")
|
| 75 |
+
self.assertEqual(row["result"], "Invalid")
|
| 76 |
+
|
| 77 |
+
def test_smtp_unavailable_is_risky(self):
|
| 78 |
+
row = classify_engine_result(
|
| 79 |
+
_result("b@gmail.test", status=ValidationStatus.RISKY,
|
| 80 |
+
smtp_verified=None, reason_code="smtp_unavailable")
|
| 81 |
+
)
|
| 82 |
+
self.assertEqual(row["grp"], "risky")
|
| 83 |
+
self.assertEqual(row["result"], "Unknown")
|
| 84 |
+
|
| 85 |
+
def test_valid_with_smtp_is_good(self):
|
| 86 |
+
row = classify_engine_result(
|
| 87 |
+
_result("c@corp.test", status=ValidationStatus.VALID,
|
| 88 |
+
smtp_verified=True, smtp_code=250, score=95)
|
| 89 |
+
)
|
| 90 |
+
self.assertEqual(row["grp"], "good")
|
| 91 |
+
self.assertEqual(row["result"], "Ok")
|
| 92 |
+
|
| 93 |
+
def test_disposable_is_bad(self):
|
| 94 |
+
row = classify_engine_result(_result("x@mailinator.test", disposable=True))
|
| 95 |
+
self.assertEqual(row["grp"], "bad")
|
| 96 |
+
self.assertEqual(row["result"], "Disposable")
|
| 97 |
+
|
| 98 |
+
def test_catch_all_is_risky(self):
|
| 99 |
+
row = classify_engine_result(
|
| 100 |
+
_result("d@ca.test", status=ValidationStatus.RISKY, catch_all=True)
|
| 101 |
+
)
|
| 102 |
+
self.assertEqual(row["grp"], "risky")
|
| 103 |
+
self.assertEqual(row["result"], "Catch-all")
|
| 104 |
+
|
| 105 |
+
def test_dns_timeout_is_risky_not_bad(self):
|
| 106 |
+
# Transient DNS states are genuinely undecidable — golden invariant.
|
| 107 |
+
row = classify_engine_result(
|
| 108 |
+
_result("e@corp.test", status=ValidationStatus.UNKNOWN,
|
| 109 |
+
dns_error="timeout", reason_code="dns_timeout")
|
| 110 |
+
)
|
| 111 |
+
self.assertEqual(row["grp"], "risky")
|
| 112 |
+
self.assertEqual(row["reason"], "dns_timeout")
|
| 113 |
+
|
| 114 |
+
|
| 115 |
+
class JobEndpointTests(unittest.TestCase):
|
| 116 |
+
"""POST /api/v1/jobs auth, ownership and lifecycle."""
|
| 117 |
+
|
| 118 |
+
def setUp(self):
|
| 119 |
+
self.job_row = {
|
| 120 |
+
"id": "11111111-1111-1111-1111-111111111111",
|
| 121 |
+
"user_id": "22222222-2222-2222-2222-222222222222",
|
| 122 |
+
"name": "t.csv",
|
| 123 |
+
"status": "queued",
|
| 124 |
+
"total_count": 3,
|
| 125 |
+
"processed_count": 0,
|
| 126 |
+
"valid_count": 0, "risky_count": 0, "bad_count": 0,
|
| 127 |
+
"credits_charged": 3, "refunded_credits": 0,
|
| 128 |
+
}
|
| 129 |
+
patcher = patch("app.api.jobs.repo", autospec=True)
|
| 130 |
+
self.repo_mock = patcher.start()
|
| 131 |
+
self.addCleanup(patcher.stop)
|
| 132 |
+
# The route 503s without persistence config; tests run without a real
|
| 133 |
+
# Supabase, so present the route with a configured project.
|
| 134 |
+
for field, value in (("supabase_url", "https://test.supabase.co"),
|
| 135 |
+
("supabase_service_role_key", "svc-key-test")):
|
| 136 |
+
p = patch.object(jobs_mod.settings, field, value)
|
| 137 |
+
p.start()
|
| 138 |
+
self.addCleanup(p.stop)
|
| 139 |
+
self.repo_mock.get_verifier_job.return_value = self.job_row
|
| 140 |
+
self.repo_mock.update_verifier_job.return_value = None
|
| 141 |
+
self.repo_mock.write_verifier_checks.return_value = None
|
| 142 |
+
self.repo_mock.refund_credits.return_value = None
|
| 143 |
+
|
| 144 |
+
def _auth(self, sub="22222222-2222-2222-2222-222222222222"):
|
| 145 |
+
return patch(
|
| 146 |
+
"app.api.jobs.verify_supabase_jwt",
|
| 147 |
+
new=AsyncMock(return_value={"sub": sub, "email": "t@t.test"}),
|
| 148 |
+
)
|
| 149 |
+
|
| 150 |
+
def test_401_without_token(self):
|
| 151 |
+
res = client.post("/api/v1/jobs", json={"job_id": "x", "emails": ["a@b.test"]})
|
| 152 |
+
self.assertEqual(res.status_code, 401)
|
| 153 |
+
|
| 154 |
+
def test_404_foreign_job(self):
|
| 155 |
+
# Same-shaped id, DIFFERENT owner: the token subject does not match the
|
| 156 |
+
# row's user_id, and the route must answer 404 (never 403/200).
|
| 157 |
+
with self._auth(sub="99999999-9999-9999-9999-999999999999"):
|
| 158 |
+
res = client.post(
|
| 159 |
+
"/api/v1/jobs",
|
| 160 |
+
json={"job_id": "11111111-1111-1111-1111-111111111111",
|
| 161 |
+
"emails": ["a@b.test"]},
|
| 162 |
+
headers={"Authorization": "Bearer faketoken"},
|
| 163 |
+
)
|
| 164 |
+
self.assertEqual(res.status_code, 404)
|
| 165 |
+
|
| 166 |
+
def test_409_when_job_closed(self):
|
| 167 |
+
self.job_row["status"] = "completed"
|
| 168 |
+
with self._auth():
|
| 169 |
+
res = client.post(
|
| 170 |
+
"/api/v1/jobs",
|
| 171 |
+
json={"job_id": "11111111-1111-1111-1111-111111111111",
|
| 172 |
+
"emails": ["a@b.test"]},
|
| 173 |
+
headers={"Authorization": "Bearer faketoken"},
|
| 174 |
+
)
|
| 175 |
+
self.assertEqual(res.status_code, 409)
|
| 176 |
+
|
| 177 |
+
def test_400_empty_emails(self):
|
| 178 |
+
with self._auth():
|
| 179 |
+
res = client.post(
|
| 180 |
+
"/api/v1/jobs",
|
| 181 |
+
json={"job_id": "11111111-1111-1111-1111-111111111111", "emails": []},
|
| 182 |
+
headers={"Authorization": "Bearer faketoken"},
|
| 183 |
+
)
|
| 184 |
+
self.assertEqual(res.status_code, 400)
|
| 185 |
+
|
| 186 |
+
def test_run_finalizes_completed_and_refunds_risky(self):
|
| 187 |
+
"""Happy path: 3 emails, 1 risky (smtp unavailable) → job completed,
|
| 188 |
+
risky row refunded once (refunded_credits ledger updated).
|
| 189 |
+
The endpoint wiring is asserted on the POST response; the lifecycle is
|
| 190 |
+
asserted by driving the runner coroutine to completion on a real loop
|
| 191 |
+
(TestClient does not guarantee background-task scheduling after the
|
| 192 |
+
response returns)."""
|
| 193 |
+
|
| 194 |
+
async def fake_batch(emails, deep=True):
|
| 195 |
+
return [
|
| 196 |
+
_result(emails[0], status=ValidationStatus.VALID, smtp_verified=True,
|
| 197 |
+
smtp_code=250, score=95),
|
| 198 |
+
_result(emails[1], status=ValidationStatus.RISKY, reason_code="smtp_unavailable"),
|
| 199 |
+
_result(emails[2], status=ValidationStatus.INVALID, domain_exists=False,
|
| 200 |
+
has_mx=False, dns_error="nxdomain", reason_code="nxdomain"),
|
| 201 |
+
]
|
| 202 |
+
|
| 203 |
+
self.job_row["refunded_credits"] = 0
|
| 204 |
+
emails = ["a@x.test", "b@x.test", "c@x.test"]
|
| 205 |
+
# The endpoint spawns the runner as a fire-and-forget task; in tests
|
| 206 |
+
# that task would outlive the patch context and run the REAL engine on
|
| 207 |
+
# .test domains (all NXDOMAIN). Swallow the spawn — the lifecycle is
|
| 208 |
+
# driven deterministically below on a real loop.
|
| 209 |
+
def _no_task(coro):
|
| 210 |
+
coro.close()
|
| 211 |
+
return MagicMock()
|
| 212 |
+
with patch("app.api.jobs.engine.validate_batch", new=AsyncMock(side_effect=fake_batch)), \
|
| 213 |
+
patch("app.api.jobs.settings.bulk_sync_chunk_size", 2), \
|
| 214 |
+
patch("app.api.jobs.asyncio.create_task", side_effect=_no_task), \
|
| 215 |
+
self._auth():
|
| 216 |
+
res = client.post(
|
| 217 |
+
"/api/v1/jobs",
|
| 218 |
+
json={"job_id": "11111111-1111-1111-1111-111111111111",
|
| 219 |
+
"emails": emails},
|
| 220 |
+
headers={"Authorization": "Bearer faketoken"},
|
| 221 |
+
)
|
| 222 |
+
self.assertEqual(res.status_code, 200)
|
| 223 |
+
self.assertEqual(res.json()["accepted"], 3)
|
| 224 |
+
# Deterministic lifecycle: drive the same runner invocation to
|
| 225 |
+
# completion INSIDE the patch context (a test-client background
|
| 226 |
+
# task would outlive it and run the real engine on .test domains).
|
| 227 |
+
asyncio.run(jobs_mod._run_verifier_job(
|
| 228 |
+
"11111111-1111-1111-1111-111111111111",
|
| 229 |
+
"22222222-2222-2222-2222-222222222222", emails, True, True))
|
| 230 |
+
updates = [c.args[1] for c in self.repo_mock.update_verifier_job.call_args_list
|
| 231 |
+
if len(c.args) >= 2 and isinstance(c.args[1], dict)]
|
| 232 |
+
final = [u for u in updates if u.get("status") == "completed"]
|
| 233 |
+
self.assertTrue(final, f"runner never completed; updates={updates}")
|
| 234 |
+
last = final[-1]
|
| 235 |
+
self.assertEqual(last["bad_count"], 1) # nxdomain → bad
|
| 236 |
+
self.assertEqual(last["risky_count"], 1) # smtp unavailable → risky
|
| 237 |
+
self.assertEqual(last["valid_count"], 1)
|
| 238 |
+
# risky refund path executed server-side
|
| 239 |
+
self.repo_mock.refund_credits.assert_called_once()
|
| 240 |
+
self.assertEqual(self.repo_mock.refund_credits.call_args[0][1], 1)
|
| 241 |
+
|
| 242 |
+
def test_chunk_failure_finalizes_error_and_refunds(self):
|
| 243 |
+
"""Engine crash mid-run → job error + risky + unprocessed refund."""
|
| 244 |
+
|
| 245 |
+
async def boom(emails, deep=True):
|
| 246 |
+
raise RuntimeError("smtp stage exploded")
|
| 247 |
+
|
| 248 |
+
self.job_row["total_count"] = 5
|
| 249 |
+
self.job_row["credits_charged"] = 5
|
| 250 |
+
def _no_task(coro):
|
| 251 |
+
coro.close()
|
| 252 |
+
return MagicMock()
|
| 253 |
+
with patch("app.api.jobs.engine.validate_batch", new=AsyncMock(side_effect=boom)), \
|
| 254 |
+
patch("app.api.jobs.asyncio.create_task", side_effect=_no_task), \
|
| 255 |
+
self._auth():
|
| 256 |
+
res = client.post(
|
| 257 |
+
"/api/v1/jobs",
|
| 258 |
+
json={"job_id": "11111111-1111-1111-1111-111111111111",
|
| 259 |
+
"emails": ["a@x.test"] * 5},
|
| 260 |
+
headers={"Authorization": "Bearer faketoken"},
|
| 261 |
+
)
|
| 262 |
+
self.assertEqual(res.status_code, 200)
|
| 263 |
+
# Deterministic lifecycle inside the patch context.
|
| 264 |
+
asyncio.run(jobs_mod._run_verifier_job(
|
| 265 |
+
"11111111-1111-1111-1111-111111111111",
|
| 266 |
+
"22222222-2222-2222-2222-222222222222", ["a@x.test"] * 5, True, True))
|
| 267 |
+
updates = [c.args[1] for c in self.repo_mock.update_verifier_job.call_args_list
|
| 268 |
+
if len(c.args) >= 2 and isinstance(c.args[1], dict)]
|
| 269 |
+
err_calls = [u for u in updates if u.get("status") == "error"]
|
| 270 |
+
self.assertTrue(err_calls, f"runner never finalized the failed job; updates={updates}")
|
| 271 |
+
self.assertIn("Engine chunk failed", err_calls[-1]["error"])
|
| 272 |
+
# refund = risky(0) + unprocessed(5) clamped to charged(5)
|
| 273 |
+
self.repo_mock.refund_credits.assert_called_once()
|
| 274 |
+
self.assertEqual(self.repo_mock.refund_credits.call_args[0][1], 5)
|
| 275 |
+
|
| 276 |
+
|
| 277 |
+
if __name__ == "__main__":
|
| 278 |
+
unittest.main()
|