Spaces:
Running
feat(pdf): 新增异步作业端点 /api/pdf_jobs(提交+轮询+取件),同步端点一字节未改
Browse files起因:pro 用户 NotebookLM 导 PDF 报 ERROR(MD/JSON 正常)。PostHog 与 Space 运行日志
对账:同一份 7.78MB/354 消息文档,后端 8 次全部渲染成功(66.2–70.8s,出参 12.68MB),
客户端两次独立尝试都在发出请求后 60.28s/60.57s 抛 TypeError: Failed to fetch —— 后端还在
算、客户端已死约 10 秒,8 次重试把 cpu-basic 2 vCPU 全白烧。插件侧已实测排除(SW 可等
75s 静默、真插件跑到后端 127.7s 仍成功),且真链路实测 176.36s 静默 + 11.01MB 响应可正常
送达 ⇒ 病根不是"某处一定 60 秒断",而是**同步模式让一次长静默交互把命交给链路上任意一个
idle 型中间盒**(该用户具体凶手仍未定性)。
改动:
- lib/pdf-jobs.js(新):作业层。契约对齐 Azure Asynchronous Request-Reply / Google LRO /
RFC 9110:POST 202+Location+Retry-After;GET /:id 200 状态体(status/phase/createdAt/
lastUpdatedAt/error),完成用 **303 See Other**(规范点名不用 302:302 不保证改方法,
客户端可能重放 POST 从而重复渲染);GET /:id/download 流式(首字节即时);DELETE 取消。
幂等键=请求体 SHA-256 ⇒ 同内容重复提交复用既有作业,**结构性消灭白烧 CPU**(这是本案
实际损失最大的一条)。并发闸默认 2 并带队列/429 背压——排队时客户端只在轮询,不占长连接,
这是同步模式做不到的。结果落 os.tmpdir()/xwx-pdf-jobs,终态一律保留 ttlMs(含 FAILED,
失败原因必须仍可被轮询读到并分类),TTL 30 分钟 + 总量上限 LRU 淘汰。
- server.js:仅 require + register 两处 12 行。**/api/generate_pdf 一行未改**——它同时服务
全部已安装插件版本(近 30 天活跃设备散在 15 个版本,3.1.9 为 0 台),改成 202+JSON 会让
老客户端把 `response.ok` 判成成功、把 jobId 当 PDF 存盘,静默产出坏文件。
- worker 走进程内回环调用既有同步端点,而不是复制渲染流水线:动态超时、script 剥离、图片
检测、遥测日志、错误映射保持单一事实来源,异步层只是传输适配器。
验证:temp/test/.../verify-pdf-jobs.mjs 用假渲染端点跑契约 34/34,其中三条是本设计命门——
重复提交 renderCalls 恒 1、轮询期 503/429 继续轮询不重提交、单次交互最慢 5ms。
备份 backend-service_v3.1.2_2026-10-15T12-34-19(159 文件)。
生产 space/main **未推送**(用户指示:会影响生产环境)。
- lib/pdf-jobs.js +299 -0
- server.js +12 -0
|
@@ -0,0 +1,299 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**
|
| 2 |
+
* PDF 异步作业层(方案 A:作业制 + 轮询)
|
| 3 |
+
*
|
| 4 |
+
* 为什么要它:同步 `/api/generate_pdf` 让客户端**占着一条连接空等**——后端渲染
|
| 5 |
+
* 期间一个字节都不发。实测大文档要 66–181 秒,任何链路上的"无字节即断连"中间盒
|
| 6 |
+
* (VPN/代理/企业网关的 idle timeout)都会把请求掐死,浏览器只报
|
| 7 |
+
* `TypeError: Failed to fetch`,客户端重试又白烧一遍全量渲染。
|
| 8 |
+
* (2026-10-15 PostHog + HF 运行日志取证;扩展自身栈已被实测排除:
|
| 9 |
+
* MV3 SW 对 75s 静默、真插件对 127.7s 静默 + 11.5MB 响应都能正常完成。)
|
| 10 |
+
*
|
| 11 |
+
* 契约按行业规范(Azure Asynchronous Request-Reply / Google LRO / RFC 9110):
|
| 12 |
+
* POST /api/pdf_jobs → 202 + Location + Retry-After + {jobId,status}
|
| 13 |
+
* GET /api/pdf_jobs/:id → 200 {status,createdAt,lastUpdatedAt,phase,error}
|
| 14 |
+
* 完成时 303 See Other → /:id/download
|
| 15 |
+
* GET /api/pdf_jobs/:id/download → 200 application/pdf(立即开始吐字节)
|
| 16 |
+
* DELETE /api/pdf_jobs/:id → 200 {status:'CANCELED'}
|
| 17 |
+
*
|
| 18 |
+
* 三条设计决定,改代码前请先读:
|
| 19 |
+
* 1. **不动同步端点**:`/api/generate_pdf` 同时服务全部已安装版本(2.1.x→3.1.9),
|
| 20 |
+
* 改成 202 会让老客户端当场失效。这里是**新增**端点,老路径一字节不变。
|
| 21 |
+
* 2. **worker 走进程内回环**调用既有的 `/api/generate_pdf`,而不是把渲染流水线
|
| 22 |
+
* 复制一份。理由:单一事实来源——动态超时、script 剥离、图片检测、遥测日志、
|
| 23 |
+
* 错误映射全部留在原处;异步层只是个传输适配器。
|
| 24 |
+
* 3. **幂等键=请求体的 SHA-256**。同内容重复提交直接复用既有作业,绝不二次渲染
|
| 25 |
+
* (取证到的事故:同一份文档被完整渲染 8 次,全在 2 vCPU 免费档上白烧)。
|
| 26 |
+
*
|
| 27 |
+
* 并发闸 `concurrency` 是有意义的**背压**:作业排队时客户端只是在轮询,
|
| 28 |
+
* 不占用任何长连接——这是同步模式做不到的。
|
| 29 |
+
*/
|
| 30 |
+
|
| 31 |
+
'use strict';
|
| 32 |
+
|
| 33 |
+
const crypto = require('node:crypto');
|
| 34 |
+
const fs = require('node:fs');
|
| 35 |
+
const path = require('node:path');
|
| 36 |
+
|
| 37 |
+
const STATUS = { PENDING: 'PENDING', RUNNING: 'RUNNING', SUCCEEDED: 'SUCCEEDED', FAILED: 'FAILED', CANCELED: 'CANCELED' };
|
| 38 |
+
const TERMINAL = new Set([STATUS.SUCCEEDED, STATUS.FAILED, STATUS.CANCELED]);
|
| 39 |
+
|
| 40 |
+
function sha256(buf) {
|
| 41 |
+
return crypto.createHash('sha256').update(buf).digest('hex');
|
| 42 |
+
}
|
| 43 |
+
|
| 44 |
+
/**
|
| 45 |
+
* @param {Object} cfg
|
| 46 |
+
* @param {string} cfg.selfOrigin 本服务自身的 origin(回环调用用),如 http://127.0.0.1:7860
|
| 47 |
+
* @param {string} [cfg.syncPath] 被复用的同步渲染端点路径,默认 /api/generate_pdf
|
| 48 |
+
* @param {number} [cfg.concurrency] 同时跑的渲染数(默认 2,给同步直连用户留出浏览器池槽位)
|
| 49 |
+
* @param {number} [cfg.ttlMs] 结果保留时长(默认 30 分钟)
|
| 50 |
+
* @param {number} [cfg.maxTotalBytes] 结果总字节上限,超了按完成时间淘汰(默认 2 GiB)
|
| 51 |
+
* @param {number} [cfg.queueLimit] 排队作业上限(默认 50),超了直接 429
|
| 52 |
+
* @param {string} [cfg.resultDir] 结果落盘目录,默认 os.tmpdir()/xwx-pdf-jobs
|
| 53 |
+
* @param {(msg:string)=>void} [cfg.log]
|
| 54 |
+
* @param {typeof fetch} [cfg.fetchImpl] 仅供测试注入
|
| 55 |
+
*/
|
| 56 |
+
function createPdfJobService(cfg) {
|
| 57 |
+
const {
|
| 58 |
+
selfOrigin,
|
| 59 |
+
syncPath = '/api/generate_pdf',
|
| 60 |
+
concurrency = 2,
|
| 61 |
+
ttlMs = 30 * 60_000,
|
| 62 |
+
maxTotalBytes = 2 * 1024 * 1024 * 1024,
|
| 63 |
+
queueLimit = 50,
|
| 64 |
+
resultDir = path.join(require('node:os').tmpdir(), 'xwx-pdf-jobs'),
|
| 65 |
+
sweepIntervalMs = 60_000,
|
| 66 |
+
log = (m) => console.log(m),
|
| 67 |
+
fetchImpl = globalThis.fetch,
|
| 68 |
+
} = cfg;
|
| 69 |
+
|
| 70 |
+
if (!selfOrigin) throw new Error('createPdfJobService: selfOrigin is required');
|
| 71 |
+
fs.mkdirSync(resultDir, { recursive: true });
|
| 72 |
+
|
| 73 |
+
const jobs = new Map(); // jobId -> job
|
| 74 |
+
const byDigest = new Map(); // sha256(body) -> jobId
|
| 75 |
+
let running = 0;
|
| 76 |
+
const waiters = [];
|
| 77 |
+
|
| 78 |
+
const nowIso = () => new Date().toISOString();
|
| 79 |
+
const resultFile = (id) => path.join(resultDir, `${id}.pdf`);
|
| 80 |
+
|
| 81 |
+
function touch(job, patch) {
|
| 82 |
+
Object.assign(job, patch || {});
|
| 83 |
+
job.lastUpdatedAt = nowIso();
|
| 84 |
+
// 任何终态(含 FAILED/CANCELED)都从今天起保留 ttlMs:失败原因必须仍可被客户端
|
| 85 |
+
// 轮询读到并分类,不能一失败就 404。
|
| 86 |
+
if (TERMINAL.has(job.status) && !job.expiresAt) job.expiresAt = Date.now() + ttlMs;
|
| 87 |
+
return job;
|
| 88 |
+
}
|
| 89 |
+
|
| 90 |
+
/** 入队;并发未满则立刻开跑 */
|
| 91 |
+
function pump() {
|
| 92 |
+
while (running < concurrency && waiters.length) {
|
| 93 |
+
const job = waiters.shift();
|
| 94 |
+
if (job.status !== STATUS.PENDING) continue; // 已被取消/复用
|
| 95 |
+
running += 1;
|
| 96 |
+
execute(job).finally(() => {
|
| 97 |
+
running -= 1;
|
| 98 |
+
pump();
|
| 99 |
+
});
|
| 100 |
+
}
|
| 101 |
+
}
|
| 102 |
+
|
| 103 |
+
/** 回环调用既有同步渲染端点,把 PDF 落盘 */
|
| 104 |
+
async function execute(job) {
|
| 105 |
+
const t0 = Date.now();
|
| 106 |
+
touch(job, { status: STATUS.RUNNING, phase: 'RENDERING', startedAt: nowIso() });
|
| 107 |
+
try {
|
| 108 |
+
const resp = await fetchImpl(selfOrigin + syncPath, {
|
| 109 |
+
method: 'POST',
|
| 110 |
+
headers: { 'Content-Type': 'application/json' },
|
| 111 |
+
body: job.body,
|
| 112 |
+
});
|
| 113 |
+
const buf = Buffer.from(await resp.arrayBuffer());
|
| 114 |
+
if (!resp.ok) {
|
| 115 |
+
// 后端已给出 HTTP 语义错误(400/413/500…)——原样带进作业状态,客户端据此分类
|
| 116 |
+
const snippet = buf.subarray(0, 500).toString('utf8');
|
| 117 |
+
touch(job, { status: STATUS.FAILED, phase: 'FAILED', error: { httpStatus: resp.status, message: snippet } });
|
| 118 |
+
log(`[PDF-JOB] ${job.jobId} FAILED http=${resp.status} @${((Date.now() - t0) / 1000).toFixed(1)}s body=${snippet.slice(0, 160)}`);
|
| 119 |
+
return;
|
| 120 |
+
}
|
| 121 |
+
const file = resultFile(job.jobId);
|
| 122 |
+
await fs.promises.writeFile(file, buf);
|
| 123 |
+
touch(job, {
|
| 124 |
+
status: STATUS.SUCCEEDED, phase: 'DONE', bytes: buf.length, file,
|
| 125 |
+
finishedAt: nowIso(), expiresAt: Date.now() + ttlMs,
|
| 126 |
+
});
|
| 127 |
+
job.digest && byDigest.set(job.digest, job.jobId);
|
| 128 |
+
log(`[PDF-JOB] ${job.jobId} SUCCEEDED ${(buf.length / 1048576).toFixed(2)}MB @${((Date.now() - t0) / 1000).toFixed(1)}s (queued=${job.queueWaitMs}ms)`);
|
| 129 |
+
enforceRetention();
|
| 130 |
+
} catch (e) {
|
| 131 |
+
touch(job, { status: STATUS.FAILED, phase: 'FAILED', error: { message: `render_loopback_${e.name || 'Error'}: ${e.message}` } });
|
| 132 |
+
log(`[PDF-JOB] ${job.jobId} ERROR ${e.name}: ${e.message}`);
|
| 133 |
+
}
|
| 134 |
+
}
|
| 135 |
+
|
| 136 |
+
/** 立即回收已过期结果;并按总量上限淘汰最早完成的 */
|
| 137 |
+
function enforceRetention() {
|
| 138 |
+
const t = Date.now();
|
| 139 |
+
for (const [id, job] of jobs) {
|
| 140 |
+
if (job.file && ((job.expiresAt && job.expiresAt < t) || TERMINAL.has(job.status) && job.status !== STATUS.SUCCEEDED)) {
|
| 141 |
+
discard(id, job);
|
| 142 |
+
}
|
| 143 |
+
}
|
| 144 |
+
let total = 0;
|
| 145 |
+
const done = [];
|
| 146 |
+
for (const job of jobs.values()) if (job.bytes) { total += job.bytes; done.push(job); }
|
| 147 |
+
if (total > maxTotalBytes) {
|
| 148 |
+
done.sort((a, b) => (a.finishedAt || '').localeCompare(b.finishedAt || ''));
|
| 149 |
+
for (const job of done) {
|
| 150 |
+
if (total <= maxTotalBytes) break;
|
| 151 |
+
total -= job.bytes;
|
| 152 |
+
discard(job.jobId, job);
|
| 153 |
+
}
|
| 154 |
+
}
|
| 155 |
+
}
|
| 156 |
+
|
| 157 |
+
function discard(id, job) {
|
| 158 |
+
if (job.file) fs.promises.unlink(job.file).catch(() => {});
|
| 159 |
+
if (job.digest) byDigest.delete(job.digest);
|
| 160 |
+
jobs.delete(id);
|
| 161 |
+
}
|
| 162 |
+
|
| 163 |
+
// 兜底清扫:容器长驻时即便无人访问也要回收磁盘。
|
| 164 |
+
// sweepIntervalMs 可注入,仅为让 TTL 行为能在测试里被快速验证(生产默认 60s)。
|
| 165 |
+
const sweep = setInterval(enforceRetention, sweepIntervalMs);
|
| 166 |
+
if (typeof sweep.unref === 'function') sweep.unref();
|
| 167 |
+
|
| 168 |
+
// ── HTTP 层 ────────────────────────────────────────────────────────
|
| 169 |
+
const statusPayload = (job) => ({
|
| 170 |
+
jobId: job.jobId,
|
| 171 |
+
status: job.status,
|
| 172 |
+
phase: job.phase,
|
| 173 |
+
createdAt: job.createdAt,
|
| 174 |
+
lastUpdatedAt: job.lastUpdatedAt,
|
| 175 |
+
startedAt: job.startedAt || null,
|
| 176 |
+
finishedAt: job.finishedAt || null,
|
| 177 |
+
bytes: job.bytes || null,
|
| 178 |
+
reused: job.reused === true,
|
| 179 |
+
error: job.status === STATUS.FAILED ? job.error : undefined,
|
| 180 |
+
});
|
| 181 |
+
|
| 182 |
+
const rel = (id) => `/api/pdf_jobs/${encodeURIComponent(id)}`;
|
| 183 |
+
|
| 184 |
+
function register(app) {
|
| 185 |
+
app.post('/api/pdf_jobs', async (req, res) => {
|
| 186 |
+
let raw;
|
| 187 |
+
try {
|
| 188 |
+
raw = req.body == null ? Buffer.alloc(0) : Buffer.from(JSON.stringify(req.body));
|
| 189 |
+
} catch (e) {
|
| 190 |
+
return res.status(400).json({ error: 'unserializable_payload' });
|
| 191 |
+
}
|
| 192 |
+
if (!raw.length) return res.status(400).json({ error: 'empty_payload' });
|
| 193 |
+
|
| 194 |
+
const digest = sha256(raw);
|
| 195 |
+
const existingId = byDigest.get(digest);
|
| 196 |
+
const existing = existingId ? jobs.get(existingId) : null;
|
| 197 |
+
if (existing && (existing.status === STATUS.RUNNING || existing.status === STATUS.SUCCEEDED)) {
|
| 198 |
+
// 同内容已在渲染/已完成 ⇒ 复用,绝不重复烧 CPU
|
| 199 |
+
existing.reused = true;
|
| 200 |
+
res.set('Location', rel(existing.jobId));
|
| 201 |
+
log(`[PDF-JOB] dedupe hit → ${existing.jobId} status=${existing.status} bytes=${raw.length}`);
|
| 202 |
+
return res.status(202).json({ ...statusPayload(existing), Location: rel(existing.jobId) });
|
| 203 |
+
}
|
| 204 |
+
|
| 205 |
+
const active = [...jobs.values()].filter((j) => !TERMINAL.has(j.status)).length;
|
| 206 |
+
if (active >= queueLimit) {
|
| 207 |
+
res.set('Retry-After', '10');
|
| 208 |
+
return res.status(429).json({ error: 'queue_full', active });
|
| 209 |
+
}
|
| 210 |
+
|
| 211 |
+
const job = {
|
| 212 |
+
jobId: crypto.randomUUID(),
|
| 213 |
+
digest,
|
| 214 |
+
body: raw.toString('utf8'),
|
| 215 |
+
status: STATUS.PENDING,
|
| 216 |
+
phase: 'QUEUED',
|
| 217 |
+
createdAt: nowIso(),
|
| 218 |
+
lastUpdatedAt: nowIso(),
|
| 219 |
+
queuedAt: Date.now(),
|
| 220 |
+
};
|
| 221 |
+
job.queueWaitMs = 0;
|
| 222 |
+
jobs.set(job.jobId, job);
|
| 223 |
+
byDigest.set(digest, job.jobId);
|
| 224 |
+
waiters.push(job);
|
| 225 |
+
setImmediate(pump);
|
| 226 |
+
|
| 227 |
+
res.set('Location', rel(job.jobId));
|
| 228 |
+
res.set('Retry-After', '3');
|
| 229 |
+
log(`[PDF-JOB] accepted ${job.jobId} payloadKB=${(raw.length / 1024) | 0} active=${active + 1} running=${running}`);
|
| 230 |
+
return res.status(202).json({ jobId: job.jobId, status: job.status, phase: job.phase, Location: rel(job.jobId) });
|
| 231 |
+
});
|
| 232 |
+
|
| 233 |
+
app.get('/api/pdf_jobs/:id', (req, res) => {
|
| 234 |
+
const job = jobs.get(req.params.id);
|
| 235 |
+
if (!job) {
|
| 236 |
+
// 容器重启/回收后作业不存在——这是**可安全重提交**的状态,不是失败
|
| 237 |
+
return res.status(404).json({ error: 'job_not_found', retryable: true });
|
| 238 |
+
}
|
| 239 |
+
if (job.status === STATUS.SUCCEEDED) {
|
| 240 |
+
// 303 而非 302:302 不保证改方法,部分客户端会重放原方法(规范明确点名)
|
| 241 |
+
return res.redirect(303, rel(job.jobId) + '/download');
|
| 242 |
+
}
|
| 243 |
+
if (!TERMINAL.has(job.status)) res.set('Retry-After', String(pollIntervalSec(job)));
|
| 244 |
+
return res.status(200).json(statusPayload(job));
|
| 245 |
+
});
|
| 246 |
+
|
| 247 |
+
app.get('/api/pdf_jobs/:id/download', (req, res) => {
|
| 248 |
+
const job = jobs.get(req.params.id);
|
| 249 |
+
if (!job) return res.status(404).json({ error: 'job_not_found', retryable: true });
|
| 250 |
+
if (job.status === STATUS.FAILED) {
|
| 251 |
+
return res.status(500).json({ error: 'render_failed', ...job.error });
|
| 252 |
+
}
|
| 253 |
+
if (job.status !== STATUS.SUCCEEDED) {
|
| 254 |
+
res.set('Retry-After', String(pollIntervalSec(job)));
|
| 255 |
+
return res.status(202).json(statusPayload(job));
|
| 256 |
+
}
|
| 257 |
+
res.setHeader('Content-Type', 'application/pdf');
|
| 258 |
+
res.setHeader('Content-Disposition', 'attachment; filename=export.pdf');
|
| 259 |
+
res.setHeader('Cache-Control', 'private, no-store');
|
| 260 |
+
// 流式:首字节立刻出发,链路上任何 idle 型中间盒都不会再掐这条连接
|
| 261 |
+
const stream = fs.createReadStream(job.file);
|
| 262 |
+
stream.on('error', () => { if (!res.headersSent) res.status(410).json({ error: 'result_expired' }); else res.destroy(); });
|
| 263 |
+
stream.pipe(res);
|
| 264 |
+
});
|
| 265 |
+
|
| 266 |
+
app.delete('/api/pdf_jobs/:id', (req, res) => {
|
| 267 |
+
const job = jobs.get(req.params.id);
|
| 268 |
+
if (!job) return res.status(404).json({ error: 'job_not_found' });
|
| 269 |
+
if (TERMINAL.has(job.status)) return res.status(200).json(statusPayload(job));
|
| 270 |
+
const wi = waiters.indexOf(job);
|
| 271 |
+
if (wi >= 0) waiters.splice(wi, 1);
|
| 272 |
+
touch(job, { status: STATUS.CANCELED, phase: 'CANCELED' });
|
| 273 |
+
return res.status(200).json(statusPayload(job));
|
| 274 |
+
});
|
| 275 |
+
|
| 276 |
+
app.get('/api/pdf_jobs', (_req, res) => {
|
| 277 |
+
res.json({
|
| 278 |
+
running,
|
| 279 |
+
queued: waiters.length,
|
| 280 |
+
jobs: jobs.size,
|
| 281 |
+
concurrency,
|
| 282 |
+
ttlMs,
|
| 283 |
+
totalResultBytes: [...jobs.values()].reduce((s, j) => s + (j.bytes || 0), 0),
|
| 284 |
+
});
|
| 285 |
+
});
|
| 286 |
+
}
|
| 287 |
+
|
| 288 |
+
/** 前 3 次快、之后按已等多久放宽;上限 8s(指数退避的简化版,配合客户端 jitter) */
|
| 289 |
+
function pollIntervalSec(job) {
|
| 290 |
+
const waited = Date.now() - job.queuedAt;
|
| 291 |
+
if (waited < 10_000) return 2;
|
| 292 |
+
if (waited < 60_000) return 4;
|
| 293 |
+
return 8;
|
| 294 |
+
}
|
| 295 |
+
|
| 296 |
+
return { register, jobs, STATUS, stats: () => ({ running, queued: waiters.length, jobs: jobs.size }) };
|
| 297 |
+
}
|
| 298 |
+
|
| 299 |
+
module.exports = { createPdfJobService, STATUS };
|
|
@@ -25,6 +25,7 @@
|
|
| 25 |
const express = require('express');
|
| 26 |
const puppeteer = require('puppeteer');
|
| 27 |
const cors = require('cors');
|
|
|
|
| 28 |
const { getHighlighter } = require('shiki');
|
| 29 |
const fs = require('fs');
|
| 30 |
const path = require('path');
|
|
@@ -1437,6 +1438,17 @@ app.post('/api/render_charts', async (req, res) => {
|
|
| 1437 |
res.json({ results: renderResults });
|
| 1438 |
});
|
| 1439 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1440 |
// ─── Body-parser / JSON 解析错误兜底 ───────────────────────────
|
| 1441 |
// express.json() 解析失败(Content-Type: application/json 但 body 非法)时,
|
| 1442 |
// 会抛 entity.parse.failed 错误。外部探针/扫描器会定期向公开接口发送这类
|
|
|
|
| 25 |
const express = require('express');
|
| 26 |
const puppeteer = require('puppeteer');
|
| 27 |
const cors = require('cors');
|
| 28 |
+
const { createPdfJobService } = require('./lib/pdf-jobs');
|
| 29 |
const { getHighlighter } = require('shiki');
|
| 30 |
const fs = require('fs');
|
| 31 |
const path = require('path');
|
|
|
|
| 1438 |
res.json({ results: renderResults });
|
| 1439 |
});
|
| 1440 |
|
| 1441 |
+
// ─── 异步 PDF 作业层(方案 A:作业制 + 轮询)──────────────────────────
|
| 1442 |
+
// 同步 /api/generate_pdf 保留原样服务全部老客户端;新作业端点把"一次长静默请求"
|
| 1443 |
+
// 拆成"短提交 + 短轮询 + 短下载",链路上任何 idle 型中间盒都碰不到长连接。
|
| 1444 |
+
// 实现细节与三条设计决定见 lib/pdf-jobs.js 头注释。
|
| 1445 |
+
const pdfJobs = createPdfJobService({
|
| 1446 |
+
selfOrigin: `http://127.0.0.1:${port}`,
|
| 1447 |
+
concurrency: Number(process.env.PDF_JOB_CONCURRENCY || 2),
|
| 1448 |
+
ttlMs: Number(process.env.PDF_JOB_TTL_MS || 30 * 60 * 1000),
|
| 1449 |
+
});
|
| 1450 |
+
pdfJobs.register(app);
|
| 1451 |
+
|
| 1452 |
// ─── Body-parser / JSON 解析错误兜底 ───────────────────────────
|
| 1453 |
// express.json() 解析失败(Content-Type: application/json 但 body 非法)时,
|
| 1454 |
// 会抛 entity.parse.failed 错误。外部探针/扫描器会定期向公开接口发送这类
|