BirkhoffLee commited on
Commit
37e12f1
·
unverified ·
1 Parent(s): f8d66a3

feat: 切换到自定义翻译页面了

Browse files
Files changed (7) hide show
  1. .cnb.yml +2 -2
  2. Dockerfile +12 -12
  3. README.md +41 -18
  4. entrypoint.sh +7 -26
  5. gateway.py +0 -368
  6. src/gateway.py +1268 -0
  7. {scripts → src/scripts}/update_space_secret.py +0 -0
.cnb.yml CHANGED
@@ -29,7 +29,7 @@ main:
29
  HF_SPACE_URL: "https://huggingface.co/spaces/ServiceX/PDF"
30
  HF_SPACE_SECRET_KEY: "BASIC_AUTH_USERS"
31
  BASIC_AUTH_FILE: "basic_auth_users.txt"
32
- HF_EXCLUDE_FILES: ".cnb.yml basic_auth_users.txt scripts"
33
  stages:
34
  - name: Update HF Space Secret
35
  script: |
@@ -40,7 +40,7 @@ main:
40
  HF_SPACE_URL="${HF_SPACE_URL}" \
41
  HF_SPACE_SECRET_KEY="${HF_SPACE_SECRET_KEY}" \
42
  BASIC_AUTH_FILE="${BASIC_AUTH_FILE}" \
43
- uvx --from huggingface_hub python scripts/update_space_secret.py
44
 
45
  - name: Push Git To Huggingface Spaces
46
  script: |
 
29
  HF_SPACE_URL: "https://huggingface.co/spaces/ServiceX/PDF"
30
  HF_SPACE_SECRET_KEY: "BASIC_AUTH_USERS"
31
  BASIC_AUTH_FILE: "basic_auth_users.txt"
32
+ HF_EXCLUDE_FILES: ".cnb.yml basic_auth_users.txt src/scripts"
33
  stages:
34
  - name: Update HF Space Secret
35
  script: |
 
40
  HF_SPACE_URL="${HF_SPACE_URL}" \
41
  HF_SPACE_SECRET_KEY="${HF_SPACE_SECRET_KEY}" \
42
  BASIC_AUTH_FILE="${BASIC_AUTH_FILE}" \
43
+ uvx --from huggingface_hub python src/scripts/update_space_secret.py
44
 
45
  - name: Push Git To Huggingface Spaces
46
  script: |
Dockerfile CHANGED
@@ -19,21 +19,21 @@ COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
19
 
20
  ENV PATH="/root/.local/bin:${PATH}" \
21
  PYTHONDONTWRITEBYTECODE=1 \
22
- PYTHONUNBUFFERED=1 \
23
- GRADIO_SERVER_NAME=127.0.0.1 \
24
- GRADIO_SERVER_PORT=7861
25
 
26
- # 安装 pdf2zh-next
27
- RUN uv tool install --python 3.12 pdf2zh-next
28
-
29
- # 为网关创建独立 venv 并安装依赖
30
  RUN uv venv /opt/gateway --python 3.12 && \
31
  uv pip install --python /opt/gateway/bin/python \
32
- "fastapi>=0.115" "uvicorn[standard]>=0.32" "httpx>=0.28" \
33
- python-multipart bcrypt itsdangerous websockets
34
-
35
- # 复制网关和启动脚本
36
- COPY gateway.py /gateway.py
 
 
 
 
 
37
  COPY entrypoint.sh /entrypoint.sh
38
  RUN chmod +x /entrypoint.sh
39
 
 
19
 
20
  ENV PATH="/root/.local/bin:${PATH}" \
21
  PYTHONDONTWRITEBYTECODE=1 \
22
+ PYTHONUNBUFFERED=1
 
 
23
 
24
+ # 为网关创建独立 venv 并安装依赖(包含 pdf2zh-next
 
 
 
25
  RUN uv venv /opt/gateway --python 3.12 && \
26
  uv pip install --python /opt/gateway/bin/python \
27
+ "pdf2zh-next" \
28
+ "fastapi>=0.115" \
29
+ "uvicorn[standard]>=0.32" \
30
+ "httpx>=0.28" \
31
+ "python-multipart" \
32
+ "bcrypt" \
33
+ "itsdangerous"
34
+
35
+ # 复制应用代码和启动脚本
36
+ COPY src /src
37
  COPY entrypoint.sh /entrypoint.sh
38
  RUN chmod +x /entrypoint.sh
39
 
README.md CHANGED
@@ -9,18 +9,29 @@ pinned: false
9
 
10
  ## PDFMathTranslate-next on HuggingFace Spaces
11
 
12
- 这个仓库用于部署 `pdf2zh_next --gui`并通过 Caddy BasicAuth 做访问门控。
 
 
 
 
 
 
13
 
14
  ### 运行架构
15
 
16
- 1. `pdf2zh_next --gui` 在容器内监听 `127.0.0.1:7861`
17
- 2. Caddy 监听外部端口 `7860`
18
- 3. `/healthz` 所有请求都需要 BasicAuth
19
- 4. 认证通过后,Caddy 反向代理到 `pdf2zh_next`
20
 
21
  ### 必需 Secret
22
 
23
- 在 HuggingFace Space 设置 `BASIC_AUTH_USERS`(多行文本)
 
 
 
 
 
24
 
25
  ```text
26
  alice:your_password_1
@@ -28,22 +39,20 @@ bob:your_password_2
28
  ```
29
 
30
  规则:
 
31
  - 每行一个账号,格式 `username:password`
32
  - 空行和 `#` 开头行会被忽略
33
- - 容器启动时会把明文密码转换为 bcrypt 哈希,仓库不保存真实密码
34
 
35
- ### CI 同步到 Spaces 的排除策略
36
 
37
- - 本仓库可本地维护密码文件:`basic_auth_users.txt`
38
- - CNB `push` 分两个阶段
39
- - 阶段 1:读取 `basic_auth_users.txt`,自动更新 Space Secret `BASIC_AUTH_USERS`
40
- - 阶段 2删除排除文件后,强制推送代码到 HuggingFace Spaces
41
- - 推送前会删除以下文件
42
- - `.cnb.yml`
43
- - `basic_auth_users.txt`
44
- - `scripts/`
45
- - 排除列表在 `.cnb.yml` 的 `HF_EXCLUDE_FILES` 中配置(空格分隔)
46
- - 密钥更新脚本:`scripts/update_space_secret.py`
47
 
48
  ### 健康检查
49
 
@@ -55,5 +64,19 @@ bob:your_password_2
55
  docker build -t pdf2zh-gated .
56
  docker run --rm -p 7860:7860 \
57
  -e BASIC_AUTH_USERS=$'alice:pass1\nbob:pass2' \
 
58
  pdf2zh-gated
59
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
 
10
  ## PDFMathTranslate-next on HuggingFace Spaces
11
 
12
+ 这个仓库部署一个单体 FastAPI 服务包含:
13
+
14
+ 1. 用户登录(Session Cookie)
15
+ 2. 自研 Web UI(上传 PDF、查看任务、下载结果)
16
+ 3. 任务队列(单 worker)调用 `pdf2zh_next` Python API
17
+ 4. 内部 OpenAI 兼容代理:`/internal/openai/v1/chat/completions`
18
+ 5. 按登录用户名计费(token + USD)
19
 
20
  ### 运行架构
21
 
22
+ 1. 用户访问 `:7860` 登录并提交翻译任务
23
+ 2. 后台 worker 调用 `pdf2zh_next`,并把 OpenAI 请求发到本机内部代理
24
+ 3. 内部代理转发到 OpenAI 官方 API同时记录 token 用量
25
+ 4. 计费按 `username` 聚合,前端展示账单
26
 
27
  ### 必需 Secret
28
 
29
+ 在 HuggingFace Space 设置:
30
+
31
+ - `BASIC_AUTH_USERS`(多行文本)
32
+ - `OPENAI_API_KEY`
33
+
34
+ `BASIC_AUTH_USERS` 格式:
35
 
36
  ```text
37
  alice:your_password_1
 
39
  ```
40
 
41
  规则:
42
+
43
  - 每行一个账号,格式 `username:password`
44
  - 空行和 `#` 开头行会被忽略
45
+ - 支持明文密码 bcrypt 哈希
46
 
47
+ ### 可选环境变量
48
 
49
+ - `SESSION_SECRET`:Session 签名密钥
50
+ - `INTERNAL_KEY_SALT`:内部 key 生成盐(默认复用 `SESSION_SECRET`)
51
+ - `DEFAULT_OPENAI_MODEL`:默认模型(默认 `gpt-4o-mini`
52
+ - `DEFAULT_LANG_IN`默认源语言(默认 `en`)
53
+ - `DEFAULT_LANG_OUT`默认目标语言(默认 `zh`)
54
+ - `TRANSLATION_QPS`:翻译 QPS(默认 `4`)
55
+ - `DATA_DIR`:数据目录(默认 `/data`)
 
 
 
56
 
57
  ### 健康检查
58
 
 
64
  docker build -t pdf2zh-gated .
65
  docker run --rm -p 7860:7860 \
66
  -e BASIC_AUTH_USERS=$'alice:pass1\nbob:pass2' \
67
+ -e OPENAI_API_KEY='sk-your-openai-key' \
68
  pdf2zh-gated
69
  ```
70
+
71
+ ### CI 同步到 Spaces 的排除策略
72
+
73
+ - 本仓库可本地维护密码文件:`basic_auth_users.txt`
74
+ - CNB `push` 分两个阶段:
75
+ - 阶段 1:读取 `basic_auth_users.txt`,自动更新 Space Secret `BASIC_AUTH_USERS`
76
+ - 阶段 2:删除排除文件后,强制推送代码到 HuggingFace Spaces
77
+ - 推送前会删除以下文件:
78
+ - `.cnb.yml`
79
+ - `basic_auth_users.txt`
80
+ - `src/scripts/`
81
+ - 排除列表在 `.cnb.yml` 的 `HF_EXCLUDE_FILES` 中配置(空格分隔)
82
+ - 密钥更新脚本:`src/scripts/update_space_secret.py`
entrypoint.sh CHANGED
@@ -1,37 +1,18 @@
1
  #!/usr/bin/env bash
2
  set -euo pipefail
3
 
4
- # 验证必需的环境变量
5
  RAW_USERS="${BASIC_AUTH_USERS:-}"
6
  if [[ -z "${RAW_USERS}" ]]; then
7
  echo "[ERROR] BASIC_AUTH_USERS is required. Use one 'username:password' per line." >&2
8
  exit 1
9
  fi
10
 
11
- # 固定内部监听,确保 pdf2zh_next 只能经由网关访问
12
- export GRADIO_SERVER_NAME="${GRADIO_SERVER_NAME:-127.0.0.1}"
13
- export GRADIO_SERVER_PORT="${GRADIO_SERVER_PORT:-7861}"
14
- # 部分 Gradio 应用优先读取 PORT;强制覆盖避免与网关的 7860 冲突
15
- export PORT="${GRADIO_SERVER_PORT}"
16
-
17
- echo "[INFO] Starting pdf2zh_next on ${GRADIO_SERVER_NAME}:${GRADIO_SERVER_PORT}"
18
- pdf2zh_next --gui --server-port "${GRADIO_SERVER_PORT}" &
19
- PDF_PID=$!
20
 
21
  echo "[INFO] Starting gateway on :7860"
22
- cd / && /opt/gateway/bin/uvicorn gateway:app \
23
- --host 0.0.0.0 --port 7860 --log-level info &
24
- GW_PID=$!
25
-
26
- cleanup() {
27
- kill "${PDF_PID}" "${GW_PID}" 2>/dev/null || true
28
- }
29
- trap cleanup INT TERM EXIT
30
-
31
- # 任一进程退出都结束容器
32
- set +e
33
- wait -n "${PDF_PID}" "${GW_PID}"
34
- EXIT_CODE=$?
35
- set -e
36
- echo "[ERROR] A service exited unexpectedly. Shutting down."
37
- exit "${EXIT_CODE}"
 
1
  #!/usr/bin/env bash
2
  set -euo pipefail
3
 
 
4
  RAW_USERS="${BASIC_AUTH_USERS:-}"
5
  if [[ -z "${RAW_USERS}" ]]; then
6
  echo "[ERROR] BASIC_AUTH_USERS is required. Use one 'username:password' per line." >&2
7
  exit 1
8
  fi
9
 
10
+ if [[ -z "${OPENAI_API_KEY:-}" ]]; then
11
+ echo "[ERROR] OPENAI_API_KEY is required." >&2
12
+ exit 1
13
+ fi
 
 
 
 
 
14
 
15
  echo "[INFO] Starting gateway on :7860"
16
+ cd / && exec /opt/gateway/bin/uvicorn gateway:app \
17
+ --app-dir /src \
18
+ --host 0.0.0.0 --port 7860 --workers 1 --log-level info
 
 
 
 
 
 
 
 
 
 
 
 
 
gateway.py DELETED
@@ -1,368 +0,0 @@
1
- #!/usr/bin/env python3
2
- """FastAPI 认证网关:Session 登录 + 反向代理到 pdf2zh_next(:7861)。
3
-
4
- 架构:用户 → Gateway(:7860) [Session Auth] → pdf2zh_next(:7861)
5
- """
6
-
7
- import asyncio
8
- import logging
9
- import os
10
- import secrets
11
- from typing import Optional
12
-
13
- import bcrypt
14
- import httpx
15
- import websockets
16
- import websockets.exceptions
17
- from fastapi import FastAPI, Form, Request, WebSocket
18
- from fastapi.responses import HTMLResponse, RedirectResponse, Response, StreamingResponse
19
- from itsdangerous import BadSignature, SignatureExpired, TimestampSigner
20
- from starlette.background import BackgroundTask
21
-
22
- # ── 配置 ──────────────────────────────────────────────────────────────────────
23
- UPSTREAM_HTTP = "http://127.0.0.1:7861"
24
- UPSTREAM_WS = "ws://127.0.0.1:7861"
25
- SESSION_COOKIE = "gw_session"
26
- SESSION_MAX_AGE = 86400 # 24 小时
27
-
28
- SECRET_KEY = os.environ.get("SESSION_SECRET") or secrets.token_hex(32)
29
- signer = TimestampSigner(SECRET_KEY)
30
-
31
- # hop-by-hop headers(不透传给上游或客户端)
32
- HOP_BY_HOP = frozenset(
33
- [
34
- "connection",
35
- "keep-alive",
36
- "proxy-authenticate",
37
- "proxy-authorization",
38
- "te",
39
- "trailers",
40
- "transfer-encoding",
41
- "upgrade",
42
- ]
43
- )
44
-
45
- logging.basicConfig(
46
- level=logging.INFO,
47
- format="%(asctime)s %(levelname)s %(name)s - %(message)s",
48
- )
49
- logger = logging.getLogger("gateway")
50
-
51
-
52
- # ── 用户加载与认证 ────────────────────────────────────────────────────────────
53
- def _load_users() -> dict[str, str]:
54
- """从 BASIC_AUTH_USERS 环境变量加载用户名和明文密码。
55
-
56
- 支持格式:每行 `username:password`,支持 \\n 转义的单行形式。
57
- """
58
- raw = os.environ.get("BASIC_AUTH_USERS", "").replace("\\n", "\n")
59
- users: dict[str, str] = {}
60
- for line in raw.splitlines():
61
- line = line.strip()
62
- if not line or line.startswith("#"):
63
- continue
64
- if ":" not in line:
65
- logger.warning("Skipping invalid BASIC_AUTH_USERS line (no colon)")
66
- continue
67
- username, password = line.split(":", 1)
68
- if username and password:
69
- users[username] = password
70
- if not users:
71
- logger.error("No valid users found — authentication will always fail")
72
- return users
73
-
74
-
75
- USERS = _load_users()
76
-
77
-
78
- def _verify_credentials(username: str, password: str) -> bool:
79
- """验证用户名密码,使用 secrets.compare_digest 防止时序攻击。
80
-
81
- 支持明文密码和 bcrypt 哈希(以 $2b$ 开头)。
82
- """
83
- stored = USERS.get(username)
84
- if stored is None:
85
- return False
86
- if stored.startswith("$2"):
87
- # bcrypt 哈希
88
- return bcrypt.checkpw(password.encode(), stored.encode())
89
- # 明文对比
90
- return secrets.compare_digest(stored, password)
91
-
92
-
93
- # ── Session 签名 ──────────────────────────────────────────────────────────────
94
- def _make_session(username: str) -> str:
95
- return signer.sign(username).decode()
96
-
97
-
98
- def _verify_session(token: str) -> Optional[str]:
99
- try:
100
- return signer.unsign(token, max_age=SESSION_MAX_AGE).decode()
101
- except (BadSignature, SignatureExpired):
102
- return None
103
-
104
-
105
- def _get_session(request: Request) -> Optional[str]:
106
- token = request.cookies.get(SESSION_COOKIE)
107
- return _verify_session(token) if token else None
108
-
109
-
110
- # ── 登录页 HTML ───────────────────────────────────────────────────────────────
111
- _LOGIN_HTML = """\
112
- <!DOCTYPE html>
113
- <html lang="en">
114
- <head>
115
- <meta charset="UTF-8">
116
- <meta name="viewport" content="width=device-width, initial-scale=1.0">
117
- <title>Sign In</title>
118
- <style>
119
- *, *::before, *::after {{ box-sizing: border-box; margin: 0; padding: 0; }}
120
- body {{
121
- min-height: 100vh;
122
- display: flex;
123
- align-items: center;
124
- justify-content: center;
125
- background: linear-gradient(135deg, #f0f2f5 0%, #e4e8f0 100%);
126
- font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
127
- }}
128
- .card {{
129
- background: #fff;
130
- border-radius: 14px;
131
- box-shadow: 0 6px 32px rgba(0, 0, 0, 0.10);
132
- padding: 44px 40px;
133
- width: 100%;
134
- max-width: 400px;
135
- }}
136
- h1 {{ font-size: 1.5rem; font-weight: 700; color: #111827; margin-bottom: 6px; }}
137
- p.sub {{ font-size: 0.875rem; color: #6b7280; margin-bottom: 30px; }}
138
- label {{ display: block; font-size: 0.8rem; font-weight: 600; color: #374151; margin-bottom: 6px; }}
139
- input[type=text], input[type=password] {{
140
- width: 100%;
141
- padding: 11px 14px;
142
- border: 1.5px solid #e5e7eb;
143
- border-radius: 8px;
144
- font-size: 0.95rem;
145
- outline: none;
146
- transition: border-color 0.15s;
147
- margin-bottom: 20px;
148
- color: #111827;
149
- }}
150
- input:focus {{ border-color: #4f6ef7; box-shadow: 0 0 0 3px rgba(79,110,247,0.12); }}
151
- button {{
152
- width: 100%;
153
- padding: 12px;
154
- background: linear-gradient(135deg, #4f6ef7 0%, #3b5bdb 100%);
155
- color: #fff;
156
- border: none;
157
- border-radius: 8px;
158
- font-size: 1rem;
159
- font-weight: 600;
160
- cursor: pointer;
161
- transition: opacity 0.15s;
162
- letter-spacing: 0.01em;
163
- }}
164
- button:hover {{ opacity: 0.88; }}
165
- .error {{
166
- background: #fef2f2;
167
- border: 1.5px solid #fecaca;
168
- border-radius: 8px;
169
- padding: 10px 14px;
170
- font-size: 0.875rem;
171
- color: #dc2626;
172
- margin-bottom: 20px;
173
- }}
174
- </style>
175
- </head>
176
- <body>
177
- <div class="card">
178
- <h1>Welcome back</h1>
179
- <p class="sub">Sign in to continue</p>
180
- {error_block}
181
- <form method="post" action="/login">
182
- <label for="u">Username</label>
183
- <input id="u" type="text" name="username" autocomplete="username" required autofocus>
184
- <label for="p">Password</label>
185
- <input id="p" type="password" name="password" autocomplete="current-password" required>
186
- <button type="submit">Sign in</button>
187
- </form>
188
- </div>
189
- </body>
190
- </html>
191
- """
192
-
193
-
194
- def _login_page(error: str = "") -> str:
195
- error_block = f'<div class="error">{error}</div>' if error else ""
196
- return _LOGIN_HTML.format(error_block=error_block)
197
-
198
-
199
- # ── FastAPI App ───────────────────────────────────────────────────────────────
200
- app = FastAPI(docs_url=None, redoc_url=None, openapi_url=None)
201
-
202
- _http_client: Optional[httpx.AsyncClient] = None
203
-
204
-
205
- @app.on_event("startup")
206
- async def _startup() -> None:
207
- global _http_client
208
- _http_client = httpx.AsyncClient(
209
- base_url=UPSTREAM_HTTP,
210
- follow_redirects=False,
211
- timeout=httpx.Timeout(60.0),
212
- )
213
- logger.info("Gateway started. Upstream: %s", UPSTREAM_HTTP)
214
-
215
-
216
- @app.on_event("shutdown")
217
- async def _shutdown() -> None:
218
- if _http_client:
219
- await _http_client.aclose()
220
-
221
-
222
- # ── 路由:无需认证 ─────────────────────────────────────────────────────────────
223
- @app.get("/healthz")
224
- async def healthz() -> Response:
225
- return Response("ok", media_type="text/plain")
226
-
227
-
228
- @app.get("/login", response_class=HTMLResponse)
229
- async def login_page(request: Request) -> HTMLResponse:
230
- if _get_session(request):
231
- return RedirectResponse("/", status_code=302)
232
- return HTMLResponse(_login_page())
233
-
234
-
235
- @app.post("/login")
236
- async def login(
237
- request: Request,
238
- username: str = Form(...),
239
- password: str = Form(...),
240
- ) -> Response:
241
- next_url = request.query_params.get("next", "/")
242
- if _verify_credentials(username, password):
243
- token = _make_session(username)
244
- resp = RedirectResponse(next_url, status_code=303)
245
- resp.set_cookie(
246
- SESSION_COOKIE,
247
- token,
248
- max_age=SESSION_MAX_AGE,
249
- httponly=True,
250
- samesite="lax",
251
- )
252
- logger.info("Login successful: %s", username)
253
- return resp
254
- logger.warning("Login failed: %s", username)
255
- return HTMLResponse(_login_page("Invalid username or password."), status_code=401)
256
-
257
-
258
- @app.get("/logout")
259
- async def logout() -> Response:
260
- resp = RedirectResponse("/login", status_code=302)
261
- resp.delete_cookie(SESSION_COOKIE)
262
- return resp
263
-
264
-
265
- # ── 路由:WebSocket 透传(Gradio 依赖) ────────────────────────────────────────
266
- @app.websocket("/{path:path}")
267
- async def ws_proxy(websocket: WebSocket, path: str) -> None:
268
- """WebSocket 透传:验证 session 后桥接到上游。"""
269
- token = websocket.cookies.get(SESSION_COOKIE)
270
- if not token or not _verify_session(token):
271
- await websocket.close(code=1008) # Policy Violation
272
- return
273
-
274
- await websocket.accept()
275
-
276
- qs = websocket.scope.get("query_string", b"").decode()
277
- upstream_url = f"{UPSTREAM_WS}/{path}"
278
- if qs:
279
- upstream_url += f"?{qs}"
280
-
281
- try:
282
- async with websockets.connect(upstream_url) as upstream:
283
-
284
- async def _client_to_upstream() -> None:
285
- try:
286
- while True:
287
- data = await websocket.receive()
288
- if data["type"] == "websocket.disconnect":
289
- break
290
- msg = data.get("bytes") or data.get("text")
291
- if msg is not None:
292
- await upstream.send(msg)
293
- except Exception:
294
- pass
295
- finally:
296
- await upstream.close()
297
-
298
- async def _upstream_to_client() -> None:
299
- try:
300
- async for msg in upstream:
301
- if isinstance(msg, bytes):
302
- await websocket.send_bytes(msg)
303
- else:
304
- await websocket.send_text(msg)
305
- except Exception:
306
- pass
307
-
308
- tasks = [
309
- asyncio.create_task(_client_to_upstream()),
310
- asyncio.create_task(_upstream_to_client()),
311
- ]
312
- _done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
313
- for t in pending:
314
- t.cancel()
315
-
316
- except websockets.exceptions.WebSocketException as exc:
317
- logger.debug("WS proxy closed: %s", exc)
318
- except Exception as exc:
319
- logger.debug("WS proxy error: %s", exc)
320
-
321
-
322
- # ── 路由:HTTP 反向代理 ────────────────────────────────────────────────────────
323
- @app.api_route(
324
- "/{path:path}",
325
- methods=["GET", "POST", "PUT", "DELETE", "PATCH", "HEAD", "OPTIONS"],
326
- )
327
- async def http_proxy(request: Request, path: str) -> Response:
328
- """HTTP 反向代理:需要 session,流式转发请求和响应。"""
329
- if not _get_session(request):
330
- if request.method == "GET":
331
- dest = f"/{path}" if path else "/"
332
- return RedirectResponse(f"/login?next={dest}", status_code=302)
333
- return Response("Unauthorized", status_code=401)
334
-
335
- # 构建上游请求头
336
- headers = {
337
- k: v
338
- for k, v in request.headers.items()
339
- if k.lower() not in HOP_BY_HOP and k.lower() != "host"
340
- }
341
-
342
- url = f"/{path}"
343
- if request.url.query:
344
- url += f"?{request.url.query}"
345
-
346
- # 读取请求体(对于大文件可换成流式,但原型够用)
347
- body = await request.body()
348
-
349
- upstream_req = _http_client.build_request(
350
- request.method,
351
- url,
352
- headers=headers,
353
- content=body,
354
- )
355
- upstream_resp = await _http_client.send(upstream_req, stream=True)
356
-
357
- resp_headers = {
358
- k: v
359
- for k, v in upstream_resp.headers.items()
360
- if k.lower() not in HOP_BY_HOP
361
- }
362
-
363
- return StreamingResponse(
364
- upstream_resp.aiter_bytes(),
365
- status_code=upstream_resp.status_code,
366
- headers=resp_headers,
367
- background=BackgroundTask(upstream_resp.aclose),
368
- )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
src/gateway.py ADDED
@@ -0,0 +1,1268 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """FastAPI 应用:登录鉴权、自研 GUI、翻译任务、内部 OpenAI 代理与计费。"""
3
+
4
+ from __future__ import annotations
5
+
6
+ import asyncio
7
+ import contextlib
8
+ import html
9
+ import json
10
+ import logging
11
+ import os
12
+ import secrets
13
+ import shutil
14
+ import sqlite3
15
+ import threading
16
+ import uuid
17
+ from datetime import datetime, timezone
18
+ from pathlib import Path
19
+ from typing import Any, Optional
20
+
21
+ import bcrypt
22
+ import httpx
23
+ from fastapi import Depends, FastAPI, File, Form, HTTPException, Request, UploadFile
24
+ from fastapi.responses import (
25
+ FileResponse,
26
+ HTMLResponse,
27
+ JSONResponse,
28
+ RedirectResponse,
29
+ Response,
30
+ )
31
+ from itsdangerous import BadSignature, SignatureExpired, TimestampSigner
32
+ from pdf2zh_next import BasicSettings
33
+ from pdf2zh_next import OpenAISettings
34
+ from pdf2zh_next import PDFSettings
35
+ from pdf2zh_next import SettingsModel
36
+ from pdf2zh_next import TranslationSettings
37
+ from pdf2zh_next.high_level import do_translate_async_stream
38
+
39
+ # ── 配置 ──────────────────────────────────────────────────────────────────────
40
+ SESSION_COOKIE = "gw_session"
41
+ SESSION_MAX_AGE = 86400 # 24 hours
42
+
43
+ SECRET_KEY = os.environ.get("SESSION_SECRET") or secrets.token_hex(32)
44
+ signer = TimestampSigner(SECRET_KEY)
45
+
46
+ DATA_DIR = Path(os.environ.get("DATA_DIR", "/data"))
47
+ UPLOAD_DIR = DATA_DIR / "uploads"
48
+ JOB_DIR = DATA_DIR / "jobs"
49
+ DB_PATH = DATA_DIR / "gateway.db"
50
+
51
+ INTERNAL_OPENAI_BASE_URL = os.environ.get(
52
+ "INTERNAL_OPENAI_BASE_URL", "http://127.0.0.1:7860/internal/openai/v1"
53
+ )
54
+ OPENAI_UPSTREAM_CHAT_URL = os.environ.get(
55
+ "OPENAI_UPSTREAM_CHAT_URL", "https://api.openai.com/v1/chat/completions"
56
+ )
57
+ OPENAI_REAL_API_KEY = os.environ.get("OPENAI_API_KEY", "").strip()
58
+
59
+ DEFAULT_MODEL = os.environ.get("DEFAULT_OPENAI_MODEL", "gpt-4o-mini").strip()
60
+ DEFAULT_LANG_IN = os.environ.get("DEFAULT_LANG_IN", "en").strip()
61
+ DEFAULT_LANG_OUT = os.environ.get("DEFAULT_LANG_OUT", "zh").strip()
62
+ TRANSLATION_QPS = int(os.environ.get("TRANSLATION_QPS", "4"))
63
+
64
+ INTERNAL_KEY_SALT = (os.environ.get("INTERNAL_KEY_SALT") or SECRET_KEY).strip()
65
+
66
+ # 价格单位:USD / 1M tokens
67
+ DEFAULT_INPUT_PRICE_PER_1M = float(
68
+ os.environ.get("OPENAI_DEFAULT_INPUT_PRICE_PER_1M", "0.15")
69
+ )
70
+ DEFAULT_OUTPUT_PRICE_PER_1M = float(
71
+ os.environ.get("OPENAI_DEFAULT_OUTPUT_PRICE_PER_1M", "0.60")
72
+ )
73
+ MODEL_PRICES_PER_1M: dict[str, tuple[float, float]] = {
74
+ "gpt-4o-mini": (0.15, 0.60),
75
+ "gpt-4.1-mini": (0.40, 1.60),
76
+ "gpt-4.1": (2.00, 8.00),
77
+ "gpt-4o": (2.50, 10.00),
78
+ }
79
+
80
+ LOCALHOSTS = frozenset({"127.0.0.1", "::1", "localhost"})
81
+
82
+ logging.basicConfig(
83
+ level=logging.INFO,
84
+ format="%(asctime)s %(levelname)s %(name)s - %(message)s",
85
+ )
86
+ logger = logging.getLogger("gateway")
87
+
88
+
89
+ # ── 用户加载与认证 ────────────────────────────────────────────────────────────
90
+ def _load_users() -> dict[str, str]:
91
+ """从 BASIC_AUTH_USERS 加载用户名密码。"""
92
+ raw = os.environ.get("BASIC_AUTH_USERS", "").replace("\\n", "\n")
93
+ users: dict[str, str] = {}
94
+ for line in raw.splitlines():
95
+ line = line.strip()
96
+ if not line or line.startswith("#"):
97
+ continue
98
+ if ":" not in line:
99
+ logger.warning("Skipping invalid BASIC_AUTH_USERS line (no colon)")
100
+ continue
101
+ username, password = line.split(":", 1)
102
+ username = username.strip()
103
+ password = password.strip()
104
+ if username and password:
105
+ users[username] = password
106
+ if not users:
107
+ logger.error("No valid users found — authentication will always fail")
108
+ return users
109
+
110
+
111
+ USERS = _load_users()
112
+ INTERNAL_KEY_NAMESPACE = uuid.uuid5(uuid.NAMESPACE_DNS, INTERNAL_KEY_SALT)
113
+
114
+
115
+ def _make_internal_api_key(username: str) -> str:
116
+ """基于用户名生成稳定内部 Key(仅服务端使用)。"""
117
+ value = uuid.uuid5(INTERNAL_KEY_NAMESPACE, username)
118
+ return f"sk-{value}"
119
+
120
+
121
+ INTERNAL_KEY_TO_USER = {
122
+ _make_internal_api_key(username): username for username in USERS.keys()
123
+ }
124
+
125
+
126
+ def _verify_credentials(username: str, password: str) -> bool:
127
+ """验证用户名密码,支持明文与 bcrypt。"""
128
+ stored = USERS.get(username)
129
+ if stored is None:
130
+ return False
131
+ if stored.startswith("$2"):
132
+ return bcrypt.checkpw(password.encode(), stored.encode())
133
+ return secrets.compare_digest(stored, password)
134
+
135
+
136
+ # ── Session ──────────────────────────────────────────────────────────────────
137
+ def _make_session(username: str) -> str:
138
+ return signer.sign(username).decode()
139
+
140
+
141
+ def _verify_session(token: str) -> Optional[str]:
142
+ try:
143
+ username = signer.unsign(token, max_age=SESSION_MAX_AGE).decode()
144
+ except (BadSignature, SignatureExpired):
145
+ return None
146
+ if username not in USERS:
147
+ return None
148
+ return username
149
+
150
+
151
+ def _get_session_user(request: Request) -> Optional[str]:
152
+ token = request.cookies.get(SESSION_COOKIE)
153
+ return _verify_session(token) if token else None
154
+
155
+
156
+ def _require_user(request: Request) -> str:
157
+ username = _get_session_user(request)
158
+ if not username:
159
+ raise HTTPException(status_code=401, detail="Unauthorized")
160
+ return username
161
+
162
+
163
+ # ── 存储层 ───────────────────────────────────────────────────────────────────
164
+ _db_lock = threading.Lock()
165
+ _db_conn: sqlite3.Connection | None = None
166
+
167
+
168
+ def _now_iso() -> str:
169
+ return datetime.now(timezone.utc).isoformat()
170
+
171
+
172
+ def _ensure_data_dirs() -> None:
173
+ UPLOAD_DIR.mkdir(parents=True, exist_ok=True)
174
+ JOB_DIR.mkdir(parents=True, exist_ok=True)
175
+
176
+
177
+ def _init_db() -> None:
178
+ global _db_conn
179
+ _ensure_data_dirs()
180
+ conn = sqlite3.connect(DB_PATH, check_same_thread=False)
181
+ conn.row_factory = sqlite3.Row
182
+ with conn:
183
+ conn.execute(
184
+ """
185
+ CREATE TABLE IF NOT EXISTS jobs (
186
+ id TEXT PRIMARY KEY,
187
+ username TEXT NOT NULL,
188
+ filename TEXT NOT NULL,
189
+ input_path TEXT NOT NULL,
190
+ output_dir TEXT NOT NULL,
191
+ status TEXT NOT NULL,
192
+ progress REAL NOT NULL DEFAULT 0,
193
+ message TEXT,
194
+ error TEXT,
195
+ model TEXT NOT NULL,
196
+ lang_in TEXT NOT NULL,
197
+ lang_out TEXT NOT NULL,
198
+ cancel_requested INTEGER NOT NULL DEFAULT 0,
199
+ mono_pdf_path TEXT,
200
+ dual_pdf_path TEXT,
201
+ glossary_path TEXT,
202
+ created_at TEXT NOT NULL,
203
+ updated_at TEXT NOT NULL,
204
+ started_at TEXT,
205
+ finished_at TEXT
206
+ )
207
+ """
208
+ )
209
+ conn.execute(
210
+ """
211
+ CREATE TABLE IF NOT EXISTS usage_records (
212
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
213
+ username TEXT NOT NULL,
214
+ job_id TEXT,
215
+ model TEXT NOT NULL,
216
+ prompt_tokens INTEGER NOT NULL,
217
+ completion_tokens INTEGER NOT NULL,
218
+ total_tokens INTEGER NOT NULL,
219
+ cost_usd REAL NOT NULL,
220
+ created_at TEXT NOT NULL
221
+ )
222
+ """
223
+ )
224
+ conn.execute(
225
+ """
226
+ CREATE INDEX IF NOT EXISTS idx_jobs_user_time
227
+ ON jobs(username, created_at DESC)
228
+ """
229
+ )
230
+ conn.execute(
231
+ """
232
+ CREATE INDEX IF NOT EXISTS idx_usage_user_time
233
+ ON usage_records(username, created_at DESC)
234
+ """
235
+ )
236
+ _db_conn = conn
237
+
238
+
239
+ def _db_execute(sql: str, params: tuple[Any, ...] = ()) -> None:
240
+ if _db_conn is None:
241
+ raise RuntimeError("DB is not initialized")
242
+ with _db_lock, _db_conn:
243
+ _db_conn.execute(sql, params)
244
+
245
+
246
+ def _db_fetchone(sql: str, params: tuple[Any, ...] = ()) -> sqlite3.Row | None:
247
+ if _db_conn is None:
248
+ raise RuntimeError("DB is not initialized")
249
+ with _db_lock:
250
+ return _db_conn.execute(sql, params).fetchone()
251
+
252
+
253
+ def _db_fetchall(sql: str, params: tuple[Any, ...] = ()) -> list[sqlite3.Row]:
254
+ if _db_conn is None:
255
+ raise RuntimeError("DB is not initialized")
256
+ with _db_lock:
257
+ return _db_conn.execute(sql, params).fetchall()
258
+
259
+
260
+ def _update_job(job_id: str, **fields: Any) -> None:
261
+ if not fields:
262
+ return
263
+ fields["updated_at"] = _now_iso()
264
+ set_clause = ", ".join(f"{k} = ?" for k in fields.keys())
265
+ params = tuple(fields.values()) + (job_id,)
266
+ _db_execute(f"UPDATE jobs SET {set_clause} WHERE id = ?", params)
267
+
268
+
269
+ def _row_to_job_dict(row: sqlite3.Row) -> dict[str, Any]:
270
+ job = dict(row)
271
+ job["artifact_urls"] = {
272
+ "mono": f"/api/jobs/{job['id']}/artifacts/mono"
273
+ if job.get("mono_pdf_path")
274
+ else None,
275
+ "dual": f"/api/jobs/{job['id']}/artifacts/dual"
276
+ if job.get("dual_pdf_path")
277
+ else None,
278
+ "glossary": f"/api/jobs/{job['id']}/artifacts/glossary"
279
+ if job.get("glossary_path")
280
+ else None,
281
+ }
282
+ return job
283
+
284
+
285
+ # ── 任务执行 ───────────────────────────────────────────────────────────────────
286
+ _job_queue: asyncio.Queue[str] = asyncio.Queue()
287
+ _worker_task: asyncio.Task[None] | None = None
288
+ _running_tasks: dict[str, asyncio.Task[None]] = {}
289
+ _active_job_by_user: dict[str, str] = {}
290
+
291
+
292
+ def _calc_cost_usd(model: str, prompt_tokens: int, completion_tokens: int) -> float:
293
+ model_rates = MODEL_PRICES_PER_1M.get(model, None)
294
+ if model_rates is None:
295
+ in_rate = DEFAULT_INPUT_PRICE_PER_1M
296
+ out_rate = DEFAULT_OUTPUT_PRICE_PER_1M
297
+ else:
298
+ in_rate, out_rate = model_rates
299
+
300
+ cost = (prompt_tokens * in_rate + completion_tokens * out_rate) / 1_000_000.0
301
+ return round(cost, 8)
302
+
303
+
304
+ def _record_usage(
305
+ *,
306
+ username: str,
307
+ job_id: str | None,
308
+ model: str,
309
+ prompt_tokens: int,
310
+ completion_tokens: int,
311
+ total_tokens: int,
312
+ ) -> None:
313
+ cost_usd = _calc_cost_usd(model, prompt_tokens, completion_tokens)
314
+ _db_execute(
315
+ """
316
+ INSERT INTO usage_records(
317
+ username, job_id, model,
318
+ prompt_tokens, completion_tokens, total_tokens,
319
+ cost_usd, created_at
320
+ ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
321
+ """,
322
+ (
323
+ username,
324
+ job_id,
325
+ model,
326
+ prompt_tokens,
327
+ completion_tokens,
328
+ total_tokens,
329
+ cost_usd,
330
+ _now_iso(),
331
+ ),
332
+ )
333
+
334
+
335
+ def _build_settings_for_job(row: sqlite3.Row) -> SettingsModel:
336
+ username = row["username"]
337
+ internal_key = _make_internal_api_key(username)
338
+
339
+ settings = SettingsModel(
340
+ basic=BasicSettings(debug=False, gui=False),
341
+ translation=TranslationSettings(
342
+ lang_in=row["lang_in"],
343
+ lang_out=row["lang_out"],
344
+ output=row["output_dir"],
345
+ qps=TRANSLATION_QPS,
346
+ ),
347
+ pdf=PDFSettings(),
348
+ translate_engine_settings=OpenAISettings(
349
+ openai_model=row["model"],
350
+ openai_base_url=INTERNAL_OPENAI_BASE_URL,
351
+ openai_api_key=internal_key,
352
+ ),
353
+ )
354
+ settings.validate_settings()
355
+ return settings
356
+
357
+
358
+ async def _run_single_job(job_id: str) -> None:
359
+ row = _db_fetchone("SELECT * FROM jobs WHERE id = ?", (job_id,))
360
+ if row is None:
361
+ return
362
+ if row["status"] != "queued":
363
+ return
364
+ if row["cancel_requested"]:
365
+ _update_job(job_id, status="cancelled", message="Cancelled before start")
366
+ return
367
+
368
+ username = row["username"]
369
+ _update_job(
370
+ job_id,
371
+ status="running",
372
+ started_at=_now_iso(),
373
+ message="Translation started",
374
+ progress=0.0,
375
+ )
376
+ _active_job_by_user[username] = job_id
377
+
378
+ input_path = Path(row["input_path"])
379
+ output_dir = Path(row["output_dir"])
380
+
381
+ try:
382
+ settings = _build_settings_for_job(row)
383
+ async for event in do_translate_async_stream(settings, input_path):
384
+ event_type = event.get("type")
385
+ if event_type in {"progress_start", "progress_update", "progress_end"}:
386
+ progress = float(event.get("overall_progress", 0.0))
387
+ stage = event.get("stage", "")
388
+ _update_job(
389
+ job_id,
390
+ progress=max(0.0, min(100.0, progress)),
391
+ message=f"{stage}" if stage else "Running",
392
+ )
393
+ elif event_type == "error":
394
+ error_msg = str(event.get("error", "Unknown translation error"))
395
+ _update_job(
396
+ job_id,
397
+ status="failed",
398
+ error=error_msg,
399
+ message="Translation failed",
400
+ finished_at=_now_iso(),
401
+ )
402
+ return
403
+ elif event_type == "finish":
404
+ result = event.get("translate_result")
405
+ mono_path = str(getattr(result, "mono_pdf_path", "") or "")
406
+ dual_path = str(getattr(result, "dual_pdf_path", "") or "")
407
+ glossary_path = str(
408
+ getattr(result, "auto_extracted_glossary_path", "") or ""
409
+ )
410
+
411
+ # 兜底:如果路径为空,尝试在输出目录中扫描常见文件
412
+ if not mono_path or not dual_path:
413
+ files = list(output_dir.glob("*.pdf"))
414
+ for file in files:
415
+ name = file.name.lower()
416
+ if ".mono.pdf" in name and not mono_path:
417
+ mono_path = str(file)
418
+ elif ".dual.pdf" in name and not dual_path:
419
+ dual_path = str(file)
420
+
421
+ _update_job(
422
+ job_id,
423
+ status="succeeded",
424
+ progress=100.0,
425
+ message="Translation finished",
426
+ finished_at=_now_iso(),
427
+ mono_pdf_path=mono_path or None,
428
+ dual_pdf_path=dual_path or None,
429
+ glossary_path=glossary_path or None,
430
+ )
431
+ return
432
+
433
+ _update_job(
434
+ job_id,
435
+ status="failed",
436
+ error="Translation stream ended unexpectedly",
437
+ message="Translation failed",
438
+ finished_at=_now_iso(),
439
+ )
440
+ except asyncio.CancelledError:
441
+ _update_job(
442
+ job_id,
443
+ status="cancelled",
444
+ message="Cancelled by user",
445
+ finished_at=_now_iso(),
446
+ )
447
+ raise
448
+ except Exception as exc: # noqa: BLE001
449
+ logger.exception("Translation job failed: %s", job_id)
450
+ _update_job(
451
+ job_id,
452
+ status="failed",
453
+ error=str(exc),
454
+ message="Translation failed",
455
+ finished_at=_now_iso(),
456
+ )
457
+ finally:
458
+ if _active_job_by_user.get(username) == job_id:
459
+ _active_job_by_user.pop(username, None)
460
+
461
+
462
+ async def _job_worker() -> None:
463
+ logger.info("Job worker started")
464
+ while True:
465
+ job_id = await _job_queue.get()
466
+ try:
467
+ task = asyncio.create_task(_run_single_job(job_id), name=f"job-{job_id}")
468
+ _running_tasks[job_id] = task
469
+ await task
470
+ except asyncio.CancelledError:
471
+ raise
472
+ except Exception: # noqa: BLE001
473
+ logger.exception("Unhandled worker error for job=%s", job_id)
474
+ finally:
475
+ _running_tasks.pop(job_id, None)
476
+ _job_queue.task_done()
477
+
478
+
479
+ def _enqueue_pending_jobs() -> None:
480
+ # 服务重启后,正在运行中的任务标记失败。
481
+ restart_time = _now_iso()
482
+ _db_execute(
483
+ """
484
+ UPDATE jobs
485
+ SET status='failed',
486
+ error='Service restarted while running',
487
+ message='Failed due to restart',
488
+ finished_at=?,
489
+ updated_at=?
490
+ WHERE status='running'
491
+ """,
492
+ (restart_time, restart_time),
493
+ )
494
+
495
+ rows = _db_fetchall(
496
+ "SELECT id FROM jobs WHERE status='queued' ORDER BY created_at ASC"
497
+ )
498
+ for row in rows:
499
+ _job_queue.put_nowait(row["id"])
500
+
501
+
502
+ # ── 页面模板 ───────────────────────────────────────────────────────────────────
503
+ _LOGIN_HTML = """\
504
+ <!DOCTYPE html>
505
+ <html lang="en">
506
+ <head>
507
+ <meta charset="UTF-8">
508
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
509
+ <title>Sign In</title>
510
+ <style>
511
+ *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
512
+ body {
513
+ min-height: 100vh;
514
+ display: flex;
515
+ align-items: center;
516
+ justify-content: center;
517
+ background: linear-gradient(135deg, #f0f2f5 0%, #e4e8f0 100%);
518
+ font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
519
+ }
520
+ .card {
521
+ background: #fff;
522
+ border-radius: 14px;
523
+ box-shadow: 0 6px 32px rgba(0, 0, 0, 0.10);
524
+ padding: 44px 40px;
525
+ width: 100%;
526
+ max-width: 400px;
527
+ }
528
+ h1 { font-size: 1.5rem; font-weight: 700; color: #111827; margin-bottom: 6px; }
529
+ p.sub { font-size: 0.875rem; color: #6b7280; margin-bottom: 30px; }
530
+ label { display: block; font-size: 0.8rem; font-weight: 600; color: #374151; margin-bottom: 6px; }
531
+ input[type=text], input[type=password] {
532
+ width: 100%;
533
+ padding: 11px 14px;
534
+ border: 1.5px solid #e5e7eb;
535
+ border-radius: 8px;
536
+ font-size: 0.95rem;
537
+ outline: none;
538
+ transition: border-color 0.15s;
539
+ margin-bottom: 20px;
540
+ color: #111827;
541
+ }
542
+ input:focus { border-color: #4f6ef7; box-shadow: 0 0 0 3px rgba(79,110,247,0.12); }
543
+ button {
544
+ width: 100%;
545
+ padding: 12px;
546
+ background: linear-gradient(135deg, #4f6ef7 0%, #3b5bdb 100%);
547
+ color: #fff;
548
+ border: none;
549
+ border-radius: 8px;
550
+ font-size: 1rem;
551
+ font-weight: 600;
552
+ cursor: pointer;
553
+ transition: opacity 0.15s;
554
+ }
555
+ button:hover { opacity: 0.88; }
556
+ .error {
557
+ background: #fef2f2;
558
+ border: 1.5px solid #fecaca;
559
+ border-radius: 8px;
560
+ padding: 10px 14px;
561
+ font-size: 0.875rem;
562
+ color: #dc2626;
563
+ margin-bottom: 20px;
564
+ }
565
+ </style>
566
+ </head>
567
+ <body>
568
+ <div class="card">
569
+ <h1>Welcome back</h1>
570
+ <p class="sub">Sign in to continue</p>
571
+ __ERROR_BLOCK__
572
+ <form method="post" action="/login">
573
+ <label for="u">Username</label>
574
+ <input id="u" type="text" name="username" autocomplete="username" required autofocus>
575
+ <label for="p">Password</label>
576
+ <input id="p" type="password" name="password" autocomplete="current-password" required>
577
+ <button type="submit">Sign in</button>
578
+ </form>
579
+ </div>
580
+ </body>
581
+ </html>
582
+ """
583
+
584
+
585
+ def _login_page(error: str = "") -> str:
586
+ error_block = f'<div class="error">{html.escape(error)}</div>' if error else ""
587
+ return _LOGIN_HTML.replace("__ERROR_BLOCK__", error_block)
588
+
589
+
590
+ def _dashboard_page(username: str) -> str:
591
+ safe_user = html.escape(username)
592
+ safe_model = html.escape(DEFAULT_MODEL)
593
+ safe_lang_in = html.escape(DEFAULT_LANG_IN)
594
+ safe_lang_out = html.escape(DEFAULT_LANG_OUT)
595
+
596
+ return f"""<!DOCTYPE html>
597
+ <html lang="en">
598
+ <head>
599
+ <meta charset="UTF-8" />
600
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
601
+ <title>PDF Translation Console</title>
602
+ <style>
603
+ :root {{
604
+ --bg: #f4f7fb;
605
+ --card: #ffffff;
606
+ --ink: #0f172a;
607
+ --sub: #475569;
608
+ --line: #dbe3ee;
609
+ --brand: #0f766e;
610
+ --brand-dark: #115e59;
611
+ --danger: #b91c1c;
612
+ }}
613
+ * {{ box-sizing: border-box; }}
614
+ body {{
615
+ margin: 0;
616
+ color: var(--ink);
617
+ background: radial-gradient(circle at 15% -20%, #d5f3ef 0, #f4f7fb 52%);
618
+ font-family: "Segoe UI", Tahoma, Geneva, Verdana, sans-serif;
619
+ }}
620
+ .wrap {{ max-width: 1100px; margin: 24px auto; padding: 0 16px 40px; }}
621
+ .top {{
622
+ display: flex;
623
+ align-items: center;
624
+ justify-content: space-between;
625
+ margin-bottom: 16px;
626
+ }}
627
+ h1 {{ margin: 0; font-size: 1.5rem; }}
628
+ .user {{ color: var(--sub); font-size: 0.95rem; }}
629
+ .grid {{ display: grid; grid-template-columns: 1fr 1fr; gap: 14px; }}
630
+ .card {{
631
+ background: var(--card);
632
+ border: 1px solid var(--line);
633
+ border-radius: 14px;
634
+ box-shadow: 0 10px 28px rgba(17, 24, 39, 0.06);
635
+ padding: 16px;
636
+ }}
637
+ .card h2 {{ margin: 0 0 10px; font-size: 1.03rem; }}
638
+ .row {{ display: grid; grid-template-columns: 1fr 1fr; gap: 10px; }}
639
+ label {{ display: block; margin: 10px 0 6px; font-size: 0.86rem; color: var(--sub); }}
640
+ input[type=text], select, input[type=file] {{
641
+ width: 100%; padding: 10px 12px; border-radius: 8px;
642
+ border: 1px solid var(--line); background: #fff; color: var(--ink);
643
+ }}
644
+ button {{
645
+ border: none; border-radius: 9px; padding: 10px 14px;
646
+ font-weight: 600; cursor: pointer;
647
+ }}
648
+ .primary {{ background: var(--brand); color: #fff; }}
649
+ .primary:hover {{ background: var(--brand-dark); }}
650
+ .muted {{ background: #e2e8f0; color: #0f172a; }}
651
+ .danger {{ background: #fee2e2; color: var(--danger); }}
652
+ .hint {{ margin-top: 8px; color: var(--sub); font-size: 0.84rem; }}
653
+ .status {{ margin-top: 10px; min-height: 22px; font-size: 0.9rem; }}
654
+ table {{ width: 100%; border-collapse: collapse; margin-top: 8px; font-size: 0.88rem; }}
655
+ th, td {{ border-bottom: 1px solid var(--line); text-align: left; padding: 8px 6px; }}
656
+ th {{ color: var(--sub); font-weight: 600; }}
657
+ .mono {{ font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; font-size: 0.8rem; }}
658
+ .actions button {{ margin-right: 6px; margin-bottom: 4px; }}
659
+ .foot {{ margin-top: 20px; color: var(--sub); font-size: 0.82rem; }}
660
+ @media (max-width: 900px) {{
661
+ .grid {{ grid-template-columns: 1fr; }}
662
+ .row {{ grid-template-columns: 1fr; }}
663
+ }}
664
+ </style>
665
+ </head>
666
+ <body>
667
+ <div class="wrap">
668
+ <div class="top">
669
+ <div>
670
+ <h1>PDF Translation Console</h1>
671
+ <div class="user">Signed in as <strong>{safe_user}</strong></div>
672
+ </div>
673
+ <div><a href="/logout"><button class="muted">Sign out</button></a></div>
674
+ </div>
675
+
676
+ <div class="grid">
677
+ <section class="card">
678
+ <h2>New Job</h2>
679
+ <form id="jobForm">
680
+ <label>PDF File</label>
681
+ <input name="file" type="file" accept=".pdf" required />
682
+
683
+ <div class="row">
684
+ <div>
685
+ <label>Source Language</label>
686
+ <input name="lang_in" type="text" value="{safe_lang_in}" required />
687
+ </div>
688
+ <div>
689
+ <label>Target Language</label>
690
+ <input name="lang_out" type="text" value="{safe_lang_out}" required />
691
+ </div>
692
+ </div>
693
+
694
+ <label>OpenAI Model</label>
695
+ <input name="model" type="text" value="{safe_model}" required />
696
+
697
+ <div style="margin-top: 12px;">
698
+ <button class="primary" type="submit">Submit Job</button>
699
+ </div>
700
+ </form>
701
+ <div class="hint">Billing is based on your login user and OpenAI token usage.</div>
702
+ <div id="jobStatus" class="status"></div>
703
+ </section>
704
+
705
+ <section class="card">
706
+ <h2>My Billing</h2>
707
+ <div id="billingSummary" class="mono">Loading...</div>
708
+ <table>
709
+ <thead>
710
+ <tr>
711
+ <th>Time (UTC)</th>
712
+ <th>Model</th>
713
+ <th>Prompt</th>
714
+ <th>Completion</th>
715
+ <th>Total</th>
716
+ <th>Cost (USD)</th>
717
+ </tr>
718
+ </thead>
719
+ <tbody id="billingBody"></tbody>
720
+ </table>
721
+ </section>
722
+ </div>
723
+
724
+ <section class="card" style="margin-top: 14px;">
725
+ <h2>My Jobs</h2>
726
+ <table>
727
+ <thead>
728
+ <tr>
729
+ <th>ID</th>
730
+ <th>File</th>
731
+ <th>Status</th>
732
+ <th>Progress</th>
733
+ <th>Model</th>
734
+ <th>Updated (UTC)</th>
735
+ <th>Actions</th>
736
+ </tr>
737
+ </thead>
738
+ <tbody id="jobsBody"></tbody>
739
+ </table>
740
+ </section>
741
+
742
+ <div class="foot">Internal OpenAI endpoint is localhost-only and not exposed to end users.</div>
743
+ </div>
744
+
745
+ <script>
746
+ async function apiJson(url, options = undefined) {{
747
+ const resp = await fetch(url, options);
748
+ if (!resp.ok) {{
749
+ const data = await resp.text();
750
+ throw new Error(data || `HTTP ${{resp.status}}`);
751
+ }}
752
+ return resp.json();
753
+ }}
754
+
755
+ function esc(s) {{
756
+ return String(s || "").replace(/[&<>"']/g, (c) => ({{
757
+ '&': '&amp;',
758
+ '<': '&lt;',
759
+ '>': '&gt;',
760
+ '"': '&quot;',
761
+ "'": '&#39;'
762
+ }})[c]);
763
+ }}
764
+
765
+ async function refreshBilling() {{
766
+ const summary = await apiJson('/api/billing/me');
767
+ const rows = await apiJson('/api/billing/me/records?limit=20');
768
+
769
+ document.getElementById('billingSummary').textContent =
770
+ `total_tokens=${{summary.total_tokens}} | total_cost_usd=${{Number(summary.total_cost_usd).toFixed(6)}}`;
771
+
772
+ const body = document.getElementById('billingBody');
773
+ body.innerHTML = '';
774
+ for (const r of rows.records) {{
775
+ const tr = document.createElement('tr');
776
+ tr.innerHTML = `
777
+ <td>${{esc(r.created_at)}}</td>
778
+ <td class="mono">${{esc(r.model)}}</td>
779
+ <td>${{r.prompt_tokens}}</td>
780
+ <td>${{r.completion_tokens}}</td>
781
+ <td>${{r.total_tokens}}</td>
782
+ <td>${{Number(r.cost_usd).toFixed(6)}}</td>
783
+ `;
784
+ body.appendChild(tr);
785
+ }}
786
+ }}
787
+
788
+ function actionButtons(job) {{
789
+ const actions = [];
790
+ if (job.status === 'queued' || job.status === 'running') {{
791
+ actions.push(`<button class="danger" onclick="cancelJob('${{job.id}}')">Cancel</button>`);
792
+ }}
793
+ if (job.artifact_urls?.mono) {{
794
+ actions.push(`<a href="${{job.artifact_urls.mono}}"><button class="muted">Mono</button></a>`);
795
+ }}
796
+ if (job.artifact_urls?.dual) {{
797
+ actions.push(`<a href="${{job.artifact_urls.dual}}"><button class="muted">Dual</button></a>`);
798
+ }}
799
+ if (job.artifact_urls?.glossary) {{
800
+ actions.push(`<a href="${{job.artifact_urls.glossary}}"><button class="muted">Glossary</button></a>`);
801
+ }}
802
+ return actions.join(' ');
803
+ }}
804
+
805
+ async function refreshJobs() {{
806
+ const data = await apiJson('/api/jobs?limit=50');
807
+ const body = document.getElementById('jobsBody');
808
+ body.innerHTML = '';
809
+
810
+ for (const job of data.jobs) {{
811
+ const tr = document.createElement('tr');
812
+ tr.innerHTML = `
813
+ <td class="mono">${{esc(job.id)}}</td>
814
+ <td>${{esc(job.filename)}}</td>
815
+ <td>${{esc(job.status)}}${{job.error ? ' / ' + esc(job.error) : ''}}</td>
816
+ <td>${{Number(job.progress).toFixed(1)}}%</td>
817
+ <td class="mono">${{esc(job.model)}}</td>
818
+ <td class="mono">${{esc(job.updated_at)}}</td>
819
+ <td class="actions">${{actionButtons(job)}}</td>
820
+ `;
821
+ body.appendChild(tr);
822
+ }}
823
+ }}
824
+
825
+ async function cancelJob(jobId) {{
826
+ try {{
827
+ await apiJson(`/api/jobs/${{jobId}}/cancel`, {{ method: 'POST' }});
828
+ await refreshJobs();
829
+ }} catch (err) {{
830
+ alert(`Cancel failed: ${{err.message}}`);
831
+ }}
832
+ }}
833
+
834
+ document.getElementById('jobForm').addEventListener('submit', async (event) => {{
835
+ event.preventDefault();
836
+ const status = document.getElementById('jobStatus');
837
+ status.textContent = 'Submitting...';
838
+
839
+ const formData = new FormData(event.target);
840
+ try {{
841
+ const created = await apiJson('/api/jobs', {{ method: 'POST', body: formData }});
842
+ status.textContent = `Job queued: ${{created.job.id}}`;
843
+ event.target.reset();
844
+ await refreshJobs();
845
+ }} catch (err) {{
846
+ status.textContent = `Submit failed: ${{err.message}}`;
847
+ }}
848
+ }});
849
+
850
+ async function refreshAll() {{
851
+ await Promise.all([refreshJobs(), refreshBilling()]);
852
+ }}
853
+
854
+ refreshAll();
855
+ setInterval(refreshAll, 3000);
856
+ </script>
857
+ </body>
858
+ </html>
859
+ """
860
+
861
+
862
+ # ── FastAPI App ───────────────────────────────────────────────────────────────
863
+ app = FastAPI(docs_url=None, redoc_url=None, openapi_url=None)
864
+ _http_client: httpx.AsyncClient | None = None
865
+
866
+
867
+ @app.on_event("startup")
868
+ async def _startup() -> None:
869
+ global _http_client, _worker_task
870
+
871
+ _init_db()
872
+ _enqueue_pending_jobs()
873
+
874
+ _http_client = httpx.AsyncClient(timeout=httpx.Timeout(180.0))
875
+ _worker_task = asyncio.create_task(_job_worker(), name="job-worker")
876
+
877
+ if not OPENAI_REAL_API_KEY:
878
+ logger.warning("OPENAI_API_KEY is empty, translation jobs will fail")
879
+
880
+ logger.info("Gateway started. Data dir: %s", DATA_DIR)
881
+
882
+
883
+ @app.on_event("shutdown")
884
+ async def _shutdown() -> None:
885
+ global _worker_task, _http_client
886
+
887
+ if _worker_task:
888
+ _worker_task.cancel()
889
+ with contextlib.suppress(asyncio.CancelledError):
890
+ await _worker_task
891
+ _worker_task = None
892
+
893
+ for task in list(_running_tasks.values()):
894
+ task.cancel()
895
+
896
+ if _http_client:
897
+ await _http_client.aclose()
898
+ _http_client = None
899
+
900
+ if _db_conn:
901
+ _db_conn.close()
902
+
903
+
904
+ # ── 路由:基础与认证 ───────────────────────────────────────────────────────────
905
+ @app.get("/healthz")
906
+ async def healthz() -> Response:
907
+ return Response("ok", media_type="text/plain")
908
+
909
+
910
+ @app.get("/login", response_class=HTMLResponse)
911
+ async def login_page(request: Request) -> HTMLResponse:
912
+ if _get_session_user(request):
913
+ return RedirectResponse("/", status_code=302)
914
+ return HTMLResponse(_login_page())
915
+
916
+
917
+ @app.post("/login")
918
+ async def login(
919
+ request: Request,
920
+ username: str = Form(...),
921
+ password: str = Form(...),
922
+ ) -> Response:
923
+ next_url = request.query_params.get("next", "/")
924
+ if _verify_credentials(username, password):
925
+ token = _make_session(username)
926
+ resp = RedirectResponse(next_url, status_code=303)
927
+ resp.set_cookie(
928
+ SESSION_COOKIE,
929
+ token,
930
+ max_age=SESSION_MAX_AGE,
931
+ httponly=True,
932
+ samesite="lax",
933
+ )
934
+ logger.info("Login successful: %s", username)
935
+ return resp
936
+
937
+ logger.warning("Login failed: %s", username)
938
+ return HTMLResponse(_login_page("Invalid username or password."), status_code=401)
939
+
940
+
941
+ @app.get("/logout")
942
+ async def logout() -> Response:
943
+ resp = RedirectResponse("/login", status_code=302)
944
+ resp.delete_cookie(SESSION_COOKIE)
945
+ return resp
946
+
947
+
948
+ @app.get("/", response_class=HTMLResponse)
949
+ async def index(request: Request) -> Response:
950
+ username = _get_session_user(request)
951
+ if not username:
952
+ return RedirectResponse("/login", status_code=302)
953
+ return HTMLResponse(_dashboard_page(username))
954
+
955
+
956
+ # ── 路由:任务 API ─────────────────────────────────────────────────────────────
957
+ @app.get("/api/me")
958
+ async def api_me(username: str = Depends(_require_user)) -> dict[str, str]:
959
+ return {"username": username}
960
+
961
+
962
+ @app.get("/api/jobs")
963
+ async def api_list_jobs(
964
+ limit: int = 50,
965
+ username: str = Depends(_require_user),
966
+ ) -> dict[str, Any]:
967
+ limit = max(1, min(limit, 200))
968
+ rows = _db_fetchall(
969
+ """
970
+ SELECT * FROM jobs
971
+ WHERE username = ?
972
+ ORDER BY created_at DESC
973
+ LIMIT ?
974
+ """,
975
+ (username, limit),
976
+ )
977
+ return {"jobs": [_row_to_job_dict(row) for row in rows]}
978
+
979
+
980
+ @app.get("/api/jobs/{job_id}")
981
+ async def api_get_job(job_id: str, username: str = Depends(_require_user)) -> dict[str, Any]:
982
+ row = _db_fetchone(
983
+ "SELECT * FROM jobs WHERE id = ? AND username = ?",
984
+ (job_id, username),
985
+ )
986
+ if row is None:
987
+ raise HTTPException(status_code=404, detail="Job not found")
988
+ return {"job": _row_to_job_dict(row)}
989
+
990
+
991
+ @app.post("/api/jobs")
992
+ async def api_create_job(
993
+ file: UploadFile = File(...),
994
+ lang_in: str = Form(DEFAULT_LANG_IN),
995
+ lang_out: str = Form(DEFAULT_LANG_OUT),
996
+ model: str = Form(DEFAULT_MODEL),
997
+ username: str = Depends(_require_user),
998
+ ) -> dict[str, Any]:
999
+ filename = file.filename or "input.pdf"
1000
+ if not filename.lower().endswith(".pdf"):
1001
+ raise HTTPException(status_code=400, detail="Only PDF file is allowed")
1002
+
1003
+ if not model.strip():
1004
+ raise HTTPException(status_code=400, detail="Model is required")
1005
+
1006
+ job_id = uuid.uuid4().hex
1007
+ safe_filename = Path(filename).name
1008
+ input_path = (UPLOAD_DIR / f"{job_id}.pdf").resolve()
1009
+ output_dir = (JOB_DIR / job_id).resolve()
1010
+ output_dir.mkdir(parents=True, exist_ok=True)
1011
+
1012
+ try:
1013
+ with input_path.open("wb") as f:
1014
+ shutil.copyfileobj(file.file, f)
1015
+ finally:
1016
+ await file.close()
1017
+
1018
+ now = _now_iso()
1019
+ _db_execute(
1020
+ """
1021
+ INSERT INTO jobs(
1022
+ id, username, filename, input_path, output_dir,
1023
+ status, progress, message, error,
1024
+ model, lang_in, lang_out,
1025
+ cancel_requested,
1026
+ created_at, updated_at
1027
+ ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
1028
+ """,
1029
+ (
1030
+ job_id,
1031
+ username,
1032
+ safe_filename,
1033
+ str(input_path),
1034
+ str(output_dir),
1035
+ "queued",
1036
+ 0.0,
1037
+ "Queued",
1038
+ None,
1039
+ model.strip(),
1040
+ lang_in.strip() or DEFAULT_LANG_IN,
1041
+ lang_out.strip() or DEFAULT_LANG_OUT,
1042
+ 0,
1043
+ now,
1044
+ now,
1045
+ ),
1046
+ )
1047
+
1048
+ await _job_queue.put(job_id)
1049
+ row = _db_fetchone("SELECT * FROM jobs WHERE id = ?", (job_id,))
1050
+ return {"job": _row_to_job_dict(row)}
1051
+
1052
+
1053
+ @app.post("/api/jobs/{job_id}/cancel")
1054
+ async def api_cancel_job(
1055
+ job_id: str,
1056
+ username: str = Depends(_require_user),
1057
+ ) -> dict[str, Any]:
1058
+ row = _db_fetchone(
1059
+ "SELECT * FROM jobs WHERE id = ? AND username = ?",
1060
+ (job_id, username),
1061
+ )
1062
+ if row is None:
1063
+ raise HTTPException(status_code=404, detail="Job not found")
1064
+
1065
+ status = row["status"]
1066
+ if status in {"succeeded", "failed", "cancelled"}:
1067
+ return {"status": status, "message": "Job already finished"}
1068
+
1069
+ _update_job(job_id, cancel_requested=1, message="Cancel requested")
1070
+ if status == "queued":
1071
+ _update_job(job_id, status="cancelled", finished_at=_now_iso(), progress=0.0)
1072
+ return {"status": "cancelled", "message": "Job cancelled"}
1073
+
1074
+ task = _running_tasks.get(job_id)
1075
+ if task:
1076
+ task.cancel()
1077
+
1078
+ return {"status": "cancelling", "message": "Cancellation requested"}
1079
+
1080
+
1081
+ def _resolve_artifact_path(raw_path: str | None, output_dir: Path) -> Path | None:
1082
+ if not raw_path:
1083
+ return None
1084
+ path = Path(raw_path)
1085
+ if not path.is_absolute():
1086
+ path = (output_dir / path).resolve()
1087
+ else:
1088
+ path = path.resolve()
1089
+
1090
+ if not path.exists():
1091
+ return None
1092
+
1093
+ try:
1094
+ path.relative_to(output_dir)
1095
+ except ValueError:
1096
+ return None
1097
+ return path
1098
+
1099
+
1100
+ @app.get("/api/jobs/{job_id}/artifacts/{artifact_type}")
1101
+ async def api_download_artifact(
1102
+ job_id: str,
1103
+ artifact_type: str,
1104
+ username: str = Depends(_require_user),
1105
+ ) -> Response:
1106
+ row = _db_fetchone(
1107
+ "SELECT * FROM jobs WHERE id = ? AND username = ?",
1108
+ (job_id, username),
1109
+ )
1110
+ if row is None:
1111
+ raise HTTPException(status_code=404, detail="Job not found")
1112
+
1113
+ col_map = {
1114
+ "mono": "mono_pdf_path",
1115
+ "dual": "dual_pdf_path",
1116
+ "glossary": "glossary_path",
1117
+ }
1118
+ column = col_map.get(artifact_type)
1119
+ if column is None:
1120
+ raise HTTPException(status_code=404, detail="Unknown artifact")
1121
+
1122
+ output_dir = Path(row["output_dir"]).resolve()
1123
+ path = _resolve_artifact_path(row[column], output_dir)
1124
+ if path is None:
1125
+ raise HTTPException(status_code=404, detail="Artifact not found")
1126
+
1127
+ return FileResponse(path)
1128
+
1129
+
1130
+ # ── 路由:计费 API ─────────────────────────────────────────────────────────────
1131
+ @app.get("/api/billing/me")
1132
+ async def api_billing_summary(username: str = Depends(_require_user)) -> dict[str, Any]:
1133
+ row = _db_fetchone(
1134
+ """
1135
+ SELECT
1136
+ COALESCE(SUM(prompt_tokens), 0) AS prompt_tokens,
1137
+ COALESCE(SUM(completion_tokens), 0) AS completion_tokens,
1138
+ COALESCE(SUM(total_tokens), 0) AS total_tokens,
1139
+ COALESCE(SUM(cost_usd), 0) AS total_cost_usd
1140
+ FROM usage_records
1141
+ WHERE username = ?
1142
+ """,
1143
+ (username,),
1144
+ )
1145
+ return {
1146
+ "username": username,
1147
+ "prompt_tokens": row["prompt_tokens"],
1148
+ "completion_tokens": row["completion_tokens"],
1149
+ "total_tokens": row["total_tokens"],
1150
+ "total_cost_usd": round(float(row["total_cost_usd"]), 8),
1151
+ }
1152
+
1153
+
1154
+ @app.get("/api/billing/me/records")
1155
+ async def api_billing_records(
1156
+ limit: int = 50,
1157
+ username: str = Depends(_require_user),
1158
+ ) -> dict[str, Any]:
1159
+ limit = max(1, min(limit, 200))
1160
+ rows = _db_fetchall(
1161
+ """
1162
+ SELECT
1163
+ id, username, job_id, model,
1164
+ prompt_tokens, completion_tokens, total_tokens,
1165
+ cost_usd, created_at
1166
+ FROM usage_records
1167
+ WHERE username = ?
1168
+ ORDER BY created_at DESC
1169
+ LIMIT ?
1170
+ """,
1171
+ (username, limit),
1172
+ )
1173
+ return {
1174
+ "records": [dict(row) for row in rows],
1175
+ }
1176
+
1177
+
1178
+ # ── 路由:内部 OpenAI 兼容接口 ────────────────────────────────────────────────
1179
+ def _extract_bearer_token(request: Request) -> str:
1180
+ header = request.headers.get("authorization", "")
1181
+ if not header.lower().startswith("bearer "):
1182
+ raise HTTPException(status_code=401, detail="Missing bearer token")
1183
+ token = header[7:].strip()
1184
+ if not token:
1185
+ raise HTTPException(status_code=401, detail="Missing bearer token")
1186
+ return token
1187
+
1188
+
1189
+ def _require_localhost(request: Request) -> None:
1190
+ client = request.client
1191
+ host = client.host if client else ""
1192
+ if host not in LOCALHOSTS:
1193
+ raise HTTPException(status_code=403, detail="Internal endpoint only")
1194
+
1195
+
1196
+ @app.post("/internal/openai/v1/chat/completions")
1197
+ async def internal_openai_chat_completions(request: Request) -> Response:
1198
+ _require_localhost(request)
1199
+ token = _extract_bearer_token(request)
1200
+
1201
+ username = INTERNAL_KEY_TO_USER.get(token)
1202
+ if not username:
1203
+ raise HTTPException(status_code=401, detail="Invalid internal API key")
1204
+
1205
+ if not OPENAI_REAL_API_KEY:
1206
+ raise HTTPException(status_code=500, detail="OPENAI_API_KEY is not configured")
1207
+
1208
+ try:
1209
+ payload = await request.json()
1210
+ except json.JSONDecodeError as exc:
1211
+ raise HTTPException(status_code=400, detail=f"Invalid JSON body: {exc}") from exc
1212
+
1213
+ if payload.get("stream"):
1214
+ raise HTTPException(status_code=400, detail="stream=true is not supported")
1215
+
1216
+ if _http_client is None:
1217
+ raise HTTPException(status_code=500, detail="HTTP client is not ready")
1218
+
1219
+ headers = {
1220
+ "Authorization": f"Bearer {OPENAI_REAL_API_KEY}",
1221
+ "Content-Type": "application/json",
1222
+ }
1223
+
1224
+ try:
1225
+ upstream = await _http_client.post(
1226
+ OPENAI_UPSTREAM_CHAT_URL,
1227
+ headers=headers,
1228
+ json=payload,
1229
+ )
1230
+ except httpx.HTTPError as exc:
1231
+ logger.error("Upstream OpenAI call failed: %s", exc)
1232
+ raise HTTPException(status_code=502, detail="Upstream OpenAI request failed") from exc
1233
+
1234
+ content_type = upstream.headers.get("content-type", "")
1235
+
1236
+ response_json: dict[str, Any] | None = None
1237
+ if "application/json" in content_type.lower():
1238
+ try:
1239
+ response_json = upstream.json()
1240
+ except Exception: # noqa: BLE001
1241
+ response_json = None
1242
+
1243
+ if upstream.status_code < 400 and response_json is not None:
1244
+ usage = response_json.get("usage") or {}
1245
+ prompt_tokens = int(usage.get("prompt_tokens") or 0)
1246
+ completion_tokens = int(usage.get("completion_tokens") or 0)
1247
+ total_tokens = int(usage.get("total_tokens") or (prompt_tokens + completion_tokens))
1248
+
1249
+ model = str(response_json.get("model") or payload.get("model") or "unknown")
1250
+ job_id = _active_job_by_user.get(username)
1251
+
1252
+ _record_usage(
1253
+ username=username,
1254
+ job_id=job_id,
1255
+ model=model,
1256
+ prompt_tokens=prompt_tokens,
1257
+ completion_tokens=completion_tokens,
1258
+ total_tokens=total_tokens,
1259
+ )
1260
+
1261
+ if response_json is not None:
1262
+ return JSONResponse(response_json, status_code=upstream.status_code)
1263
+
1264
+ return Response(
1265
+ content=upstream.content,
1266
+ status_code=upstream.status_code,
1267
+ media_type=content_type or None,
1268
+ )
{scripts → src/scripts}/update_space_secret.py RENAMED
File without changes