Spaces:
Running
Running
Revert "fix(pdf): acceptInsecureCerts 加载过期证书图片 + domcontentloaded 防外部图片挂起阻塞"
Browse filesThis reverts commit 144b733936adceb8a2b407b95d01a626ce046af9.
- .gitignore +0 -17
- PERFORMANCE_ANALYSIS.md +0 -252
- PERFORMANCE_STRESS_TEST_2026-08-03.md +0 -177
- benchmark.js +259 -0
- browser-pool.js +0 -184
- docker-compose.yml +3 -10
- make/backup-source.js +0 -64
- package-lock.json +2 -2
- package.json +1 -1
- reproduce-realistic.js +844 -0
- reproduce-timeout.js +159 -0
- server.js +89 -178
- test-long.js +154 -0
- tools/compare-dirs.js +0 -57
.gitignore
CHANGED
|
@@ -3,20 +3,3 @@ node_modules
|
|
| 3 |
*.bat
|
| 4 |
*.txt
|
| 5 |
# docker-compose.yml
|
| 6 |
-
|
| 7 |
-
# Hugging Face credentials (NEVER commit)
|
| 8 |
-
HF_TOKEN.md
|
| 9 |
-
hf_token.md
|
| 10 |
-
*.token.md
|
| 11 |
-
|
| 12 |
-
# Local dev/test artifacts (NOT source code)
|
| 13 |
-
temp/
|
| 14 |
-
test-results/
|
| 15 |
-
tests/
|
| 16 |
-
|
| 17 |
-
# One-off local tooling (not part of the backend)
|
| 18 |
-
lib/
|
| 19 |
-
make/_clean*.py
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
|
|
|
| 3 |
*.bat
|
| 4 |
*.txt
|
| 5 |
# docker-compose.yml
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
PERFORMANCE_ANALYSIS.md
DELETED
|
@@ -1,252 +0,0 @@
|
|
| 1 |
-
# Claude PDF Export Performance Analysis
|
| 2 |
-
|
| 3 |
-
## Problem: Performance Regression After CDN Localization
|
| 4 |
-
|
| 5 |
-
### Test Results
|
| 6 |
-
|
| 7 |
-
| Test ID | Time | Widget Render | Description |
|
| 8 |
-
|---------|------|---------------|-------------|
|
| 9 |
-
| 17:02:05 | 22,923ms | 17,063ms | **Before optimization** |
|
| 10 |
-
| 18:08:55 | 25,469ms | 19,612ms | After browser pool fix |
|
| 11 |
-
| 20:35:52 | 27,282ms | 19,410ms | **After CDN localization** |
|
| 12 |
-
|
| 13 |
-
**Result: CDN localization made performance WORSE (+2,347ms)**
|
| 14 |
-
|
| 15 |
-
## Root Cause Analysis
|
| 16 |
-
|
| 17 |
-
### What the CDN Interception Does
|
| 18 |
-
|
| 19 |
-
From backend logs:
|
| 20 |
-
```
|
| 21 |
-
[WIDGET] Chart.js CDN interception: ENABLED (local version: 200.3KB)
|
| 22 |
-
[WIDGET] Intercepting Chart.js request: https://cdnjs.cloudflare.com/ajax/libs/Chart.js/4.4.1/chart.umd.js
|
| 23 |
-
[WIDGET] Chart.js served from local file (200.3KB)
|
| 24 |
-
[WIDGET-PERF] setContent + networkidle0: 1957ms
|
| 25 |
-
[WIDGET-PERF] render: 1105ms
|
| 26 |
-
```
|
| 27 |
-
|
| 28 |
-
### Why It Made Things Worse
|
| 29 |
-
|
| 30 |
-
#### 1. Browser Request Interception Has Overhead
|
| 31 |
-
|
| 32 |
-
The code uses `BrowserContext.setServerInterception()` which:
|
| 33 |
-
- Adds overhead per network request (interception check + response handling)
|
| 34 |
-
- Each widget makes multiple requests (HTML + Chart.js + other resources)
|
| 35 |
-
- Total: 15 widgets × multiple requests × interception overhead
|
| 36 |
-
|
| 37 |
-
#### 2. Chart.js CDN Already Fast
|
| 38 |
-
|
| 39 |
-
Cloudflare CDN caches Chart.js aggressively:
|
| 40 |
-
- Chart.js (200KB) likely loads in <100ms from CDN
|
| 41 |
-
- Local file serving has similar or worse overhead (filesystem I/O + interception handling)
|
| 42 |
-
|
| 43 |
-
#### 3. Real Bottleneck is Chart.js EXECUTION
|
| 44 |
-
|
| 45 |
-
Backend timing breakdown per widget:
|
| 46 |
-
```
|
| 47 |
-
setContent + networkidle0: 600-2000ms (includes CDN + interception)
|
| 48 |
-
render: 2000-2500ms ← CHART.JS EXECUTION IS THE BOTTLENECK
|
| 49 |
-
screenshot: 40-220ms
|
| 50 |
-
```
|
| 51 |
-
|
| 52 |
-
The `render` phase (2-2.5 seconds) is where Chart.js:
|
| 53 |
-
- Parses the JavaScript code
|
| 54 |
-
- Initializes the Chart object
|
| 55 |
-
- Computes chart layout
|
| 56 |
-
- Draws bars/lines/pies to canvas
|
| 57 |
-
- Applies animations and styling
|
| 58 |
-
|
| 59 |
-
**CDN localization only saves the download time, not the execution time.**
|
| 60 |
-
|
| 61 |
-
### Concurrency Analysis
|
| 62 |
-
|
| 63 |
-
With MAX_CONCURRENT=3 and 15 widgets:
|
| 64 |
-
```
|
| 65 |
-
Batch 1: widgets 0-2 (~4 seconds)
|
| 66 |
-
Batch 2: widgets 3-5 (~4 seconds)
|
| 67 |
-
Batch 3: widgets 6-8 (~4 seconds)
|
| 68 |
-
Batch 4: widgets 9-11 (~4 seconds)
|
| 69 |
-
Batch 5: widgets 12-14 (~4 seconds)
|
| 70 |
-
Total: ~20 seconds
|
| 71 |
-
```
|
| 72 |
-
|
| 73 |
-
This matches the actual test time (19.4 seconds), confirming the bottleneck is per-widget rendering time, not network.
|
| 74 |
-
|
| 75 |
-
## Why CDN Localization Failed
|
| 76 |
-
|
| 77 |
-
### Initial Hypothesis (Wrong)
|
| 78 |
-
|
| 79 |
-
```
|
| 80 |
-
CDN download (2-3s) → Widget render (1s)
|
| 81 |
-
Local file (0s) + Widget render (1s)
|
| 82 |
-
Expected savings: 2-3s per widget
|
| 83 |
-
```
|
| 84 |
-
|
| 85 |
-
### Actual Behavior (Reality)
|
| 86 |
-
|
| 87 |
-
```
|
| 88 |
-
CDN download (0.1s) + interception overhead (0.2s) + render (2.5s) = 2.8s
|
| 89 |
-
Local file (0.1s) + interception overhead (0.2s) + render (2.5s) = 2.8s
|
| 90 |
-
Savings: 0s, but added interception overhead = WORSE
|
| 91 |
-
```
|
| 92 |
-
|
| 93 |
-
Cloudflare CDN already caches Chart.js efficiently, so the download time was already minimal.
|
| 94 |
-
|
| 95 |
-
### What the Interception Actually Does
|
| 96 |
-
|
| 97 |
-
From server.js:
|
| 98 |
-
```typescript
|
| 99 |
-
// BrowserContext-level request interception
|
| 100 |
-
await browserContext.setServerInterception({
|
| 101 |
-
urlPattern: '**/Chart.js/**/*.js',
|
| 102 |
-
handler: async (route) => {
|
| 103 |
-
const chartJsContent = fs.readFileSync('/app/lib/chart.umd.js', 'utf8');
|
| 104 |
-
await route.fulfill({
|
| 105 |
-
status: 200,
|
| 106 |
-
contentType: 'application/javascript',
|
| 107 |
-
body: chartJsContent
|
| 108 |
-
});
|
| 109 |
-
}
|
| 110 |
-
});
|
| 111 |
-
```
|
| 112 |
-
|
| 113 |
-
The interception handler:
|
| 114 |
-
1. Catches the Chart.js request
|
| 115 |
-
2. Reads local file (filesystem I/O)
|
| 116 |
-
3. Returns response to browser
|
| 117 |
-
|
| 118 |
-
Each step adds latency, and with 15 widgets making these requests, the overhead compounds.
|
| 119 |
-
|
| 120 |
-
## Performance Bottleneck Breakdown
|
| 121 |
-
|
| 122 |
-
### Current Bottleneck Distribution
|
| 123 |
-
|
| 124 |
-
```
|
| 125 |
-
┌─────────────────────────────────────────────────────────┐
|
| 126 |
-
│ Widget Render Time: ~4 seconds per widget │
|
| 127 |
-
├─────────────────────────────────────────────────────────┤
|
| 128 |
-
│ setContent + networkidle: 600-2000ms (network + DOM) │
|
| 129 |
-
│ Chart.js DOWNLOAD: 100-200ms (already fast) │
|
| 130 |
-
│ Chart.js EXECUTION: 2000-2500ms ← BOTTLENECK │
|
| 131 |
-
│ Screenshot: 40-220ms │
|
| 132 |
-
└─────────────────────────────────────────────────────────┘
|
| 133 |
-
```
|
| 134 |
-
|
| 135 |
-
### Where the Time Goes
|
| 136 |
-
|
| 137 |
-
1. **Chart.js Initialization** (~500ms)
|
| 138 |
-
- Parse JavaScript code
|
| 139 |
-
- Set up Chart namespace and utilities
|
| 140 |
-
- Configure defaults and helpers
|
| 141 |
-
|
| 142 |
-
2. **Chart Computation** (~1000ms)
|
| 143 |
-
- Calculate scales and axes
|
| 144 |
-
- Compute bar/line/pie positions
|
| 145 |
-
- Apply data transformations
|
| 146 |
-
|
| 147 |
-
3. **Canvas Drawing** (~1000ms)
|
| 148 |
-
- Draw grid lines and labels
|
| 149 |
-
- Render bars/lines/pies
|
| 150 |
-
- Apply colors and gradients
|
| 151 |
-
- Draw legends and tooltips
|
| 152 |
-
|
| 153 |
-
## What Doesn't Work
|
| 154 |
-
|
| 155 |
-
### ❌ CDN Localization
|
| 156 |
-
|
| 157 |
-
**Reason**: Cloudflare CDN already fast; interception overhead negates benefit
|
| 158 |
-
|
| 159 |
-
### ❌ Increasing MAX_CONCURRENT
|
| 160 |
-
|
| 161 |
-
**Reason**: Each widget already takes CPU time; more concurrency = more CPU contention
|
| 162 |
-
|
| 163 |
-
### ❌ Disabling Chart.js Animations
|
| 164 |
-
|
| 165 |
-
**Reason**: Already disabled in current implementation
|
| 166 |
-
|
| 167 |
-
## What Could Work (Future Optimization)
|
| 168 |
-
|
| 169 |
-
### Option 1: Chart.js Worker Pool
|
| 170 |
-
|
| 171 |
-
Pre-render charts in web workers to avoid blocking:
|
| 172 |
-
- Complex to implement
|
| 173 |
-
- Limited benefit (still need to wait for rendering)
|
| 174 |
-
|
| 175 |
-
### Option 2: Chart.js Caching
|
| 176 |
-
|
| 177 |
-
Cache rendered chart images to avoid re-rendering identical charts:
|
| 178 |
-
- Cache key: (chart type, data JSON, dimensions)
|
| 179 |
-
- Benefit: Repeated charts render instantly
|
| 180 |
-
- Drawback: First render still slow
|
| 181 |
-
|
| 182 |
-
### Option 3: Use Lightweight Chart Libraries
|
| 183 |
-
|
| 184 |
-
Replace Chart.js with lighter alternatives:
|
| 185 |
-
- D3.js: More flexible but similarly complex
|
| 186 |
-
- Lightweight charting libraries: May not support all chart types
|
| 187 |
-
- Custom canvas rendering: Most work, most control
|
| 188 |
-
|
| 189 |
-
### Option 4: Accept Current Performance
|
| 190 |
-
|
| 191 |
-
**Rationale**: 19 seconds for 15 charts is reasonable
|
| 192 |
-
- Each chart takes ~1.3 seconds average
|
| 193 |
-
- CDN overhead with interception: 0ms (interception overhead negates benefit)
|
| 194 |
-
- Actual per-widget time: ~1.3 seconds
|
| 195 |
-
|
| 196 |
-
## Recommendation
|
| 197 |
-
|
| 198 |
-
**Do not deploy CDN localization to production.**
|
| 199 |
-
|
| 200 |
-
Instead:
|
| 201 |
-
1. **Keep current CDN approach**: Cloudflare CDN is already efficient
|
| 202 |
-
2. **Document current performance**: 19 seconds for 15 charts is acceptable
|
| 203 |
-
3. **Monitor for regression**: Track performance over time
|
| 204 |
-
4. **Consider user experience**: Add progress indicator if not already present
|
| 205 |
-
|
| 206 |
-
## Technical Details
|
| 207 |
-
|
| 208 |
-
### Files Modified
|
| 209 |
-
|
| 210 |
-
1. `backend-service/Dockerfile` - Added Chart.js library
|
| 211 |
-
2. `backend-service/server.js` - Added CDN interception handler
|
| 212 |
-
|
| 213 |
-
### Rollback Steps
|
| 214 |
-
|
| 215 |
-
To revert to pre-optimization state:
|
| 216 |
-
```bash
|
| 217 |
-
# Remove Chart.js library
|
| 218 |
-
rm backend-service/lib/chart.umd.js
|
| 219 |
-
|
| 220 |
-
# Remove Dockerfile COPY line
|
| 221 |
-
# Remove server.js interception handler
|
| 222 |
-
|
| 223 |
-
# Rebuild and restart
|
| 224 |
-
docker build -t backend-service .
|
| 225 |
-
docker restart pdf-test
|
| 226 |
-
```
|
| 227 |
-
|
| 228 |
-
### Performance Metrics
|
| 229 |
-
|
| 230 |
-
| Metric | Before | After | Change |
|
| 231 |
-
|--------|--------|-------|--------|
|
| 232 |
-
| Total time | 22.9s | 27.3s | +4.4s (worse) |
|
| 233 |
-
| Widget render | 17.1s | 19.4s | +2.3s (worse) |
|
| 234 |
-
| Per-widget avg | ~1.1s | ~1.3s | +0.2s (worse) |
|
| 235 |
-
|
| 236 |
-
## Conclusion
|
| 237 |
-
|
| 238 |
-
The CDN localization optimization was based on the incorrect assumption that Chart.js download time was the bottleneck. In reality:
|
| 239 |
-
|
| 240 |
-
1. **Cloudflare CDN already serves Chart.js quickly** (<100ms)
|
| 241 |
-
2. **Chart.js execution is the real bottleneck** (2-2.5 seconds)
|
| 242 |
-
3. **Request interception adds overhead** that negates any benefit
|
| 243 |
-
|
| 244 |
-
The optimization should be reverted, and future optimization efforts should focus on:
|
| 245 |
-
- Chart.js execution optimization (difficult)
|
| 246 |
-
- Caching rendered charts (complex)
|
| 247 |
-
- Accepting current performance (practical)
|
| 248 |
-
|
| 249 |
-
---
|
| 250 |
-
|
| 251 |
-
**Analysis Date**: 2026-06-23
|
| 252 |
-
**Analyst**: AI Assistant (Claude Code)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
PERFORMANCE_STRESS_TEST_2026-08-03.md
DELETED
|
@@ -1,177 +0,0 @@
|
|
| 1 |
-
# PDF 导出并发压力测试与性能优化报告
|
| 2 |
-
|
| 3 |
-
**日期**: 2026-08-03
|
| 4 |
-
**测试环境**: 本机 Docker（`pdf-test` 容器，16 核 / 32GB，Docker 可见 ~15.5GB）
|
| 5 |
-
**对比目标**: Hugging Face 免费版（CPU Basic = 2 vCPU / 16GB）
|
| 6 |
-
**工具**: `backend-service/tests/stress/`（README 见 `tests/stress/README.md`）
|
| 7 |
-
|
| 8 |
-
---
|
| 9 |
-
|
| 10 |
-
## 一、结论摘要
|
| 11 |
-
|
| 12 |
-
| 指标 | 优化前 | 优化后 | 提升 |
|
| 13 |
-
|------|--------|--------|------|
|
| 14 |
-
| 单请求延迟 (medium, 并发1) | 4906ms | **1572ms** | **3.1x** |
|
| 15 |
-
| 单请求延迟 (small, 并发1) | 3380ms | **926ms** | **3.7x** |
|
| 16 |
-
| 吞吐 (medium, 并发1) | ~12/min | **38/min** | **3.2x** |
|
| 17 |
-
| 吞吐峰值 (medium) | ~10/min（并发4） | **83/min**（并发8） | **8x** |
|
| 18 |
-
| 并发承载（无失败） | 6 并发即延迟飙升至 8.9s | **8 并发全部成功，延迟 3.6s** | 稳定 |
|
| 19 |
-
| 内存峰值 | 每请求新建浏览器（泄漏僵尸进程） | **4 个常驻浏览器，峰值 0.91GiB** | 受控 |
|
| 20 |
-
|
| 21 |
-
**核心答案**: 优化后本机 docker 上 **4 个用户可同时导出 PDF**（=浏览器池大小），更多并发用户自动排队，不会雪崩。Hugging Face 免费版（2 vCPU）建议 `PDF_POOL_SIZE=2`，即 **2 个用户同时导出**，超出排队。
|
| 22 |
-
|
| 23 |
-
---
|
| 24 |
-
|
| 25 |
-
## 二、根因分析（为什么原来并发差）
|
| 26 |
-
|
| 27 |
-
### 2.1 每个 PDF 请求启动一个全新 Chromium（最大瓶颈）
|
| 28 |
-
|
| 29 |
-
`server.js` 原代码在 `/api/generate_pdf` 中**每次请求都 `puppeteer.launch()`**：
|
| 30 |
-
|
| 31 |
-
- 启动一个 Chromium 耗时 ~0.3–1.5s、占用 ~200–300MB
|
| 32 |
-
- 并发 N 个请求 = N 个 Chromium 进程同时竞争 CPU/内存
|
| 33 |
-
- 请求结束 `browser.close()` 还会在容器里留下**僵尸进程**（`chrome_crashpad`/`chromium <defunct>` 累积）
|
| 34 |
-
- 实测 6 并发时延迟从 4.9s 飙到 8.9s，吞吐反而下降
|
| 35 |
-
|
| 36 |
-
> 行业共识（见参考资料）：`page.pdf()` 是 CPU 密集操作，**并发数≈CPU 核数**，永远不要每请求启动浏览器。
|
| 37 |
-
|
| 38 |
-
### 2.2 setContent 的 networkidle0 等待浪费 ~2s
|
| 39 |
-
|
| 40 |
-
用临时实验程序（`temp/setcontent-wait-experiment.js`，在容器内实测）：
|
| 41 |
-
|
| 42 |
-
| waitUntil 策略 | 耗时 |
|
| 43 |
-
|----------------|------|
|
| 44 |
-
| `setContent(load)` | **3–7ms** |
|
| 45 |
-
| `setContent(load + networkidle0)` | **~1970ms** |
|
| 46 |
-
| `setContent(load) + waitForNetworkIdle(500)` | ~505ms |
|
| 47 |
-
| 原代码两者都做 | **~2470ms** |
|
| 48 |
-
|
| 49 |
-
原代码既在 `setContent` 里 `networkidle0`，又在后面 `waitForNetworkIdle(500)`，对一个无外链资源的纯本地 HTML 白白等待 ~2.5s。
|
| 50 |
-
|
| 51 |
-
### 2.3 Widget 渲染每个 widget 启动一个浏览器
|
| 52 |
-
|
| 53 |
-
`renderWidgetPuppeteer()` / `_renderFullHtml()` 原代码**每个 widget 都 launch 一个新 Chromium**。一次导出 15 个 widget = 15 次浏览器启动。`WidgetRenderer._widgetBrowser` 单例存在但从未被使用（死代码）。
|
| 54 |
-
|
| 55 |
-
### 2.4 事件循环被同步操作阻塞
|
| 56 |
-
|
| 57 |
-
- 大 HTML 临时文件的 `fs.writeFileSync` / `fs.unlinkSync` 同步阻塞主线程
|
| 58 |
-
- 优化后改为 `fs.promises` 异步版本
|
| 59 |
-
|
| 60 |
-
### 2.5 响应被错误 JSON 序列化（我引入后立即修复）
|
| 61 |
-
|
| 62 |
-
`page.pdf()` 在 Puppeteer 24 返回 **`Uint8Array` 而非 Buffer**。`res.send(u8array)` 时 Express 不识别为二进制，会 `JSON.stringify` 成 `{"0":37,"1":80,...}`（约 13 倍体积）。原代码 `Buffer.from(pdfBuffer)` 正是为此。**任何修改都不能去掉这一步**。
|
| 63 |
-
|
| 64 |
-
---
|
| 65 |
-
|
| 66 |
-
## 三、实施的优化
|
| 67 |
-
|
| 68 |
-
### 3.1 新增 `browser-pool.js` — PDF 浏览器池
|
| 69 |
-
|
| 70 |
-
- 常驻 `PDF_POOL_SIZE` 个 Chromium（默认 4，env 可配）
|
| 71 |
-
- 每次请求 `acquire()` 一个浏览器 → 创建 page → 渲染 → 关闭 page → `release()` 归还
|
| 72 |
-
- 浏览器在 N 个任务后回收重建（`PDF_RECYCLE_AFTER=30`），防止 Chromium 长期运行内存膨胀
|
| 73 |
-
- 浏览器崩溃自动重建；池满时请求排队（`PDF_ACQUIRE_TIMEOUT_MS=120s`）
|
| 74 |
-
- 池大小 = 最大并发 PDF 数
|
| 75 |
-
|
| 76 |
-
### 3.2 Widget 渲染复用单例浏览器
|
| 77 |
-
|
| 78 |
-
`renderWidgetPuppeteer()` 与 `_renderFullHtml()` 改为使用 `getWidgetBrowser()` 共享浏览器（每个 widget 独立 page，渲染后只关 page 不关浏览器），并加了竞态保护与崩溃自动重建。
|
| 79 |
-
|
| 80 |
-
### 3.3 等待策略优化
|
| 81 |
-
|
| 82 |
-
- `setContent` / `goto`：`networkidle0` → `'load'`（每个请求省 ~2s）
|
| 83 |
-
- `waitForNetworkIdle`：idleTime 500 → 300（保留作为安全网）
|
| 84 |
-
- Widget 渲染同样处理
|
| 85 |
-
|
| 86 |
-
### 3.4 其他
|
| 87 |
-
|
| 88 |
-
- 大文件读写改异步 `fs.promises`
|
| 89 |
-
- 恢复 `res.send(Buffer.from(pdfBuffer))`（关键，见 2.5）
|
| 90 |
-
- 补上 `page.on('dialog')` 自动关弹窗（解决方案备忘录 24 的兜底，防止 XSS/异常 HTML 卡死）
|
| 91 |
-
- 配置全部 env 化：`PDF_POOL_SIZE` / `PDF_RECYCLE_AFTER` / `WIDGET_MAX_CONCURRENT`（docker-compose.yml 已配置：生产 2，测试 4）
|
| 92 |
-
|
| 93 |
-
---
|
| 94 |
-
|
| 95 |
-
## 四、压力测试数据
|
| 96 |
-
|
| 97 |
-
### 4.1 medium 负载（375KB 文本 + 3 张 base64 图）
|
| 98 |
-
|
| 99 |
-
| 并发 | 优化前 avg | 优化后 avg | 优化后 p95 | 优化后吞吐/min | 成功率 |
|
| 100 |
-
|------|-----------|-----------|-----------|---------------|--------|
|
| 101 |
-
| 1 | 4906ms | **1572ms** | 1843ms | 38 | 100% |
|
| 102 |
-
| 2 | 5283ms | **1882ms** | 2131ms | 62 | 100% |
|
| 103 |
-
| 4 | 5918ms | **2433ms** | 3752ms | 67 | 100% |
|
| 104 |
-
| 6 | 8919ms | **2758ms** | 4163ms | 81 | 100% |
|
| 105 |
-
| 8 | 未测 | **3569ms** | 5763ms | 83 | 100% |
|
| 106 |
-
|
| 107 |
-
### 4.2 small 负载（62KB，纯文本）
|
| 108 |
-
|
| 109 |
-
| 并发 | 优化前 avg | 优化后 avg | 优化后吞吐/min |
|
| 110 |
-
|------|-----------|-----------|---------------|
|
| 111 |
-
| 1 | 3380ms | **926ms** | 65 |
|
| 112 |
-
| 4 | — | **1513ms** | 117 |
|
| 113 |
-
| 8 | — | **2521ms** | 118 |
|
| 114 |
-
|
| 115 |
-
### 4.3 large 负载（1.24MB + 10 图，走临时文件路径）
|
| 116 |
-
|
| 117 |
-
| 并发 | 优化后 avg | 优化后吞吐/min | 成功率 |
|
| 118 |
-
|------|-----------|---------------|--------|
|
| 119 |
-
| 1 | 4988ms | 12 | 100% |
|
| 120 |
-
| 4 | 7748ms | 20 | 100% |
|
| 121 |
-
|
| 122 |
-
### 4.4 资源监控（optimized, medium, 4 浏览器常驻）
|
| 123 |
-
|
| 124 |
-
- CPU 峰值 ~392%（4 个浏览器同时渲染 ≈ 4 核忙）
|
| 125 |
-
- 内存峰值 **0.91GiB**，空闲 0.57GiB（4 个常驻浏览器 + Node）
|
| 126 |
-
- 相比优化前每请求新建浏览器、僵尸进程累积，资源完全受控
|
| 127 |
-
|
| 128 |
-
### 4.5 Widget 渲染
|
| 129 |
-
|
| 130 |
-
- 5 个 widget（4 chart + 1 mermaid）批量渲染：**7.03s，5/5 成功**，日志确认只启动 **1 次**浏览器（优化前每 widget 1 次）
|
| 131 |
-
- 单次 PDF 校验：输出为真实 PDF（`%PDF-1.4` 魔数，~113 页，1.6MB）
|
| 132 |
-
|
| 133 |
-
---
|
| 134 |
-
|
| 135 |
-
## 五、对 Hugging Face 免费版（2 vCPU）的预测
|
| 136 |
-
|
| 137 |
-
本机 16 核 docker 测出的容量是 **HF 的上界**，不能直接套用。按行业经验（每 page.pdf() 约占满 1 核）：
|
| 138 |
-
|
| 139 |
-
- **HF 建议 `PDF_POOL_SIZE=2`**（docker-compose 已配置）→ 2 个用户同时导出，超出排队
|
| 140 |
-
- 单请求延迟在 HF 上会比本机高（CPU 弱），medium 预计 ~3–4s
|
| 141 |
-
- 吞吐预计 ~20–30/min（HF 2 核，CPU 是瓶颈）
|
| 142 |
-
- **结论**：当前 1000 用户规模绰绰有余；若未来用户数大幅增长，HF 免费版 2 vCPU 会成为瓶颈，需升级 `CPU Upgrade`（8 vCPU/32GB，$0.03/时）或使用 GPU Space
|
| 143 |
-
|
| 144 |
-
> ⚠️ **Docker 不能完全模拟 HuggingFace 机器**：CPU 核数（16 vs 2）差异巨大，PDF 渲染是 CPU 密集任务，因此本机测试结果只能作为相对对比（优化前后提升倍数），绝对并发数在 HF 上需按 2 vCPU 重新评估。
|
| 145 |
-
|
| 146 |
-
---
|
| 147 |
-
|
| 148 |
-
## 六、验证过的兼容性（避免复杂问题重现）
|
| 149 |
-
|
| 150 |
-
对照解决方案备忘录逐项确认优化未破坏：
|
| 151 |
-
|
| 152 |
-
| 备忘录 | 关注点 | 验证 |
|
| 153 |
-
|--------|--------|------|
|
| 154 |
-
| 24 (Puppeteer 超时/XSS) | dialog 卡死 setContent | ✅ 补回 `page.on('dialog')` 自动关闭 |
|
| 155 |
-
| 32 (Widget 渲染性能) | CDN 本地化失败、缓存方案 | ✅ 未重蹈 `setServerInterception` 覆辙；widget 单例浏览器按备忘录原设计实现 |
|
| 156 |
-
| 31 (表格串行 bug) | 前端导出流程 | ✅ 后端响应格式未变 |
|
| 157 |
-
| 05/06/26 (PDF/图表) | Shiki 高亮、Mermaid 版本固定 | ✅ 相关逻辑未改动，实测 PDF 与 widget 输出正确 |
|
| 158 |
-
|
| 159 |
-
---
|
| 160 |
-
|
| 161 |
-
## 七、后续建议（如需进一步优化）
|
| 162 |
-
|
| 163 |
-
1. **Shiki 高亮移入 Worker 线程**：大 HTML（>1MB）大量代码块时主线程阻塞；可放入 `worker_threads`。当前 medium/small 负载下 <100ms，非瓶颈。
|
| 164 |
-
2. **Node.js 多进程（cluster/PM2）**：充分利用多核处理主线程工作。注意每个 worker 需独立浏览器池。
|
| 165 |
-
3. **外部图片加载超时优化**：非 base64 图片失败时会等 15s 兜底（备忘录 03/04 相关）。可考虑对 http 图片缩短等待或降级。
|
| 166 |
-
4. **widget 渲染缓存**：备忘录 32 提到 CDN 响应内存缓存可提升 28%（此前实现丢失，本次未恢复，因为 `setServerInterception` 开销曾被证明为负收益；如需恢复需重新验证）。
|
| 167 |
-
5. **大规模扩展**：超过单机能力后，将 PDF 服务拆分为独立可水平扩展的服务（参考 medium 文章的直接 CDP + 队列架构）。
|
| 168 |
-
|
| 169 |
-
---
|
| 170 |
-
|
| 171 |
-
## 八、参考资料（行业最佳实践）
|
| 172 |
-
|
| 173 |
-
- [Puppeteer PDF: Common Problems and How to Fix](https://blog.pdfloom.com/puppeteer-pdf-problems/) — 浏览器池模式、内存 150-300MB/实例、并发限制
|
| 174 |
-
- [Optimizing Puppeteer PDF generation](https://www.codepasta.com/2024/04/19/optimizing-puppeteer-pdf-generation) — 并发数≈核数，队列限流
|
| 175 |
-
- [Designing a High-Performance HTML-to-PDF Service](https://medium.com/@harishrawat93/designing-a-high-performance-html-to-pdf-service-for-production-1a70099e3ccb) — 浏览器/页面池 + 队列架构
|
| 176 |
-
- [How to Fix Puppeteer Memory Leaks](https://www.grabbit.live/blog/puppeteer-memory-leak) — 浏览器按 N 任务回收（disposable Chromium）
|
| 177 |
-
- Hugging Face Spaces 免费版规格：CPU Basic = **2 vCPU / 16GB**（[spaces-overview](https://huggingface.co/docs/hub/en/spaces-overview)）
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
benchmark.js
ADDED
|
@@ -0,0 +1,259 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**
|
| 2 |
+
* Benchmark: 测量不同 HTML 大小下 page.pdf() 的实际渲染时间
|
| 3 |
+
*/
|
| 4 |
+
const http = require('http');
|
| 5 |
+
const fs = require('fs');
|
| 6 |
+
|
| 7 |
+
const BACKEND_PORT = process.env.PORT || 7861;
|
| 8 |
+
const BACKEND_URL = `http://localhost:${BACKEND_PORT}`;
|
| 9 |
+
|
| 10 |
+
// CSS from buildDocumentCss
|
| 11 |
+
const CSS = `@media print { @page { size: A4; margin: 15mm 10mm; } body { -webkit-print-color-adjust: exact; } }
|
| 12 |
+
body { font-family: -apple-system, sans-serif; font-size: 14px; line-height: 1.6; max-width: 746px; margin: 0 auto; padding: 20px; }
|
| 13 |
+
h1,h2,h3 { font-weight: 600; margin: 16px 0 8px; }
|
| 14 |
+
pre { background: #f6f8fa; padding: 16px; border-radius: 6px; overflow-x: auto; border: 1px solid #e1e4e8; }
|
| 15 |
+
code { font-family: monospace; font-size: 13px; }
|
| 16 |
+
p { margin: 8px 0; } table { border-collapse: collapse; width: 100%; margin: 12px 0; }
|
| 17 |
+
th,td { border: 1px solid #dfe2e5; padding: 8px 12px; } th { background: #f1f3f4; }
|
| 18 |
+
.chat-container { display: flex; flex-direction: column; gap: 16px; }
|
| 19 |
+
.message-row { display: flex; gap: 10px; } .message-bubble { max-width: 85%; padding: 12px 16px; border-radius: 12px; }
|
| 20 |
+
.ai-bubble { background: #fff; border: 1px solid #eee; } .user-bubble { background: #e8f0fe; }
|
| 21 |
+
.avatar { width: 32px; height: 32px; border-radius: 50%; display: flex; align-items: center; justify-content: center; }`;
|
| 22 |
+
|
| 23 |
+
function E(s) { return s.replace(/&/g,'&').replace(/</g,'<').replace(/>/g,'>'); }
|
| 24 |
+
|
| 25 |
+
// Shiki-style colored code token
|
| 26 |
+
function shikiWrap(token, color) {
|
| 27 |
+
return `<span style="color:${color}">${E(token)}</span>`;
|
| 28 |
+
}
|
| 29 |
+
|
| 30 |
+
// Generate code with Shiki-style spans (simulating real output)
|
| 31 |
+
function generateShikiCode(lines) {
|
| 32 |
+
return lines.map(line => {
|
| 33 |
+
const parts = [];
|
| 34 |
+
const tokens = line.split(/(\s+|[^a-zA-Z0-9_\s]+)/g);
|
| 35 |
+
for (const tok of tokens) {
|
| 36 |
+
if (!tok) continue;
|
| 37 |
+
if (/^\s+$/.test(tok)) { parts.push(tok); continue; }
|
| 38 |
+
let color = '#c9d1d9'; // plain
|
| 39 |
+
if (/^(const|let|var|function|return|if|else|for|async|await|import|from|export|class|extends|new|try|catch|throw|typeof|this|switch|case|break|continue|while|do|of|in|static|get|set|super|interface|type|void|null|undefined|true|false)$/.test(tok)) color = '#ff7b72';
|
| 40 |
+
else if (/^(console|log|error|fetch|Promise|Math|Date|JSON|Map|Set|Array|Object|String|Number|Error|setTimeout|clearTimeout|AbortController|AbortSignal|require|module|exports|process)$/.test(tok)) color = '#d2a8ff';
|
| 41 |
+
else if (/^\d+$/.test(tok)) color = '#79c0ff';
|
| 42 |
+
else if (/^[{}()\[\];,\.:=+\-*/<>!&|?%@~^'"`]+$/.test(tok)) color = '#c9d1d9';
|
| 43 |
+
else if (/^[A-Z]/.test(tok) && tok.length > 1) color = '#ffa657';
|
| 44 |
+
parts.push(shikiWrap(tok, color));
|
| 45 |
+
}
|
| 46 |
+
return parts.join('');
|
| 47 |
+
}).join('\n');
|
| 48 |
+
}
|
| 49 |
+
|
| 50 |
+
const CODE_LINES = [
|
| 51 |
+
'async function fetchData(url, options = {}) {',
|
| 52 |
+
' const controller = new AbortController();',
|
| 53 |
+
' const timeout = setTimeout(() => controller.abort(), 30000);',
|
| 54 |
+
' try {',
|
| 55 |
+
' const response = await fetch(url, {',
|
| 56 |
+
' ...options,',
|
| 57 |
+
' signal: controller.signal,',
|
| 58 |
+
" headers: { 'Content-Type': 'application/json' },",
|
| 59 |
+
' });',
|
| 60 |
+
' if (!response.ok) {',
|
| 61 |
+
' throw new Error(`HTTP ${response.status}: ${response.statusText}`);',
|
| 62 |
+
' }',
|
| 63 |
+
' const data = await response.json();',
|
| 64 |
+
" console.log('Data received:', data);",
|
| 65 |
+
' return data;',
|
| 66 |
+
' } catch (error) {',
|
| 67 |
+
" console.error('Fetch failed:', error.message);",
|
| 68 |
+
' throw error;',
|
| 69 |
+
' } finally {',
|
| 70 |
+
' clearTimeout(timeout);',
|
| 71 |
+
' }',
|
| 72 |
+
'}',
|
| 73 |
+
];
|
| 74 |
+
|
| 75 |
+
function buildHtml(targetSizeMB, opts = {}) {
|
| 76 |
+
const shikiRatio = opts.shikiRatio || 0.5; // fraction of content that is Shiki code
|
| 77 |
+
const textContent = opts.textContent || '<p>This is a typical AI response explaining a concept with some details and examples.</p>';
|
| 78 |
+
|
| 79 |
+
const shikiCode = generateShikiCode(CODE_LINES);
|
| 80 |
+
const codeBlock = `<pre data-language="javascript"><code class="language-javascript">${shikiCode}</code></pre>`;
|
| 81 |
+
const codeBlockSize = Buffer.byteLength(codeBlock, 'utf8');
|
| 82 |
+
|
| 83 |
+
const textBlockSize = Buffer.byteLength(textContent, 'utf8');
|
| 84 |
+
|
| 85 |
+
const overhead = 2048; // HTML wrapper + CSS
|
| 86 |
+
const targetBytes = targetSizeMB * 1024 * 1024 - overhead;
|
| 87 |
+
|
| 88 |
+
let html = `<!DOCTYPE html><html><head><meta charset="UTF-8"><style>${CSS}</style></head><body><div class="chat-container">`;
|
| 89 |
+
|
| 90 |
+
// Build a mix of code blocks and text
|
| 91 |
+
const codeWeight = shikiRatio;
|
| 92 |
+
const textWeight = 1 - shikiRatio;
|
| 93 |
+
const codeBytes = targetBytes * codeWeight;
|
| 94 |
+
const textBytes = targetBytes * textWeight;
|
| 95 |
+
|
| 96 |
+
const codeIterations = Math.ceil(codeBytes / codeBlockSize);
|
| 97 |
+
const textIterations = Math.ceil(textBytes / textBlockSize);
|
| 98 |
+
|
| 99 |
+
let i = 0, j = 0;
|
| 100 |
+
const totalIterations = codeIterations + textIterations;
|
| 101 |
+
|
| 102 |
+
while (i < codeIterations || j < textIterations) {
|
| 103 |
+
// Alternate between code and text
|
| 104 |
+
if (i < codeIterations) {
|
| 105 |
+
html += `<div class="message-row"><div class="avatar">🤖</div><div class="message-bubble ai-bubble">${codeBlock}</div></div>`;
|
| 106 |
+
i++;
|
| 107 |
+
}
|
| 108 |
+
if (j < textIterations) {
|
| 109 |
+
html += `<div class="message-row"><div class="avatar">👤</div><div class="message-bubble user-bubble">${textContent}</div></div>`;
|
| 110 |
+
j++;
|
| 111 |
+
}
|
| 112 |
+
}
|
| 113 |
+
|
| 114 |
+
html += '</div></body></html>';
|
| 115 |
+
|
| 116 |
+
const actualSize = (Buffer.byteLength(html, 'utf8') / 1024 / 1024).toFixed(2);
|
| 117 |
+
return html;
|
| 118 |
+
}
|
| 119 |
+
|
| 120 |
+
async function sendPdfRequest(html, label) {
|
| 121 |
+
const payload = JSON.stringify({
|
| 122 |
+
html,
|
| 123 |
+
codeTheme: 'github',
|
| 124 |
+
showWatermark: false,
|
| 125 |
+
imageCount: 0,
|
| 126 |
+
totalImageSizeMB: 0,
|
| 127 |
+
platform: 'Benchmark',
|
| 128 |
+
language: 'en-US',
|
| 129 |
+
extensionVersion: '2.0.2',
|
| 130 |
+
exportCount: 0, exportPdf: 0, exportMd: 0,
|
| 131 |
+
exportTxt: 0, exportDocx: 0, exportJson: 0,
|
| 132 |
+
exportClipboard: 0, exportNotion: 0,
|
| 133 |
+
});
|
| 134 |
+
|
| 135 |
+
return new Promise((resolve, reject) => {
|
| 136 |
+
const startTime = Date.now();
|
| 137 |
+
const options = {
|
| 138 |
+
hostname: 'localhost',
|
| 139 |
+
port: BACKEND_PORT,
|
| 140 |
+
path: '/api/generate_pdf',
|
| 141 |
+
method: 'POST',
|
| 142 |
+
headers: { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(payload) },
|
| 143 |
+
timeout: 600000,
|
| 144 |
+
};
|
| 145 |
+
|
| 146 |
+
const req = http.request(options, (res) => {
|
| 147 |
+
const chunks = [];
|
| 148 |
+
res.on('data', (chunk) => chunks.push(chunk));
|
| 149 |
+
res.on('end', () => {
|
| 150 |
+
const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
|
| 151 |
+
if (res.statusCode === 200) {
|
| 152 |
+
const pdfSizeMB = (Buffer.concat(chunks).length / 1024 / 1024).toFixed(2);
|
| 153 |
+
resolve({ ok: true, elapsed: parseFloat(elapsed), pdfSizeMB });
|
| 154 |
+
} else {
|
| 155 |
+
const body = Buffer.concat(chunks).toString();
|
| 156 |
+
const errorMatch = body.match(/details":"([^"]+)"/);
|
| 157 |
+
resolve({ ok: false, elapsed: parseFloat(elapsed), error: errorMatch ? errorMatch[1] : body, status: res.statusCode });
|
| 158 |
+
}
|
| 159 |
+
});
|
| 160 |
+
});
|
| 161 |
+
|
| 162 |
+
req.on('error', (e) => reject(e));
|
| 163 |
+
req.on('timeout', () => { req.destroy(); reject(new Error('HTTP timeout')); });
|
| 164 |
+
req.write(payload);
|
| 165 |
+
req.end();
|
| 166 |
+
});
|
| 167 |
+
}
|
| 168 |
+
|
| 169 |
+
async function runBenchmark() {
|
| 170 |
+
console.log('=== PDF Rendering Benchmark ===\n');
|
| 171 |
+
console.log(`Backend: ${BACKEND_URL}\n`);
|
| 172 |
+
|
| 173 |
+
const testCases = [
|
| 174 |
+
{ size: 0.5, shikiRatio: 0, label: '0.5 MB, 纯文本' },
|
| 175 |
+
{ size: 0.5, shikiRatio: 0.5, label: '0.5 MB, 50% Shiki' },
|
| 176 |
+
{ size: 1, shikiRatio: 0, label: '1.0 MB, 纯文本' },
|
| 177 |
+
{ size: 1, shikiRatio: 0.5, label: '1.0 MB, 50% Shiki' },
|
| 178 |
+
{ size: 2, shikiRatio: 0, label: '2.0 MB, 纯文本' },
|
| 179 |
+
{ size: 2, shikiRatio: 0.5, label: '2.0 MB, 50% Shiki' },
|
| 180 |
+
{ size: 3, shikiRatio: 0, label: '3.0 MB, 纯文本' },
|
| 181 |
+
{ size: 3, shikiRatio: 0.5, label: '3.0 MB, 50% Shiki' },
|
| 182 |
+
{ size: 5, shikiRatio: 0, label: '5.0 MB, 纯文本' },
|
| 183 |
+
{ size: 5, shikiRatio: 0.5, label: '5.0 MB, 50% Shiki' },
|
| 184 |
+
];
|
| 185 |
+
|
| 186 |
+
const results = [];
|
| 187 |
+
|
| 188 |
+
for (const tc of testCases) {
|
| 189 |
+
const html = buildHtml(tc.size, { shikiRatio: tc.shikiRatio });
|
| 190 |
+
const actualSizeMB = (Buffer.byteLength(html, 'utf8') / 1024 / 1024).toFixed(2);
|
| 191 |
+
|
| 192 |
+
process.stdout.write(`\n[${tc.label}] HTML=${actualSizeMB} MB ... `);
|
| 193 |
+
|
| 194 |
+
try {
|
| 195 |
+
const result = await sendPdfRequest(html, tc.label);
|
| 196 |
+
results.push({
|
| 197 |
+
label: tc.label,
|
| 198 |
+
htmlSizeMB: parseFloat(actualSizeMB),
|
| 199 |
+
success: result.ok,
|
| 200 |
+
time: result.elapsed,
|
| 201 |
+
pdfSizeMB: result.pdfSizeMB || 'N/A',
|
| 202 |
+
error: result.error || '',
|
| 203 |
+
});
|
| 204 |
+
|
| 205 |
+
if (result.ok) {
|
| 206 |
+
process.stdout.write(`OK ${result.elapsed}s (PDF: ${result.pdfSizeMB} MB)\n`);
|
| 207 |
+
} else {
|
| 208 |
+
process.stdout.write(`FAIL ${result.elapsed}s - ${result.error}\n`);
|
| 209 |
+
}
|
| 210 |
+
} catch (err) {
|
| 211 |
+
process.stdout.write(`ERROR: ${err.message}\n`);
|
| 212 |
+
results.push({ label: tc.label, htmlSizeMB: parseFloat(actualSizeMB), success: false, time: 0, error: err.message });
|
| 213 |
+
}
|
| 214 |
+
}
|
| 215 |
+
|
| 216 |
+
// Summary table
|
| 217 |
+
console.log('\n\n=== Results Summary ===\n');
|
| 218 |
+
console.log('| HTML Size | Content Type | Time (s) | PDF Size | Status |');
|
| 219 |
+
console.log('|-----------|-------------|----------|----------|--------|');
|
| 220 |
+
|
| 221 |
+
for (const r of results) {
|
| 222 |
+
const status = r.success ? 'OK' : `FAIL (${r.error})`;
|
| 223 |
+
console.log(`| ${r.htmlSizeMB} MB | ${r.label.split(', ')[1]} | ${r.time.toFixed(1)}s | ${r.pdfSizeMB || '-'} MB | ${status} |`);
|
| 224 |
+
}
|
| 225 |
+
|
| 226 |
+
// Calculate per-MB render time
|
| 227 |
+
console.log('\n=== Per-MB Analysis ===\n');
|
| 228 |
+
const successResults = results.filter(r => r.success);
|
| 229 |
+
if (successResults.length >= 2) {
|
| 230 |
+
// Linear regression: time = base + rate * size
|
| 231 |
+
const n = successResults.length;
|
| 232 |
+
let sumX = 0, sumY = 0, sumXY = 0, sumX2 = 0;
|
| 233 |
+
for (const r of successResults) {
|
| 234 |
+
sumX += r.htmlSizeMB;
|
| 235 |
+
sumY += r.time;
|
| 236 |
+
sumXY += r.htmlSizeMB * r.time;
|
| 237 |
+
sumX2 += r.htmlSizeMB * r.htmlSizeMB;
|
| 238 |
+
}
|
| 239 |
+
const rate = (n * sumXY - sumX * sumY) / (n * sumX2 - sumX * sumX);
|
| 240 |
+
const base = (sumY - rate * sumX) / n;
|
| 241 |
+
|
| 242 |
+
console.log(`Base overhead (browser launch etc.): ${base.toFixed(1)}s`);
|
| 243 |
+
console.log(`Per-MB PDF render time: ${rate.toFixed(2)}s/MB`);
|
| 244 |
+
console.log('');
|
| 245 |
+
console.log('Recommended timeout formula:');
|
| 246 |
+
console.log(` pdfTimeout = ${Math.ceil(base)}s + sizeMB * ${Math.ceil(rate * 3)}s (3x safety margin)`);
|
| 247 |
+
console.log('');
|
| 248 |
+
|
| 249 |
+
// Predicted timeouts at various sizes
|
| 250 |
+
console.log('Predicted timeouts (with 3x safety margin):');
|
| 251 |
+
for (const size of [1, 2, 3, 5, 7, 10]) {
|
| 252 |
+
const predicted = base + rate * size;
|
| 253 |
+
const withMargin = Math.ceil(base + rate * size * 3);
|
| 254 |
+
console.log(` ${size} MB: actual ~${predicted.toFixed(0)}s, recommended timeout = ${withMargin}s`);
|
| 255 |
+
}
|
| 256 |
+
}
|
| 257 |
+
}
|
| 258 |
+
|
| 259 |
+
runBenchmark();
|
browser-pool.js
DELETED
|
@@ -1,184 +0,0 @@
|
|
| 1 |
-
/**
|
| 2 |
-
* BrowserPool — a small, robust pool of reusable Puppeteer browsers.
|
| 3 |
-
*
|
| 4 |
-
* Why: launching a fresh Chromium per request is the #1 cause of poor
|
| 5 |
-
* PDF-export concurrency (each instance ~0.3–1.5s to start + 200–300MB RAM).
|
| 6 |
-
* This pool keeps `size` browsers warm; a job checks out a browser, does its
|
| 7 |
-
* work, then returns it. Browsers are recycled after `recycleAfter` jobs so a
|
| 8 |
-
* long-lived Chromium never accumulates too much memory.
|
| 9 |
-
*
|
| 10 |
-
* Design notes (industry best practice, verified by web research):
|
| 11 |
-
* - Concurrency = pool size. PDF/page rendering is CPU-bound; running more
|
| 12 |
-
* parallel jobs than the CPU count only degrades latency.
|
| 13 |
-
* - One job per browser at a time (no shared page juggling) keeps isolation
|
| 14 |
-
* and error handling trivial.
|
| 15 |
-
* - Browser crash / disconnect → replaced lazily on next acquire.
|
| 16 |
-
* - Callers must ALWAYS release in a finally block.
|
| 17 |
-
*/
|
| 18 |
-
|
| 19 |
-
const puppeteer = require('puppeteer');
|
| 20 |
-
|
| 21 |
-
const MAX_LAUNCH_FAILURES_PER_SLOT = 3;
|
| 22 |
-
|
| 23 |
-
class BrowserPool {
|
| 24 |
-
/**
|
| 25 |
-
* @param {object} opts
|
| 26 |
-
* @param {string} opts.name label for logs
|
| 27 |
-
* @param {number} opts.size number of browsers to keep
|
| 28 |
-
* @param {object} opts.launchOptions puppeteer.launch() options
|
| 29 |
-
* @param {number} [opts.recycleAfter] jobs per browser before recycle (default 30)
|
| 30 |
-
* @param {number} [opts.acquireTimeoutMs] how long a waiter waits for a free browser (default 120000)
|
| 31 |
-
* @param {Function} [opts.log]
|
| 32 |
-
*/
|
| 33 |
-
constructor(opts) {
|
| 34 |
-
this.name = opts.name || 'pool';
|
| 35 |
-
this.size = Math.max(1, Math.min(16, Math.floor(opts.size) || 1));
|
| 36 |
-
this.launchOptions = opts.launchOptions || {};
|
| 37 |
-
this.recycleAfter = opts.recycleAfter || 30;
|
| 38 |
-
this.acquireTimeoutMs = opts.acquireTimeoutMs || 120000;
|
| 39 |
-
this.log = opts.log || (() => {});
|
| 40 |
-
this._slots = [];
|
| 41 |
-
this._waiters = [];
|
| 42 |
-
this.stats = { acquires: 0, launches: 0, recycles: 0, waits: 0, waitTimeouts: 0, errors: 0 };
|
| 43 |
-
}
|
| 44 |
-
|
| 45 |
-
_log(msg) {
|
| 46 |
-
this.log(`[POOL:${this.name}] ${msg}`);
|
| 47 |
-
}
|
| 48 |
-
|
| 49 |
-
_newSlot() {
|
| 50 |
-
return { browser: null, jobs: 0, available: true, closed: false, launching: false, launchFailures: 0 };
|
| 51 |
-
}
|
| 52 |
-
|
| 53 |
-
async _launch() {
|
| 54 |
-
this.stats.launches++;
|
| 55 |
-
this._log(`launching browser (total launches=${this.stats.launches})`);
|
| 56 |
-
const browser = await puppeteer.launch(this.launchOptions);
|
| 57 |
-
const slot = this._newSlot();
|
| 58 |
-
slot.browser = browser;
|
| 59 |
-
browser.on('disconnected', () => {
|
| 60 |
-
slot.closed = true;
|
| 61 |
-
slot.available = true;
|
| 62 |
-
this._log('browser disconnected (crash/kill); slot marked closed');
|
| 63 |
-
});
|
| 64 |
-
return slot;
|
| 65 |
-
}
|
| 66 |
-
|
| 67 |
-
_pump() {
|
| 68 |
-
// 1. Hand free browsers to waiting jobs.
|
| 69 |
-
while (this._waiters.length > 0) {
|
| 70 |
-
const slot = this._slots.find((s) => s.available && !s.closed && s.browser);
|
| 71 |
-
if (!slot) break;
|
| 72 |
-
const waiter = this._waiters.shift();
|
| 73 |
-
clearTimeout(waiter.timer);
|
| 74 |
-
slot.available = false;
|
| 75 |
-
// IMPORTANT: resolve with the wrapper ({browser, release}), same shape
|
| 76 |
-
// as the warm-path return in acquire(). Resolving with the raw slot made
|
| 77 |
-
// callers' `acquired.release` undefined → slots never released → pool
|
| 78 |
-
// deadlocked (3/4 busy forever).
|
| 79 |
-
waiter.resolve(this._wrap(slot));
|
| 80 |
-
}
|
| 81 |
-
// 2. Grow the pool (launch one browser per pump pass).
|
| 82 |
-
if (this._waiters.length > 0 && this._slots.length < this.size) {
|
| 83 |
-
const empty = this._slots.find((s) => !s.browser && !s.launching);
|
| 84 |
-
if (empty) {
|
| 85 |
-
empty.launching = true;
|
| 86 |
-
this._launch().then((slot) => {
|
| 87 |
-
const i = this._slots.indexOf(empty);
|
| 88 |
-
if (i === -1) { slot.browser.close().catch(() => {}); return; }
|
| 89 |
-
this._slots[i] = slot;
|
| 90 |
-
this._pump();
|
| 91 |
-
}).catch((err) => {
|
| 92 |
-
this.stats.errors++;
|
| 93 |
-
this._log(`browser launch failed: ${err.message}`);
|
| 94 |
-
empty.launchFailures++;
|
| 95 |
-
const i = this._slots.indexOf(empty);
|
| 96 |
-
if (i !== -1) {
|
| 97 |
-
if (empty.launchFailures >= MAX_LAUNCH_FAILURES_PER_SLOT) {
|
| 98 |
-
this._slots.splice(i, 1);
|
| 99 |
-
const waiter = this._waiters.shift();
|
| 100 |
-
if (waiter) {
|
| 101 |
-
clearTimeout(waiter.timer);
|
| 102 |
-
waiter.reject(new Error(`[POOL:${this.name}] browser launch failed: ${err.message}`));
|
| 103 |
-
}
|
| 104 |
-
} else {
|
| 105 |
-
empty.launching = false; // allow retry
|
| 106 |
-
}
|
| 107 |
-
}
|
| 108 |
-
this._pump();
|
| 109 |
-
});
|
| 110 |
-
}
|
| 111 |
-
}
|
| 112 |
-
}
|
| 113 |
-
|
| 114 |
-
/**
|
| 115 |
-
* Check out a browser slot for one job.
|
| 116 |
-
* @returns {Promise<{browser: object, release: Function}>}
|
| 117 |
-
*/
|
| 118 |
-
async acquire() {
|
| 119 |
-
this.stats.acquires++;
|
| 120 |
-
const warm = this._slots.find((s) => s.available && !s.closed && s.browser);
|
| 121 |
-
if (warm) {
|
| 122 |
-
warm.available = false;
|
| 123 |
-
this._log(`acquire: warm slot (inUse=${this._inUse()}/${this.size})`);
|
| 124 |
-
return this._wrap(warm);
|
| 125 |
-
}
|
| 126 |
-
// Reserve capacity to grow the pool.
|
| 127 |
-
if (this._slots.length < this.size) {
|
| 128 |
-
this._slots.push(this._newSlot());
|
| 129 |
-
this._pump();
|
| 130 |
-
}
|
| 131 |
-
this.stats.waits++;
|
| 132 |
-
this._log(`acquire: no free slot, queued (inUse=${this._inUse()}/${this.size})`);
|
| 133 |
-
return new Promise((resolve, reject) => {
|
| 134 |
-
const timer = setTimeout(() => {
|
| 135 |
-
const i = this._waiters.indexOf(waiter);
|
| 136 |
-
if (i !== -1) this._waiters.splice(i, 1);
|
| 137 |
-
this.stats.waitTimeouts++;
|
| 138 |
-
this._log(`acquire timed out after ${this.acquireTimeoutMs}ms`);
|
| 139 |
-
reject(new Error(`[POOL:${this.name}] no free browser within ${this.acquireTimeoutMs}ms (busy=${this._inUse()}/${this.size})`));
|
| 140 |
-
}, this.acquireTimeoutMs);
|
| 141 |
-
const waiter = { resolve, reject, timer };
|
| 142 |
-
this._waiters.push(waiter);
|
| 143 |
-
this._pump();
|
| 144 |
-
});
|
| 145 |
-
}
|
| 146 |
-
|
| 147 |
-
_inUse() {
|
| 148 |
-
return this._slots.filter((s) => !s.available).length;
|
| 149 |
-
}
|
| 150 |
-
|
| 151 |
-
_wrap(slot) {
|
| 152 |
-
return {
|
| 153 |
-
browser: slot.browser,
|
| 154 |
-
release: async () => {
|
| 155 |
-
slot.jobs++;
|
| 156 |
-
if (slot.closed || slot.jobs >= this.recycleAfter) {
|
| 157 |
-
this.stats.recycles++;
|
| 158 |
-
this._log(`recycling browser after ${slot.jobs} jobs (recycles=${this.stats.recycles})`);
|
| 159 |
-
const i = this._slots.indexOf(slot);
|
| 160 |
-
if (i !== -1) this._slots.splice(i, 1);
|
| 161 |
-
try { await slot.browser.close(); } catch (e) {}
|
| 162 |
-
slot.closed = true;
|
| 163 |
-
} else {
|
| 164 |
-
slot.available = true;
|
| 165 |
-
this._log(`released browser (jobs=${slot.jobs}, inUse=${this._inUse()}/${this.size})`);
|
| 166 |
-
}
|
| 167 |
-
this._pump();
|
| 168 |
-
},
|
| 169 |
-
};
|
| 170 |
-
}
|
| 171 |
-
|
| 172 |
-
async close() {
|
| 173 |
-
const slots = this._slots.splice(0);
|
| 174 |
-
for (const s of slots) {
|
| 175 |
-
if (s.browser) { try { await s.browser.close(); } catch (e) {} }
|
| 176 |
-
}
|
| 177 |
-
for (const w of this._waiters.splice(0)) {
|
| 178 |
-
clearTimeout(w.timer);
|
| 179 |
-
w.reject(new Error(`[POOL:${this.name}] pool closed`));
|
| 180 |
-
}
|
| 181 |
-
}
|
| 182 |
-
}
|
| 183 |
-
|
| 184 |
-
module.exports = { BrowserPool };
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
docker-compose.yml
CHANGED
|
@@ -1,3 +1,5 @@
|
|
|
|
|
|
|
|
| 1 |
services:
|
| 2 |
pdf-prod:
|
| 3 |
build: .
|
|
@@ -8,10 +10,6 @@ services:
|
|
| 8 |
environment:
|
| 9 |
- NODE_ENV=production
|
| 10 |
- PORT=7860
|
| 11 |
-
# Hugging Face free tier = 2 vCPU / 16GB → PDF browser pool of 2.
|
| 12 |
-
# page.pdf() is CPU-bound; pool size = max concurrent PDF jobs.
|
| 13 |
-
- PDF_POOL_SIZE=2
|
| 14 |
-
- WIDGET_MAX_CONCURRENT=3
|
| 15 |
logging:
|
| 16 |
driver: "json-file"
|
| 17 |
options:
|
|
@@ -23,17 +21,12 @@ services:
|
|
| 23 |
container_name: pdf-test
|
| 24 |
restart: always
|
| 25 |
ports:
|
| 26 |
-
- "
|
| 27 |
environment:
|
| 28 |
- NODE_ENV=test
|
| 29 |
- PORT=7860
|
| 30 |
-
# Local test machine: multi-core → larger pool for finding max capacity.
|
| 31 |
-
# Tune with: docker compose up -d --build pdf-test (edit here or override)
|
| 32 |
-
- PDF_POOL_SIZE=4
|
| 33 |
-
- WIDGET_MAX_CONCURRENT=3
|
| 34 |
logging:
|
| 35 |
driver: "json-file"
|
| 36 |
options:
|
| 37 |
max-size: "10m"
|
| 38 |
max-file: "3"
|
| 39 |
-
|
|
|
|
| 1 |
+
version: '3.8'
|
| 2 |
+
|
| 3 |
services:
|
| 4 |
pdf-prod:
|
| 5 |
build: .
|
|
|
|
| 10 |
environment:
|
| 11 |
- NODE_ENV=production
|
| 12 |
- PORT=7860
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
logging:
|
| 14 |
driver: "json-file"
|
| 15 |
options:
|
|
|
|
| 21 |
container_name: pdf-test
|
| 22 |
restart: always
|
| 23 |
ports:
|
| 24 |
+
- "7861:7860"
|
| 25 |
environment:
|
| 26 |
- NODE_ENV=test
|
| 27 |
- PORT=7860
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
logging:
|
| 29 |
driver: "json-file"
|
| 30 |
options:
|
| 31 |
max-size: "10m"
|
| 32 |
max-file: "3"
|
|
|
make/backup-source.js
DELETED
|
@@ -1,64 +0,0 @@
|
|
| 1 |
-
const fs = require('fs');
|
| 2 |
-
const path = require('path');
|
| 3 |
-
|
| 4 |
-
const BACKUP_BASE = path.resolve(__dirname, '..', '..');
|
| 5 |
-
const SRC_DIR = path.resolve(__dirname, '..');
|
| 6 |
-
|
| 7 |
-
const EXCLUDE_DIRS = ['node_modules', '.git', '.claude', 'dist'];
|
| 8 |
-
|
| 9 |
-
function shouldExclude(full, srcDir) {
|
| 10 |
-
const relative = path.relative(srcDir, full).replace(/\\/g, '/');
|
| 11 |
-
const basename = path.basename(full);
|
| 12 |
-
for (const d of EXCLUDE_DIRS) {
|
| 13 |
-
if (relative === d || relative.startsWith(d + '/')) return true;
|
| 14 |
-
}
|
| 15 |
-
return false;
|
| 16 |
-
}
|
| 17 |
-
|
| 18 |
-
function collectFiles(srcDir) {
|
| 19 |
-
const files = [];
|
| 20 |
-
function walk(dir) {
|
| 21 |
-
const entries = fs.readdirSync(dir);
|
| 22 |
-
for (const entry of entries) {
|
| 23 |
-
const full = path.join(dir, entry);
|
| 24 |
-
if (shouldExclude(full, srcDir)) continue;
|
| 25 |
-
const stat = fs.statSync(full);
|
| 26 |
-
if (stat.isDirectory()) walk(full);
|
| 27 |
-
else files.push(full);
|
| 28 |
-
}
|
| 29 |
-
}
|
| 30 |
-
walk(srcDir);
|
| 31 |
-
return files;
|
| 32 |
-
}
|
| 33 |
-
|
| 34 |
-
function copyFiles(files, srcDir, destDir) {
|
| 35 |
-
for (const full of files) {
|
| 36 |
-
const relative = path.relative(srcDir, full);
|
| 37 |
-
const dest = path.join(destDir, relative);
|
| 38 |
-
fs.mkdirSync(path.dirname(dest), { recursive: true });
|
| 39 |
-
fs.copyFileSync(full, dest);
|
| 40 |
-
}
|
| 41 |
-
}
|
| 42 |
-
|
| 43 |
-
const pkgPath = path.join(SRC_DIR, 'package.json');
|
| 44 |
-
const pkg = JSON.parse(fs.readFileSync(pkgPath, 'utf-8'));
|
| 45 |
-
const version = pkg.version || 'unknown';
|
| 46 |
-
|
| 47 |
-
const now = new Date();
|
| 48 |
-
const ts = now.toISOString().replace(/[:.]/g, '-').slice(0, 19);
|
| 49 |
-
|
| 50 |
-
const folderName = 'backend-service_v' + version + '_' + ts;
|
| 51 |
-
const backupPath = path.join(BACKUP_BASE, folderName);
|
| 52 |
-
|
| 53 |
-
console.log('Version: ' + version);
|
| 54 |
-
console.log('Creating backup: ' + backupPath);
|
| 55 |
-
|
| 56 |
-
const files = collectFiles(SRC_DIR);
|
| 57 |
-
console.log('Found ' + files.length + ' source files');
|
| 58 |
-
|
| 59 |
-
copyFiles(files, SRC_DIR, backupPath);
|
| 60 |
-
console.log('Backup complete: ' + backupPath);
|
| 61 |
-
|
| 62 |
-
const serverJs = fs.readFileSync(path.join(backupPath, 'server.js'), 'utf-8');
|
| 63 |
-
const hasNewColors = serverJs.includes('#339CFF');
|
| 64 |
-
console.log('New ChatGPT colors in backup: ' + (hasNewColors ? 'YES' : 'NO'));
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
package-lock.json
CHANGED
|
@@ -1,12 +1,12 @@
|
|
| 1 |
{
|
| 2 |
"name": "pdf-server",
|
| 3 |
-
"version": "2.
|
| 4 |
"lockfileVersion": 3,
|
| 5 |
"requires": true,
|
| 6 |
"packages": {
|
| 7 |
"": {
|
| 8 |
"name": "pdf-server",
|
| 9 |
-
"version": "2.
|
| 10 |
"dependencies": {
|
| 11 |
"@mermaid-js/mermaid-cli": "^11.0.0",
|
| 12 |
"chart.js": "^4.4.8",
|
|
|
|
| 1 |
{
|
| 2 |
"name": "pdf-server",
|
| 3 |
+
"version": "2.0.0",
|
| 4 |
"lockfileVersion": 3,
|
| 5 |
"requires": true,
|
| 6 |
"packages": {
|
| 7 |
"": {
|
| 8 |
"name": "pdf-server",
|
| 9 |
+
"version": "2.0.0",
|
| 10 |
"dependencies": {
|
| 11 |
"@mermaid-js/mermaid-cli": "^11.0.0",
|
| 12 |
"chart.js": "^4.4.8",
|
package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
{
|
| 2 |
"name": "pdf-server",
|
| 3 |
-
"version": "2.
|
| 4 |
"description": "Puppeteer PDF + Widget Renderer for XWX AI Chat Exporter",
|
| 5 |
"main": "server.js",
|
| 6 |
"dependencies": {
|
|
|
|
| 1 |
{
|
| 2 |
"name": "pdf-server",
|
| 3 |
+
"version": "2.0.7",
|
| 4 |
"description": "Puppeteer PDF + Widget Renderer for XWX AI Chat Exporter",
|
| 5 |
"main": "server.js",
|
| 6 |
"dependencies": {
|
reproduce-realistic.js
ADDED
|
@@ -0,0 +1,844 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**
|
| 2 |
+
* 真实复现 PDF 生成超时 bug — v2
|
| 3 |
+
*
|
| 4 |
+
* 用法: node reproduce-realistic.js
|
| 5 |
+
*
|
| 6 |
+
* 基于真实超长对话测试用例构造:
|
| 7 |
+
* - 完整 CSS (buildDocumentCss 输出)
|
| 8 |
+
* - 真实消息结构 (avatar, bubble, timestamp)
|
| 9 |
+
* - Shiki 高亮后的代码块 (每个 token 带内联颜色 <span>)
|
| 10 |
+
* - Markdown 渲染后的 HTML (p, strong, em, ul, blockquote, table 等)
|
| 11 |
+
*
|
| 12 |
+
* 参考: [gemini]★超长对话_2026-05-26-11-43-53_api.json
|
| 13 |
+
* 102 条原始消息, 3.74 MB JSON, 估算转换后 HTML 约 5-8 MB
|
| 14 |
+
*/
|
| 15 |
+
|
| 16 |
+
const http = require('http');
|
| 17 |
+
const fs = require('fs');
|
| 18 |
+
const path = require('path');
|
| 19 |
+
|
| 20 |
+
const BACKEND_PORT = process.env.PORT || 7861;
|
| 21 |
+
const BACKEND_URL = `http://localhost:${BACKEND_PORT}`;
|
| 22 |
+
|
| 23 |
+
// ─── 完整 CSS (从 buildDocumentCss 提取) ───
|
| 24 |
+
const REAL_CSS = `
|
| 25 |
+
@media print { @page { size: A4; margin: 15mm 10mm; } body { -webkit-print-color-adjust: exact; print-color-adjust: exact; } }
|
| 26 |
+
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif; font-size: 14px; line-height: 1.6; color: #333; max-width: 746px; margin: 0 auto; padding: 20px; background: #fff; }
|
| 27 |
+
h1 { font-size: 24px; font-weight: 600; margin: 24px 0 12px; padding-bottom: 8px; border-bottom: 2px solid #eee; color: #1a1a1a; }
|
| 28 |
+
h2 { font-size: 20px; font-weight: 600; margin: 20px 0 10px; color: #1a1a1a; }
|
| 29 |
+
h3 { font-size: 17px; font-weight: 600; margin: 16px 0 8px; color: #1a1a1a; }
|
| 30 |
+
h4 { font-size: 15px; font-weight: 600; margin: 12px 0 6px; }
|
| 31 |
+
p { margin: 8px 0; }
|
| 32 |
+
a { color: #0366d6; text-decoration: none; }
|
| 33 |
+
a:hover { text-decoration: underline; }
|
| 34 |
+
strong { font-weight: 600; }
|
| 35 |
+
em { font-style: italic; }
|
| 36 |
+
code { font-family: 'SFMono-Regular', Consolas, 'Liberation Mono', Menlo, monospace; font-size: 0.9em; background: rgba(27,31,35,0.05); padding: 0.2em 0.4em; border-radius: 3px; }
|
| 37 |
+
pre { background: #f6f8fa; padding: 16px; border-radius: 6px; overflow-x: auto; line-height: 1.45; margin: 12px 0; border: 1px solid #e1e4e8; }
|
| 38 |
+
pre code { background: none; padding: 0; font-size: 13px; color: #24292e; }
|
| 39 |
+
pre[data-language]::before { content: attr(data-language); display: block; font-size: 11px; color: #999; margin-bottom: 4px; text-transform: uppercase; letter-spacing: 0.5px; }
|
| 40 |
+
blockquote { margin: 12px 0; padding: 8px 16px; border-left: 4px solid #dfe2e5; color: #6a737d; background: #f9f9f9; }
|
| 41 |
+
ul, ol { padding-left: 2em; margin: 8px 0; }
|
| 42 |
+
li { margin: 4px 0; }
|
| 43 |
+
table { border-collapse: collapse; width: 100%; margin: 12px 0; overflow-x: auto; display: block; }
|
| 44 |
+
th, td { border: 1px solid #dfe2e5; padding: 8px 12px; text-align: left; }
|
| 45 |
+
th { background: #f1f3f4; font-weight: 600; }
|
| 46 |
+
tr:nth-child(even) { background: #fafbfc; }
|
| 47 |
+
hr { border: none; border-top: 1px solid #eee; margin: 24px 0; }
|
| 48 |
+
img { max-width: 100%; height: auto; margin: 8px 0; border-radius: 4px; }
|
| 49 |
+
.chat-container { display: flex; flex-direction: column; gap: 16px; }
|
| 50 |
+
.message-row { display: flex; align-items: flex-start; gap: 10px; }
|
| 51 |
+
.user-row { flex-direction: row-reverse; }
|
| 52 |
+
.avatar { flex-shrink: 0; width: 32px; height: 32px; border-radius: 50%; display: flex; align-items: center; justify-content: center; font-size: 16px; margin-top: 4px; }
|
| 53 |
+
.user-avatar { background: #e8f0fe; }
|
| 54 |
+
.ai-avatar { background: #fce8e6; }
|
| 55 |
+
.message-bubble { max-width: 85%; padding: 12px 16px; border-radius: 12px; }
|
| 56 |
+
.user-bubble { background: #e8f0fe; color: #1f1f1f; border-bottom-right-radius: 4px; }
|
| 57 |
+
.ai-bubble { background: #ffffff; color: #333; border: 1px solid #eee; border-bottom-left-radius: 4px; }
|
| 58 |
+
.message-time { font-size: 11px; color: #999; margin-bottom: 4px; }
|
| 59 |
+
.user-message-time { font-size: 11px; color: #999; text-align: right; margin-bottom: 4px; }
|
| 60 |
+
.thinking-block { margin: 8px 0; }
|
| 61 |
+
.thinking-header { font-size: 12px; color: #999; margin-bottom: 4px; font-style: italic; }
|
| 62 |
+
.thinking-content { font-size: 13px; color: #666; padding: 8px; background: #f9f9f9; border-radius: 4px; border-left: 3px solid #ddd; }
|
| 63 |
+
.main-title { text-align: center; margin-bottom: 32px; }
|
| 64 |
+
.source-link { font-size: 12px; color: #999; }
|
| 65 |
+
.pdf-footer { margin-top: 40px; padding-top: 16px; border-top: 1px solid #eee; font-size: 12px; color: #999; text-align: center; }
|
| 66 |
+
.pdf-footer a { color: #0366d6; }
|
| 67 |
+
.toc { background: #f9f9f9; padding: 16px; border-radius: 8px; margin: 16px 0; border: 1px solid #eee; }
|
| 68 |
+
.toc h3 { margin: 0 0 8px; font-size: 16px; }
|
| 69 |
+
.toc ul { list-style: none; padding: 0; margin: 0; }
|
| 70 |
+
.toc li { margin: 4px 0; }
|
| 71 |
+
.toc a { color: #0366d6; text-decoration: none; }
|
| 72 |
+
.toc .toc-h2 { padding-left: 20px; }
|
| 73 |
+
.toc .toc-h3 { padding-left: 40px; }
|
| 74 |
+
.c-purple { fill: #EEEDFE; stroke: #534AB7; color: #3C3489; }
|
| 75 |
+
.c-teal { fill: #E1F5EE; stroke: #0F6E56; color: #085041; }
|
| 76 |
+
.c-coral { fill: #FAECE7; stroke: #993C1D; color: #712B13; }
|
| 77 |
+
.c-pink { fill: #FBEAF0; stroke: #993556; color: #72243E; }
|
| 78 |
+
.c-gray { fill: #F1EFE8; stroke: #5F5E5A; color: #444441; }
|
| 79 |
+
.c-blue { fill: #E6F1FB; stroke: #185FA5; color: #0C447C; }
|
| 80 |
+
.c-green { fill: #EAF3DE; stroke: #3B6D11; color: #27500A; }
|
| 81 |
+
.c-amber { fill: #FAEEDA; stroke: #854F0B; color: #633806; }
|
| 82 |
+
.c-red { fill: #FCEBEB; stroke: #A32D2D; color: #791F1F; }
|
| 83 |
+
.th, text.th { font-weight: 500; fill: #212121; }
|
| 84 |
+
.ts, text.ts { font-size: 12px; fill: #6b7280; }
|
| 85 |
+
.t, text.t { font-size: 14px; fill: #212121; }
|
| 86 |
+
svg .arr { fill: none; stroke: #888780; stroke-width: 1.5; }
|
| 87 |
+
svg .leader { fill: none; stroke: #888780; stroke-width: 0.5; stroke-dasharray: 2 2; }
|
| 88 |
+
svg .box { fill: #f9f9f9; stroke: #e5e5e5; }
|
| 89 |
+
`;
|
| 90 |
+
|
| 91 |
+
// ─── Shiki github-dark token colors ───
|
| 92 |
+
const KW = '#ff7b72'; // keywords (const, let, function, return, if, for, async, await, etc.)
|
| 93 |
+
const OBJ = '#d2a8ff'; // objects/classes (Promise, Math, console, etc.)
|
| 94 |
+
const FN = '#d2a8ff'; // function names
|
| 95 |
+
const STR = '#a5d6ff'; // strings
|
| 96 |
+
const NUM = '#79c0ff'; // numbers
|
| 97 |
+
const CMT = '#8b949e'; // comments
|
| 98 |
+
const PCT = '#c9d1d9'; // punctuation
|
| 99 |
+
const PLAIN = '#c9d1d9'; // plain text
|
| 100 |
+
const BOOL = '#79c0ff'; // true, false, null, undefined
|
| 101 |
+
|
| 102 |
+
function E(s) { return s.replace(/&/g,'&').replace(/</g,'<').replace(/>/g,'>').replace(/"/g,'"'); }
|
| 103 |
+
|
| 104 |
+
// Simulate Shiki: wrap each token in a colored span
|
| 105 |
+
function shikiColorCode(code) {
|
| 106 |
+
return code.split('\n').map(line => {
|
| 107 |
+
// Handle comments
|
| 108 |
+
const ci = line.indexOf('//');
|
| 109 |
+
const mainPart = ci >= 0 ? line.substring(0, ci) : line;
|
| 110 |
+
const commentPart = ci >= 0 ? line.substring(ci) : '';
|
| 111 |
+
|
| 112 |
+
let result = '';
|
| 113 |
+
// Tokenize
|
| 114 |
+
const tokens = mainPart.split(/(\s+|[{}()\[\];,\.:=+\-*/<>!&|?%@~^'"`])/g);
|
| 115 |
+
for (const tok of tokens) {
|
| 116 |
+
if (!tok) continue;
|
| 117 |
+
if (/^\s+$/.test(tok)) { result += tok; continue; }
|
| 118 |
+
// Keywords
|
| 119 |
+
if (/^(const|let|var|function|return|if|else|for|while|do|switch|case|break|continue|new|class|extends|import|from|export|default|async|await|try|catch|finally|throw|typeof|instanceof|void|this|yield|of|in|static|get|set|super)$/i.test(tok)) {
|
| 120 |
+
result += `<span style="color:${KW}">${E(tok)}</span>`;
|
| 121 |
+
} else if (/^(true|false|null|undefined|NaN|Infinity)$/i.test(tok)) {
|
| 122 |
+
result += `<span style="color:${BOOL}">${E(tok)}</span>`;
|
| 123 |
+
} else if (/^(console|log|error|warn|info|debug|Promise|resolve|reject|Math|Date|JSON|Buffer|require|module|exports|process|setTimeout|setInterval|clearTimeout|clearInterval|fetch|AbortSignal|AbortController|Map|Set|Array|Object|String|Number|Boolean|Error|TypeError|RangeError|SyntaxError)$/i.test(tok)) {
|
| 124 |
+
result += `<span style="color:${OBJ}">${E(tok)}</span>`;
|
| 125 |
+
} else if (/^\d+\.?\d*$/.test(tok)) {
|
| 126 |
+
result += `<span style="color:${NUM}">${tok}</span>`;
|
| 127 |
+
} else if (/^[{}()\[\];,\.:=+\-*/<>!&|?%@~^'"`]+$/.test(tok)) {
|
| 128 |
+
result += `<span style="color:${PCT}">${E(tok)}</span>`;
|
| 129 |
+
} else {
|
| 130 |
+
result += `<span style="color:${PLAIN}">${E(tok)}</span>`;
|
| 131 |
+
}
|
| 132 |
+
}
|
| 133 |
+
if (commentPart) {
|
| 134 |
+
result += `<span style="color:${CMT}">${E(commentPart)}</span>`;
|
| 135 |
+
}
|
| 136 |
+
return result;
|
| 137 |
+
}).join('\n');
|
| 138 |
+
}
|
| 139 |
+
|
| 140 |
+
// String-highlight helper for inline strings in code
|
| 141 |
+
function colorStrings(code) {
|
| 142 |
+
// Simple: wrap content between quotes in STR color
|
| 143 |
+
return code.replace(/(['"`])(.*?)\1/g, `<span style="color:${STR}">$1$2$1</span>`);
|
| 144 |
+
}
|
| 145 |
+
|
| 146 |
+
// ─── Real conversation content samples ───
|
| 147 |
+
|
| 148 |
+
const CODE_SAMPLES = [
|
| 149 |
+
// JavaScript async/await
|
| 150 |
+
`async function fetchData(url, options = {}) {
|
| 151 |
+
const controller = new AbortController();
|
| 152 |
+
const timeout = setTimeout(() => controller.abort(), 30000);
|
| 153 |
+
try {
|
| 154 |
+
const response = await fetch(url, {
|
| 155 |
+
...options,
|
| 156 |
+
signal: controller.signal,
|
| 157 |
+
headers: { 'Content-Type': 'application/json' },
|
| 158 |
+
});
|
| 159 |
+
if (!response.ok) {
|
| 160 |
+
throw new Error(\`HTTP \${response.status}: \${response.statusText}\`);
|
| 161 |
+
}
|
| 162 |
+
const data = await response.json();
|
| 163 |
+
console.log('Data received:', data);
|
| 164 |
+
return data;
|
| 165 |
+
} catch (error) {
|
| 166 |
+
if (error.name === 'AbortError') {
|
| 167 |
+
console.error('Request timed out');
|
| 168 |
+
} else {
|
| 169 |
+
console.error('Fetch failed:', error.message);
|
| 170 |
+
}
|
| 171 |
+
throw error;
|
| 172 |
+
} finally {
|
| 173 |
+
clearTimeout(timeout);
|
| 174 |
+
}
|
| 175 |
+
}`,
|
| 176 |
+
|
| 177 |
+
// TypeScript class
|
| 178 |
+
`interface CacheEntry<T> {
|
| 179 |
+
value: T;
|
| 180 |
+
expiresAt: number;
|
| 181 |
+
accessCount: number;
|
| 182 |
+
}
|
| 183 |
+
|
| 184 |
+
class SmartCache<T> {
|
| 185 |
+
private cache: Map<string, CacheEntry<T>> = new Map();
|
| 186 |
+
private defaultTTL: number;
|
| 187 |
+
private maxSize: number;
|
| 188 |
+
|
| 189 |
+
constructor(defaultTTL: number = 300000, maxSize: number = 1000) {
|
| 190 |
+
this.defaultTTL = defaultTTL;
|
| 191 |
+
this.maxSize = maxSize;
|
| 192 |
+
}
|
| 193 |
+
|
| 194 |
+
set(key: string, value: T, ttl?: number): void {
|
| 195 |
+
if (this.cache.size >= this.maxSize) {
|
| 196 |
+
this.evictOldest();
|
| 197 |
+
}
|
| 198 |
+
const expiresAt = Date.now() + (ttl ?? this.defaultTTL);
|
| 199 |
+
this.cache.set(key, { value, expiresAt, accessCount: 0 });
|
| 200 |
+
}
|
| 201 |
+
|
| 202 |
+
get(key: string): T | undefined {
|
| 203 |
+
const entry = this.cache.get(key);
|
| 204 |
+
if (!entry) return undefined;
|
| 205 |
+
if (Date.now() > entry.expiresAt) {
|
| 206 |
+
this.cache.delete(key);
|
| 207 |
+
return undefined;
|
| 208 |
+
}
|
| 209 |
+
entry.accessCount++;
|
| 210 |
+
return entry.value;
|
| 211 |
+
}
|
| 212 |
+
|
| 213 |
+
private evictOldest(): void {
|
| 214 |
+
let oldestKey: string | null = null;
|
| 215 |
+
let oldestTime = Infinity;
|
| 216 |
+
for (const [key, entry] of this.cache) {
|
| 217 |
+
if (entry.expiresAt < oldestTime) {
|
| 218 |
+
oldestTime = entry.expiresAt;
|
| 219 |
+
oldestKey = key;
|
| 220 |
+
}
|
| 221 |
+
}
|
| 222 |
+
if (oldestKey) this.cache.delete(oldestKey);
|
| 223 |
+
}
|
| 224 |
+
|
| 225 |
+
clear(): void { this.cache.clear(); }
|
| 226 |
+
get size(): number { return this.cache.size; }
|
| 227 |
+
}`,
|
| 228 |
+
|
| 229 |
+
// Python data pipeline
|
| 230 |
+
`import asyncio
|
| 231 |
+
import aiohttp
|
| 232 |
+
from typing import List, Dict, Any, Optional
|
| 233 |
+
from dataclasses import dataclass, field
|
| 234 |
+
from datetime import datetime, timedelta
|
| 235 |
+
import logging
|
| 236 |
+
|
| 237 |
+
logger = logging.getLogger(__name__)
|
| 238 |
+
|
| 239 |
+
@dataclass
|
| 240 |
+
class DataPoint:
|
| 241 |
+
timestamp: datetime
|
| 242 |
+
value: float
|
| 243 |
+
source: str
|
| 244 |
+
metadata: Dict[str, Any] = field(default_factory=dict)
|
| 245 |
+
|
| 246 |
+
@dataclass
|
| 247 |
+
class PipelineConfig:
|
| 248 |
+
max_concurrent: int = 10
|
| 249 |
+
timeout_seconds: float = 30.0
|
| 250 |
+
retry_count: int = 3
|
| 251 |
+
batch_size: int = 100
|
| 252 |
+
retry_delay: float = 1.0
|
| 253 |
+
|
| 254 |
+
class DataPipeline:
|
| 255 |
+
def __init__(self, config: Optional[PipelineConfig] = None):
|
| 256 |
+
self.config = config or PipelineConfig()
|
| 257 |
+
self.semaphore = asyncio.Semaphore(self.config.max_concurrent)
|
| 258 |
+
self.results: List[DataPoint] = []
|
| 259 |
+
self._session: Optional[aiohttp.ClientSession] = None
|
| 260 |
+
|
| 261 |
+
async def fetch_data(
|
| 262 |
+
self,
|
| 263 |
+
session: aiohttp.ClientSession,
|
| 264 |
+
url: str,
|
| 265 |
+
params: Optional[Dict[str, Any]] = None
|
| 266 |
+
) -> Dict[str, Any]:
|
| 267 |
+
async with self.semaphore:
|
| 268 |
+
for attempt in range(self.config.retry_count):
|
| 269 |
+
try:
|
| 270 |
+
async with session.get(
|
| 271 |
+
url,
|
| 272 |
+
params=params,
|
| 273 |
+
timeout=aiohttp.ClientTimeout(total=self.config.timeout_seconds)
|
| 274 |
+
) as response:
|
| 275 |
+
response.raise_for_status()
|
| 276 |
+
return await response.json()
|
| 277 |
+
except (aiohttp.ClientError, asyncio.TimeoutError) as e:
|
| 278 |
+
if attempt == self.config.retry_count - 1:
|
| 279 |
+
logger.error(f"Failed after {self.config.retry_count} retries: {e}")
|
| 280 |
+
raise
|
| 281 |
+
await asyncio.sleep(self.config.retry_delay * (attempt + 1))
|
| 282 |
+
|
| 283 |
+
async def process_batch(self, urls: List[str]) -> List[DataPoint]:
|
| 284 |
+
async with aiohttp.ClientSession() as session:
|
| 285 |
+
tasks = [self.fetch_data(session, url) for url in urls]
|
| 286 |
+
raw_results = await asyncio.gather(*tasks, return_exceptions=True)
|
| 287 |
+
|
| 288 |
+
points = []
|
| 289 |
+
for i, result in enumerate(raw_results):
|
| 290 |
+
if isinstance(result, Exception):
|
| 291 |
+
logger.error(f"URL {urls[i]} failed: {result}")
|
| 292 |
+
continue
|
| 293 |
+
points.append(DataPoint(
|
| 294 |
+
timestamp=datetime.utcnow(),
|
| 295 |
+
value=result.get("value", 0.0),
|
| 296 |
+
source=result.get("source", "unknown"),
|
| 297 |
+
metadata={"url": urls[i], "index": i}
|
| 298 |
+
))
|
| 299 |
+
|
| 300 |
+
self.results.extend(points)
|
| 301 |
+
return points
|
| 302 |
+
|
| 303 |
+
def get_statistics(self) -> Dict[str, Any]:
|
| 304 |
+
if not self.results:
|
| 305 |
+
return {"count": 0, "avg": 0.0, "min": 0.0, "max": 0.0}
|
| 306 |
+
values = [p.value for p in self.results]
|
| 307 |
+
return {
|
| 308 |
+
"count": len(values),
|
| 309 |
+
"avg": sum(values) / len(values),
|
| 310 |
+
"min": min(values),
|
| 311 |
+
"max": max(values),
|
| 312 |
+
"total": sum(values),
|
| 313 |
+
}`,
|
| 314 |
+
|
| 315 |
+
// Rust
|
| 316 |
+
`use std::collections::HashMap;
|
| 317 |
+
use std::sync::{Arc, Mutex};
|
| 318 |
+
use tokio::sync::Semaphore;
|
| 319 |
+
use std::sync::atomic::{AtomicUsize, Ordering};
|
| 320 |
+
|
| 321 |
+
#[derive(Debug, Clone)]
|
| 322 |
+
struct CacheEntry<T> {
|
| 323 |
+
value: T,
|
| 324 |
+
expires_at: std::time::Instant,
|
| 325 |
+
access_count: AtomicUsize,
|
| 326 |
+
}
|
| 327 |
+
|
| 328 |
+
struct SmartCache<T: Clone + Send + Sync> {
|
| 329 |
+
entries: Arc<Mutex<HashMap<String, CacheEntry<T>>>>,
|
| 330 |
+
default_ttl: std::time::Duration,
|
| 331 |
+
max_size: usize,
|
| 332 |
+
}
|
| 333 |
+
|
| 334 |
+
impl<T: Clone + Send + Sync + 'static> SmartCache<T> {
|
| 335 |
+
fn new(default_ttl: std::time::Duration, max_size: usize) -> Self {
|
| 336 |
+
Self {
|
| 337 |
+
entries: Arc::new(Mutex::new(HashMap::with_capacity(max_size))),
|
| 338 |
+
default_ttl,
|
| 339 |
+
max_size,
|
| 340 |
+
}
|
| 341 |
+
}
|
| 342 |
+
|
| 343 |
+
fn set(&self, key: String, value: T) {
|
| 344 |
+
let mut entries = self.entries.lock().unwrap();
|
| 345 |
+
if entries.len() >= self.max_size {
|
| 346 |
+
// Evict the least recently accessed entry
|
| 347 |
+
let lru_key = entries
|
| 348 |
+
.iter()
|
| 349 |
+
.min_by_key(|(_, entry)| entry.access_count.load(Ordering::Relaxed))
|
| 350 |
+
.map(|(key, _)| key.clone());
|
| 351 |
+
if let Some(lru_key) = lru_key {
|
| 352 |
+
entries.remove(&lru_key);
|
| 353 |
+
}
|
| 354 |
+
}
|
| 355 |
+
entries.insert(key, CacheEntry {
|
| 356 |
+
value,
|
| 357 |
+
expires_at: std::time::Instant::now() + self.default_ttl,
|
| 358 |
+
access_count: AtomicUsize::new(0),
|
| 359 |
+
});
|
| 360 |
+
}
|
| 361 |
+
|
| 362 |
+
fn get(&self, key: &str) -> Option<T> {
|
| 363 |
+
let entries = self.entries.lock().unwrap();
|
| 364 |
+
entries.get(key).and_then(|entry| {
|
| 365 |
+
if std::time::Instant::now() > entry.expires_at {
|
| 366 |
+
None
|
| 367 |
+
} else {
|
| 368 |
+
entry.access_count.fetch_add(1, Ordering::Relaxed);
|
| 369 |
+
Some(entry.value.clone())
|
| 370 |
+
}
|
| 371 |
+
})
|
| 372 |
+
}
|
| 373 |
+
}`,
|
| 374 |
+
|
| 375 |
+
// Go
|
| 376 |
+
`package main
|
| 377 |
+
|
| 378 |
+
import (
|
| 379 |
+
"context"
|
| 380 |
+
"encoding/json"
|
| 381 |
+
"fmt"
|
| 382 |
+
"log"
|
| 383 |
+
"net/http"
|
| 384 |
+
"sync"
|
| 385 |
+
"time"
|
| 386 |
+
)
|
| 387 |
+
|
| 388 |
+
type CacheEntry struct {
|
| 389 |
+
Value interface{}
|
| 390 |
+
ExpiresAt time.Time
|
| 391 |
+
AccessCount int
|
| 392 |
+
}
|
| 393 |
+
|
| 394 |
+
type SmartCache struct {
|
| 395 |
+
mu sync.RWMutex
|
| 396 |
+
entries map[string]*CacheEntry
|
| 397 |
+
defaultTTL time.Duration
|
| 398 |
+
maxSize int
|
| 399 |
+
}
|
| 400 |
+
|
| 401 |
+
func NewSmartCache(defaultTTL time.Duration, maxSize int) *SmartCache {
|
| 402 |
+
return &SmartCache{
|
| 403 |
+
entries: make(map[string]*CacheEntry, maxSize),
|
| 404 |
+
defaultTTL: defaultTTL,
|
| 405 |
+
maxSize: maxSize,
|
| 406 |
+
}
|
| 407 |
+
}
|
| 408 |
+
|
| 409 |
+
func (c *SmartCache) Set(key string, value interface{}) {
|
| 410 |
+
c.mu.Lock()
|
| 411 |
+
defer c.mu.Unlock()
|
| 412 |
+
|
| 413 |
+
if len(c.entries) >= c.maxSize {
|
| 414 |
+
c.evictOldest()
|
| 415 |
+
}
|
| 416 |
+
|
| 417 |
+
c.entries[key] = &CacheEntry{
|
| 418 |
+
Value: value,
|
| 419 |
+
ExpiresAt: time.Now().Add(c.defaultTTL),
|
| 420 |
+
}
|
| 421 |
+
}
|
| 422 |
+
|
| 423 |
+
func (c *SmartCache) Get(key string) (interface{}, bool) {
|
| 424 |
+
c.mu.RLock()
|
| 425 |
+
defer c.mu.RUnlock()
|
| 426 |
+
|
| 427 |
+
entry, exists := c.entries[key]
|
| 428 |
+
if !exists {
|
| 429 |
+
return nil, false
|
| 430 |
+
}
|
| 431 |
+
|
| 432 |
+
if time.Now().After(entry.ExpiresAt) {
|
| 433 |
+
return nil, false
|
| 434 |
+
}
|
| 435 |
+
|
| 436 |
+
entry.AccessCount++
|
| 437 |
+
return entry.Value, true
|
| 438 |
+
}
|
| 439 |
+
|
| 440 |
+
func (c *SmartCache) evictOldest() {
|
| 441 |
+
var oldestKey string
|
| 442 |
+
oldestTime := time.Now().Add(time.Hour)
|
| 443 |
+
|
| 444 |
+
for key, entry := range c.entries {
|
| 445 |
+
if entry.ExpiresAt.Before(oldestTime) {
|
| 446 |
+
oldestTime = entry.ExpiresAt
|
| 447 |
+
oldestKey = key
|
| 448 |
+
}
|
| 449 |
+
}
|
| 450 |
+
|
| 451 |
+
if oldestKey != "" {
|
| 452 |
+
delete(c.entries, oldestKey)
|
| 453 |
+
}
|
| 454 |
+
}
|
| 455 |
+
|
| 456 |
+
func fetchWithRetry(ctx context.Context, url string, maxRetries int) ([]byte, error) {
|
| 457 |
+
client := &http.Client{Timeout: 30 * time.Second}
|
| 458 |
+
var lastErr error
|
| 459 |
+
|
| 460 |
+
for i := 0; i < maxRetries; i++ {
|
| 461 |
+
req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
|
| 462 |
+
if err != nil {
|
| 463 |
+
return nil, fmt.Errorf("creating request: %w", err)
|
| 464 |
+
}
|
| 465 |
+
|
| 466 |
+
resp, err := client.Do(req)
|
| 467 |
+
if err == nil && resp.StatusCode == http.StatusOK {
|
| 468 |
+
defer resp.Body.Close()
|
| 469 |
+
var result map[string]interface{}
|
| 470 |
+
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
|
| 471 |
+
return nil, fmt.Errorf("decoding JSON: %w", err)
|
| 472 |
+
}
|
| 473 |
+
return json.Marshal(result)
|
| 474 |
+
}
|
| 475 |
+
|
| 476 |
+
lastErr = err
|
| 477 |
+
if resp != nil {
|
| 478 |
+
lastErr = fmt.Errorf("HTTP %d", resp.StatusCode)
|
| 479 |
+
resp.Body.Close()
|
| 480 |
+
}
|
| 481 |
+
|
| 482 |
+
log.Printf("Attempt %d failed: %v, retrying...", i+1, lastErr)
|
| 483 |
+
time.Sleep(time.Duration(i+1) * time.Second)
|
| 484 |
+
}
|
| 485 |
+
|
| 486 |
+
return nil, fmt.Errorf("after %d retries: %w", maxRetries, lastErr)
|
| 487 |
+
}`,
|
| 488 |
+
];
|
| 489 |
+
|
| 490 |
+
const EXPLANATION_TEXTS = [
|
| 491 |
+
`这是一个非常好的问题！让我详细解释一下。
|
| 492 |
+
|
| 493 |
+
首先，我们需要理解这个问题的核心概念。**异步编程**是现代 JavaScript 中非常重要的一部分，尤其是在处理网络请求、文件 I/O 和定时器等操作时。
|
| 494 |
+
|
| 495 |
+
### 关键点说明
|
| 496 |
+
|
| 497 |
+
1. **async/await 语法** — 让异步代码看起来像同步代码，提高了可读性
|
| 498 |
+
2. **错误处理** — 使用 try/catch 块捕获异步操作中的异常
|
| 499 |
+
3. **Promise 链式调用** — \`.then()\`/\`.catch()\` 是传统的 Promise 处理方式
|
| 500 |
+
|
| 501 |
+
> 注意：在实际生产环境中，建议添加重试机制和超时处理。
|
| 502 |
+
|
| 503 |
+
还有一些其他需要考虑的因素：
|
| 504 |
+
|
| 505 |
+
- 网络延迟和超时设置
|
| 506 |
+
- 并发请求的限制
|
| 507 |
+
- 内存泄漏的预防
|
| 508 |
+
- 错误恢复策略`,
|
| 509 |
+
|
| 510 |
+
`好的，让我为你提供一个完整的解决方案。
|
| 511 |
+
|
| 512 |
+
在实际项目中，我们经常会遇到性能瓶颈。下面是一个经过优化的实现：
|
| 513 |
+
|
| 514 |
+
### 架构设计
|
| 515 |
+
|
| 516 |
+
这个方案有几个关键优势：
|
| 517 |
+
|
| 518 |
+
| 特性 | 描述 |
|
| 519 |
+
|------|------|
|
| 520 |
+
| 类型安全 | 完整的 TypeScript 类型注解 |
|
| 521 |
+
| 自动过期 | 基于 TTL 的自动清理机制 |
|
| 522 |
+
| 容量控制 | 最大条目数限制 + LRU 淘汰 |
|
| 523 |
+
| 线程安全 | 适合并发环境使用 |
|
| 524 |
+
|
| 525 |
+
### 性能对比
|
| 526 |
+
|
| 527 |
+
| 方法 | 响应时间 | 内存占用 |
|
| 528 |
+
|------|---------|---------|
|
| 529 |
+
| 直接查询 | ~200ms | 低 |
|
| 530 |
+
| 缓存查询 | ~2ms | 中 |
|
| 531 |
+
| 带过期缓存 | ~2ms | 可控 |
|
| 532 |
+
|
| 533 |
+
如果你需要处理更大规模的数据，可以考虑：
|
| 534 |
+
- 使用消息队列（RabbitMQ/Kafka）
|
| 535 |
+
- 引入分布式缓存（Redis）
|
| 536 |
+
- 采用微服务架构拆分模块`,
|
| 537 |
+
|
| 538 |
+
`让我用一个更贴近实际的例子来说明。
|
| 539 |
+
|
| 540 |
+
假设我们正在构建一个 **实时数据处理管道**：
|
| 541 |
+
|
| 542 |
+
### 设计思路
|
| 543 |
+
|
| 544 |
+
这个设计模式有几个关键优势：
|
| 545 |
+
|
| 546 |
+
1. **并发控制** — 通过 Semaphore 限制最大并发数，防止服务器过载
|
| 547 |
+
2. **批量处理** — 使用 \`asyncio.gather\` 并行处理多个请求
|
| 548 |
+
3. **错误隔离** — \`return_exceptions=True\` 确保单个失败不影响��体
|
| 549 |
+
4. **类型提示** — 完整的类型注解提高代码可维护性
|
| 550 |
+
|
| 551 |
+
### 扩展建议
|
| 552 |
+
|
| 553 |
+
- 添加监控和告警机制
|
| 554 |
+
- 实现优雅关闭（graceful shutdown）
|
| 555 |
+
- 支持动态配置热加载
|
| 556 |
+
- 集成链路追踪（OpenTelemetry）
|
| 557 |
+
|
| 558 |
+
---
|
| 559 |
+
|
| 560 |
+
希望这个方案对你有帮助。如果需要进一步讨论，随时告诉我！`,
|
| 561 |
+
|
| 562 |
+
`我理解你的困惑。让我们从基础开始，一步步来分析。
|
| 563 |
+
|
| 564 |
+
### 什么是闭包？
|
| 565 |
+
|
| 566 |
+
闭包（Closure）是一个函数和其词法环境的组合。简单来说，就是函数可以记住并访问它被创建时所在的词法作用域。
|
| 567 |
+
|
| 568 |
+
### 实际应用场景
|
| 569 |
+
|
| 570 |
+
闭包在前端开发中有广泛应用：
|
| 571 |
+
|
| 572 |
+
| 场景 | 示例 |
|
| 573 |
+
|------|------|
|
| 574 |
+
| 数据封装 | 私有变量和计数器 |
|
| 575 |
+
| 事件处理 | 记住事件触发时的状态 |
|
| 576 |
+
| 函数工厂 | 根据参数生成不同行为的函数 |
|
| 577 |
+
| 模块模式 | 创建私有变量和方法 |
|
| 578 |
+
|
| 579 |
+
---
|
| 580 |
+
|
| 581 |
+
希望这个解释对你有帮助！`,
|
| 582 |
+
];
|
| 583 |
+
|
| 584 |
+
const USER_QUERIES = [
|
| 585 |
+
'请问 JavaScript 中 async/await 和 Promise 有什么区别？在实际项目中应该怎么选择？能给出一些具体的例子吗？',
|
| 586 |
+
'我想要实现一个带过期时间的缓存，有什么好的方案吗？最好是用 TypeScript，需要支持最大容量限制和自动淘汰。',
|
| 587 |
+
'能帮我看看这段 Python 代码的性能瓶颈在哪里？我需要处理大量并发请求，每个请求都有 30 秒超时限制。',
|
| 588 |
+
'我在学习 JavaScript 闭包，能举个实际的例子说明它的用途吗？最好能解释一下为什么需要闭包。',
|
| 589 |
+
];
|
| 590 |
+
|
| 591 |
+
function markdownToHtml(text) {
|
| 592 |
+
const lines = text.split('\n');
|
| 593 |
+
const html = [];
|
| 594 |
+
let inUl = false;
|
| 595 |
+
let inParagraph = false;
|
| 596 |
+
|
| 597 |
+
for (const line of lines) {
|
| 598 |
+
if (line.trim() === '') {
|
| 599 |
+
if (inUl) { html.push('</ul>'); inUl = false; }
|
| 600 |
+
if (inParagraph) { html.push('</p>'); inParagraph = false; }
|
| 601 |
+
continue;
|
| 602 |
+
}
|
| 603 |
+
if (line.startsWith('### ')) {
|
| 604 |
+
if (inUl) { html.push('</ul>'); inUl = false; }
|
| 605 |
+
if (inParagraph) { html.push('</p>'); inParagraph = false; }
|
| 606 |
+
html.push(`<h3>${inlineFormat(line.substring(4))}</h3>`);
|
| 607 |
+
} else if (line.startsWith('## ')) {
|
| 608 |
+
if (inUl) { html.push('</ul>'); inUl = false; }
|
| 609 |
+
if (inParagraph) { html.push('</p>'); inParagraph = false; }
|
| 610 |
+
html.push(`<h2>${inlineFormat(line.substring(3))}</h2>`);
|
| 611 |
+
} else if (line.startsWith('> ')) {
|
| 612 |
+
if (inUl) { html.push('</ul>'); inUl = false; }
|
| 613 |
+
if (inParagraph) { html.push('</p>'); inParagraph = false; }
|
| 614 |
+
html.push(`<blockquote><p>${inlineFormat(line.substring(2))}</p></blockquote>`);
|
| 615 |
+
} else if (line.startsWith('- ')) {
|
| 616 |
+
if (!inUl) { html.push('<ul>'); inUl = true; }
|
| 617 |
+
html.push(`<li>${inlineFormat(line.substring(2))}</li>`);
|
| 618 |
+
} else if (line.startsWith('|')) {
|
| 619 |
+
// Skip table rows - handled separately
|
| 620 |
+
continue;
|
| 621 |
+
} else if (line.startsWith('---')) {
|
| 622 |
+
if (inUl) { html.push('</ul>'); inUl = false; }
|
| 623 |
+
if (inParagraph) { html.push('</p>'); inParagraph = false; }
|
| 624 |
+
html.push('<hr>');
|
| 625 |
+
} else {
|
| 626 |
+
if (inUl) { html.push('</ul>'); inUl = false; }
|
| 627 |
+
if (!inParagraph) { html.push('<p>'); inParagraph = true; }
|
| 628 |
+
html.push(inlineFormat(line));
|
| 629 |
+
html.push('<br>');
|
| 630 |
+
}
|
| 631 |
+
}
|
| 632 |
+
if (inUl) html.push('</ul>');
|
| 633 |
+
if (inParagraph) html.push('</p>');
|
| 634 |
+
return html.join('\n');
|
| 635 |
+
}
|
| 636 |
+
|
| 637 |
+
function inlineFormat(text) {
|
| 638 |
+
return text
|
| 639 |
+
.replace(/\*\*(.*?)\*\*/g, '<strong>$1</strong>')
|
| 640 |
+
.replace(/\*(.*?)\*/g, '<em>$1</em>')
|
| 641 |
+
.replace(/`([^`]+)`/g, '<code>$1</code>')
|
| 642 |
+
.replace(/\|([^|]+)\|([^|]+)\|/g, (m, col1, col2) => {
|
| 643 |
+
if (col1.trim() === '---' || col2.trim() === '---') return '';
|
| 644 |
+
return m;
|
| 645 |
+
});
|
| 646 |
+
}
|
| 647 |
+
|
| 648 |
+
function generateMessageHTML(userQuery, aiExplanation, codeSample, msgIndex) {
|
| 649 |
+
const idx = msgIndex * 2;
|
| 650 |
+
const ts1 = new Date(Date.now() - (50 - msgIndex) * 3600000).toISOString().replace('T', ' ').substring(0, 19);
|
| 651 |
+
const ts2 = new Date(Date.now() - (50 - msgIndex) * 3600000 + 5000).toISOString().replace('T', ' ').substring(0, 19);
|
| 652 |
+
|
| 653 |
+
// User message
|
| 654 |
+
const userHtml = `
|
| 655 |
+
<div id="msg-${idx}" class="message-row user-row">
|
| 656 |
+
<div class="user-message-time"><div class="message-time">${ts1}</div></div>
|
| 657 |
+
<div class="message-bubble user-bubble">
|
| 658 |
+
<div class="message-time">${ts1}</div>
|
| 659 |
+
<p>${E(userQuery)}</p>
|
| 660 |
+
</div>
|
| 661 |
+
<div class="avatar user-avatar">👤</div>
|
| 662 |
+
</div>`;
|
| 663 |
+
|
| 664 |
+
// AI message with Shiki-colored code
|
| 665 |
+
const shikiCode = colorStrings(shikiColorCode(codeSample));
|
| 666 |
+
const explanationHtml = markdownToHtml(aiExplanation);
|
| 667 |
+
|
| 668 |
+
const aiHtml = `
|
| 669 |
+
<div id="msg-${idx + 1}" class="message-row ai-row">
|
| 670 |
+
<div class="avatar ai-avatar">🤖</div>
|
| 671 |
+
<div class="message-bubble ai-bubble">
|
| 672 |
+
<div class="message-time">${ts2}</div>
|
| 673 |
+
${explanationHtml}
|
| 674 |
+
<pre data-language="javascript"><code class="language-javascript">${shikiCode}</code></pre>
|
| 675 |
+
</div>
|
| 676 |
+
</div>`;
|
| 677 |
+
|
| 678 |
+
return userHtml + aiHtml;
|
| 679 |
+
}
|
| 680 |
+
|
| 681 |
+
function generateTableHtml() {
|
| 682 |
+
return `
|
| 683 |
+
<table>
|
| 684 |
+
<thead><tr><th>指标</th><th>方案 A</th><th>方案 B</th><th>方案 C</th></tr></thead>
|
| 685 |
+
<tbody>
|
| 686 |
+
<tr><td>响应时间</td><td>~200ms</td><td>~50ms</td><td>~5ms</td></tr>
|
| 687 |
+
<tr><td>吞吐量</td><td>100 req/s</td><td>500 req/s</td><td>2000 req/s</td></tr>
|
| 688 |
+
<tr><td>内存占用</td><td>低</td><td>中</td><td>高</td></tr>
|
| 689 |
+
<tr><td>可扩展性</td><td>一般</td><td>好</td><td>优秀</td></tr>
|
| 690 |
+
<tr><td>维护成本</td><td>低</td><td>中</td><td>高</td></tr>
|
| 691 |
+
</tbody>
|
| 692 |
+
</table>`;
|
| 693 |
+
}
|
| 694 |
+
|
| 695 |
+
function generateLargeHtml(targetSizeMB) {
|
| 696 |
+
const headHtml = \`<!DOCTYPE html>
|
| 697 |
+
<html>
|
| 698 |
+
<head>
|
| 699 |
+
<meta charset="UTF-8">
|
| 700 |
+
<meta name="color-scheme" content="light">
|
| 701 |
+
<title>Gemini Chat Export</title>
|
| 702 |
+
<style>\${REAL_CSS}</style>
|
| 703 |
+
</head>
|
| 704 |
+
<body>
|
| 705 |
+
<h1 class="main-title">Gemini Chat Export</h1>
|
| 706 |
+
<p class="source-link">Source: <a href="https://gemini.google.com">https://gemini.google.com</a></p>
|
| 707 |
+
<br>
|
| 708 |
+
<nav class="toc">
|
| 709 |
+
<h3>Table of Contents</h3>
|
| 710 |
+
<ul>
|
| 711 |
+
<li class="toc-h2"><a href="#msg-1">Async/Await vs Promise</a></li>
|
| 712 |
+
<li class="toc-h2"><a href="#msg-3">Smart Cache Implementation</a></li>
|
| 713 |
+
<li class="toc-h2"><a href="#msg-5">Python Data Pipeline</a></li>
|
| 714 |
+
<li class="toc-h2"><a href="#msg-7">JavaScript Closures</a></li>
|
| 715 |
+
</ul>
|
| 716 |
+
</nav>
|
| 717 |
+
<hr style="border:0;border-top:1px dashed #ccc;margin:40px 0;">
|
| 718 |
+
<div class="chat-container">\`;
|
| 719 |
+
|
| 720 |
+
const footerHtml = \` </div>
|
| 721 |
+
<div class="pdf-footer"><span>Powered by <a href="https://xwxexporter.com/tools/chat-exporter" target="_blank">XWX AI Chat Exporter</a></span></div>
|
| 722 |
+
</body>
|
| 723 |
+
</html>\`;
|
| 724 |
+
|
| 725 |
+
const overhead = Buffer.byteLength(headHtml, 'utf8') + Buffer.byteLength(footerHtml, 'utf8');
|
| 726 |
+
const targetContent = targetSizeMB * 1024 * 1024 - overhead;
|
| 727 |
+
|
| 728 |
+
// Build one "round" of conversation: 4 Q&A pairs + table
|
| 729 |
+
let oneRound = '';
|
| 730 |
+
for (let i = 0; i < 4; i++) {
|
| 731 |
+
oneRound += generateMessageHTML(USER_QUERIES[i], EXPLANATION_TEXTS[i], CODE_SAMPLES[i], i);
|
| 732 |
+
}
|
| 733 |
+
oneRound += generateTableHtml();
|
| 734 |
+
|
| 735 |
+
const roundSize = Buffer.byteLength(oneRound, 'utf8');
|
| 736 |
+
const iterations = Math.ceil(targetContent / roundSize);
|
| 737 |
+
|
| 738 |
+
process.stdout.write(\`每轮对话大小: \${(roundSize / 1024).toFixed(1)} KB, 需要 \${iterations} 轮达到 ~\${targetSizeMB} MB\`);
|
| 739 |
+
|
| 740 |
+
let bodyHtml = '';
|
| 741 |
+
for (let i = 0; i < iterations; i++) {
|
| 742 |
+
bodyHtml += oneRound.replace(/id="msg-(\\d+)"/g, (m, n) => \`id="msg-\${parseInt(n) + i * 10}"\`);
|
| 743 |
+
if ((i + 1) % 10 === 0) process.stdout.write('.');
|
| 744 |
+
}
|
| 745 |
+
process.stdout.write('\\n');
|
| 746 |
+
|
| 747 |
+
const fullHtml = headHtml + bodyHtml + footerHtml;
|
| 748 |
+
const actualSizeMB = (Buffer.byteLength(fullHtml, 'utf8') / 1024 / 1024).toFixed(2);
|
| 749 |
+
console.log(\`生成 HTML 大小: \${actualSizeMB} MB\`);
|
| 750 |
+
return fullHtml;
|
| 751 |
+
}
|
| 752 |
+
|
| 753 |
+
async function sendPdfRequest(html) {
|
| 754 |
+
const payload = JSON.stringify({
|
| 755 |
+
html,
|
| 756 |
+
codeTheme: 'github',
|
| 757 |
+
showWatermark: false,
|
| 758 |
+
imageCount: 0,
|
| 759 |
+
totalImageSizeMB: 0,
|
| 760 |
+
platform: 'Gemini',
|
| 761 |
+
language: 'en-US',
|
| 762 |
+
extensionVersion: '2.0.2',
|
| 763 |
+
exportCount: 18,
|
| 764 |
+
exportPdf: 3,
|
| 765 |
+
exportMd: 7,
|
| 766 |
+
exportTxt: 0,
|
| 767 |
+
exportDocx: 8,
|
| 768 |
+
exportJson: 0,
|
| 769 |
+
exportClipboard: 0,
|
| 770 |
+
exportNotion: 0,
|
| 771 |
+
});
|
| 772 |
+
|
| 773 |
+
console.log(\`\\n发送请求 payload 大小: \${(Buffer.byteLength(payload) / 1024 / 1024).toFixed(2)} MB\`);
|
| 774 |
+
const startTime = Date.now();
|
| 775 |
+
console.log(\`开始时间: \${new Date().toISOString()}\`);
|
| 776 |
+
|
| 777 |
+
return new Promise((resolve, reject) => {
|
| 778 |
+
const url = new URL('/api/generate_pdf', BACKEND_URL);
|
| 779 |
+
const options = {
|
| 780 |
+
hostname: url.hostname,
|
| 781 |
+
port: url.port,
|
| 782 |
+
path: url.pathname,
|
| 783 |
+
method: 'POST',
|
| 784 |
+
headers: {
|
| 785 |
+
'Content-Type': 'application/json',
|
| 786 |
+
'Content-Length': Buffer.byteLength(payload),
|
| 787 |
+
},
|
| 788 |
+
timeout: 600000,
|
| 789 |
+
};
|
| 790 |
+
|
| 791 |
+
const req = http.request(options, (res) => {
|
| 792 |
+
const chunks = [];
|
| 793 |
+
res.on('data', (chunk) => chunks.push(chunk));
|
| 794 |
+
res.on('end', () => {
|
| 795 |
+
const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
|
| 796 |
+
console.log(\`\\n结束时间: \${new Date().toISOString()}\`);
|
| 797 |
+
console.log(\`总耗时: \${elapsed}s\`);
|
| 798 |
+
console.log(\`HTTP 状态码: \${res.statusCode}\`);
|
| 799 |
+
|
| 800 |
+
if (res.statusCode === 200) {
|
| 801 |
+
const pdfSizeMB = (Buffer.concat(chunks).length / 1024 / 1024).toFixed(2);
|
| 802 |
+
console.log(\`PDF 文件大小: \${pdfSizeMB} MB\`);
|
| 803 |
+
resolve({ status: 200, pdfSizeMB });
|
| 804 |
+
} else {
|
| 805 |
+
const body = Buffer.concat(chunks).toString();
|
| 806 |
+
console.log(\`错误响应: \${body}\`);
|
| 807 |
+
resolve({ status: res.statusCode, error: body });
|
| 808 |
+
}
|
| 809 |
+
});
|
| 810 |
+
});
|
| 811 |
+
|
| 812 |
+
req.on('error', (e) => reject(e));
|
| 813 |
+
req.on('timeout', () => {
|
| 814 |
+
req.destroy();
|
| 815 |
+
reject(new Error(\`请求超时 (\${options.timeout}ms)\`));
|
| 816 |
+
});
|
| 817 |
+
|
| 818 |
+
req.write(payload);
|
| 819 |
+
req.end();
|
| 820 |
+
});
|
| 821 |
+
}
|
| 822 |
+
|
| 823 |
+
async function main() {
|
| 824 |
+
console.log('=== PDF 超��问题复现（真实 HTML 结构 v2）===\\n');
|
| 825 |
+
console.log(\`后端地址: \${BACKEND_URL} (端口 \${BACKEND_PORT})\`);
|
| 826 |
+
console.log(\`PORT=7860 → 生产环境, PORT=7861(默认) → 测试环境\`);
|
| 827 |
+
|
| 828 |
+
const html = generateLargeHtml(7);
|
| 829 |
+
|
| 830 |
+
try {
|
| 831 |
+
console.log('\\n正在发送 PDF 生成请求...');
|
| 832 |
+
const result = await sendPdfRequest(html);
|
| 833 |
+
|
| 834 |
+
if (result.status === 200) {
|
| 835 |
+
console.log('\\n✅ PDF 生成成功');
|
| 836 |
+
} else {
|
| 837 |
+
console.log(\`\\n❌ PDF 生成失败: HTTP \${result.status}\`);
|
| 838 |
+
}
|
| 839 |
+
} catch (err) {
|
| 840 |
+
console.error(\`\\n❌ 请求异常: \${err.message}\`);
|
| 841 |
+
}
|
| 842 |
+
}
|
| 843 |
+
|
| 844 |
+
main();
|
reproduce-timeout.js
ADDED
|
@@ -0,0 +1,159 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/**
|
| 2 |
+
* 复现 PDF 生成超时 bug
|
| 3 |
+
*
|
| 4 |
+
* 用法: node reproduce-timeout.js
|
| 5 |
+
*
|
| 6 |
+
* 原理: 生成一个 ~7MB 的纯文本/代码 HTML（不含图片），
|
| 7 |
+
* 发送给后端 /api/generate_pdf，观察是否在 page.pdf() 阶段超时 120s。
|
| 8 |
+
*/
|
| 9 |
+
|
| 10 |
+
const http = require('http');
|
| 11 |
+
|
| 12 |
+
// 端口: 测试环境 = 7861, 生产环境 = 7860
|
| 13 |
+
const BACKEND_PORT = process.env.PORT || 7861;
|
| 14 |
+
const BACKEND_URL = `http://localhost:${BACKEND_PORT}`;
|
| 15 |
+
|
| 16 |
+
// 生成指定大小的重复文本块
|
| 17 |
+
function generateLargeHtml(targetSizeMB) {
|
| 18 |
+
// 模拟 Gemini 对话：大量代码块 + 长文本
|
| 19 |
+
const codeBlock = `<pre><code class="language-javascript">function fibonacci(n) {
|
| 20 |
+
if (n <= 1) return n;
|
| 21 |
+
return fibonacci(n - 1) + fibonacci(n - 2);
|
| 22 |
+
}
|
| 23 |
+
|
| 24 |
+
// 这是一个很长的代码注释，用来增加 HTML 体积
|
| 25 |
+
// Lorem ipsum dolor sit amet, consectetur adipiscing elit.
|
| 26 |
+
// Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
|
| 27 |
+
// Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris.
|
| 28 |
+
const result = [];
|
| 29 |
+
for (let i = 0; i < 100; i++) {
|
| 30 |
+
result.push(fibonacci(i));
|
| 31 |
+
}
|
| 32 |
+
console.log(result);
|
| 33 |
+
</code></pre>`;
|
| 34 |
+
|
| 35 |
+
const messageBlock = `<div class="message"><p>这是一段很长的对话内容，用来模拟真实场景下的文本体积。在实际使用中，Gemini 用户可能会产生非常长的对话，包含大量代码块、解释文本、数学公式等。这些内容累积起来会形成非常大的 HTML 文档。</p><p>Additional text to simulate real conversation output from AI assistants. The more verbose the responses, the larger the HTML becomes. This is especially common with Gemini which tends to produce lengthy, detailed answers.</p></div>`;
|
| 36 |
+
|
| 37 |
+
let html = `<!DOCTYPE html><html><head><meta charset="utf-8"><style>
|
| 38 |
+
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; max-width: 800px; margin: 0 auto; padding: 20px; line-height: 1.6; }
|
| 39 |
+
pre { background: #f6f8fa; padding: 16px; border-radius: 6px; overflow-x: auto; }
|
| 40 |
+
code { font-family: 'SFMono-Regular', Consolas, 'Liberation Mono', Menlo, monospace; font-size: 14px; }
|
| 41 |
+
.message { margin-bottom: 24px; padding: 12px; border-bottom: 1px solid #eee; }
|
| 42 |
+
</style></head><body>`;
|
| 43 |
+
|
| 44 |
+
const chunk = codeBlock + messageBlock;
|
| 45 |
+
const chunkSize = Buffer.byteLength(chunk, 'utf8');
|
| 46 |
+
const targetBytes = targetSizeMB * 1024 * 1024;
|
| 47 |
+
const iterations = Math.ceil(targetBytes / chunkSize);
|
| 48 |
+
|
| 49 |
+
console.log(`每块大小: ${(chunkSize / 1024).toFixed(1)} KB, 需要重复 ${iterations} 次达到 ${targetSizeMB} MB`);
|
| 50 |
+
|
| 51 |
+
for (let i = 0; i < iterations; i++) {
|
| 52 |
+
html += `<div class="message-block">${chunk}</div>`;
|
| 53 |
+
if ((i + 1) % 50 === 0) {
|
| 54 |
+
process.stdout.write('.');
|
| 55 |
+
}
|
| 56 |
+
}
|
| 57 |
+
|
| 58 |
+
html += '</body></html>';
|
| 59 |
+
|
| 60 |
+
const actualSizeMB = (Buffer.byteLength(html, 'utf8') / 1024 / 1024).toFixed(2);
|
| 61 |
+
console.log(`\n生成 HTML 大小: ${actualSizeMB} MB`);
|
| 62 |
+
return html;
|
| 63 |
+
}
|
| 64 |
+
|
| 65 |
+
async function sendPdfRequest(html) {
|
| 66 |
+
const payload = JSON.stringify({
|
| 67 |
+
html,
|
| 68 |
+
codeTheme: 'github',
|
| 69 |
+
showWatermark: false,
|
| 70 |
+
imageCount: 0, // 关键: 0 张图片
|
| 71 |
+
totalImageSizeMB: 0, // 关键: 0 MB 图片
|
| 72 |
+
platform: 'Gemini',
|
| 73 |
+
language: 'en-US',
|
| 74 |
+
extensionVersion: '2.0.2',
|
| 75 |
+
exportCount: 18,
|
| 76 |
+
exportPdf: 3,
|
| 77 |
+
exportMd: 7,
|
| 78 |
+
exportTxt: 0,
|
| 79 |
+
exportDocx: 8,
|
| 80 |
+
exportJson: 0,
|
| 81 |
+
exportClipboard: 0,
|
| 82 |
+
exportNotion: 0,
|
| 83 |
+
});
|
| 84 |
+
|
| 85 |
+
console.log(`\n发送请求 payload 大小: ${(Buffer.byteLength(payload) / 1024 / 1024).toFixed(2)} MB`);
|
| 86 |
+
console.log(`开始时间: ${new Date().toISOString()}`);
|
| 87 |
+
|
| 88 |
+
return new Promise((resolve, reject) => {
|
| 89 |
+
const url = new URL('/api/generate_pdf', BACKEND_URL);
|
| 90 |
+
const options = {
|
| 91 |
+
hostname: url.hostname,
|
| 92 |
+
port: url.port,
|
| 93 |
+
path: url.pathname,
|
| 94 |
+
method: 'POST',
|
| 95 |
+
headers: {
|
| 96 |
+
'Content-Type': 'application/json',
|
| 97 |
+
'Content-Length': Buffer.byteLength(payload),
|
| 98 |
+
},
|
| 99 |
+
// PDF 生成可能很慢，node HTTP client 默认 timeout 较短
|
| 100 |
+
timeout: 600000, // 10 min
|
| 101 |
+
};
|
| 102 |
+
|
| 103 |
+
const startTime = Date.now();
|
| 104 |
+
|
| 105 |
+
const req = http.request(options, (res) => {
|
| 106 |
+
const chunks = [];
|
| 107 |
+
res.on('data', (chunk) => chunks.push(chunk));
|
| 108 |
+
res.on('end', () => {
|
| 109 |
+
const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
|
| 110 |
+
console.log(`\n结束时间: ${new Date().toISOString()}`);
|
| 111 |
+
console.log(`总耗时: ${elapsed}s`);
|
| 112 |
+
console.log(`HTTP 状态码: ${res.statusCode}`);
|
| 113 |
+
|
| 114 |
+
if (res.statusCode === 200) {
|
| 115 |
+
const pdfSizeMB = (Buffer.concat(chunks).length / 1024 / 1024).toFixed(2);
|
| 116 |
+
console.log(`PDF 文件大小: ${pdfSizeMB} MB`);
|
| 117 |
+
resolve({ status: 200, pdfSizeMB });
|
| 118 |
+
} else {
|
| 119 |
+
const body = Buffer.concat(chunks).toString();
|
| 120 |
+
console.log(`错误响应: ${body}`);
|
| 121 |
+
resolve({ status: res.statusCode, error: body });
|
| 122 |
+
}
|
| 123 |
+
});
|
| 124 |
+
});
|
| 125 |
+
|
| 126 |
+
req.on('error', (e) => reject(e));
|
| 127 |
+
req.on('timeout', () => {
|
| 128 |
+
req.destroy();
|
| 129 |
+
reject(new Error(`请求超时 (${options.timeout}ms)`));
|
| 130 |
+
});
|
| 131 |
+
|
| 132 |
+
req.write(payload);
|
| 133 |
+
req.end();
|
| 134 |
+
});
|
| 135 |
+
}
|
| 136 |
+
|
| 137 |
+
async function main() {
|
| 138 |
+
console.log('=== PDF 超时问题复现脚本 ===\n');
|
| 139 |
+
console.log(`后端地址: ${BACKEND_URL} (端口 ${BACKEND_PORT})`);
|
| 140 |
+
console.log(`环境变量 PORT=7860 → 生产环境, PORT=7861(默认) → 测试环境`);
|
| 141 |
+
|
| 142 |
+
// 生成 ~7MB HTML（对齐日志中的 6.89 MB）
|
| 143 |
+
const html = generateLargeHtml(7);
|
| 144 |
+
|
| 145 |
+
try {
|
| 146 |
+
console.log('\n正在发送 PDF 生成请求...');
|
| 147 |
+
const result = await sendPdfRequest(html);
|
| 148 |
+
|
| 149 |
+
if (result.status === 200) {
|
| 150 |
+
console.log('\n✅ PDF 生成成功');
|
| 151 |
+
} else {
|
| 152 |
+
console.log(`\n❌ PDF 生成失败: HTTP ${result.status}`);
|
| 153 |
+
}
|
| 154 |
+
} catch (err) {
|
| 155 |
+
console.error(`\n❌ 请求异常: ${err.message}`);
|
| 156 |
+
}
|
| 157 |
+
}
|
| 158 |
+
|
| 159 |
+
main();
|
server.js
CHANGED
|
@@ -29,7 +29,6 @@ const { getHighlighter } = require('shiki');
|
|
| 29 |
const fs = require('fs');
|
| 30 |
const path = require('path');
|
| 31 |
const os = require('os');
|
| 32 |
-
const { BrowserPool } = require('./browser-pool');
|
| 33 |
|
| 34 |
let ChartJSNodeCanvas = null;
|
| 35 |
try {
|
|
@@ -123,51 +122,6 @@ const isTest = process.env.NODE_ENV === 'test';
|
|
| 123 |
|
| 124 |
const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms));
|
| 125 |
|
| 126 |
-
// ─── Performance configuration (env overridable) ────────────────────
|
| 127 |
-
// PDF_POOL_SIZE: how many Chromium browsers to keep warm. Each browser
|
| 128 |
-
// renders ONE PDF at a time (page.pdf() is CPU-bound), so this is also the
|
| 129 |
-
// max concurrent PDF jobs. Default 4 works well on multi-core; set to 2 for
|
| 130 |
-
// Hugging Face free tier (2 vCPU / 16GB).
|
| 131 |
-
// PDF_RECYCLE_AFTER: recycle a browser after N jobs to bound memory growth.
|
| 132 |
-
// WIDGET_MAX_CONCURRENT: concurrent widget renders inside the shared widget
|
| 133 |
-
// browser (each widget gets its own page).
|
| 134 |
-
const PDF_POOL_SIZE = parseInt(process.env.PDF_POOL_SIZE || '4', 10);
|
| 135 |
-
const PDF_RECYCLE_AFTER = parseInt(process.env.PDF_RECYCLE_AFTER || '30', 10);
|
| 136 |
-
const PDF_ACQUIRE_TIMEOUT_MS = parseInt(process.env.PDF_ACQUIRE_TIMEOUT_MS || '120000', 10);
|
| 137 |
-
const WIDGET_MAX_CONCURRENT = parseInt(process.env.WIDGET_MAX_CONCURRENT || '3', 10);
|
| 138 |
-
|
| 139 |
-
const PDF_LAUNCH_ARGS = [
|
| 140 |
-
'--no-sandbox',
|
| 141 |
-
'--disable-setuid-sandbox',
|
| 142 |
-
'--disable-dev-shm-usage',
|
| 143 |
-
'--font-render-hinting=none',
|
| 144 |
-
'--disable-gpu',
|
| 145 |
-
'--disable-software-rasterizer',
|
| 146 |
-
'--memory-pressure-off'
|
| 147 |
-
];
|
| 148 |
-
|
| 149 |
-
const PDF_LAUNCH_OPTIONS = {
|
| 150 |
-
executablePath: '/usr/bin/chromium',
|
| 151 |
-
protocolTimeout: 0,
|
| 152 |
-
// Load images served with expired/invalid SSL certificates.
|
| 153 |
-
// Third-party image CDNs (e.g. imgs.sbkko.com) can have certificate issues;
|
| 154 |
-
// the PDF must still render those images the user saw in the conversation.
|
| 155 |
-
acceptInsecureCerts: true,
|
| 156 |
-
args: PDF_LAUNCH_ARGS,
|
| 157 |
-
headless: 'shell'
|
| 158 |
-
};
|
| 159 |
-
|
| 160 |
-
const pdfPool = new BrowserPool({
|
| 161 |
-
name: 'pdf',
|
| 162 |
-
size: PDF_POOL_SIZE,
|
| 163 |
-
launchOptions: PDF_LAUNCH_OPTIONS,
|
| 164 |
-
recycleAfter: PDF_RECYCLE_AFTER,
|
| 165 |
-
acquireTimeoutMs: PDF_ACQUIRE_TIMEOUT_MS,
|
| 166 |
-
log: (msg) => console.log(`[PERF] ${msg}`)
|
| 167 |
-
});
|
| 168 |
-
|
| 169 |
-
console.log(`[PERF] PDF browser pool: size=${PDF_POOL_SIZE}, recycleAfter=${PDF_RECYCLE_AFTER}, widgetConcurrent=${WIDGET_MAX_CONCURRENT}`);
|
| 170 |
-
|
| 171 |
app.use(cors());
|
| 172 |
app.use(express.json({ limit: '50mb' }));
|
| 173 |
|
|
@@ -196,9 +150,6 @@ app.post('/api/generate_pdf', async (req, res) => {
|
|
| 196 |
|
| 197 |
const getElapsed = () => ((Date.now() - startTime) / 1000).toFixed(2) + 's';
|
| 198 |
let browser = null;
|
| 199 |
-
let acquired = null; // pooled browser slot (release in finally)
|
| 200 |
-
let page = null;
|
| 201 |
-
let tempFilePath = null;
|
| 202 |
|
| 203 |
try {
|
| 204 |
if (!html) {
|
|
@@ -311,21 +262,27 @@ app.post('/api/generate_pdf', async (req, res) => {
|
|
| 311 |
const estMinutes = (pdfTimeout / 60000).toFixed(0);
|
| 312 |
console.log(`[PDF-GEN] [${getElapsed()}] ⚠️ 大文件预警: 纯文本=${effectiveSizeMB.toFixed(2)} MB HTML 预计需要 ${estMinutes} 分钟`);
|
| 313 |
}
|
| 314 |
-
console.log(`[PDF-GEN] [${getElapsed()}] 正在
|
| 315 |
// protocolTimeout: 0 = 禁用 CDP 协议层超时
|
| 316 |
// 参考: https://github.com/puppeteer/puppeteer/issues/9927
|
| 317 |
// PDF 超时时由应用层 Promise.race 控制，不依赖协议层超时
|
| 318 |
-
|
| 319 |
-
|
| 320 |
-
|
| 321 |
-
|
| 322 |
-
|
| 323 |
-
|
| 324 |
-
|
| 325 |
-
|
| 326 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 327 |
});
|
|
|
|
| 328 |
|
|
|
|
| 329 |
// 设置 viewport 满足大部分页面渲染需求
|
| 330 |
await page.setViewport({ width: 1200, height: 800 });
|
| 331 |
console.log(`[PDF-GEN] [${getElapsed()}] Viewport: 1200x800`);
|
|
@@ -333,34 +290,25 @@ app.post('/api/generate_pdf', async (req, res) => {
|
|
| 333 |
// 大 HTML (> 5 MB) 使用临时文件法，避免 CDP WebSocket 传输限制
|
| 334 |
// 参考: https://danindu.medium.com/optimizing-puppeteer-for-pdf-generation-8b7777edbeca
|
| 335 |
const isLargeHtml = htmlSizeMBNum > 5;
|
|
|
|
| 336 |
|
| 337 |
console.log(`[PDF-GEN] [${getElapsed()}] 正在${isLargeHtml ? '通过临时文件加载' : '填充'}页面内容...`);
|
| 338 |
-
// waitUntil 策略：用 'domcontentloaded' 而非 'load'。
|
| 339 |
-
// 原因：'load' 会等待页面所有子资源（含外部图片）加载完成。若某张外部图片
|
| 340 |
-
// 的 CDN 响应挂起（既不成功也不失败，例如证书/网络异常），load 事件永不触发，
|
| 341 |
-
// 导致整个 PDF 请求超时失败。改为 domcontentloaded 后 DOM 就绪即继续，
|
| 342 |
-
// 图片由下方的 per-image 有界等待逻辑兜底，单张慢图不会阻塞整个 PDF。
|
| 343 |
if (isLargeHtml) {
|
| 344 |
tempFilePath = path.join(os.tmpdir(), `xwx-pdf-${Date.now()}.html`);
|
| 345 |
-
|
| 346 |
await page.goto(`file://${tempFilePath}`, {
|
| 347 |
-
waitUntil: '
|
| 348 |
timeout: setContentTimeout
|
| 349 |
});
|
| 350 |
} else {
|
| 351 |
await page.setContent(htmlToUse, {
|
| 352 |
-
waitUntil: '
|
| 353 |
timeout: setContentTimeout
|
| 354 |
});
|
| 355 |
}
|
| 356 |
-
|
| 357 |
-
// 此处超时仅告警不中断，防止挂起的外部资源阻塞整个 PDF 生成。
|
| 358 |
-
try {
|
| 359 |
-
await page.waitForNetworkIdle({ idleTime: 300, timeout: Math.min(networkTimeout, 15000) });
|
| 360 |
-
} catch (e) {
|
| 361 |
-
console.log(`[PDF-GEN] [${getElapsed()}] waitForNetworkIdle 超时（非致命，图片检测将继续）: ${e.message}`);
|
| 362 |
-
}
|
| 363 |
console.log(`[PDF-GEN] [${getElapsed()}] 页面内容加载完成`);
|
|
|
|
| 364 |
// 等待 base64 图片完全渲染（检测实际加载状态）
|
| 365 |
let loadedImages = null;
|
| 366 |
if (imgCount > 0) {
|
|
@@ -385,25 +333,16 @@ app.post('/api/generate_pdf', async (req, res) => {
|
|
| 385 |
results.push({ src: srcPreview, status, width: img.naturalWidth, isBase64 });
|
| 386 |
}
|
| 387 |
|
| 388 |
-
//
|
| 389 |
-
// - base64 图片随 DOM 同步解码，无需等待（headless shell 模式 complete 也不准确）
|
| 390 |
-
// - 外部图片受 CDN 网络影响，统一等待 onload/onerror，最多 15s 兜底
|
| 391 |
-
// - 这样单张挂起的外部图片不会阻塞整个 PDF（与 waitUntil:'domcontentloaded' 配合）
|
| 392 |
-
const externalImages = Array.from(images).filter(img => {
|
| 393 |
-
const src = img.getAttribute('src') || '';
|
| 394 |
-
return !src.startsWith('data:image/');
|
| 395 |
-
});
|
| 396 |
const base64Images = Array.from(images).filter(img => {
|
| 397 |
const src = img.getAttribute('src') || '';
|
| 398 |
return src.startsWith('data:image/') && src.length > 100;
|
| 399 |
});
|
| 400 |
-
|
| 401 |
if (base64Images.length > 0) {
|
| 402 |
-
console.log(` 检测到 ${base64Images.length} 张base64图片
|
| 403 |
-
}
|
| 404 |
-
|
| 405 |
-
console.log(` 等待 ${externalImages.length} 张外部图片加载（每张最多 15s 兜底）...`);
|
| 406 |
-
await Promise.all(externalImages.map(img => {
|
| 407 |
if (img.complete && img.naturalWidth > 0) {
|
| 408 |
return Promise.resolve();
|
| 409 |
}
|
|
@@ -520,12 +459,12 @@ app.post('/api/generate_pdf', async (req, res) => {
|
|
| 520 |
|
| 521 |
// 清理临时文件
|
| 522 |
if (tempFilePath) {
|
| 523 |
-
try {
|
| 524 |
console.log(`[PDF-GEN] [${getElapsed()}] 已清理临时文件`);
|
| 525 |
}
|
| 526 |
|
| 527 |
const pdfSizeMB = (pdfBuffer.length / 1024 / 1024).toFixed(2);
|
| 528 |
-
console.log(`[PDF-GEN] [${getElapsed()}] PDF 生成成功 (${pdfSizeMB} MB)，正在
|
| 529 |
|
| 530 |
// Pass image loading summary as response header for frontend debugging
|
| 531 |
if (loadedImages && loadedImages.final) {
|
|
@@ -534,49 +473,23 @@ app.post('/api/generate_pdf', async (req, res) => {
|
|
| 534 |
res.setHeader('X-Image-Count', String(loadedImages.final.length));
|
| 535 |
}
|
| 536 |
|
| 537 |
-
|
| 538 |
-
if (page) {
|
| 539 |
-
try { await page.close(); } catch (e) {}
|
| 540 |
-
page = null;
|
| 541 |
-
}
|
| 542 |
-
if (acquired) {
|
| 543 |
-
if (typeof acquired.release === 'function') {
|
| 544 |
-
try { await acquired.release(); } catch (e) {}
|
| 545 |
-
} else {
|
| 546 |
-
console.error(`[PDF-GEN] [${getElapsed()}] BUG: acquired slot has no release() method — pool will leak!`);
|
| 547 |
-
}
|
| 548 |
-
acquired = null;
|
| 549 |
-
}
|
| 550 |
browser = null;
|
| 551 |
|
| 552 |
console.log(`[PDF-GEN] [${getElapsed()}] >>> 任务全��完成 <<<`);
|
| 553 |
|
| 554 |
res.setHeader('Content-Type', 'application/pdf');
|
| 555 |
res.setHeader('Content-Disposition', 'attachment; filename=export.pdf');
|
| 556 |
-
// page.pdf() 返回 Uint8Array（非 Buffer）。Express 的 res.send() 只对真正的
|
| 557 |
-
// Buffer 走二进制路径，否则会 JSON 序列化（{"0":37,"1":80,...}）。
|
| 558 |
-
// 必须用 Buffer.from() 转换回 Buffer 才能正确返回二进制 PDF。
|
| 559 |
res.send(Buffer.from(pdfBuffer));
|
| 560 |
|
| 561 |
} catch (error) {
|
| 562 |
console.error(`[PDF-GEN] [${getElapsed()}] 发生错误:`, error);
|
| 563 |
-
|
| 564 |
-
|
| 565 |
-
try { await page.close(); } catch (e) {}
|
| 566 |
-
page = null;
|
| 567 |
}
|
| 568 |
-
if (acquired) {
|
| 569 |
-
if (typeof acquired.release === 'function') {
|
| 570 |
-
try { await acquired.release(); } catch (e) {}
|
| 571 |
-
} else {
|
| 572 |
-
console.error(`[PDF-GEN] [${getElapsed()}] BUG: acquired slot has no release() method — pool will leak!`);
|
| 573 |
-
}
|
| 574 |
-
acquired = null;
|
| 575 |
-
}
|
| 576 |
-
browser = null;
|
| 577 |
// 清理临时文件
|
| 578 |
if (tempFilePath) {
|
| 579 |
-
try {
|
| 580 |
}
|
| 581 |
res.status(500).json({ error: 'Internal Server Error', details: error.message });
|
| 582 |
}
|
|
@@ -588,49 +501,29 @@ class WidgetRenderer {
|
|
| 588 |
constructor() {
|
| 589 |
this._chartInstances = new Map();
|
| 590 |
this._widgetBrowser = null;
|
| 591 |
-
this._widgetLaunchPromise = null;
|
| 592 |
}
|
| 593 |
|
| 594 |
async getWidgetBrowser() {
|
| 595 |
if (this._widgetBrowser && this._widgetBrowser.isConnected()) {
|
| 596 |
return this._widgetBrowser;
|
| 597 |
}
|
| 598 |
-
|
| 599 |
-
|
| 600 |
-
|
| 601 |
-
|
| 602 |
-
|
| 603 |
-
|
| 604 |
-
|
| 605 |
-
|
| 606 |
-
|
| 607 |
-
|
| 608 |
-
|
| 609 |
-
|
| 610 |
-
|
| 611 |
-
|
| 612 |
-
|
| 613 |
-
'--disable-gpu',
|
| 614 |
-
'--disable-software-rasterizer',
|
| 615 |
-
'--enable-webgl',
|
| 616 |
-
'--use-gl=angle',
|
| 617 |
-
'--use-angle=swiftshader',
|
| 618 |
-
'--memory-pressure-off'
|
| 619 |
-
],
|
| 620 |
-
headless: 'shell'
|
| 621 |
-
});
|
| 622 |
-
this._widgetBrowser = browser;
|
| 623 |
-
// 竞态修复 + 崩溃自动重建：断开后清空引用，下次调用重新启动
|
| 624 |
-
browser.on('disconnected', () => {
|
| 625 |
-
this._widgetBrowser = null;
|
| 626 |
-
this._widgetLaunchPromise = null;
|
| 627 |
-
});
|
| 628 |
-
console.log('[WIDGET] singleton widget browser ready');
|
| 629 |
-
return browser;
|
| 630 |
-
})().finally(() => {
|
| 631 |
-
this._widgetLaunchPromise = null;
|
| 632 |
});
|
| 633 |
-
return this.
|
| 634 |
}
|
| 635 |
|
| 636 |
async close() {
|
|
@@ -952,20 +845,31 @@ class WidgetRenderer {
|
|
| 952 |
const startTime = Date.now();
|
| 953 |
console.log(`[WIDGET] _renderFullHtml START: title=${title}, timeout=${renderTimeout}ms, chartType=${chartType}`);
|
| 954 |
|
| 955 |
-
let
|
| 956 |
try {
|
| 957 |
-
|
| 958 |
-
|
| 959 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 960 |
|
| 961 |
// CRITICAL: Use a compact viewport height to avoid excessive whitespace
|
| 962 |
// in fullPage screenshots. Width 800 gives enough room for 650px chart.
|
| 963 |
await page.setViewport({ width: 800, height: 500 });
|
| 964 |
|
| 965 |
// Set complete HTML document directly
|
| 966 |
-
|
| 967 |
-
await page.setContent(htmlContent, { waitUntil: 'load', timeout: 15000 });
|
| 968 |
-
await page.waitForNetworkIdle({ idleTime: 300, timeout: 15000 });
|
| 969 |
|
| 970 |
// Wait for React/Recharts to render (check window._widgetRendered flag)
|
| 971 |
const rendered = await page.waitForFunction(
|
|
@@ -996,9 +900,8 @@ class WidgetRenderer {
|
|
| 996 |
console.log(`[WIDGET] _renderFullHtml ERROR: ${e.message}`);
|
| 997 |
return null;
|
| 998 |
} finally {
|
| 999 |
-
|
| 1000 |
-
|
| 1001 |
-
try { await page.close(); } catch (e) {}
|
| 1002 |
}
|
| 1003 |
}
|
| 1004 |
}
|
|
@@ -1416,15 +1319,26 @@ window._widgetRendered = false;
|
|
| 1416 |
const startTime = Date.now();
|
| 1417 |
console.log(`[WIDGET] renderPuppeteer START: type=${type}, title=${title}, timeout=${renderTimeout}ms`);
|
| 1418 |
|
| 1419 |
-
let
|
| 1420 |
try {
|
| 1421 |
const finalTimeout = renderTimeout || 3000;
|
| 1422 |
|
| 1423 |
-
|
| 1424 |
-
|
| 1425 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1426 |
|
| 1427 |
-
page = await browser.newPage();
|
| 1428 |
await page.setViewport({ width: 1024, height: 768 });
|
| 1429 |
|
| 1430 |
const templatesDir = path.join(__dirname, 'templates');
|
|
@@ -1460,9 +1374,7 @@ window._widgetRendered = false;
|
|
| 1460 |
let fullHtml = template
|
| 1461 |
.replace('%%WIDGET_CODE%%', widgetCode);
|
| 1462 |
|
| 1463 |
-
|
| 1464 |
-
await page.setContent(fullHtml, { waitUntil: 'load', timeout: 15000 });
|
| 1465 |
-
await page.waitForNetworkIdle({ idleTime: 300, timeout: 15000 });
|
| 1466 |
|
| 1467 |
await page.evaluate(() => { window._widgetRendered = false; });
|
| 1468 |
|
|
@@ -1600,9 +1512,8 @@ window._widgetRendered = false;
|
|
| 1600 |
console.log(`[WIDGET] renderPuppeteer ERROR: ${e.message}`);
|
| 1601 |
return null;
|
| 1602 |
} finally {
|
| 1603 |
-
|
| 1604 |
-
|
| 1605 |
-
try { await page.close(); } catch (e) {}
|
| 1606 |
}
|
| 1607 |
}
|
| 1608 |
}
|
|
@@ -1630,7 +1541,7 @@ window._widgetRendered = false;
|
|
| 1630 |
}
|
| 1631 |
|
| 1632 |
const widgetRenderer = new WidgetRenderer();
|
| 1633 |
-
const MAX_CONCURRENT_RENDER =
|
| 1634 |
let activeRenderCount = 0;
|
| 1635 |
const renderQueue = [];
|
| 1636 |
|
|
|
|
| 29 |
const fs = require('fs');
|
| 30 |
const path = require('path');
|
| 31 |
const os = require('os');
|
|
|
|
| 32 |
|
| 33 |
let ChartJSNodeCanvas = null;
|
| 34 |
try {
|
|
|
|
| 122 |
|
| 123 |
const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms));
|
| 124 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 125 |
app.use(cors());
|
| 126 |
app.use(express.json({ limit: '50mb' }));
|
| 127 |
|
|
|
|
| 150 |
|
| 151 |
const getElapsed = () => ((Date.now() - startTime) / 1000).toFixed(2) + 's';
|
| 152 |
let browser = null;
|
|
|
|
|
|
|
|
|
|
| 153 |
|
| 154 |
try {
|
| 155 |
if (!html) {
|
|
|
|
| 262 |
const estMinutes = (pdfTimeout / 60000).toFixed(0);
|
| 263 |
console.log(`[PDF-GEN] [${getElapsed()}] ⚠️ 大文件预警: 纯文本=${effectiveSizeMB.toFixed(2)} MB HTML 预计需要 ${estMinutes} 分钟`);
|
| 264 |
}
|
| 265 |
+
console.log(`[PDF-GEN] [${getElapsed()}] 正在启动浏览器...`);
|
| 266 |
// protocolTimeout: 0 = 禁用 CDP 协议层超时
|
| 267 |
// 参考: https://github.com/puppeteer/puppeteer/issues/9927
|
| 268 |
// PDF 超时时由应用层 Promise.race 控制，不依赖协议层超时
|
| 269 |
+
browser = await puppeteer.launch({
|
| 270 |
+
executablePath: '/usr/bin/chromium',
|
| 271 |
+
protocolTimeout: 0,
|
| 272 |
+
args: [
|
| 273 |
+
'--no-sandbox',
|
| 274 |
+
'--disable-setuid-sandbox',
|
| 275 |
+
'--disable-dev-shm-usage',
|
| 276 |
+
'--font-render-hinting=none',
|
| 277 |
+
'--disable-gpu',
|
| 278 |
+
'--disable-software-rasterizer',
|
| 279 |
+
'--memory-pressure-off'
|
| 280 |
+
],
|
| 281 |
+
headless: 'shell'
|
| 282 |
});
|
| 283 |
+
console.log(`[PDF-GEN] [${getElapsed()}] 浏览器启动成功`);
|
| 284 |
|
| 285 |
+
const page = await browser.newPage();
|
| 286 |
// 设置 viewport 满足大部分页面渲染需求
|
| 287 |
await page.setViewport({ width: 1200, height: 800 });
|
| 288 |
console.log(`[PDF-GEN] [${getElapsed()}] Viewport: 1200x800`);
|
|
|
|
| 290 |
// 大 HTML (> 5 MB) 使用临时文件法，避免 CDP WebSocket 传输限制
|
| 291 |
// 参考: https://danindu.medium.com/optimizing-puppeteer-for-pdf-generation-8b7777edbeca
|
| 292 |
const isLargeHtml = htmlSizeMBNum > 5;
|
| 293 |
+
let tempFilePath = null;
|
| 294 |
|
| 295 |
console.log(`[PDF-GEN] [${getElapsed()}] 正在${isLargeHtml ? '通过临时文件加载' : '填充'}页面内容...`);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 296 |
if (isLargeHtml) {
|
| 297 |
tempFilePath = path.join(os.tmpdir(), `xwx-pdf-${Date.now()}.html`);
|
| 298 |
+
fs.writeFileSync(tempFilePath, htmlToUse, 'utf8');
|
| 299 |
await page.goto(`file://${tempFilePath}`, {
|
| 300 |
+
waitUntil: ['load', 'networkidle0'],
|
| 301 |
timeout: setContentTimeout
|
| 302 |
});
|
| 303 |
} else {
|
| 304 |
await page.setContent(htmlToUse, {
|
| 305 |
+
waitUntil: ['load', 'networkidle0'],
|
| 306 |
timeout: setContentTimeout
|
| 307 |
});
|
| 308 |
}
|
| 309 |
+
await page.waitForNetworkIdle({ idleTime: 500, timeout: networkTimeout });
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 310 |
console.log(`[PDF-GEN] [${getElapsed()}] 页面内容加载完成`);
|
| 311 |
+
|
| 312 |
// 等待 base64 图片完全渲染（检测实际加载状态）
|
| 313 |
let loadedImages = null;
|
| 314 |
if (imgCount > 0) {
|
|
|
|
| 333 |
results.push({ src: srcPreview, status, width: img.naturalWidth, isBase64 });
|
| 334 |
}
|
| 335 |
|
| 336 |
+
// 对于base64图片，跳过等待onload（headless模式不准确）
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 337 |
const base64Images = Array.from(images).filter(img => {
|
| 338 |
const src = img.getAttribute('src') || '';
|
| 339 |
return src.startsWith('data:image/') && src.length > 100;
|
| 340 |
});
|
| 341 |
+
|
| 342 |
if (base64Images.length > 0) {
|
| 343 |
+
console.log(` 检测到 ${base64Images.length} 张base64图片，跳过onload等待（headless模式不准确）`);
|
| 344 |
+
} else {
|
| 345 |
+
await Promise.all(Array.from(images).map(img => {
|
|
|
|
|
|
|
| 346 |
if (img.complete && img.naturalWidth > 0) {
|
| 347 |
return Promise.resolve();
|
| 348 |
}
|
|
|
|
| 459 |
|
| 460 |
// 清理临时文件
|
| 461 |
if (tempFilePath) {
|
| 462 |
+
try { fs.unlinkSync(tempFilePath); } catch {}
|
| 463 |
console.log(`[PDF-GEN] [${getElapsed()}] 已清理临时文件`);
|
| 464 |
}
|
| 465 |
|
| 466 |
const pdfSizeMB = (pdfBuffer.length / 1024 / 1024).toFixed(2);
|
| 467 |
+
console.log(`[PDF-GEN] [${getElapsed()}] PDF 生成成功 (${pdfSizeMB} MB)，正在关闭浏览器...`);
|
| 468 |
|
| 469 |
// Pass image loading summary as response header for frontend debugging
|
| 470 |
if (loadedImages && loadedImages.final) {
|
|
|
|
| 473 |
res.setHeader('X-Image-Count', String(loadedImages.final.length));
|
| 474 |
}
|
| 475 |
|
| 476 |
+
await browser.close();
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 477 |
browser = null;
|
| 478 |
|
| 479 |
console.log(`[PDF-GEN] [${getElapsed()}] >>> 任务全��完成 <<<`);
|
| 480 |
|
| 481 |
res.setHeader('Content-Type', 'application/pdf');
|
| 482 |
res.setHeader('Content-Disposition', 'attachment; filename=export.pdf');
|
|
|
|
|
|
|
|
|
|
| 483 |
res.send(Buffer.from(pdfBuffer));
|
| 484 |
|
| 485 |
} catch (error) {
|
| 486 |
console.error(`[PDF-GEN] [${getElapsed()}] 发生错误:`, error);
|
| 487 |
+
if (browser) {
|
| 488 |
+
try { await browser.close(); } catch (e) {}
|
|
|
|
|
|
|
| 489 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 490 |
// 清理临时文件
|
| 491 |
if (tempFilePath) {
|
| 492 |
+
try { fs.unlinkSync(tempFilePath); } catch {}
|
| 493 |
}
|
| 494 |
res.status(500).json({ error: 'Internal Server Error', details: error.message });
|
| 495 |
}
|
|
|
|
| 501 |
constructor() {
|
| 502 |
this._chartInstances = new Map();
|
| 503 |
this._widgetBrowser = null;
|
|
|
|
| 504 |
}
|
| 505 |
|
| 506 |
async getWidgetBrowser() {
|
| 507 |
if (this._widgetBrowser && this._widgetBrowser.isConnected()) {
|
| 508 |
return this._widgetBrowser;
|
| 509 |
}
|
| 510 |
+
this._widgetBrowser = await puppeteer.launch({
|
| 511 |
+
executablePath: '/usr/bin/chromium',
|
| 512 |
+
args: [
|
| 513 |
+
'--no-sandbox',
|
| 514 |
+
'--disable-setuid-sandbox',
|
| 515 |
+
'--disable-dev-shm-usage',
|
| 516 |
+
'--font-render-hinting=none',
|
| 517 |
+
'--disable-gpu',
|
| 518 |
+
'--disable-software-rasterizer',
|
| 519 |
+
'--enable-webgl',
|
| 520 |
+
'--use-gl=angle',
|
| 521 |
+
'--use-angle=swiftshader',
|
| 522 |
+
'--memory-pressure-off'
|
| 523 |
+
],
|
| 524 |
+
headless: 'shell'
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 525 |
});
|
| 526 |
+
return this._widgetBrowser;
|
| 527 |
}
|
| 528 |
|
| 529 |
async close() {
|
|
|
|
| 845 |
const startTime = Date.now();
|
| 846 |
console.log(`[WIDGET] _renderFullHtml START: title=${title}, timeout=${renderTimeout}ms, chartType=${chartType}`);
|
| 847 |
|
| 848 |
+
let browser = null;
|
| 849 |
try {
|
| 850 |
+
const launchArgs = [
|
| 851 |
+
'--no-sandbox',
|
| 852 |
+
'--disable-setuid-sandbox',
|
| 853 |
+
'--disable-dev-shm-usage',
|
| 854 |
+
'--disable-gpu',
|
| 855 |
+
'--disable-software-rasterizer',
|
| 856 |
+
'--memory-pressure-off'
|
| 857 |
+
];
|
| 858 |
+
|
| 859 |
+
browser = await puppeteer.launch({
|
| 860 |
+
executablePath: '/usr/bin/chromium',
|
| 861 |
+
args: launchArgs,
|
| 862 |
+
headless: 'shell'
|
| 863 |
+
});
|
| 864 |
+
|
| 865 |
+
const page = await browser.newPage();
|
| 866 |
|
| 867 |
// CRITICAL: Use a compact viewport height to avoid excessive whitespace
|
| 868 |
// in fullPage screenshots. Width 800 gives enough room for 650px chart.
|
| 869 |
await page.setViewport({ width: 800, height: 500 });
|
| 870 |
|
| 871 |
// Set complete HTML document directly
|
| 872 |
+
await page.setContent(htmlContent, { waitUntil: 'networkidle0', timeout: 15000 });
|
|
|
|
|
|
|
| 873 |
|
| 874 |
// Wait for React/Recharts to render (check window._widgetRendered flag)
|
| 875 |
const rendered = await page.waitForFunction(
|
|
|
|
| 900 |
console.log(`[WIDGET] _renderFullHtml ERROR: ${e.message}`);
|
| 901 |
return null;
|
| 902 |
} finally {
|
| 903 |
+
if (browser) {
|
| 904 |
+
try { await browser.close(); } catch (e) {}
|
|
|
|
| 905 |
}
|
| 906 |
}
|
| 907 |
}
|
|
|
|
| 1319 |
const startTime = Date.now();
|
| 1320 |
console.log(`[WIDGET] renderPuppeteer START: type=${type}, title=${title}, timeout=${renderTimeout}ms`);
|
| 1321 |
|
| 1322 |
+
let browser = null;
|
| 1323 |
try {
|
| 1324 |
const finalTimeout = renderTimeout || 3000;
|
| 1325 |
|
| 1326 |
+
const launchArgs = [
|
| 1327 |
+
'--no-sandbox',
|
| 1328 |
+
'--disable-setuid-sandbox',
|
| 1329 |
+
'--disable-dev-shm-usage',
|
| 1330 |
+
'--disable-gpu',
|
| 1331 |
+
'--disable-software-rasterizer',
|
| 1332 |
+
'--memory-pressure-off'
|
| 1333 |
+
];
|
| 1334 |
+
|
| 1335 |
+
browser = await puppeteer.launch({
|
| 1336 |
+
executablePath: '/usr/bin/chromium',
|
| 1337 |
+
args: launchArgs,
|
| 1338 |
+
headless: 'shell'
|
| 1339 |
+
});
|
| 1340 |
|
| 1341 |
+
const page = await browser.newPage();
|
| 1342 |
await page.setViewport({ width: 1024, height: 768 });
|
| 1343 |
|
| 1344 |
const templatesDir = path.join(__dirname, 'templates');
|
|
|
|
| 1374 |
let fullHtml = template
|
| 1375 |
.replace('%%WIDGET_CODE%%', widgetCode);
|
| 1376 |
|
| 1377 |
+
await page.setContent(fullHtml, { waitUntil: 'networkidle0', timeout: 15000 });
|
|
|
|
|
|
|
| 1378 |
|
| 1379 |
await page.evaluate(() => { window._widgetRendered = false; });
|
| 1380 |
|
|
|
|
| 1512 |
console.log(`[WIDGET] renderPuppeteer ERROR: ${e.message}`);
|
| 1513 |
return null;
|
| 1514 |
} finally {
|
| 1515 |
+
if (browser) {
|
| 1516 |
+
try { await browser.close(); } catch (e) {}
|
|
|
|
| 1517 |
}
|
| 1518 |
}
|
| 1519 |
}
|
|
|
|
| 1541 |
}
|
| 1542 |
|
| 1543 |
const widgetRenderer = new WidgetRenderer();
|
| 1544 |
+
const MAX_CONCURRENT_RENDER = 3;
|
| 1545 |
let activeRenderCount = 0;
|
| 1546 |
const renderQueue = [];
|
| 1547 |
|
test-long.js
ADDED
|
@@ -0,0 +1,154 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
const http = require('http');
|
| 2 |
+
const fs = require('fs');
|
| 3 |
+
|
| 4 |
+
const BACKEND_PORT = process.env.PORT || 7861;
|
| 5 |
+
const CSS = `@media print { @page { size: A4; margin: 15mm 10mm; } body { -webkit-print-color-adjust: exact; } }
|
| 6 |
+
body { font-family: -apple-system, sans-serif; font-size: 14px; line-height: 1.6; max-width: 746px; margin: 0 auto; padding: 20px; }
|
| 7 |
+
h1,h2,h3 { font-weight: 600; margin: 16px 0 8px; }
|
| 8 |
+
pre { background: #f6f8fa; padding: 16px; border-radius: 6px; overflow-x: auto; border: 1px solid #e1e4e8; }
|
| 9 |
+
code { font-family: monospace; font-size: 13px; }
|
| 10 |
+
p { margin: 8px 0; } table { border-collapse: collapse; width: 100%; margin: 12px 0; }
|
| 11 |
+
th,td { border: 1px solid #dfe2e5; padding: 8px 12px; } th { background: #f1f3f4; }
|
| 12 |
+
.chat-container { display: flex; flex-direction: column; gap: 16px; }
|
| 13 |
+
.message-row { display: flex; gap: 10px; } .message-bubble { max-width: 85%; padding: 12px 16px; border-radius: 12px; }
|
| 14 |
+
.ai-bubble { background: #fff; border: 1px solid #eee; } .user-bubble { background: #e8f0fe; }
|
| 15 |
+
.avatar { width: 32px; height: 32px; border-radius: 50%; display: flex; align-items: center; justify-content: center; }`;
|
| 16 |
+
|
| 17 |
+
function E(s) { return s.replace(/&/g,'&').replace(/</g,'<').replace(/>/g,'>'); }
|
| 18 |
+
|
| 19 |
+
function shikiWrap(token, color) {
|
| 20 |
+
return `<span style="color:${color}">${E(token)}</span>`;
|
| 21 |
+
}
|
| 22 |
+
|
| 23 |
+
function generateShikiCode(lines) {
|
| 24 |
+
return lines.map(line => {
|
| 25 |
+
const parts = [];
|
| 26 |
+
const tokens = line.split(/(\s+|[^a-zA-Z0-9_\s]+)/g);
|
| 27 |
+
for (const tok of tokens) {
|
| 28 |
+
if (!tok) continue;
|
| 29 |
+
if (/^\s+$/.test(tok)) { parts.push(tok); continue; }
|
| 30 |
+
let color = '#c9d1d9';
|
| 31 |
+
if (/^(const|let|var|function|return|if|else|for|async|await|import|from|export|class|extends|new|try|catch|throw|typeof|this|switch|case|break|continue|while|do|of|in|static|get|set|super|interface|type|void|null|undefined|true|false)$/.test(tok)) color = '#ff7b72';
|
| 32 |
+
else if (/^(console|log|error|fetch|Promise|Math|Date|JSON|Map|Set|Array|Object|String|Number|Error|setTimeout|clearTimeout|AbortController|AbortSignal|require|module|exports|process)$/.test(tok)) color = '#d2a8ff';
|
| 33 |
+
else if (/^\d+$/.test(tok)) color = '#79c0ff';
|
| 34 |
+
else if (/^[{}()\[\];,\.:=+\-*/<>!&|?%@~^'"`]+$/.test(tok)) color = '#c9d1d9';
|
| 35 |
+
else if (/^[A-Z]/.test(tok) && tok.length > 1) color = '#ffa657';
|
| 36 |
+
parts.push(shikiWrap(tok, color));
|
| 37 |
+
}
|
| 38 |
+
return parts.join('');
|
| 39 |
+
}).join('\n');
|
| 40 |
+
}
|
| 41 |
+
|
| 42 |
+
const CODE_LINES = [
|
| 43 |
+
'async function fetchData(url, options = {}) {',
|
| 44 |
+
' const controller = new AbortController();',
|
| 45 |
+
' const timeout = setTimeout(() => controller.abort(), 30000);',
|
| 46 |
+
' try {',
|
| 47 |
+
' const response = await fetch(url, {',
|
| 48 |
+
' ...options,',
|
| 49 |
+
' signal: controller.signal,',
|
| 50 |
+
" headers: { 'Content-Type': 'application/json' },",
|
| 51 |
+
' });',
|
| 52 |
+
' if (!response.ok) {',
|
| 53 |
+
' throw new Error(`HTTP ${response.status}: ${response.statusText}`);',
|
| 54 |
+
' }',
|
| 55 |
+
' const data = await response.json();',
|
| 56 |
+
" console.log('Data received:', data);",
|
| 57 |
+
' return data;',
|
| 58 |
+
' } catch (error) {',
|
| 59 |
+
" console.error('Fetch failed:', error.message);",
|
| 60 |
+
' throw error;',
|
| 61 |
+
' } finally {',
|
| 62 |
+
' clearTimeout(timeout);',
|
| 63 |
+
' }',
|
| 64 |
+
'}',
|
| 65 |
+
];
|
| 66 |
+
|
| 67 |
+
const TEXT = '<p>This is a typical AI response explaining a concept with some details and examples that users commonly see in chat conversations.</p>';
|
| 68 |
+
|
| 69 |
+
function buildHtml(targetSizeMB) {
|
| 70 |
+
const shikiCode = generateShikiCode(CODE_LINES);
|
| 71 |
+
const codeBlock = `<pre data-language="javascript"><code class="language-javascript">${shikiCode}</code></pre>`;
|
| 72 |
+
const codeBlockSize = Buffer.byteLength(codeBlock, 'utf8');
|
| 73 |
+
const textBlockSize = Buffer.byteLength(TEXT, 'utf8');
|
| 74 |
+
const overhead = 2048;
|
| 75 |
+
const targetBytes = targetSizeMB * 1024 * 1024 - overhead;
|
| 76 |
+
|
| 77 |
+
let html = `<!DOCTYPE html><html><head><meta charset="UTF-8"><style>${CSS}</style></head><body><div class="chat-container">`;
|
| 78 |
+
|
| 79 |
+
const codeIterations = Math.ceil((targetBytes * 0.5) / codeBlockSize);
|
| 80 |
+
const textIterations = Math.ceil((targetBytes * 0.5) / textBlockSize);
|
| 81 |
+
let i = 0, j = 0;
|
| 82 |
+
|
| 83 |
+
while (i < codeIterations || j < textIterations) {
|
| 84 |
+
if (i < codeIterations) {
|
| 85 |
+
html += `<div class="message-row"><div class="avatar">🤖</div><div class="message-bubble ai-bubble">${codeBlock}</div></div>`;
|
| 86 |
+
i++;
|
| 87 |
+
}
|
| 88 |
+
if (j < textIterations) {
|
| 89 |
+
html += `<div class="message-row"><div class="avatar">👤</div><div class="message-bubble user-bubble">${TEXT}</div></div>`;
|
| 90 |
+
j++;
|
| 91 |
+
}
|
| 92 |
+
}
|
| 93 |
+
|
| 94 |
+
html += '</div></body></html>';
|
| 95 |
+
return html;
|
| 96 |
+
}
|
| 97 |
+
|
| 98 |
+
async function sendPdfRequest(html) {
|
| 99 |
+
const payload = JSON.stringify({
|
| 100 |
+
html, codeTheme: 'github', showWatermark: false,
|
| 101 |
+
imageCount: 0, totalImageSizeMB: 0,
|
| 102 |
+
platform: 'Benchmark', language: 'en-US', extensionVersion: '2.0.2',
|
| 103 |
+
exportCount: 0, exportPdf: 0, exportMd: 0, exportTxt: 0,
|
| 104 |
+
exportDocx: 0, exportJson: 0, exportClipboard: 0, exportNotion: 0,
|
| 105 |
+
});
|
| 106 |
+
|
| 107 |
+
return new Promise((resolve, reject) => {
|
| 108 |
+
const startTime = Date.now();
|
| 109 |
+
const req = http.request({
|
| 110 |
+
hostname: 'localhost', port: BACKEND_PORT, path: '/api/generate_pdf',
|
| 111 |
+
method: 'POST',
|
| 112 |
+
headers: { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(payload) },
|
| 113 |
+
timeout: 1800000, // 30 min
|
| 114 |
+
}, (res) => {
|
| 115 |
+
const chunks = [];
|
| 116 |
+
res.on('data', (chunk) => chunks.push(chunk));
|
| 117 |
+
res.on('end', () => {
|
| 118 |
+
const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
|
| 119 |
+
if (res.statusCode === 200) {
|
| 120 |
+
resolve({ ok: true, elapsed, pdfSizeMB: (Buffer.concat(chunks).length / 1024 / 1024).toFixed(2) });
|
| 121 |
+
} else {
|
| 122 |
+
resolve({ ok: false, elapsed, status: res.statusCode, error: Buffer.concat(chunks).toString() });
|
| 123 |
+
}
|
| 124 |
+
});
|
| 125 |
+
});
|
| 126 |
+
req.on('error', reject);
|
| 127 |
+
req.on('timeout', () => { req.destroy(); reject(new Error('HTTP timeout')); });
|
| 128 |
+
req.write(payload);
|
| 129 |
+
req.end();
|
| 130 |
+
});
|
| 131 |
+
}
|
| 132 |
+
|
| 133 |
+
async function main() {
|
| 134 |
+
const size = parseFloat(process.argv[2] || '4.92');
|
| 135 |
+
const html = buildHtml(size);
|
| 136 |
+
const actualSizeMB = (Buffer.byteLength(html, 'utf8') / 1024 / 1024).toFixed(2);
|
| 137 |
+
console.log(`Testing ${size} MB target, actual ${actualSizeMB} MB HTML...`);
|
| 138 |
+
console.log(`Start: ${new Date().toISOString()}`);
|
| 139 |
+
|
| 140 |
+
try {
|
| 141 |
+
const result = await sendPdfRequest(html);
|
| 142 |
+
console.log(`End: ${new Date().toISOString()}`);
|
| 143 |
+
if (result.ok) {
|
| 144 |
+
console.log(`SUCCESS in ${result.elapsed}s, PDF: ${result.pdfSizeMB} MB`);
|
| 145 |
+
} else {
|
| 146 |
+
console.log(`FAIL: HTTP ${result.status}, ${result.elapsed}s`);
|
| 147 |
+
console.log(result.error);
|
| 148 |
+
}
|
| 149 |
+
} catch (err) {
|
| 150 |
+
console.error(`ERROR: ${err.message}`);
|
| 151 |
+
}
|
| 152 |
+
}
|
| 153 |
+
|
| 154 |
+
main();
|
tools/compare-dirs.js
DELETED
|
@@ -1,57 +0,0 @@
|
|
| 1 |
-
/**
|
| 2 |
-
* 递归对比两个目录，列出差异文件（排除 node_modules/.git/dist 等）。
|
| 3 |
-
* 用法: node compare-dirs.js <dirA> <dirB>
|
| 4 |
-
*/
|
| 5 |
-
const fs = require('fs');
|
| 6 |
-
const path = require('path');
|
| 7 |
-
const crypto = require('crypto');
|
| 8 |
-
|
| 9 |
-
const EXCLUDE_DIRS = new Set(['node_modules', '.git', 'temp', 'test-results', 'make', 'dist', 'tools', 'tests', 'stress', 'hf-trend', '.sisyphus', '.opencode', 'output', 'test-outputs']);
|
| 10 |
-
const EXCLUDE_FILES = new Set(['PERFORMANCE_ANALYSIS.md', 'PERFORMANCE_STRESS_TEST_2026-08-03.md', 'HF_TOKEN.md']);
|
| 11 |
-
|
| 12 |
-
const [A, B] = process.argv.slice(2);
|
| 13 |
-
if (!A || !B) { console.error('need two dirs'); process.exit(1); }
|
| 14 |
-
|
| 15 |
-
const md5 = (p) => crypto.createHash('md5').update(fs.readFileSync(p)).digest('hex');
|
| 16 |
-
|
| 17 |
-
function walk(d, base) {
|
| 18 |
-
const out = new Map();
|
| 19 |
-
if (!fs.existsSync(d)) return out;
|
| 20 |
-
for (const f of fs.readdirSync(d)) {
|
| 21 |
-
const fp = path.join(d, f);
|
| 22 |
-
if (fs.statSync(fp).isDirectory()) {
|
| 23 |
-
if (EXCLUDE_DIRS.has(f)) continue;
|
| 24 |
-
for (const [rel, h] of walk(fp, base)) out.set(rel, h);
|
| 25 |
-
} else {
|
| 26 |
-
if (EXCLUDE_FILES.has(f)) continue;
|
| 27 |
-
const rel = path.relative(base, fp).replace(/\\/g, '/');
|
| 28 |
-
out.set(rel, md5(fp));
|
| 29 |
-
}
|
| 30 |
-
}
|
| 31 |
-
return out;
|
| 32 |
-
}
|
| 33 |
-
|
| 34 |
-
const mapA = walk(A, A);
|
| 35 |
-
const mapB = walk(B, B);
|
| 36 |
-
|
| 37 |
-
const all = new Set([...mapA.keys(), ...mapB.keys()]);
|
| 38 |
-
const changed = [];
|
| 39 |
-
const onlyA = [];
|
| 40 |
-
const onlyB = [];
|
| 41 |
-
for (const rel of all) {
|
| 42 |
-
const ha = mapA.get(rel);
|
| 43 |
-
const hb = mapB.get(rel);
|
| 44 |
-
if (ha && hb) { if (ha !== hb) changed.push(rel); }
|
| 45 |
-
else if (ha && !hb) onlyA.push(rel);
|
| 46 |
-
else onlyB.push(rel);
|
| 47 |
-
}
|
| 48 |
-
|
| 49 |
-
changed.sort(); onlyA.sort(); onlyB.sort();
|
| 50 |
-
console.log(`=== A=${A}`);
|
| 51 |
-
console.log(`=== B=${B}`);
|
| 52 |
-
console.log(`\n[不同内容] ${changed.length} 个:`);
|
| 53 |
-
for (const f of changed) console.log(` M ${f}`);
|
| 54 |
-
console.log(`\n[仅 A 有] ${onlyA.length} 个:`);
|
| 55 |
-
for (const f of onlyA) console.log(` A ${f}`);
|
| 56 |
-
console.log(`\n[仅 B 有] ${onlyB.length} 个:`);
|
| 57 |
-
for (const f of onlyB) console.log(` B ${f}`);
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|