somratpro Claude Opus 4.7 commited on
Commit
a1ea643
ยท
1 Parent(s): a83da60

fix: fresh clone + filtered pnpm install in frontend stage to cut peak RSS

Browse files

Root cause: COPY --from=stage1 /build /build in Stage 2 imports ~2 GB of
node_modules (3817 packages). OS page-caches the unpacked tree; webpack then
traverses the full module graph on top โ†’ combined RSS exceeds builder cgroup
limit โ†’ OOMKilled (exit 137) even with a separate build stage.

Fix: Stage 2 does a fresh git clone and installs only the frontend package's
dependency tree (pnpm --filter "./apps/frontend..."). This skips NestJS,
Prisma, bcrypt, Bull, and other server-only packages, producing a much smaller
node_modules โ†’ lower OS page-cache pressure โ†’ next build fits in RAM.

Also adds swcMinify: false to force Terser (pure JS, V8-heap-bounded) over the
native SWC binary, which adds RSS outside the V8 heap limit.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Files changed (1) hide show
  1. Dockerfile +64 -59
Dockerfile CHANGED
@@ -3,33 +3,33 @@
3
  #
4
  # Three-stage build to beat the HF Space builder memory limit:
5
  #
6
- # Stage 1 (postiz-builder): clone, patch, install deps,
7
- # build backend + workers + cron.
8
- # Stage 2 (postiz-frontend): copy tree from Stage 1, build ONLY the
9
- # Next.js frontend in a clean process.
10
- # Stage 1's processes are dead โ†’ their RSS
11
- # is fully freed before `next build` starts.
12
- # Stage 3 (runtime): copy server build from Stage 1,
13
- # overlay frontend .next from Stage 2.
14
  #
15
- # Why three stages (not two):
16
- # Three NestJS builds (backend+workers+cron) leave ~1-2 GB of residual
17
- # RSS in the same container even after each `pnpm run build:*` exits,
18
- # because the OS hasn't reclaimed all pages. `next build` alone needs
19
- # ~3-4 GB RSS (V8 heap + SWC + native addons). Together they exceed
20
- # the HF builder cgroup limit โ†’ OOMKilled (exit 137).
21
- # Splitting frontend into its own stage gives it a clean address space.
 
22
  #
23
  # Container layout at runtime:
24
- # - nginx (port 5000, internal) โ€” Postiz frontend + backend + uploads
25
- # - PM2 โ†’ 4 Postiz procs (backend/frontend/workers/cron)
26
  # - postgres (port 5432, internal)
27
  # - redis (port 6379, internal)
28
- # - postiz-sync.py loop โ€” backup DB + uploads to HF Dataset
29
  # - health-server.js (port 7860, public) โ€” dashboard + reverse proxy
30
  # ============================================================================
31
 
32
- # โ”€โ”€ Stage 1: Clone, patch, install deps, build server apps โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
33
  FROM node:22.20-alpine AS postiz-builder
34
 
35
  WORKDIR /build
@@ -49,22 +49,25 @@ RUN npm install -g pnpm@10.6.1
49
  # Pinned to v2.11.3 โ€” last release before Temporal became a hard requirement.
50
  RUN git clone --depth=1 --branch v2.11.3 https://github.com/gitroomhq/postiz-app.git .
51
 
52
- # Patch Next.js config:
53
- # 1. basePath/assetPrefix=/app โ†’ Postiz UI mounts at /app; dashboard owns /
54
- # 2. productionBrowserSourceMaps: false โ†’ saves ~500 MB RSS during emit
55
- # 3. Sentry sourcemap plugin: disable: true โ†’ saves another ~300 MB
56
- # 4. experimental.cpus=1 + workerThreads=false โ†’ single-thread webpack;
57
- # no parallel worker copies of the module graph in memory
58
- RUN sed -i "s|const nextConfig = {|const nextConfig = {\n basePath: '/app',\n assetPrefix: '/app',|" apps/frontend/next.config.js \
 
 
 
59
  && sed -i "s|productionBrowserSourceMaps: true|productionBrowserSourceMaps: false|" apps/frontend/next.config.js \
60
  && sed -i "s|disable: false,|disable: true,|" apps/frontend/next.config.js \
61
  && sed -i "s|experimental: {|experimental: {\n cpus: 1,\n workerThreads: false,|" apps/frontend/next.config.js \
62
  && grep -q "basePath: '/app'" apps/frontend/next.config.js \
63
  && grep -q "productionBrowserSourceMaps: false" apps/frontend/next.config.js \
 
64
  && grep -q "cpus: 1" apps/frontend/next.config.js \
65
  || (echo "PATCH FAILED โ€” next.config.js shape changed upstream"; exit 1)
66
 
67
- # Sentry env stubs โ€” keep transitive Sentry imports from doing network calls.
68
  ENV SENTRY_DSN="" \
69
  SENTRY_AUTH_TOKEN="" \
70
  SENTRY_ORG="" \
@@ -73,44 +76,56 @@ ENV SENTRY_DSN="" \
73
  NEXT_TELEMETRY_DISABLED=1 \
74
  NEXT_PRIVATE_SKIP_SIZE_MINIMIZATION=true
75
 
76
- # Install all deps (shared pnpm virtual store for all workspace packages).
77
  RUN pnpm install --frozen-lockfile=false
78
 
79
- # Build server-side apps sequentially at 3 GB heap each.
80
- # Frontend is intentionally excluded โ€” built in its own stage below.
81
  RUN NODE_OPTIONS="--max-old-space-size=3072" pnpm run build:backend
82
  RUN NODE_OPTIONS="--max-old-space-size=3072" pnpm run build:workers
83
  RUN NODE_OPTIONS="--max-old-space-size=3072" pnpm run build:cron
84
 
85
- # Clean up dev artefacts before Stage 3 copies this tree into the runtime image.
86
  RUN find . -name ".git" -type d -prune -exec rm -rf {} + 2>/dev/null || true \
87
  && rm -rf .github reports Jenkins .devcontainer 2>/dev/null || true
88
 
89
 
90
- # โ”€โ”€ Stage 2: Build Next.js frontend in isolation โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
91
  FROM node:22.20-alpine AS postiz-frontend
92
 
93
  WORKDIR /build
94
 
95
- # pnpm must be present to run workspace scripts.
96
  RUN npm install -g pnpm@10.6.1
97
 
98
- # Copy the full build tree from Stage 1:
99
- # - patched apps/frontend/next.config.js
100
- # - node_modules (pnpm virtual store, all symlinks intact within the tree)
101
- # - already-built server apps (needed for any cross-package type references)
102
- # Stage 1's processes are dead here โ†’ its RSS is freed by the OS.
103
- # next build therefore starts with a clean address space.
104
- COPY --from=postiz-builder /build /build
105
-
106
- ENV NEXT_TELEMETRY_DISABLED=1 \
107
- NEXT_PRIVATE_SKIP_SIZE_MINIMIZATION=true \
108
- SENTRY_DSN="" \
 
 
 
109
  SENTRY_AUTH_TOKEN="" \
110
  SENTRY_ORG="" \
111
  SENTRY_PROJECT="" \
112
- NEXT_PUBLIC_SENTRY_DSN=""
 
 
 
 
 
 
 
 
113
 
 
 
114
  RUN NODE_OPTIONS="--max-old-space-size=3072" pnpm run build:frontend
115
 
116
 
@@ -119,8 +134,6 @@ FROM node:22.20-alpine
119
 
120
  WORKDIR /app
121
 
122
- # System deps โ€” same set as upstream's Dockerfile.dev (bash, nginx, py3-pip)
123
- # plus postgres + redis + extras we need.
124
  RUN apk add --no-cache \
125
  bash \
126
  curl \
@@ -140,41 +153,33 @@ RUN adduser -D -g 'www' www \
140
  && mkdir -p /var/lib/nginx /var/log/nginx \
141
  && chown -R www:www /var/lib/nginx
142
 
143
- # pnpm + pm2 to run Postiz processes the same way upstream does
144
  RUN npm install -g pnpm@10.6.1 pm2
145
 
146
- # Python deps for HF Dataset sync
147
  RUN pip install --no-cache-dir --break-system-packages \
148
  huggingface_hub \
149
  PyYAML
150
 
151
- # Copy server-side build (backend + workers + cron + node_modules, cleaned).
152
  COPY --from=postiz-builder /build /app
153
 
154
- # Overlay the compiled Next.js frontend from its isolated build stage.
 
155
  COPY --from=postiz-frontend /build/apps/frontend/.next /app/apps/frontend/.next
156
 
157
  # Use upstream's nginx.conf โ€” routes /apiโ†’3000, /uploadsโ†’fs, /โ†’4200.
158
- # health-server strips /app before forwarding, so nginx sees expected paths.
159
  COPY --from=postiz-builder /build/var/docker/nginx.conf /etc/nginx/nginx.conf
160
 
161
- # Health-server lives outside /app so its node_modules don't collide with
162
- # Postiz's pnpm workspaces.
163
  RUN mkdir -p /opt/healthsrv && cd /opt/healthsrv && \
164
  npm init -y >/dev/null && \
165
  npm install --no-save --no-audit --no-fund express@4 cors morgan
166
 
167
- # Postgres/Redis/uploads dirs โ€” all under /postiz so postiz-sync.py can
168
- # include them in the backup tarball.
169
  RUN mkdir -p /var/run/postgresql /postiz/pgdata /postiz/redis /postiz/uploads /postiz/.secrets \
170
  && chown -R postgres:postgres /var/run/postgresql /postiz/pgdata \
171
  && chmod 700 /postiz/pgdata
172
 
173
- # Symlink /uploads โ†’ /postiz/uploads so nginx's `alias /uploads/` picks up
174
- # media stored in the persisted tree.
175
  RUN ln -sf /postiz/uploads /uploads
176
 
177
- # Copy orchestration files
178
  COPY start.sh /opt/start.sh
179
  COPY health-server.js /opt/healthsrv/health-server.js
180
  COPY postiz-sync.py /opt/postiz-sync.py
 
3
  #
4
  # Three-stage build to beat the HF Space builder memory limit:
5
  #
6
+ # Stage 1 (postiz-builder): clone โ†’ patch โ†’ full install โ†’
7
+ # build backend + workers + cron
8
+ # Stage 2 (postiz-frontend): fresh clone โ†’ patch โ†’ FILTERED install
9
+ # (frontend dep tree only, skips NestJS/
10
+ # Prisma/bcrypt/etc.) โ†’ build Next.js
11
+ # Stage 3 (runtime): COPY server build from Stage 1
12
+ # overlay .next from Stage 2
 
13
  #
14
+ # Why fresh clone in Stage 2 (not COPY from Stage 1):
15
+ # COPY --from=stage1 /build /build copies ~2 GB of node_modules (3817
16
+ # packages). BuildKit decompresses that as a layer; the OS page-caches it.
17
+ # Then next build loads its own module graph on top. Combined RSS exceeds
18
+ # the builder cgroup limit โ†’ exit 137 OOMKilled.
19
+ # A filtered pnpm install in a fresh Stage 2 pulls only the frontend
20
+ # package's npm dependency tree โ€” maybe 30-50% of the full install โ€”
21
+ # so peak RSS stays within limits.
22
  #
23
  # Container layout at runtime:
24
+ # - nginx (port 5000, internal) โ€” Postiz frontend + backend + uploads
25
+ # - PM2 โ†’ 4 Postiz procs (backend / frontend / workers / cron)
26
  # - postgres (port 5432, internal)
27
  # - redis (port 6379, internal)
28
+ # - postiz-sync.py loop โ€” backup DB + uploads to HF Dataset
29
  # - health-server.js (port 7860, public) โ€” dashboard + reverse proxy
30
  # ============================================================================
31
 
32
+ # โ”€โ”€ Stage 1: Clone, patch, full install, build server apps โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
33
  FROM node:22.20-alpine AS postiz-builder
34
 
35
  WORKDIR /build
 
49
  # Pinned to v2.11.3 โ€” last release before Temporal became a hard requirement.
50
  RUN git clone --depth=1 --branch v2.11.3 https://github.com/gitroomhq/postiz-app.git .
51
 
52
+ # Patch Next.js config (applied here so Stage 2's fresh clone also patches).
53
+ # Stage 2 re-applies the same sed commands on its own clone.
54
+ # 1. basePath/assetPrefix=/app โ†’ Postiz UI at /app; dashboard owns /
55
+ # 2. productionBrowserSourceMaps: false โ†’ shaves ~500 MB RSS during emit
56
+ # 3. Sentry sourcemap plugin: disable: true โ†’ saves ~300 MB
57
+ # 4. swcMinify: false โ†’ forces Terser (pure JS, V8-heap-bounded) instead
58
+ # of the native SWC binary that adds RSS outside the V8 heap limit
59
+ # 5. experimental.cpus=1 + workerThreads=false โ†’ single-thread webpack;
60
+ # no parallel worker copies of the module graph eating extra RAM
61
+ RUN sed -i "s|const nextConfig = {|const nextConfig = {\n basePath: '/app',\n assetPrefix: '/app',\n swcMinify: false,|" apps/frontend/next.config.js \
62
  && sed -i "s|productionBrowserSourceMaps: true|productionBrowserSourceMaps: false|" apps/frontend/next.config.js \
63
  && sed -i "s|disable: false,|disable: true,|" apps/frontend/next.config.js \
64
  && sed -i "s|experimental: {|experimental: {\n cpus: 1,\n workerThreads: false,|" apps/frontend/next.config.js \
65
  && grep -q "basePath: '/app'" apps/frontend/next.config.js \
66
  && grep -q "productionBrowserSourceMaps: false" apps/frontend/next.config.js \
67
+ && grep -q "swcMinify: false" apps/frontend/next.config.js \
68
  && grep -q "cpus: 1" apps/frontend/next.config.js \
69
  || (echo "PATCH FAILED โ€” next.config.js shape changed upstream"; exit 1)
70
 
 
71
  ENV SENTRY_DSN="" \
72
  SENTRY_AUTH_TOKEN="" \
73
  SENTRY_ORG="" \
 
76
  NEXT_TELEMETRY_DISABLED=1 \
77
  NEXT_PRIVATE_SKIP_SIZE_MINIMIZATION=true
78
 
79
+ # Full install โ€” backend, workers, cron all need the complete dep tree.
80
  RUN pnpm install --frozen-lockfile=false
81
 
82
+ # Build server-side apps only. Frontend is built in its own isolated stage.
 
83
  RUN NODE_OPTIONS="--max-old-space-size=3072" pnpm run build:backend
84
  RUN NODE_OPTIONS="--max-old-space-size=3072" pnpm run build:workers
85
  RUN NODE_OPTIONS="--max-old-space-size=3072" pnpm run build:cron
86
 
87
+ # Remove dev artefacts before Stage 3 copies this tree into the runtime image.
88
  RUN find . -name ".git" -type d -prune -exec rm -rf {} + 2>/dev/null || true \
89
  && rm -rf .github reports Jenkins .devcontainer 2>/dev/null || true
90
 
91
 
92
+ # โ”€โ”€ Stage 2: Build Next.js frontend with minimal dep tree โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
93
  FROM node:22.20-alpine AS postiz-frontend
94
 
95
  WORKDIR /build
96
 
97
+ RUN apk add --no-cache git bash
98
  RUN npm install -g pnpm@10.6.1
99
 
100
+ # Fresh clone โ€” gives a clean slate with no Stage 1 memory residue.
101
+ RUN git clone --depth=1 --branch v2.11.3 https://github.com/gitroomhq/postiz-app.git .
102
+
103
+ # Apply the same patches as Stage 1.
104
+ RUN sed -i "s|const nextConfig = {|const nextConfig = {\n basePath: '/app',\n assetPrefix: '/app',\n swcMinify: false,|" apps/frontend/next.config.js \
105
+ && sed -i "s|productionBrowserSourceMaps: true|productionBrowserSourceMaps: false|" apps/frontend/next.config.js \
106
+ && sed -i "s|disable: false,|disable: true,|" apps/frontend/next.config.js \
107
+ && sed -i "s|experimental: {|experimental: {\n cpus: 1,\n workerThreads: false,|" apps/frontend/next.config.js \
108
+ && grep -q "basePath: '/app'" apps/frontend/next.config.js \
109
+ && grep -q "swcMinify: false" apps/frontend/next.config.js \
110
+ && grep -q "cpus: 1" apps/frontend/next.config.js \
111
+ || (echo "PATCH FAILED โ€” next.config.js shape changed upstream"; exit 1)
112
+
113
+ ENV SENTRY_DSN="" \
114
  SENTRY_AUTH_TOKEN="" \
115
  SENTRY_ORG="" \
116
  SENTRY_PROJECT="" \
117
+ NEXT_PUBLIC_SENTRY_DSN="" \
118
+ NEXT_TELEMETRY_DISABLED=1 \
119
+ NEXT_PRIVATE_SKIP_SIZE_MINIMIZATION=true
120
+
121
+ # Filtered install โ€” pulls only packages in the frontend's dependency tree.
122
+ # Skips NestJS, Prisma, bcrypt, Bull, and other server-only packages.
123
+ # Results in a much smaller node_modules โ†’ less OS page cache pressure
124
+ # โ†’ lower peak RSS during next build.
125
+ RUN pnpm install --filter "./apps/frontend..." --frozen-lockfile=false
126
 
127
+ # Build Next.js frontend in isolation.
128
+ # Stage 1's processes are dead; Stage 2 starts with a clean address space.
129
  RUN NODE_OPTIONS="--max-old-space-size=3072" pnpm run build:frontend
130
 
131
 
 
134
 
135
  WORKDIR /app
136
 
 
 
137
  RUN apk add --no-cache \
138
  bash \
139
  curl \
 
153
  && mkdir -p /var/lib/nginx /var/log/nginx \
154
  && chown -R www:www /var/lib/nginx
155
 
 
156
  RUN npm install -g pnpm@10.6.1 pm2
157
 
 
158
  RUN pip install --no-cache-dir --break-system-packages \
159
  huggingface_hub \
160
  PyYAML
161
 
162
+ # Copy server build (backend + workers + cron + full node_modules, cleaned).
163
  COPY --from=postiz-builder /build /app
164
 
165
+ # Overlay the compiled Next.js frontend from the isolated build stage.
166
+ # This overwrites the empty apps/frontend/.next placeholder in the tree above.
167
  COPY --from=postiz-frontend /build/apps/frontend/.next /app/apps/frontend/.next
168
 
169
  # Use upstream's nginx.conf โ€” routes /apiโ†’3000, /uploadsโ†’fs, /โ†’4200.
 
170
  COPY --from=postiz-builder /build/var/docker/nginx.conf /etc/nginx/nginx.conf
171
 
172
+ # Health-server outside /app to avoid pnpm workspace collisions.
 
173
  RUN mkdir -p /opt/healthsrv && cd /opt/healthsrv && \
174
  npm init -y >/dev/null && \
175
  npm install --no-save --no-audit --no-fund express@4 cors morgan
176
 
 
 
177
  RUN mkdir -p /var/run/postgresql /postiz/pgdata /postiz/redis /postiz/uploads /postiz/.secrets \
178
  && chown -R postgres:postgres /var/run/postgresql /postiz/pgdata \
179
  && chmod 700 /postiz/pgdata
180
 
 
 
181
  RUN ln -sf /postiz/uploads /uploads
182
 
 
183
  COPY start.sh /opt/start.sh
184
  COPY health-server.js /opt/healthsrv/health-server.js
185
  COPY postiz-sync.py /opt/postiz-sync.py