Files changed (4) hide show
  1. .env.example +5 -0
  2. Dockerfile +9 -9
  3. README.md +39 -332
  4. docker-compose.yml +3 -3
.env.example CHANGED
@@ -7,6 +7,11 @@ HOST=0.0.0.0
7
  PORT=8000
8
  DEBUG=false
9
 
 
 
 
 
 
10
  # ── Model ─────────────────────────────────────────────────────────────────────
11
  # Recommended CPU models (smallest to largest):
12
  #
 
7
  PORT=8000
8
  DEBUG=false
9
 
10
+ # ── Hugging Face ──────────────────────────────────────────────────────────────
11
+ # Get your free token at https://huggingface.co/settings/tokens
12
+ # Removes the rate-limit warning and speeds up model file resolution
13
+ HF_TOKEN=
14
+
15
  # ── Model ─────────────────────────────────────────────────────────────────────
16
  # Recommended CPU models (smallest to largest):
17
  #
Dockerfile CHANGED
@@ -39,21 +39,21 @@ COPY . .
39
  # Create runtime directories
40
  RUN mkdir -p outputs model_cache
41
 
42
- # Non-root user for security
43
  RUN useradd -m -u 1000 appuser \
44
  && chown -R appuser:appuser /app
45
  USER appuser
46
 
47
- # Expose API port
48
- EXPOSE 8000
49
 
50
- # Health check — hits the /health endpoint every 30 s
51
- HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 \
52
- CMD curl -f http://localhost:8000/health || exit 1
53
 
54
- # Default env (override with docker run -e or docker-compose)
55
  ENV HOST=0.0.0.0 \
56
- PORT=8000 \
57
  MODEL_ID=nota-ai/bk-sdm-small \
58
  MODEL_CACHE_DIR=/app/model_cache \
59
  AUTO_LOAD_MODEL=true \
@@ -70,4 +70,4 @@ ENV HOST=0.0.0.0 \
70
  JOB_TTL_SECONDS=0
71
 
72
  CMD ["python", "-m", "uvicorn", "main:app", \
73
- "--host", "0.0.0.0", "--port", "8000", "--workers", "1"]
 
39
  # Create runtime directories
40
  RUN mkdir -p outputs model_cache
41
 
42
+ # HF Spaces runs containers as UID 1000 — match it
43
  RUN useradd -m -u 1000 appuser \
44
  && chown -R appuser:appuser /app
45
  USER appuser
46
 
47
+ # ── Port 7860 is REQUIRED by Hugging Face Spaces ──────────────────────────────
48
+ EXPOSE 7860
49
 
50
+ # Health check — longer start_period for model download on cold start
51
+ HEALTHCHECK --interval=30s --timeout=10s --start-period=120s --retries=5 \
52
+ CMD curl -f http://localhost:7860/health || exit 1
53
 
54
+ # Default env (override via HF Spaces Secrets or docker-compose)
55
  ENV HOST=0.0.0.0 \
56
+ PORT=7860 \
57
  MODEL_ID=nota-ai/bk-sdm-small \
58
  MODEL_CACHE_DIR=/app/model_cache \
59
  AUTO_LOAD_MODEL=true \
 
70
  JOB_TTL_SECONDS=0
71
 
72
  CMD ["python", "-m", "uvicorn", "main:app", \
73
+ "--host", "0.0.0.0", "--port", "7860", "--workers", "1"]
README.md CHANGED
@@ -1,352 +1,59 @@
1
  ---
2
- license: mit
3
- title: hf-image-api
 
 
4
  sdk: docker
5
- emoji: 💻
6
- colorFrom: purple
7
- colorTo: yellow
8
- pinned: true
9
  ---
 
10
  # HF CPU Image Generation API
11
 
12
- A production-ready REST API for **Stable Diffusion image generation on CPU** using Hugging Face Diffusers. Built on top of the architecture patterns from [Local AI Image Generator](https://github.com/techjarves/Local-AI-Image-Generator) and redesigned as a stateless, job-queue-based HTTP API.
13
 
14
- ---
15
 
16
- ## Architecture Overview
17
 
18
  ```
19
- Client
20
- │
21
- │ POST /generate → { job_id, status: "queued" }
22
- │
23
- │ GET /jobs/{id} → { status: "processing", progress: 45 }
24
- │ GET /jobs/{id} → { status: "completed", download_url: "..." }
25
- │
26
- │ GET /jobs/{id}/download → image.png
27
- │
28
- │ DELETE /jobs/{id} → { ok: true }
29
- │
30
- ▼
31
-
32
- ┌─────────────────────────────────────────────────────────────┐
33
- │ FastAPI (main.py) │
34
- │ ├── POST /generate → enqueue job, return job_id │
35
- │ ├── GET /jobs → list / filter jobs │
36
- │ ├── GET /jobs/{id} → status + progress │
37
- │ ├── GET /jobs/{id}/download → FileResponse (PNG) │
38
- │ ├── DELETE /jobs/{id} → delete job + image │
39
- │ ├── GET /health → pipeline & queue status │
40
- │ └── POST /models/load → hot-swap model │
41
- │ │
42
- │ job_store.py — thread-safe in-memory job registry │
43
- │ worker.py — background daemon thread + FIFO queue │
44
- │ pipeline_manager.py — HF Diffusers pipeline (CPU-tuned) │
45
- └─────────────────────────────────────────────────────────────┘
46
- │
47
- ┌────────▼────────┐
48
- │ outputs/ │
49
- │ (PNG files) │
50
- └─────────────────┘
51
  ```
52
 
53
- ---
54
-
55
- ## Quick Start
56
-
57
- ### Prerequisites
58
- - Python 3.9 or higher
59
- - 4 GB+ RAM (8 GB recommended for SD v1.5)
60
- - Internet connection on first run (to download model weights)
61
-
62
- ### 1. Install
63
 
64
- ```bash
65
- git clone <this-repo>
66
- cd hf-image-api
67
-
68
- # Linux / macOS — one command setup:
69
- bash start.sh
70
-
71
- # Windows:
72
- python -m venv .venv
73
- .venv\Scripts\activate
74
- pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
75
- pip install -r requirements.txt
76
- copy .env.example .env
77
- python -m uvicorn main:app --host 0.0.0.0 --port 8000 --workers 1
78
- ```
79
 
80
- ### 2. Open the interactive docs
81
 
82
- ```
83
- http://localhost:8000/docs
84
- ```
 
 
85
 
86
- ---
87
 
88
- ## CPU Performance Guide
89
 
90
- | Model | Size | 512×512 @ 20 steps | Notes |
91
- |---|---|---|---|
92
- | `nota-ai/bk-sdm-small` | ~500 MB | **~60–120 s** | Best for CPU — default |
93
- | `runwayml/stable-diffusion-v1-5` | ~4 GB | ~150–300 s | Standard quality |
94
- | `stabilityai/stable-diffusion-2-1-base` | ~5 GB | ~200–400 s | Highest quality |
95
 
96
- ### Optimisations applied automatically
97
 
98
- | Optimisation | Effect |
99
  |---|---|
100
- | `torch.float32` | Mandatory for CPU (float16 causes NaN) |
101
- | `low_cpu_mem_usage=True` | Streams weights into RAM — avoids double-buffering |
102
- | `enable_attention_slicing(1)` | Lowers peak RAM ~30 %, minimal speed cost |
103
- | `DPMSolverMultistepScheduler` + Karras sigmas | Converges well at 15–20 steps |
104
- | `safety_checker=None` | Skips NSFW classifier, saves ~300 MB RAM |
105
- | `torch.set_num_threads(N)` | Saturates all logical CPU cores |
106
- | `torch.inference_mode()` | Disables autograd, reduces overhead |
107
-
108
- ### Tips to speed up CPU generation
109
- - Use **`nota-ai/bk-sdm-small`** — it is specifically distilled for speed
110
- - Keep images at **512×512** (larger = much slower on SD v1.x)
111
- - Use **20 steps with DPM++** (better than 50 steps with PNDM)
112
- - Set `CPU_THREADS=0` in `.env` to use all logical cores
113
- - Avoid running other CPU-heavy workloads during generation
114
-
115
- ---
116
-
117
- ## API Reference
118
-
119
- ### Submit a generation job
120
-
121
- ```
122
- POST /generate
123
- ```
124
-
125
- **Request body:**
126
- ```json
127
- {
128
- "prompt": "a photorealistic mountain landscape at golden hour",
129
- "negative_prompt": "blurry, low quality, watermark",
130
- "width": 512,
131
- "height": 512,
132
- "num_inference_steps": 20,
133
- "guidance_scale": 7.5,
134
- "seed": -1,
135
- "scheduler": "dpm++"
136
- }
137
- ```
138
-
139
- **Schedulers:** `dpm++` (best), `euler_a`, `euler`, `pndm`
140
-
141
- **Response `202 Accepted`:**
142
- ```json
143
- {
144
- "job_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
145
- "status": "queued",
146
- "created_at": "2024-06-01T12:00:00Z",
147
- "queue_position": 1,
148
- "message": "Job is next in queue and will start immediately."
149
- }
150
- ```
151
-
152
- ---
153
-
154
- ### Check job status
155
-
156
- ```
157
- GET /jobs/{job_id}
158
- ```
159
-
160
- **While processing:**
161
- ```json
162
- {
163
- "job_id": "3fa85f64-...",
164
- "status": "processing",
165
- "progress": 45,
166
- "current_step": 9,
167
- "total_steps": 20,
168
- "started_at": "2024-06-01T12:00:01Z"
169
- }
170
- ```
171
-
172
- **When completed:**
173
- ```json
174
- {
175
- "job_id": "3fa85f64-...",
176
- "status": "completed",
177
- "progress": 100,
178
- "download_url": "/jobs/3fa85f64-.../download",
179
- "image_filename": "20240601_120045_3fa85f64_42.png",
180
- "duration_seconds": 87.3,
181
- "completed_at": "2024-06-01T12:01:28Z"
182
- }
183
- ```
184
-
185
- ---
186
-
187
- ### Download the image
188
-
189
- ```
190
- GET /jobs/{job_id}/download
191
- ```
192
-
193
- Returns the PNG as a file attachment. Returns `202` if still processing.
194
-
195
- ---
196
-
197
- ### Delete a job and its image
198
-
199
- ```
200
- DELETE /jobs/{job_id}
201
- ```
202
-
203
- Removes the job record and deletes the PNG from disk.
204
-
205
- ```json
206
- {
207
- "ok": true,
208
- "job_id": "3fa85f64-...",
209
- "image_deleted": true,
210
- "message": "Job deleted. Image file removed from disk."
211
- }
212
- ```
213
-
214
- ---
215
-
216
- ### List all jobs
217
-
218
- ```
219
- GET /jobs?status=completed&limit=50&offset=0
220
- ```
221
-
222
- Status filter options: `queued`, `processing`, `completed`, `failed`
223
-
224
- ---
225
-
226
- ### Health check
227
-
228
- ```
229
- GET /health
230
- ```
231
-
232
- ```json
233
- {
234
- "status": "ok",
235
- "model_id": "nota-ai/bk-sdm-small",
236
- "pipeline_loaded": true,
237
- "loading": false,
238
- "queue_size": 2,
239
- "jobs_total": 15,
240
- "jobs_by_status": {
241
- "queued": 2,
242
- "processing": 1,
243
- "completed": 11,
244
- "failed": 1
245
- },
246
- "cpu_threads": 8
247
- }
248
- ```
249
-
250
- ---
251
-
252
- ### Hot-swap model
253
-
254
- ```
255
- POST /models/load
256
- Content-Type: application/json
257
-
258
- { "model_id": "runwayml/stable-diffusion-v1-5" }
259
- ```
260
-
261
- Loads the new model in the background. Poll `GET /health` to confirm `pipeline_loaded: true`.
262
-
263
- ---
264
-
265
- ## Example: Full workflow with curl
266
-
267
- ```bash
268
- # 1. Check service is ready
269
- curl http://localhost:8000/health | jq .pipeline_loaded
270
-
271
- # 2. Submit a generation job
272
- JOB=$(curl -s -X POST http://localhost:8000/generate \
273
- -H "Content-Type: application/json" \
274
- -d '{
275
- "prompt": "a serene Japanese garden with koi pond, watercolor style",
276
- "negative_prompt": "low quality, blurry, cartoon",
277
- "width": 512,
278
- "height": 512,
279
- "num_inference_steps": 20,
280
- "guidance_scale": 7.5,
281
- "seed": 42,
282
- "scheduler": "dpm++"
283
- }')
284
- echo $JOB | jq .
285
-
286
- JOB_ID=$(echo $JOB | jq -r .job_id)
287
-
288
- # 3. Poll until completed
289
- while true; do
290
- STATUS=$(curl -s http://localhost:8000/jobs/$JOB_ID | jq -r .status)
291
- PROGRESS=$(curl -s http://localhost:8000/jobs/$JOB_ID | jq -r .progress)
292
- echo "Status: $STATUS Progress: $PROGRESS%"
293
- [ "$STATUS" = "completed" ] || [ "$STATUS" = "failed" ] && break
294
- sleep 5
295
- done
296
-
297
- # 4. Download the image
298
- curl -s http://localhost:8000/jobs/$JOB_ID/download -o output.png
299
- echo "Saved to output.png"
300
-
301
- # 5. Delete the job + image
302
- curl -s -X DELETE http://localhost:8000/jobs/$JOB_ID | jq .
303
- ```
304
-
305
- ---
306
-
307
- ## Configuration (`.env`)
308
-
309
- | Variable | Default | Description |
310
- |---|---|---|
311
- | `MODEL_ID` | `nota-ai/bk-sdm-small` | Hugging Face model repo ID |
312
- | `MODEL_CACHE_DIR` | `./model_cache` | Where weights are cached |
313
- | `AUTO_LOAD_MODEL` | `true` | Load model at startup |
314
- | `CPU_THREADS` | `0` (all cores) | PyTorch thread count |
315
- | `ENABLE_ATTENTION_SLICING` | `true` | Lower peak RAM usage |
316
- | `DEFAULT_STEPS` | `20` | Default inference steps |
317
- | `MAX_STEPS` | `50` | Request cap |
318
- | `DEFAULT_WIDTH/HEIGHT` | `512` | Default image dimensions |
319
- | `MAX_WIDTH/HEIGHT` | `768` | Request cap |
320
- | `OUTPUTS_DIR` | `./outputs` | Where PNGs are saved |
321
- | `PORT` | `8000` | HTTP port |
322
- | `DEBUG` | `false` | Verbose logging + auto-reload |
323
-
324
- ---
325
-
326
- ## File Structure
327
-
328
- ```
329
- hf-image-api/
330
- ├── main.py # FastAPI app + lifespan (startup/shutdown)
331
- ├── config.py # Settings from environment / .env
332
- ├── schemas.py # All Pydantic request & response models
333
- ├── job_store.py # Thread-safe in-memory job registry
334
- ├── pipeline_manager.py # HuggingFace Diffusers pipeline (CPU-tuned)
335
- ├── worker.py # Background worker thread + FIFO job queue
336
- ├── routes/
337
- │ ├── generate.py # POST /generate
338
- │ ├── jobs.py # GET/DELETE /jobs and /jobs/{id}/download
339
- │ └── health.py # GET /health, GET /models, POST /models/load
340
- ├── outputs/ # Generated PNG files (auto-created)
341
- ├── model_cache/ # HuggingFace model weights cache (auto-created)
342
- ├── requirements.txt
343
- ├── .env.example
344
- ├── start.sh
345
- └── README.md
346
- ```
347
-
348
- ---
349
-
350
- ## License
351
-
352
- MIT — free to use and modify.
 
1
  ---
2
+ title: HF Image API
3
+ emoji: 🖼️
4
+ colorFrom: blue
5
+ colorTo: purple
6
  sdk: docker
7
+ pinned: false
8
+ app_port: 7860
 
 
9
  ---
10
+
11
  # HF CPU Image Generation API
12
 
13
+ A job-queue REST API for Stable Diffusion image generation on CPU using Hugging Face Diffusers.
14
 
15
+ **Swagger UI:** `https://your-space-url.hf.space/docs`
16
 
17
+ ## Job flow
18
 
19
  ```
20
+ POST /generate → { job_id, status: "queued" }
21
+ GET /jobs/{id} → { status: "processing", progress: 45 }
22
+ GET /jobs/{id} → { status: "completed", download_url: "..." }
23
+ GET /jobs/{id}/download → image.png
24
+ DELETE /jobs/{id} → removes job + image
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
  ```
26
 
27
+ ## API endpoints
 
 
 
 
 
 
 
 
 
28
 
29
+ | Method | Path | Description |
30
+ |---|---|---|
31
+ | `POST` | `/generate` | Submit a generation job |
32
+ | `GET` | `/jobs` | List all jobs |
33
+ | `GET` | `/jobs/{id}` | Poll job status & progress |
34
+ | `GET` | `/jobs/{id}/download` | Download the PNG |
35
+ | `DELETE` | `/jobs/{id}` | Delete job + image |
36
+ | `GET` | `/health` | Pipeline & queue status |
37
+ | `GET` | `/models` | List supported models |
38
+ | `POST` | `/models/load` | Hot-swap model |
 
 
 
 
 
39
 
40
+ ## CPU optimisations
41
 
42
+ - `torch.float32` (mandatory for CPU)
43
+ - `DPMSolverMultistepScheduler` — converges at 20 steps
44
+ - `enable_attention_slicing` — lower peak RAM
45
+ - `low_cpu_mem_usage=True` — no double-buffering
46
+ - `safety_checker=None` — saves ~300 MB RAM
47
 
48
+ ## Default model
49
 
50
+ `nota-ai/bk-sdm-small` (~500 MB) — fastest on CPU, ~60–120s per 512×512 image.
51
 
52
+ ## HF Spaces Secrets
 
 
 
 
53
 
54
+ Set these under **Settings → Secrets** in your Space:
55
 
56
+ | Secret | Description |
57
  |---|---|
58
+ | `HF_TOKEN` | Your HuggingFace token (faster downloads) |
59
+ | `MODEL_ID` | Override model (e.g. `runwayml/stable-diffusion-v1-5`) |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
docker-compose.yml CHANGED
@@ -9,7 +9,7 @@ services:
9
  container_name: hf-image-api
10
 
11
  ports:
12
- - "8000:8000"
13
 
14
  # ── Persistent volumes ─────────────────────────────────────────────────────
15
  volumes:
@@ -50,7 +50,7 @@ services:
50
  restart: unless-stopped
51
 
52
  healthcheck:
53
- test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
54
  interval: 30s
55
  timeout: 10s
56
  retries: 3
@@ -59,4 +59,4 @@ services:
59
  # ── Named volumes ──────────────────────────────────────────────────────────────
60
  volumes:
61
  model_cache:
62
- driver: local
 
9
  container_name: hf-image-api
10
 
11
  ports:
12
+ - "8000:7860"
13
 
14
  # ── Persistent volumes ─────────────────────────────────────────────────────
15
  volumes:
 
50
  restart: unless-stopped
51
 
52
  healthcheck:
53
+ test: ["CMD", "curl", "-f", "http://localhost:7860/health"]
54
  interval: 30s
55
  timeout: 10s
56
  retries: 3
 
59
  # ── Named volumes ──────────────────────────────────────────────────────────────
60
  volumes:
61
  model_cache:
62
+ driver: local