Change to subprocess

Dockerfile

@@ -12,7 +12,8 @@ ENV DEBIAN_FRONTEND=noninteractive \
     PIP_NO_BUILD_ISOLATION=1 \
     FRONTEND_VENV=/opt/frontend-py310 \
     BACKEND_VENV=/opt/backend-py39 \
-    BEATNET_SOCKET=/tmp/beatnet_backend.sock \
+    BACKEND_PYTHON=/opt/backend-py39/bin/python \
+    BACKEND_SCRIPT=/app/backend_worker.py \
     PORT=7860
 
 RUN apt-get update && apt-get install -y --no-install-recommends \

README.md

@@ -16,9 +16,9 @@ needs two Python interpreters in one container:
 
 - **Frontend:** Python 3.10 virtual environment for Gradio / pyharp.
 - **Backend:** Python 3.9 virtual environment for BeatNet and madmom.
-- **Transport:** JSON over a Unix-domain socket, with uploaded audio passed as a
-  file path. This keeps large audio payloads out of IPC and avoids importing
-  BeatNet/madmom in the frontend interpreter.
+- **Transport:** on-demand Python 3.9 subprocess calls, with uploaded audio
+  passed as a file path. This avoids importing BeatNet/madmom in the frontend
+  interpreter without keeping a backend process alive.
 
 The public `teamup-tech/BeatNet` Space currently uses a single
 `python:3.10-slim` Docker image and starts `app.py` directly. Replace that setup
@@ -31,23 +31,23 @@ with these files, or adapt the same pattern in-place.
 | `Dockerfile` | Builds separate Python 3.9 and 3.10 virtual environments. |
 | `requirements-backend-py39.txt` | BeatNet/madmom dependencies only. |
 | `requirements-frontend-py310.txt` | Gradio/pyharp dependencies only. |
-| `backend_worker.py` | Long-lived BeatNet worker running under Python 3.9. |
+| `backend_worker.py` | One-shot BeatNet subprocess runner under Python 3.9. |
 | `frontend_app.py` | Gradio app running under Python 3.10. |
-| `start.sh` | Starts the backend worker, then execs the frontend. |
+| `start.sh` | Starts the frontend. |
 
-## Why sockets instead of importing both stacks?
+## Why a subprocess instead of importing both stacks?
 
 `madmom` and BeatNet can be sensitive to Python, NumPy, Cython, and build
 isolation versions. Running BeatNet/madmom in a backend-only Python 3.9
 environment avoids the madmom compatibility patching needed by newer Python
 versions, while the public web layer runs independently on Python 3.10. The
-frontend and backend exchange only stable JSON data:
+frontend invokes the backend script only when processing audio:
 
-```json
-{"audio_path": "/tmp/gradio/upload.wav"}
+```sh
+/opt/backend-py39/bin/python /app/backend_worker.py /tmp/gradio/upload.wav
 ```
 
-The backend responds with:
+The backend prints stable JSON to stdout:
 
 ```json
 {"ok": true, "beats": [{"time": 0.42, "beat": 1.0}]}
@@ -64,10 +64,9 @@ The backend responds with:
 4. Push to the Space. Hugging Face will build the Docker image and route traffic
    to `PORT=7860`.
 
-## Alternative: file-queue transport
+## Alternative: persistent worker transport
 
-If Unix sockets are not desired, keep the same two virtual environments and have
-the frontend write request JSON to `/tmp/beatnet_queue/inbox/<uuid>.json`; the
-backend atomically moves the request into a processing directory and writes the
-response to `/tmp/beatnet_queue/outbox/<uuid>.json`. Sockets are preferred here
-because there is no polling delay and no stale-file cleanup path.
+If startup latency becomes a problem, keep the same two virtual environments and
+run `backend_worker.py` as a long-lived process behind a Unix socket or file
+queue. The subprocess approach uses less idle memory, while a persistent worker
+can reuse the loaded BeatNet model across requests.

backend_worker.py

@@ -1,73 +1,48 @@
 #!/usr/bin/env python3
-"""BeatNet backend worker for the Python 3.9 environment.
+"""One-shot BeatNet backend runner for the Python 3.9 environment.
 
-The worker owns the BeatNet import and model initialization.  The frontend sends
-small JSON messages over a Unix-domain socket and passes uploaded audio by file
-path, avoiding cross-interpreter object sharing.
+The frontend invokes this script as a subprocess and passes uploaded audio by
+file path, avoiding cross-interpreter object sharing without keeping a backend
+worker process alive.
 """
 
 from __future__ import annotations
 
+import contextlib
 import json
-import os
-import socket
-from pathlib import Path
+import sys
 from typing import Any
 
-from BeatNet.BeatNet import BeatNet
 
-SOCKET_PATH = Path(os.environ.get("BEATNET_SOCKET", "/tmp/beatnet_backend.sock"))
+def run_beatnet(audio_path: str) -> list[dict[str, float]]:
+    with contextlib.redirect_stdout(sys.stderr):
+        from BeatNet.BeatNet import BeatNet
 
+        estimator = BeatNet(1, mode="offline", inference_model="DBN", plot=[], thread=False)
+        beats = estimator.process(audio_path)
 
-def _read_message(conn: socket.socket) -> dict[str, Any]:
-    chunks: list[bytes] = []
-    while True:
-        chunk = conn.recv(65536)
-        if not chunk:
-            break
-        chunks.append(chunk)
-    if not chunks:
-        return {}
-    return json.loads(b"".join(chunks).decode("utf-8"))
+    return [
+        {"time": float(timestamp), "beat": float(beat)}
+        for timestamp, beat in beats
+    ]
 
 
-def _send_message(conn: socket.socket, payload: dict[str, Any]) -> None:
-    conn.sendall(json.dumps(payload).encode("utf-8"))
+def main(argv: list[str]) -> int:
+    if len(argv) != 2:
+        print(
+            json.dumps({"ok": False, "error": "Usage: backend_worker.py <audio_path>"}),
+            flush=True,
+        )
+        return 2
 
+    try:
+        payload: dict[str, Any] = {"ok": True, "beats": run_beatnet(argv[1])}
+    except Exception as exc:
+        payload = {"ok": False, "error": repr(exc)}
 
-def serve() -> None:
-    SOCKET_PATH.parent.mkdir(parents=True, exist_ok=True)
-    if SOCKET_PATH.exists():
-        SOCKET_PATH.unlink()
-
-    estimator = BeatNet(1, mode="offline", inference_model="DBN", plot=[], thread=False)
-
-    with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as server:
-        server.bind(str(SOCKET_PATH))
-        server.listen(16)
-        os.chmod(SOCKET_PATH, 0o666)
-        print(f"BeatNet backend listening on {SOCKET_PATH}", flush=True)
-
-        while True:
-            conn, _ = server.accept()
-            with conn:
-                try:
-                    request = _read_message(conn)
-                    audio_path = request["audio_path"]
-                    beats = estimator.process(audio_path)
-                    _send_message(
-                        conn,
-                        {
-                            "ok": True,
-                            "beats": [
-                                {"time": float(timestamp), "beat": float(beat)}
-                                for timestamp, beat in beats
-                            ],
-                        },
-                    )
-                except Exception as exc:  # Keep worker alive across bad requests.
-                    _send_message(conn, {"ok": False, "error": repr(exc)})
+    print(json.dumps(payload), flush=True)
+    return 0 if payload["ok"] else 1
 
 
 if __name__ == "__main__":
-    serve()
+    raise SystemExit(main(sys.argv))

frontend_app.py

@@ -5,41 +5,37 @@ from __future__ import annotations
 
 import json
 import os
-import socket
-import time
-from pathlib import Path
+import subprocess
 from typing import Any
 
 import gradio as gr
 from pyharp import AudioLabel, LabelList, OutputLabel
 from pyharp.core import ModelCard, build_endpoint
 
-SOCKET_PATH = Path(os.environ.get("BEATNET_SOCKET", "/tmp/beatnet_backend.sock"))
+BACKEND_PYTHON = os.environ.get("BACKEND_PYTHON", "/opt/backend-py39/bin/python")
+BACKEND_SCRIPT = os.environ.get("BACKEND_SCRIPT", "/app/backend_worker.py")
 
 
 def call_backend(audio_path: str, timeout_s: float = 120.0) -> list[dict[str, float]]:
-    deadline = time.time() + timeout_s
-    while not SOCKET_PATH.exists():
-        if time.time() > deadline:
-            raise TimeoutError(f"Timed out waiting for BeatNet backend at {SOCKET_PATH}")
-        time.sleep(0.1)
-
-    with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client:
-        client.settimeout(timeout_s)
-        client.connect(str(SOCKET_PATH))
-        client.sendall(json.dumps({"audio_path": audio_path}).encode("utf-8"))
-        client.shutdown(socket.SHUT_WR)
-
-        chunks: list[bytes] = []
-        while True:
-            chunk = client.recv(65536)
-            if not chunk:
-                break
-            chunks.append(chunk)
-
-    response: dict[str, Any] = json.loads(b"".join(chunks).decode("utf-8"))
+    completed = subprocess.run(
+        [BACKEND_PYTHON, BACKEND_SCRIPT, audio_path],
+        capture_output=True,
+        check=False,
+        text=True,
+        timeout=timeout_s,
+    )
+
+    try:
+        response: dict[str, Any] = json.loads(completed.stdout)
+    except json.JSONDecodeError as exc:
+        raise RuntimeError(
+            "BeatNet backend returned invalid JSON. "
+            f"stdout={completed.stdout!r} stderr={completed.stderr!r}"
+        ) from exc
+
     if not response.get("ok"):
-        raise RuntimeError(response.get("error", "BeatNet backend failed"))
+        error = response.get("error") or completed.stderr or "BeatNet backend failed"
+        raise RuntimeError(error)
     return response["beats"]
 
 

start.sh

@@ -1,14 +1,4 @@
 #!/usr/bin/env bash
 set -euo pipefail
 
-rm -f "${BEATNET_SOCKET:-/tmp/beatnet_backend.sock}"
-
-"${BACKEND_VENV:-/opt/backend-py39}/bin/python" -u /app/backend_worker.py &
-backend_pid="$!"
-
-cleanup() {
-  kill "$backend_pid" 2>/dev/null || true
-}
-trap cleanup EXIT INT TERM
-
 exec "${FRONTEND_VENV:-/opt/frontend-py310}/bin/python" -u /app/frontend_app.py