HuggingRun: Run anything on Hugging Face — generic Docker + persistence

.env.example

@@ -0,0 +1,20 @@
+# HuggingRun — set as Repository secrets in HF Space (or .env for local Docker)
+
+# Required for persistence (optional for demo-only)
+HF_TOKEN=
+HF_DATASET_REPO=
+
+# Optional: auto-create dataset repo on first run
+AUTO_CREATE_DATASET=false
+
+# Persistence path (default /data). Your app should read/write here.
+PERSIST_PATH=/data
+
+# Your app command. Empty = run default demo (HTTP server on 7860).
+RUN_CMD=
+
+# Sync interval in seconds
+SYNC_INTERVAL=60
+
+# Port for default demo (HF Spaces expect 7860)
+PORT=7860

.gitignore

@@ -0,0 +1,10 @@
+.env
+.venv
+__pycache__
+*.pyc
+.pytest_cache
+.coverage
+htmlcov/
+*.egg-info/
+dist/
+build/

DESIGN.md

@@ -0,0 +1,39 @@
+# HuggingRun 设计
+
+**Slogan:** Run anything on Hugging Face.
+
+## 目标
+
+- 把 Hugging Face Spaces 当成**通用容器**：只要应用能 Docker 化，就能在 HF 上跑。
+- 参考 HuggingClaw 的持久化与入口模式，做成**可配置**的通用运行时。
+- 通过至少一个「难例」部署验证（如 FastAPI+SQLite 或带状态的 Web 应用）。
+
+## 架构
+
+1. **单一 Docker 镜像**
+   - 使用 `ARG BASE_IMAGE` 选择基础镜像（默认 `python:3.11-slim`）。
+   - 在基础镜像上安装 Python3 + `huggingface_hub`（若基础镜像无），并加入通用脚本。
+   - 通过 `ENV RUN_CMD` 指定要执行的命令（应用由用户通过 CMD/环境变量提供）。
+
+2. **入口脚本 `entrypoint.sh`**
+   - 调用 `sync_hf.py` 做启动时恢复与周期性上传（可选）。
+   - 最后 `exec` 执行 `RUN_CMD`（或 Dockerfile 的 CMD），使主进程为 PID 1。
+
+3. **持久化 `sync_hf.py`**
+   - 可配置持久化目录 `PERSIST_PATH`（默认 `/data`）。
+   - 可配置 HF Dataset repo：`HF_DATASET_REPO` 或由 `SPACE_ID` 推导 `{SPACE_ID}-data`。
+   - 启动时：若存在 `HF_TOKEN` 与 repo，则 `snapshot_download` 到 `PERSIST_PATH`。
+   - 后台线程：按 `SYNC_INTERVAL` 将 `PERSIST_PATH` 上传到 Dataset。
+   - 退出时：最后一次上传。
+
+4. **HF Spaces 约定**
+   - `sdk: docker`，`app_port: 7860`。
+   - 用户需在 Space 中设置 `RUN_CMD`（以及可选 `HF_TOKEN`、`HF_DATASET_REPO`、`PERSIST_PATH`）。
+   - 若使用默认 demo，`RUN_CMD` 启动监听 7860 的简单服务并读写 `PERSIST_PATH`，用于验证持久化。
+
+## 完成标准（迭代开发）
+
+- [ ] 本地 `docker build` 与 `docker run` 成功，默认 demo 在 7860 响应。
+- [ ] 持久化：重启容器后，写入 `PERSIST_PATH` 的内容仍在（通过 HF Dataset 恢复）。
+- [ ] 至少一个难例（如 FastAPI+SQLite 或 Gradio+状态）在 README 中说明并在 HF 上可复现。
+- [ ] 代码推送到 HF Space，且在该 Space 中成功运行并通过验证。

Dockerfile

@@ -0,0 +1,35 @@
+# HuggingRun — Run anything on Hugging Face
+# Optional build arg: BASE_IMAGE (default python:3.11-slim)
+ARG BASE_IMAGE=python:3.11-slim
+FROM ${BASE_IMAGE}
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+# Ensure Python and pip for sync script (base may not have them if BASE_IMAGE is e.g. node)
+RUN (command -v python3 >/dev/null 2>&1 || (apt-get update && apt-get install -y --no-install-recommends python3 python3-pip && rm -rf /var/lib/apt/lists/*)) \
+    && pip3 install --no-cache-dir --break-system-packages huggingface_hub 2>/dev/null || pip3 install --no-cache-dir huggingface_hub \
+    && pip3 install --no-cache-dir --break-system-packages fastapi uvicorn 2>/dev/null || pip3 install --no-cache-dir fastapi uvicorn
+
+# HF Spaces run as user 1000
+RUN useradd -m -u 1000 user
+ENV HOME=/home/user
+WORKDIR /home/user
+
+COPY --chown=user:user scripts /scripts
+COPY --chown=user:user app /app
+RUN chmod +x /scripts/entrypoint.sh
+
+# Default: run demo app. Override with RUN_CMD in Space secrets.
+ENV PERSIST_PATH=/data
+ENV RUN_CMD=""
+ENV PORT=7860
+ENV PYTHONPATH=/app
+
+# Persist path must be writable by user 1000
+RUN mkdir -p /data && chown user:user /data
+
+USER user
+EXPOSE 7860
+ENTRYPOINT ["/scripts/entrypoint.sh"]

README.md

@@ -0,0 +1,76 @@
+---
+title: HuggingRun
+emoji: 🚀
+colorFrom: blue
+colorTo: indigo
+sdk: docker
+pinned: false
+license: mit
+app_port: 7860
+tags:
+  - docker
+  - huggingface-spaces
+  - persistence
+  - deployment
+  - run-anything
+---
+
+# HuggingRun
+
+**Run anything on Hugging Face.**
+
+Turn Hugging Face Spaces into a generic container: run any Docker-friendly app with optional persistence via HF Datasets. No Mac Mini, no paid cloud — 2 vCPU, 16GB RAM, 50GB storage, free tier.
+
+## Why HuggingRun?
+
+- **Generic**: Not tied to one app. Set `RUN_CMD` to run your server, API, or bot.
+- **Persistent**: Sync a directory (e.g. `/data`) to a private HF Dataset so state survives restarts.
+- **One Dockerfile**: Parameterized with `BASE_IMAGE` and `RUN_CMD`; duplicate the Space and configure.
+
+## Quick Start
+
+### 1. Duplicate this Space
+
+Click **Duplicate this Space** on the HuggingRun Space page. After duplicating, set your Space **Repository secrets**:
+
+| Secret | Required | Description |
+|--------|----------|-------------|
+| `HF_TOKEN` | For persistence | HF token with write access ([create](https://huggingface.co/settings/tokens)) |
+| `HF_DATASET_REPO` | Optional | Dataset repo for backup (e.g. `your-name/HuggingRun-data`). Default: `{SPACE_ID}-data` |
+| `RUN_CMD` | Optional | Command to run (e.g. `python3 my_app.py`). Empty = default demo on port 7860 |
+
+### 2. Persistence (optional)
+
+- **Manual**: Create a private Dataset repo (e.g. `your-name/HuggingRun-data`), set `HF_DATASET_REPO` and `HF_TOKEN`. Data under `PERSIST_PATH` (default `/data`) syncs every `SYNC_INTERVAL` seconds (default 60).
+- **Auto**: Set `AUTO_CREATE_DATASET=true` and `HF_TOKEN`; a private dataset is created on first run.
+
+### 3. Open your Space
+
+Visit your Space URL. You’ll see the default demo (visit counter with persistence). To run your own app, set `RUN_CMD` in secrets and rebuild.
+
+## Environment variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `RUN_CMD` | (default demo) | Command to run (e.g. `uvicorn app:app --host 0.0.0.0 --port 7860`) |
+| `PERSIST_PATH` | `/data` | Directory to sync to the HF Dataset |
+| `HF_TOKEN` | — | HF token for persistence |
+| `HF_DATASET_REPO` | `{SPACE_ID}-data` | Dataset repo ID |
+| `AUTO_CREATE_DATASET` | `false` | Create dataset repo if missing |
+| `SYNC_INTERVAL` | `60` | Seconds between uploads |
+| `PORT` | `7860` | Port for default demo (HF expects 7860) |
+
+## Hard example: FastAPI + SQLite
+
+This repo includes a **hard example** (FastAPI + SQLite) in the image. To run it:
+
+1. In your Space **Repository secrets**, set:
+   - `RUN_CMD=uvicorn app.fastapi_sqlite:app --host 0.0.0.0 --port 7860`
+2. (Optional) Set `HF_TOKEN` and `AUTO_CREATE_DATASET=true` for persistence so the SQLite DB in `/data` survives restarts.
+3. Rebuild the Space. Open the app: you’ll see a visit counter and `/api` returning JSON. Restart the Space and refresh — the count persists.
+
+This proves a stateful, “hard” app (FastAPI + SQLite + persistence) runs on HuggingRun.
+
+## License
+
+MIT

app/demo_app.py

@@ -0,0 +1,57 @@
+#!/usr/bin/env python3
+"""
+HuggingRun default demo: minimal HTTP app on port 7860 with persistence.
+- Serves a simple page showing visit counter and PERSIST_PATH status.
+- Counter is stored in PERSIST_PATH/counter.txt so it survives restarts (via HF Dataset sync).
+"""
+import os
+import http.server
+import socketserver
+from pathlib import Path
+
+PORT = int(os.environ.get("PORT", "7860"))
+PERSIST_PATH = Path(os.environ.get("PERSIST_PATH", "/data"))
+COUNTER_FILE = PERSIST_PATH / "counter.txt"
+
+
+def get_counter():
+    PERSIST_PATH.mkdir(parents=True, exist_ok=True)
+    try:
+        return int(COUNTER_FILE.read_text().strip())
+    except (FileNotFoundError, ValueError):
+        return 0
+
+
+def inc_counter():
+    c = get_counter() + 1
+    PERSIST_PATH.mkdir(parents=True, exist_ok=True)
+    COUNTER_FILE.write_text(str(c))
+    return c
+
+
+class DemoHandler(http.server.BaseHTTPRequestHandler):
+    def do_GET(self):
+        count = inc_counter()
+        body = f"""<!DOCTYPE html>
+<html><head><meta charset="utf-8"><title>HuggingRun</title></head>
+<body style="font-family:sans-serif;max-width:600px;margin:2em auto;padding:1em;">
+<h1>Run anything on Hugging Face.</h1>
+<p>This is the default HuggingRun demo. Persistence path: <code>{PERSIST_PATH}</code></p>
+<p><strong>Visit count (persisted):</strong> {count}</p>
+<p>Set <code>RUN_CMD</code> in your Space secrets to run your own app.</p>
+</body></html>"""
+        self.send_response(200)
+        self.send_header("Content-type", "text/html; charset=utf-8")
+        self.send_header("Content-Length", str(len(body.encode("utf-8"))))
+        self.end_headers()
+        self.wfile.write(body.encode("utf-8"))
+
+    def log_message(self, format, *args):
+        print(f"[demo] {args[0]}")
+
+
+if __name__ == "__main__":
+    PERSIST_PATH.mkdir(parents=True, exist_ok=True)
+    with socketserver.TCPServer(("0.0.0.0", PORT), DemoHandler) as httpd:
+        print(f"[HuggingRun demo] Listening on 0.0.0.0:{PORT}")
+        httpd.serve_forever()

app/fastapi_sqlite.py

@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+"""
+HuggingRun hard example: FastAPI + SQLite with persistence in PERSIST_PATH.
+Run with: RUN_CMD=uvicorn app.fastapi_sqlite:app --host 0.0.0.0 --port 7860
+"""
+import os
+from pathlib import Path
+from fastapi import FastAPI
+from fastapi.responses import HTMLResponse
+import sqlite3
+
+app = FastAPI(title="HuggingRun FastAPI+SQLite")
+DB_PATH = Path(os.environ.get("PERSIST_PATH", "/data")) / "huggingrun.db"
+
+
+def get_visits():
+    DB_PATH.parent.mkdir(parents=True, exist_ok=True)
+    conn = sqlite3.connect(str(DB_PATH))
+    conn.execute("CREATE TABLE IF NOT EXISTS visits (id INTEGER PRIMARY KEY, count INTEGER)")
+    conn.commit()
+    n = conn.execute("SELECT COALESCE(SUM(count), 0) FROM visits").fetchone()[0]
+    conn.close()
+    return n
+
+
+def inc_visits():
+    DB_PATH.parent.mkdir(parents=True, exist_ok=True)
+    conn = sqlite3.connect(str(DB_PATH))
+    conn.execute("CREATE TABLE IF NOT EXISTS visits (id INTEGER PRIMARY KEY, count INTEGER)")
+    conn.execute("INSERT INTO visits (count) VALUES (1)")
+    conn.commit()
+    n = conn.execute("SELECT COALESCE(SUM(count), 0) FROM visits").fetchone()[0]
+    conn.close()
+    return n
+
+
+@app.get("/", response_class=HTMLResponse)
+def root():
+    n = inc_visits()
+    return f"""<!DOCTYPE html><html><head><meta charset="utf-8"><title>HuggingRun + FastAPI + SQLite</title></head>
+<body style="font-family:sans-serif;max-width:600px;margin:2em auto;">
+<h1>Run anything on Hugging Face.</h1>
+<p><strong>Hard example:</strong> FastAPI + SQLite. DB path: <code>{DB_PATH}</code></p>
+<p><strong>Total visits (persisted):</strong> {n}</p>
+</body></html>"""
+
+
+@app.get("/api")
+def api():
+    n = get_visits()
+    return {"message": "HuggingRun FastAPI+SQLite", "total_visits": n}

requirements.txt

@@ -0,0 +1 @@
+huggingface_hub>=0.20.0

scripts/entrypoint.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+set -e
+echo "[HuggingRun] Entrypoint — persistence + RUN_CMD"
+exec python3 -u /scripts/sync_hf.py

scripts/sync_hf.py

@@ -0,0 +1,228 @@
+#!/usr/bin/env python3
+"""
+HuggingRun — Generic persistence for any app on Hugging Face Spaces
+====================================================================
+
+- Startup: snapshot_download from HF Dataset → PERSIST_PATH
+- Periodic: upload_folder PERSIST_PATH → dataset
+- Shutdown: final upload
+- Then: exec user's RUN_CMD (or default demo app)
+"""
+
+import os
+import sys
+import time
+import threading
+import subprocess
+import signal
+import shutil
+import tempfile
+import traceback
+from pathlib import Path
+from datetime import datetime
+
+os.environ.setdefault("HF_HUB_DOWNLOAD_TIMEOUT", "300")
+os.environ.setdefault("HF_HUB_UPLOAD_TIMEOUT", "600")
+
+from huggingface_hub import HfApi, snapshot_download
+
+# ── Configuration ───────────────────────────────────────────────────────────
+
+HF_TOKEN = os.environ.get("HF_TOKEN")
+PERSIST_PATH = Path(os.environ.get("PERSIST_PATH", "/data"))
+RUN_CMD = os.environ.get("RUN_CMD", "")
+SYNC_INTERVAL = int(os.environ.get("SYNC_INTERVAL", "60"))
+AUTO_CREATE_DATASET = os.environ.get("AUTO_CREATE_DATASET", "false").lower() in ("true", "1", "yes")
+# In dataset we store under this path_in_repo subfolder
+DATASET_SUBFOLDER = "data"
+
+SPACE_ID = os.environ.get("SPACE_ID", "")
+HF_REPO_ID = os.environ.get("HF_DATASET_REPO", "")
+if not HF_REPO_ID and SPACE_ID:
+    HF_REPO_ID = f"{SPACE_ID}-data"
+if not HF_REPO_ID and HF_TOKEN:
+    try:
+        api = HfApi(token=HF_TOKEN)
+        uname = api.whoami()["name"]
+        HF_REPO_ID = f"{uname}/HuggingRun-data"
+        del api, uname
+    except Exception:
+        HF_REPO_ID = ""
+
+
+class GenericSync:
+    """Upload/download PERSIST_PATH to/from HF Dataset."""
+
+    def __init__(self):
+        self.enabled = False
+        self.dataset_exists = False
+        self.api = None
+
+        if not HF_TOKEN:
+            print("[HuggingRun] HF_TOKEN not set. Persistence disabled.")
+            return
+        if not HF_REPO_ID:
+            print("[HuggingRun] HF_DATASET_REPO/SPACE_ID not set. Persistence disabled.")
+            return
+
+        self.enabled = True
+        self.api = HfApi(token=HF_TOKEN)
+        self.dataset_exists = self._ensure_repo_exists()
+
+    def _ensure_repo_exists(self):
+        try:
+            self.api.repo_info(repo_id=HF_REPO_ID, repo_type="dataset")
+            print(f"[HuggingRun] Dataset found: {HF_REPO_ID}")
+            return True
+        except Exception:
+            if not AUTO_CREATE_DATASET:
+                print(f"[HuggingRun] Dataset not found: {HF_REPO_ID}. Set AUTO_CREATE_DATASET=true to create.")
+                return False
+            try:
+                self.api.create_repo(
+                    repo_id=HF_REPO_ID,
+                    repo_type="dataset",
+                    private=True,
+                )
+                print(f"[HuggingRun] Created dataset: {HF_REPO_ID}")
+                return True
+            except Exception as e:
+                print(f"[HuggingRun] Failed to create dataset: {e}")
+                return False
+
+    def load_from_repo(self):
+        """Download dataset → PERSIST_PATH."""
+        if not self.enabled or not self.dataset_exists:
+            PERSIST_PATH.mkdir(parents=True, exist_ok=True)
+            return
+
+        try:
+            files = self.api.list_repo_files(repo_id=HF_REPO_ID, repo_type="dataset")
+            prefix = f"{DATASET_SUBFOLDER}/"
+            data_files = [f for f in files if f.startswith(prefix)]
+            if not data_files:
+                print(f"[HuggingRun] No {prefix} in dataset. Starting fresh.")
+                PERSIST_PATH.mkdir(parents=True, exist_ok=True)
+                return
+
+            print(f"[HuggingRun] Restoring {PERSIST_PATH} from {HF_REPO_ID} ...")
+            PERSIST_PATH.mkdir(parents=True, exist_ok=True)
+            with tempfile.TemporaryDirectory() as tmpdir:
+                snapshot_download(
+                    repo_id=HF_REPO_ID,
+                    repo_type="dataset",
+                    allow_patterns=f"{prefix}**",
+                    local_dir=tmpdir,
+                    token=HF_TOKEN,
+                )
+                src = Path(tmpdir) / DATASET_SUBFOLDER
+                if src.exists():
+                    for item in src.rglob("*"):
+                        if item.is_file():
+                            rel = item.relative_to(src)
+                            dest = PERSIST_PATH / rel
+                            dest.parent.mkdir(parents=True, exist_ok=True)
+                            shutil.copy2(str(item), str(dest))
+                    print("[HuggingRun] Restore completed.")
+        except Exception as e:
+            print(f"[HuggingRun] Restore failed: {e}")
+            traceback.print_exc()
+            PERSIST_PATH.mkdir(parents=True, exist_ok=True)
+
+    def save_to_repo(self):
+        """Upload PERSIST_PATH → dataset."""
+        if not self.enabled or not self.dataset_exists:
+            return
+        if not PERSIST_PATH.exists():
+            return
+
+        try:
+            if not self._ensure_repo_exists():
+                return
+            self.api.upload_folder(
+                folder_path=str(PERSIST_PATH),
+                path_in_repo=DATASET_SUBFOLDER,
+                repo_id=HF_REPO_ID,
+                repo_type="dataset",
+                token=HF_TOKEN,
+                commit_message=f"HuggingRun sync — {datetime.now().isoformat()}",
+                ignore_patterns=["__pycache__", "*.pyc", ".git"],
+            )
+            print(f"[HuggingRun] Upload completed.")
+        except Exception as e:
+            print(f"[HuggingRun] Upload failed: {e}")
+            traceback.print_exc()
+
+    def background_sync_loop(self, stop_event):
+        while not stop_event.is_set():
+            if stop_event.wait(timeout=SYNC_INTERVAL):
+                break
+            self.save_to_repo()
+
+    def run_user_cmd(self):
+        """Run RUN_CMD (or default demo). Returns Popen process or None."""
+        cmd = (RUN_CMD or "python3 /app/demo_app.py").strip()
+        if not cmd:
+            print("[HuggingRun] RUN_CMD empty and no default demo. Exiting.")
+            return None
+        print(f"[HuggingRun] Running: {cmd}")
+        try:
+            process = subprocess.Popen(
+                cmd,
+                shell=True,
+                env=os.environ.copy(),
+                stdout=sys.stdout,
+                stderr=sys.stderr,
+            )
+            print(f"[HuggingRun] Process started PID={process.pid}")
+            return process
+        except Exception as e:
+            print(f"[HuggingRun] Failed to start: {e}")
+            traceback.print_exc()
+            return None
+
+
+def main():
+    try:
+        sync = GenericSync()
+        sync.load_from_repo()
+
+        stop_event = threading.Event()
+        t = threading.Thread(target=sync.background_sync_loop, args=(stop_event,), daemon=True)
+        t.start()
+
+        process = sync.run_user_cmd()
+        if process is None:
+            stop_event.set()
+            t.join(timeout=5)
+            sys.exit(1)
+
+        def on_signal(sig, frame):
+            print(f"\n[HuggingRun] Signal {sig}. Shutting down...")
+            stop_event.set()
+            t.join(timeout=10)
+            if process.poll() is None:
+                process.terminate()
+                try:
+                    process.wait(timeout=5)
+                except subprocess.TimeoutExpired:
+                    process.kill()
+            sync.save_to_repo()
+            sys.exit(0)
+
+        signal.signal(signal.SIGINT, on_signal)
+        signal.signal(signal.SIGTERM, on_signal)
+
+        code = process.wait()
+        stop_event.set()
+        t.join(timeout=10)
+        sync.save_to_repo()
+        sys.exit(code)
+    except Exception as e:
+        print(f"[HuggingRun] FATAL: {e}")
+        traceback.print_exc()
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()