Upload folder using huggingface_hub

README.md

@@ -12,44 +12,97 @@ Daemonless Docker image builder using Kaniko. Builds and pushes images to GHCR w
 
 Set these as HuggingFace Secrets:
 
-- `REGISTRY_USER`: GitHub username
-- `REGISTRY_PASSWORD`: GitHub PAT with `packages:write` scope
-- `GITHUB_TOKEN`: (optional) GitHub PAT for cloning private repositories
-- `UPSTASH_REDIS_REST_URL`: (optional) Redis URL for build queue
-- `UPSTASH_REDIS_REST_TOKEN`: (optional) Redis token
+| Secret | Required | Description |
+|--------|----------|-------------|
+| `REGISTRY_USER` | Yes | GitHub username |
+| `REGISTRY_PASSWORD` | Yes | GitHub PAT with `packages:write` scope |
+| `GITHUB_TOKEN` | For private repos | GitHub PAT for cloning private repositories |
+| `GITHUB_WEBHOOK_SECRET` | Recommended | Secret for validating GitHub webhook payloads |
+| `DEFAULT_IMAGE_NAME` | For webhooks | Default image name (e.g., `jonathanagustin/lawforge`) |
+| `UPSTASH_REDIS_REST_URL` | Optional | Redis URL for build queue |
+| `UPSTASH_REDIS_REST_TOKEN` | Optional | Redis token |
 
 ## Usage
 
+### Via GitHub Webhook (Automatic Builds)
+
+Set up automatic builds when you push to your repository:
+
+1. Go to your GitHub repo → **Settings** → **Webhooks** → **Add webhook**
+2. **Payload URL**: `https://jonathanagustin-builder.hf.space/webhook/github`
+3. **Content type**: `application/json`
+4. **Secret**: Same value as `GITHUB_WEBHOOK_SECRET` (generate with `openssl rand -hex 32`)
+5. **Events**: Select "Just the push event"
+
+The builder will automatically:
+- Trigger on pushes to the default branch (main/master)
+- Only build when relevant files change (Dockerfile, src/**, pyproject.toml, etc.)
+- Skip builds for non-Docker-related changes (docs, tests, etc.)
+
+#### File Patterns That Trigger Builds
+
+```
+Dockerfile, docker/*, src/**/*.py, pyproject.toml, uv.lock, requirements*.txt, .dockerignore
+```
+
+#### Optional Webhook Headers
+
+For per-repo configuration, add custom headers to your webhook:
+
+| Header | Description |
+|--------|-------------|
+| `X-Builder-Token` | GitHub token for cloning private repos |
+| `X-Builder-Image` | Override image name (default: repo name) |
+| `X-Builder-Tags` | Comma-separated tags (default: "latest") |
+
 ### Via Web UI
+
 Navigate to the Space and use the build form.
 
 ### Via API
+
 ```bash
 curl -X POST https://jonathanagustin-builder.hf.space/build \
-  -H "Authorization: Bearer $HF_TOKEN" \
   -H "Content-Type: application/json" \
   -d '{
-    "repo_url": "https://github.com/jonathanagustin/lawforge",
-    "image_name": "jonathanagustin/lawforge-worker",
+    "repo_url": "https://github.com/owner/repo",
+    "image_name": "owner/repo",
     "branch": "main",
     "tags": ["latest"],
-    "dockerfile": "Dockerfile",
-    "context_path": "jobs/worker"
+    "dockerfile": "Dockerfile"
   }'
 ```
 
-### Via Taskfile
-```bash
-# Build worker image
-task builder:build:worker
+### Via Test Endpoint
 
-# Build any image
-REPO_URL=https://github.com/user/repo IMAGE_NAME=user/image task builder:build
+Trigger a build without webhook signature validation:
+
+```bash
+curl -X POST https://jonathanagustin-builder.hf.space/webhook/test \
+  -H "Content-Type: application/json" \
+  -d '{
+    "repo_url": "https://github.com/owner/repo",
+    "image_name": "owner/repo"
+  }'
 ```
 
 ## Endpoints
 
-- `GET /` - Web UI
-- `GET /api/status` - Builder status JSON
-- `POST /build` - Trigger a build
-- `POST /api/queue` - Queue a build (requires Redis)
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/` | GET | Web UI |
+| `/api/status` | GET | Builder status JSON |
+| `/build` | POST | Trigger a build manually |
+| `/webhook/github` | POST | GitHub webhook endpoint |
+| `/webhook/test` | POST | Test build trigger (no signature) |
+| `/api/queue` | POST | Queue a build (requires Redis) |
+
+## How It Works
+
+1. **Webhook received**: GitHub sends push event to `/webhook/github`
+2. **Signature verified**: HMAC-SHA256 validation using `GITHUB_WEBHOOK_SECRET`
+3. **Files checked**: Only builds if Docker-relevant files changed
+4. **Repo cloned**: Uses `GITHUB_TOKEN` (or per-repo `X-Builder-Token`) for private repos
+5. **Image built**: Kaniko builds without Docker daemon
+6. **Image pushed**: Pushes to GHCR using `REGISTRY_USER`/`REGISTRY_PASSWORD`
+7. **Cleanup**: Temporary files removed

app.py

@@ -2,16 +2,28 @@
 """Docker image builder using Kaniko - no Docker daemon required.
 
 Builds Docker images and pushes to container registries (GHCR, Docker Hub, etc.)
-Can be triggered via API or run builds from a queue in Redis.
+Can be triggered via API, GitHub webhooks, or run builds from a queue in Redis.
 
 Environment variables:
   - REGISTRY_USER: Registry username (e.g., GitHub username for GHCR)
   - REGISTRY_PASSWORD: Registry password/token (e.g., GitHub PAT with packages:write)
   - REGISTRY_URL: Registry URL (default: ghcr.io)
+  - GITHUB_TOKEN: GitHub token for cloning private repos
+  - GITHUB_WEBHOOK_SECRET: Secret for validating GitHub webhook payloads
   - UPSTASH_REDIS_REST_URL: Redis URL for build queue
   - UPSTASH_REDIS_REST_TOKEN: Redis token
+
+Webhook setup:
+  1. Go to your GitHub repo → Settings → Webhooks → Add webhook
+  2. Payload URL: https://your-space.hf.space/webhook/github
+  3. Content type: application/json
+  4. Secret: Same value as GITHUB_WEBHOOK_SECRET
+  5. Events: Just the push event
 """
 
+import fnmatch
+import hashlib
+import hmac
 import json
 import os
 import shutil
@@ -23,7 +35,7 @@ import uuid
 from datetime import datetime, timezone
 from pathlib import Path
 
-from flask import Flask, jsonify, render_template_string, request
+from flask import Flask, jsonify, render_template_string, request, abort
 import git
 
 # =============================================================================
@@ -36,8 +48,25 @@ REGISTRY_URL = os.environ.get("REGISTRY_URL", "ghcr.io")
 REGISTRY_USER = os.environ.get("REGISTRY_USER", "")
 REGISTRY_PASSWORD = os.environ.get("REGISTRY_PASSWORD", "")
 GITHUB_TOKEN = os.environ.get("GITHUB_TOKEN", "")  # For cloning private repos
+GITHUB_WEBHOOK_SECRET = os.environ.get("GITHUB_WEBHOOK_SECRET", "")  # For webhook validation
 AUTO_START = os.environ.get("AUTO_START", "false").lower() == "true"
 
+# Default image name for webhook-triggered builds (owner/repo format)
+DEFAULT_IMAGE_NAME = os.environ.get("DEFAULT_IMAGE_NAME", "")
+
+# File patterns that should trigger a Docker rebuild when changed
+# Uses glob-style patterns
+BUILD_TRIGGER_PATTERNS = [
+    "Dockerfile",
+    "docker/*",
+    "docker/**/*",
+    "src/**/*.py",
+    "pyproject.toml",
+    "uv.lock",
+    "requirements*.txt",
+    ".dockerignore",
+]
+
 # Global state
 state = {
     "status": "idle",
@@ -869,6 +898,199 @@ def api_queue():
     return jsonify({"status": "queued", "build_id": build_id}), 202
 
 
+# =============================================================================
+# GitHub Webhook
+# =============================================================================
+
+def verify_webhook_signature(payload: bytes, signature: str) -> bool:
+    """Verify GitHub webhook signature using HMAC-SHA256."""
+    if not GITHUB_WEBHOOK_SECRET:
+        log("⚠️ GITHUB_WEBHOOK_SECRET not set - skipping signature verification")
+        return True  # Allow if no secret configured (not recommended for production)
+
+    if not signature or not signature.startswith("sha256="):
+        return False
+
+    expected = hmac.new(
+        GITHUB_WEBHOOK_SECRET.encode(),
+        payload,
+        hashlib.sha256
+    ).hexdigest()
+
+    return hmac.compare_digest(f"sha256={expected}", signature)
+
+
+def should_trigger_build(changed_files: list[str]) -> tuple[bool, list[str]]:
+    """Check if any changed files match build trigger patterns.
+
+    Returns:
+        Tuple of (should_build, matching_files)
+    """
+    matching = []
+    for filepath in changed_files:
+        for pattern in BUILD_TRIGGER_PATTERNS:
+            if fnmatch.fnmatch(filepath, pattern):
+                matching.append(filepath)
+                break
+    return len(matching) > 0, matching
+
+
+def extract_changed_files(payload: dict) -> list[str]:
+    """Extract list of changed files from GitHub push payload."""
+    files = set()
+    for commit in payload.get("commits", []):
+        files.update(commit.get("added", []))
+        files.update(commit.get("modified", []))
+        files.update(commit.get("removed", []))
+    return list(files)
+
+
+@app.route("/webhook/github", methods=["POST"])
+def github_webhook():
+    """Handle GitHub webhook events from any repository.
+
+    Triggers a build when:
+    - Event is a push to the default branch (main/master)
+    - Changed files match BUILD_TRIGGER_PATTERNS
+
+    Works with any GitHub repository - extracts repo info from payload.
+
+    Optional headers for per-repo configuration:
+    - X-Builder-Token: GitHub token for cloning private repos (overrides env GITHUB_TOKEN)
+    - X-Builder-Image: Override image name (default: uses repo full_name)
+    - X-Builder-Tags: Comma-separated tags (default: "latest")
+    """
+    # Verify signature
+    signature = request.headers.get("X-Hub-Signature-256", "")
+    if not verify_webhook_signature(request.data, signature):
+        log("✗ Webhook signature verification failed")
+        abort(401, "Invalid signature")
+
+    # Only handle push events
+    event = request.headers.get("X-GitHub-Event", "")
+    if event == "ping":
+        log("✓ Webhook ping received")
+        return jsonify({"status": "pong"}), 200
+
+    if event != "push":
+        log(f"Ignoring non-push event: {event}")
+        return jsonify({"status": "ignored", "reason": f"event type: {event}"}), 200
+
+    payload = request.json
+    if not payload:
+        abort(400, "Missing payload")
+
+    # Extract repo info from webhook payload (works for any repo)
+    repo = payload.get("repository", {})
+    repo_url = repo.get("clone_url", "")
+    repo_full_name = repo.get("full_name", "")  # e.g., "owner/repo"
+    default_branch = repo.get("default_branch", "main")
+    is_private = repo.get("private", False)
+
+    # Get the ref that was pushed
+    ref = payload.get("ref", "")
+    pushed_branch = ref.replace("refs/heads/", "") if ref.startswith("refs/heads/") else ""
+
+    log(f"Webhook: push to {repo_full_name}/{pushed_branch} (private={is_private})")
+
+    # Only build on pushes to default branch
+    if pushed_branch != default_branch:
+        log(f"Ignoring push to non-default branch: {pushed_branch} (default: {default_branch})")
+        return jsonify({
+            "status": "ignored",
+            "reason": f"branch {pushed_branch} is not default branch {default_branch}"
+        }), 200
+
+    # Check if any relevant files changed
+    changed_files = extract_changed_files(payload)
+    should_build, matching_files = should_trigger_build(changed_files)
+
+    if not should_build:
+        log(f"No build-relevant files changed in {len(changed_files)} files")
+        return jsonify({
+            "status": "ignored",
+            "reason": "no build-relevant files changed",
+            "changed_files": changed_files[:20]  # Limit for response size
+        }), 200
+
+    log(f"Build triggered by {len(matching_files)} matching files: {matching_files[:5]}")
+
+    # Per-repo overrides via headers
+    override_token = request.headers.get("X-Builder-Token", "")
+    override_image = request.headers.get("X-Builder-Image", "")
+    override_tags = request.headers.get("X-Builder-Tags", "")
+
+    # Determine image name: header override > env default > repo name
+    image_name = override_image or DEFAULT_IMAGE_NAME or repo_full_name
+
+    # Determine tags
+    if override_tags:
+        tags = [t.strip() for t in override_tags.split(",") if t.strip()]
+    else:
+        tags = ["latest"]
+
+    # Build configuration
+    build_config = {
+        "repo_url": repo_url,
+        "branch": pushed_branch,
+        "image_name": image_name,
+        "tags": tags,
+        "trigger": "webhook",
+        "matching_files": matching_files[:10],
+    }
+
+    # Add per-repo token if provided (for private repos)
+    if override_token:
+        build_config["github_token"] = override_token
+        log(f"Using per-repo token for {repo_full_name}")
+
+    # Check if already building
+    if state["status"] == "building":
+        log("Build already in progress - queueing webhook build")
+        if redis_client:
+            build_id = queue_build(build_config)
+            return jsonify({"status": "queued", "build_id": build_id}), 202
+        else:
+            return jsonify({"status": "busy", "reason": "build in progress and no queue configured"}), 409
+
+    threading.Thread(target=run_build, args=(build_config,), daemon=True).start()
+
+    return jsonify({
+        "status": "started",
+        "repo": repo_full_name,
+        "branch": pushed_branch,
+        "image": f"{REGISTRY_URL}/{image_name}",
+        "tags": tags,
+        "matching_files": matching_files
+    }), 202
+
+
+@app.route("/webhook/test", methods=["POST"])
+def test_webhook():
+    """Test endpoint to simulate a webhook (no signature required)."""
+    if state["status"] == "building":
+        return jsonify({"error": "Build already in progress"}), 409
+
+    # Allow testing with minimal payload
+    payload = request.json or {}
+    repo_url = payload.get("repo_url", "https://github.com/jonathanagustin/lawforge")
+    branch = payload.get("branch", "main")
+    image_name = payload.get("image_name", DEFAULT_IMAGE_NAME or "jonathanagustin/lawforge")
+
+    build_config = {
+        "repo_url": repo_url,
+        "branch": branch,
+        "image_name": image_name,
+        "tags": ["latest"],
+        "trigger": "test",
+    }
+
+    log(f"Test webhook triggered: {image_name}")
+    threading.Thread(target=run_build, args=(build_config,), daemon=True).start()
+
+    return jsonify({"status": "started", "config": build_config}), 202
+
+
 # =============================================================================
 # Startup
 # =============================================================================
@@ -886,11 +1108,21 @@ def startup():
     else:
         log("⚠️ REGISTRY_USER not set - pushes will fail")
 
+    # Webhook configuration status
+    if GITHUB_WEBHOOK_SECRET:
+        log("✓ GitHub webhook secret configured")
+    else:
+        log("⚠️ GITHUB_WEBHOOK_SECRET not set - webhooks will accept unsigned payloads")
+
+    if DEFAULT_IMAGE_NAME:
+        log(f"✓ Default image: {REGISTRY_URL}/{DEFAULT_IMAGE_NAME}")
+
     # Start queue worker if Redis is configured
     if redis_client:
         threading.Thread(target=queue_worker, daemon=True).start()
 
     log("Ready for builds!")
+    log(f"Webhook endpoint: /webhook/github")
 
 
 threading.Thread(target=startup, daemon=True).start()