| # DibaGuard — load-balancer integrations |
|
|
| > **Open-release copy** ([DibaAi/DibaGuard-Open](https://huggingface.co/DibaAi/DibaGuard-Open)). This document was written for the private build of DibaGuard. Numbers that involve the HTTP request model (`http_v2`) were measured with that model, which is **not** part of the open release; the open release ships the licence-clean network models described in `CLEAN_MODELS.md`. Licence statements inside this document are superseded by `DATA_LICENSES.md`, `CLEAN_MODELS.md` and the repository's `LICENSE`, `LICENSE-WEIGHTS` and `NOTICE` files. Sections of the original that were written in Persian were removed from this copy; nothing else was edited. |
|
|
| DibaGuard can sit behind the load balancer you already run instead of putting `dibaguard-proxy` inline. Two |
| mechanisms, usable together: |
|
|
| 1. **Verdict service** — `dibaguard-authz` answers "may this request pass?" for nginx `auth_request`, |
| ingress-nginx `auth-url`, Envoy `ext_authz`, Traefik `ForwardAuth`, Caddy `forward_auth` and HAProxy lua |
| auth-request. The verdict is exactly the inline proxy's decision (`dibaguard.Decider`, `go/dibaguard/decide.go`): |
| the sidecar's standing decision for the client (block / challenge / rate-limit, polled from |
| `guard_fast_state`), the payload signatures (`MatchSigs`) and the site-conditioned HTTP model. |
| 2. **HAProxy blocklist sync** — `dibaguard-authz -haproxy-socket ...` mirrors the sidecar's block set into an |
| HAProxy ACL (or map) through the Runtime API, so HAProxy drops blocked sources in its own data path with **no |
| per-request round trip**. |
|
|
| Everything below binds to loopback / a private network. The verdict service is not meant to be reachable from the |
| internet. |
|
|
| | load balancer | mechanism | endpoint | status of this guide | |
| |---|---|---|---| |
| | nginx | `auth_request` | `/nginx` | **verified against nginx 1.30.6** (§13) + contract tests | |
| | Kubernetes ingress-nginx | `auth-url` (+ `auth-signin` for the challenge) | `/nginx`, `/_dibaguard/challenge` | contract tests | |
| | Envoy | `ext_authz`, HTTP service | `/envoy/...` (`path_prefix`) | contract tests | |
| | Traefik | `ForwardAuth` | `/forward` | contract tests | |
| | Caddy | `forward_auth` | `/forward` | contract tests | |
| | HAProxy | Runtime API ACL/map sync | — | tested against a fake Runtime API | |
| | HAProxy | lua auth-request (haproxy-lua-http) | `/forward` | contract tests | |
|
|
| --- |
|
|
| ## 1. Running `dibaguard-authz` |
|
|
| ```sh |
| cd guard/go/dibaguard |
| go build -o dibaguard-authz ./cmd/dibaguard-authz # stdlib only |
| |
| ./dibaguard-authz -listen 127.0.0.1:9300 -models ./models -sidecar http://127.0.0.1:8787 \ |
| -mode report -trusted 127.0.0.1/32 -challenge-secret "$(cat /etc/dibaguard/challenge.secret)" |
| ``` |
|
|
| Start in `-mode report` (never denies, only reports; §10), switch to `-mode enforce` when the verdict headers look |
| right. |
|
|
| **Feed token.** The sidecar refuses traffic feeds that carry sources unless they bring the feed token (validation |
| finding V-16: otherwise anyone who can reach it could make the layer block an address of their choice). Set the same |
| `DIBAGUARD_FEED_TOKEN` in the environment of the sidecar and of every `dibaguard-proxy` / `dibaguard-authz`; they send |
| it as `X-DibaGuard-Feed-Token` with their request summaries. Polling the standing decisions (`guard_fast_state`) needs |
| no token. |
|
|
| | flag | default | meaning | |
| |---|---|---| |
| | `-listen` | `127.0.0.1:9300` | address; `unix:/run/dibaguard.sock` for a unix socket; `""` = HAProxy syncer only | |
| | `-models` / `-model` | `models` / — | model dir (`http_v2/http_tier1_v2.bin`) or explicit model file | |
| | `-sidecar` | `http://127.0.0.1:8787` | Python sidecar (`python -m dibaguard.llm http`); `""` = none | |
| | `-mode` | `report` | `report` \| `enforce` | |
| | `-inline-block` | `true` | refuse on content (hard signatures, very high scores); `false` = content only flags | |
| | `-challenge-secret` | random | HMAC key of the challenge cookie — **share it across replicas** | |
| | `-rate` | `5` | req/s for a rate-limited client (burst 2×) | |
| | `-trusted` | `127.0.0.0/8,::1/128` | CIDRs of the load balancers whose forwarding headers are believed (§3) | |
| | `-ip-headers` | `X-Forwarded-For` | client-address headers read from trusted peers, first present wins: `X-Forwarded-For`, `Forwarded`, `X-Real-IP` — list only headers your LB sets or overwrites (§3) | |
| | `-proxy-protocol` | `false` | connections start with a PROXY v1/v2 header from a `-trusted` peer; others are dropped (§3) | |
| | `-ipv6-block-prefix` | `64` | a sidecar block on an IPv6 address covers its /N, here and in the HAProxy ACL (`128` = the address only) | |
| | `-ipv6-ratelimit-prefix` | `128` | rate-limit entries and token buckets per IPv6 /N (`64` = one bucket per /64) | |
| | `-envoy-prefix` | `/envoy` | Envoy `path_prefix` | |
| | `-budget` | `25ms` | per-request budget (body read + decision); over it the fail mode applies | |
| | `-fail-closed` | `false` | deny when the decision cannot be made (default: fail open) | |
| | `-state-stale` | `10s` | sidecar state older than this counts as unavailable | |
| | `-max-inflight` | 64 × GOMAXPROCS | concurrent decisions; no slot within the budget = `overload` | |
| | `-max-body` | 1 MiB | body bytes inspected when the LB sends the body (Envoy `with_request_body`) | |
| | `-poll` | `2s` | sidecar state poll interval | |
| | `-haproxy-socket` | — | Runtime API: `127.0.0.1:9999`, `ipv4@host:port`, `unix:/run/haproxy/admin.sock`, `/path.sock` | |
| | `-haproxy-kind` | `acl` | `acl` \| `map` | |
| | `-haproxy-file` | `/etc/haproxy/dibaguard-block.acl` | the ACL/map exactly as `haproxy.cfg` loads it (or `#<id>`) | |
| | `-haproxy-actions` | `block` | which standing decisions are mirrored (`block,challenge,rate_limit` for a map) | |
| | `-haproxy-every` / `-haproxy-resync` | `2s` / `1m` | sync interval / full re-read interval | |
|
|
| `GET /healthz` returns counters (`checks, allowed, blocked, challenged, rate_limited, flagged, fail_open, |
| fail_closed_denied, timeouts, overloads, panics`), `degraded`, `sidecar_state_age_s` and the HAProxy syncer's stats. |
| |
| --- |
| |
| ## 2. The verdict contract |
| |
| | verdict | `/nginx` (auth_request, ingress-nginx) | `/forward` (Traefik, Caddy, HAProxy lua) and `/envoy/...` | |
| |---|---|---| |
| | allow | **200** | **200** (Envoy: only 200 allows) | |
| | block | **403** | **403** `forbidden` | |
| | challenge | **401** (render with `error_page 401` → `/challenge`) | **429** + the JS challenge page (HTML) | |
| | rate_limit | **401** (same `error_page`; `/challenge` answers 429 `slow down`) | **429** `slow down`, `Retry-After: 1` | |
| | fail-closed deny | 403 | `/forward` 503, `/envoy` 403 (Envoy lets a 5xx through under `failure_mode_allow`) | |
| | any verdict in report mode | 200 | 200 | |
|
|
| nginx's `auth_request` only understands 2xx, 401 and 403 — any other status (429 included) becomes a 500. That is |
| why challenge and rate_limit share 401 on `/nginx` and are told apart by `X-DibaGuard-Verdict`. |
| |
| Response headers (every answer): |
| |
| | header | value | |
| |---|---| |
| | `X-DibaGuard-Verdict` | `allow` \| `block` \| `challenge` \| `rate_limit` — in report mode: what enforce would do | |
| | `X-DibaGuard-Reason` | `sidecar:<action>` \| `sig:<id>` (first signature) \| `model` \| `fail_open:<cause>` \| `fail_closed:<cause>` | |
| | `X-DibaGuard-Sigs` | every signature hit, e.g. `A15,A18` | |
| | `X-DibaGuard-Score` | HTTP model attack logit | |
| | `X-DibaGuard` | `flagged` (as `dibaguard-proxy` sets it) | |
| | `X-DibaGuard-Mode` | `report` \| `enforce` | |
| | `X-DibaGuard-Degraded` | `model_unavailable` \| `sidecar_unavailable` (fail-open while degraded) | |
|
|
| How the original request is reconstructed: |
|
|
| | | method | URI | host | |
| |---|---|---|---| |
| | `/nginx`, `/forward` | `X-Original-Method`, else `X-Forwarded-Method`, else the request's | `X-Original-URI`, else `X-Forwarded-Uri`, else `X-Original-URL` (absolute, ingress-nginx) | `X-Forwarded-Host`, else from `X-Original-URL`, else `Host` | |
| | `/envoy/...` | the request's (Envoy keeps it) | the raw path minus `-envoy-prefix` (never cleaned or redirected) | `:authority` | |
|
|
| All other request headers are the original client headers (all of them for nginx/Traefik/Caddy/HAProxy, Envoy's |
| `allowed_headers`). The decision reads `User-Agent, Cookie, Content-Type, Accept, Referer, X-Forwarded-For, |
| Authorization, X-Api-Version, Origin` — the same canonical text as the inline proxy. |
|
|
| Decoding follows the application, not Go's URL parser (the same in `dibaguard-proxy` and `dibaguard-authz`): the |
| query is split on `&` only (a `;` is data) and every pair is kept; well-formed `%XX` and IIS-style `%uXXXX` escapes |
| are decoded (up to 4 rounds, double encoding included) and a malformed one (`%zz`, `%2`, a lone `%`) is left as it |
| is, as PHP and most stacks read it and as the Python canonicaliser (`dibaguard/http/canon.py`) does. Before, |
| `?id=%zz1' or 1=1-- -` and `?host=127.0.0.1;cat /etc/passwd` were dropped by Go's query parser and never scored. |
| A URI whose *path* Go cannot parse (bad escape) is decoded the same lenient way. |
|
|
| `/challenge` (also `/_dibaguard/challenge`) renders the JS challenge for the client: the page sets the `dg_ch` |
| cookie (HMAC of client address + UTC day) and reloads — or, with `?rd=/local/path`, goes back to that path (only |
| local paths are honoured; anything else just reloads). It answers 429 `slow down` instead when |
| `X-DibaGuard-Verdict: rate_limit` is sent or, without that header, when the sidecar rate-limits the client. |
|
|
| --- |
|
|
| ## 3. Client identity and trusted proxies |
|
|
| The client address is what every sidecar decision, rate-limit bucket and challenge cookie is keyed on, so it must |
| not be forgeable. |
|
|
| The same rules apply to `dibaguard-authz` and `dibaguard-proxy` (which takes the same `-trusted`, `-ip-headers`, |
| `-proxy-protocol`, `-ipv6-block-prefix` and `-ipv6-ratelimit-prefix` flags; its old `-trust-xff`, which believed the |
| leftmost `X-Forwarded-For` entry from anyone, is removed and refuses to start: set `-trusted` instead). |
|
|
| * Forwarding headers are read **only when the TCP peer is in `-trusted`** (default: loopback only; or a |
| unix-socket peer). Any other peer is identified by its own address whatever it sends: it can neither escape its |
| own block nor frame another address (both are tested, and verified through real nginx and the real proxy in §13). |
| * `X-Forwarded-For` and `Forwarded` (RFC 7239, its `for=` values, quoted IPv6 with ports included) are walked |
| **right to left past trusted hops**: `"<client-written>, <real>"` yields `<real>`. A hop that cannot be read |
| (`unknown`, an obfuscated `_id`, an element without `for=`, garbage) stops the walk. `X-Real-IP` holds one address. |
| * **List only headers your load balancer sets or overwrites** in `-ip-headers` (default `X-Forwarded-For`). A header |
| the LB passes through untouched is written by the client: most LBs pass `Forwarded` and `X-Real-IP` through |
| unless configured to set them, so they are opt-in. |
| * **PROXY protocol** (`-proxy-protocol`): for an L4 load balancer (HAProxy `send-proxy`/`send-proxy-v2`, AWS NLB, |
| Envoy's upstream `proxy_protocol`), the v1 or v2 header gives the client address. It is mandatory on that listener |
| and believed only from `-trusted` peers; a connection without a valid header, or from any other peer, is closed |
| without an answer. `LOCAL`/`UNKNOWN` headers (health checks) keep the peer address. Verified with curl's |
| `--haproxy-protocol` (§13). |
| * Addresses are normalized before they are used as keys: IPv4-mapped IPv6 (`::ffff:1.2.3.4`) is the IPv4 address, |
| IPv6 is compressed lower-case, zone ids (`fe80::1%eth0`) are dropped, ports are stripped. |
| * **IPv6 networks.** An IPv6 client usually owns a /64 and can rotate addresses inside it, so by default a sidecar |
| block on one IPv6 address covers its /64 (`-ipv6-block-prefix 64`, also applied to the HAProxy ACL, which then |
| holds `2001:db8:1:2::/64`). Rate limits stay per address unless `-ipv6-ratelimit-prefix 64` (then the /64 shares |
| one standing decision and one token bucket). Challenges stay per address (the cookie is bound to the address). |
| CIDR entries in the sidecar state (`10.0.0.0/8`, `2001:db8::/48`) are honoured too. Summaries sent to the sidecar |
| keep the exact address. |
| * Put the load balancer(s) — and only them — in `-trusted`: `127.0.0.1/32` when co-located, the ingress pod CIDR in |
| Kubernetes, the Docker network for a containerised Traefik. |
| * Each LB below **overwrites or appends** these headers itself (`$proxy_add_x_forwarded_for`, Envoy |
| `use_remote_address`, Traefik/Caddy rewrite XFF for untrusted clients, HAProxy `set-header`). If the LB itself |
| sits behind another proxy, configure that LB's own real-IP handling first (nginx `realip`, Envoy |
| `xff_num_trusted_hops`, Traefik `forwardedHeaders.trustedIPs`, Caddy `trusted_proxies`). |
|
|
| --- |
|
|
| ## 4. nginx (`auth_request`) |
| |
| The `auth_request` module is in the official builds (`nginx -V | grep auth_request`). This is the configuration |
| verified in §13. |
| |
| ```nginx |
| upstream dibaguard_authz { server 127.0.0.1:9300; keepalive 32; } |
| upstream app { server 127.0.0.1:3000; keepalive 32; } |
|
|
| server { |
| listen 80; |
| |
| location / { |
| auth_request /_dibaguard_auth; |
| auth_request_set $dg_verdict $upstream_http_x_dibaguard_verdict; |
| auth_request_set $dg_reason $upstream_http_x_dibaguard_reason; |
| auth_request_set $dg_status $upstream_status; |
| error_page 401 = /_dibaguard_challenge; # challenge and rate_limit |
| error_page 500 = @dibaguard_failopen; # DibaGuard unreachable: serve anyway (remove this line to fail closed) |
| add_header X-DibaGuard-Verdict $dg_verdict always; # optional, for debugging |
| proxy_set_header X-DibaGuard-Verdict $dg_verdict; # the app sees the verdict (overwrites a client's) |
| proxy_http_version 1.1; |
| proxy_set_header Connection ""; |
| proxy_pass http://app; |
| } |
| |
| location = /_dibaguard_auth { |
| internal; |
| proxy_pass http://dibaguard_authz/nginx; |
| proxy_http_version 1.1; # keep-alive to the verdict service (else one TCP connect per request) |
| proxy_set_header Connection ""; |
| proxy_pass_request_body off; # auth_request does not pass the body |
| proxy_set_header Content-Length ""; |
| proxy_set_header X-Original-Method $request_method; |
| proxy_set_header X-Original-URI $request_uri; |
| proxy_set_header X-Forwarded-Host $host; |
| proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; |
| proxy_set_header X-Real-IP $remote_addr; |
| proxy_connect_timeout 100ms; # > -budget (25ms): the service answers (fail-open) before nginx gives up |
| proxy_send_timeout 200ms; |
| proxy_read_timeout 200ms; |
| } |
| |
| location = /_dibaguard_challenge { |
| internal; |
| proxy_pass http://dibaguard_authz/challenge; |
| proxy_http_version 1.1; |
| proxy_set_header Connection ""; |
| proxy_method GET; |
| proxy_pass_request_body off; |
| proxy_set_header Content-Length ""; |
| proxy_set_header X-DibaGuard-Verdict $dg_verdict; # challenge -> JS page, rate_limit -> 429 |
| proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; |
| proxy_set_header X-Real-IP $remote_addr; |
| } |
| |
| location @dibaguard_failopen { |
| # Fail open ONLY when DibaGuard could not be reached (502 refused / 504 timed out). An answer it did give is |
| # not a pass: Go's HTTP server answers 400 to a header nginx accepts but HTTP forbids (a 0x01 byte), and |
| # nginx turns that into a 500 - without this guard such a request reaches the app unchecked (shown in §13). |
| if ($dg_status !~ "^(502|504)") { return 403; } |
| proxy_http_version 1.1; |
| proxy_set_header Connection ""; |
| proxy_pass http://app; |
| } |
| } |
| ``` |
| |
| Run the service with `-trusted 127.0.0.1/32` (nginx's address as the service sees it). `auth_request_set` variables |
| survive the `error_page` redirect (verified). Log the verdict with |
| `log_format dg '$remote_addr "$request" $status verdict=$dg_verdict reason=$dg_reason';`. |
|
|
| --- |
|
|
| ## 5. Kubernetes ingress-nginx |
|
|
| ingress-nginx's external auth is `auth_request` underneath (2xx / 401 / 403 only). It sends `X-Original-URL` |
| (absolute), `X-Original-Method`, `X-Real-IP` and `X-Forwarded-For` plus the client's headers; `/nginx` reads all of |
| them. A 401 is turned into a redirect to `auth-signin`, which is pointed at the challenge page. |
|
|
| ```yaml |
| apiVersion: networking.k8s.io/v1 |
| kind: Ingress |
| metadata: |
| name: shop |
| annotations: |
| nginx.ingress.kubernetes.io/auth-url: "http://dibaguard-authz.shop.svc.cluster.local:9300/nginx" |
| nginx.ingress.kubernetes.io/auth-response-headers: "X-DibaGuard-Verdict,X-DibaGuard-Score,X-DibaGuard-Reason" |
| # challenge / rate_limit (401): send the browser to the challenge page, which comes back to the original path |
| nginx.ingress.kubernetes.io/auth-signin: "https://$host/_dibaguard/challenge?rd=$escaped_request_uri" |
| spec: |
| ingressClassName: nginx |
| rules: |
| - host: shop.example |
| http: |
| paths: |
| - path: / |
| pathType: Prefix |
| backend: { service: { name: shop, port: { number: 80 } } } |
| --- |
| # the challenge page, served by dibaguard-authz on the same host, NOT behind auth-url |
| apiVersion: networking.k8s.io/v1 |
| kind: Ingress |
| metadata: |
| name: shop-dibaguard-challenge |
| spec: |
| ingressClassName: nginx |
| rules: |
| - host: shop.example |
| http: |
| paths: |
| - path: /_dibaguard/challenge |
| pathType: Exact |
| backend: { service: { name: dibaguard-authz, port: { number: 9300 } } } |
| ``` |
|
|
| * Run `dibaguard-authz` with `-trusted <pod CIDR>` (e.g. `10.244.0.0/16`): the requests come from the controller |
| pods. Ingress backends must be in the Ingress's namespace — deploy the service there (or use an `ExternalName`). |
| * All ingresses at once: the controller ConfigMap's `global-auth-url` (and `global-auth-signin`). |
| * For a rate-limited client the challenge page answers 429 `slow down` (it looks the client up in the sidecar state). |
| * ingress-nginx has no fail-open switch: if the service is unreachable the request gets a 500. Run 2+ replicas with |
| a PodDisruptionBudget and the same `-challenge-secret` (a Secret mounted in every replica). |
|
|
| --- |
|
|
| ## 6. Envoy (`ext_authz`, HTTP service) |
| |
| ```yaml |
| http_filters: |
| - name: envoy.filters.http.ext_authz |
| typed_config: |
| "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz |
| transport_api_version: V3 |
| failure_mode_allow: true # DibaGuard unreachable / timed out: let the request through |
| failure_mode_allow_header_add: true # ... and mark it x-envoy-auth-failure-mode-allowed: true |
| with_request_body: # optional: Envoy (unlike nginx) can show DibaGuard the body |
| max_request_bytes: 8192 |
| allow_partial_message: true |
| allowed_headers: # what the decision reads (older Envoy: http_service.authorization_request.allowed_headers) |
| patterns: |
| - { exact: user-agent } |
| - { exact: cookie } |
| - { exact: content-type } |
| - { exact: accept } |
| - { exact: referer } |
| - { exact: x-forwarded-for } |
| - { exact: authorization } |
| - { exact: x-api-version } |
| - { exact: origin } |
| http_service: |
| server_uri: |
| uri: http://127.0.0.1:9300 |
| cluster: dibaguard_authz |
| timeout: 0.1s |
| path_prefix: /envoy # = dibaguard-authz -envoy-prefix |
| authorization_response: |
| allowed_upstream_headers: # on allow: verdict and score to the upstream request (overwrites) |
| patterns: [ { prefix: x-dibaguard } ] |
| allowed_client_headers: # on deny: what the client gets besides status and body |
| patterns: |
| - { exact: content-type } |
| - { exact: cache-control } |
| - { exact: retry-after } |
| - { prefix: x-dibaguard } |
| - name: envoy.filters.http.router |
| typed_config: { "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router } |
| |
| # clusters: |
| - name: dibaguard_authz |
| type: STATIC |
| connect_timeout: 0.05s |
| load_assignment: |
| cluster_name: dibaguard_authz |
| endpoints: |
| - lb_endpoints: |
| - endpoint: { address: { socket_address: { address: 127.0.0.1, port_value: 9300 } } } |
| ``` |
| |
| * The check request keeps the original method and is sent to `path_prefix + original path` |
| (`/envoy/items?id=1`). The service strips the prefix from the raw request target — `..` segments reach the |
| decision as sent, no path cleaning or redirect. |
| * Only **200** allows; any other status, with the allowed headers and the body (the challenge page, `slow down`), |
| goes to the client. A **5xx** from the service counts as an error, which `failure_mode_allow` lets through — so the |
| service never answers 5xx to Envoy (its fail-closed deny is a 403). |
| * Client address: with `use_remote_address: true` on the HTTP connection manager Envoy appends the downstream |
| address to `x-forwarded-for`; trust Envoy's own address (`-trusted 127.0.0.1/32` when co-located). |
| |
| --- |
| |
| ## 7. Traefik (`ForwardAuth`) |
| |
| ```yaml |
| # dynamic configuration |
| http: |
| middlewares: |
| dibaguard: |
| forwardAuth: |
| address: "http://127.0.0.1:9300/forward" |
| trustForwardHeader: false # true only if Traefik itself is behind a trusted proxy |
| authResponseHeaders: ["X-DibaGuard-Verdict", "X-DibaGuard-Score", "X-DibaGuard-Reason"] |
| routers: |
| shop: |
| rule: "Host(`shop.example`)" |
| middlewares: ["dibaguard"] |
| service: shop |
| services: |
| shop: |
| loadBalancer: |
| servers: [ { url: "http://127.0.0.1:3000" } ] |
| ``` |
| |
| Docker labels: `traefik.http.middlewares.dibaguard.forwardauth.address=http://dibaguard-authz:9300/forward`, |
| `traefik.http.middlewares.dibaguard.forwardauth.authResponseHeaders=X-DibaGuard-Verdict,X-DibaGuard-Score`, |
| `traefik.http.routers.shop.middlewares=dibaguard`. Kubernetes: a `traefik.io/v1alpha1` `Middleware` with the same |
| `forwardAuth` block. |
|
|
| Traefik sends `X-Forwarded-Method/-Uri/-Host/-Proto/-For` and all original headers; on a non-2xx it returns the |
| service's response (status, headers, body — so the challenge page works as is). Trust Traefik's address |
| (`-trusted 172.16.0.0/12` for a Docker network). There is no fail-open switch: an unreachable service is an error |
| for the client — run replicas. |
|
|
| --- |
|
|
| ## 8. Caddy (`forward_auth`) |
| |
| ```caddyfile |
| shop.example { |
| forward_auth 127.0.0.1:9300 { |
| uri /forward |
| copy_headers X-DibaGuard-Verdict X-DibaGuard-Score X-DibaGuard-Reason |
| } |
| reverse_proxy 127.0.0.1:3000 |
| } |
| ``` |
| |
| `forward_auth` sends `X-Forwarded-Method`, `X-Forwarded-Uri`, `X-Forwarded-For` (Caddy ignores a client's XFF unless |
| the client is in `trusted_proxies`), `X-Forwarded-Host/-Proto` and the original headers. On 2xx the `copy_headers` |
| are set on the proxied request; anything else is returned to the client as the service sent it. No fail-open |
| switch (an unreachable service is a 502) — run replicas or several upstreams in `forward_auth`. |
|
|
| --- |
|
|
| ## 9. HAProxy |
|
|
| ### 9a. Blocklist in HAProxy's own data path (Runtime API sync) |
|
|
| ```haproxy |
| global |
| stats socket ipv4@127.0.0.1:9999 level admin # or: stats socket /run/haproxy/admin.sock mode 600 level admin |
| stats timeout 30s |
| |
| frontend web |
| bind :80 |
| # sources the DibaGuard sidecar blocked: refused at accept, before any HTTP parsing; zero per-request round trip |
| tcp-request connection reject if { src -f /etc/haproxy/dibaguard-block.acl } |
| # behind another proxy use the address it saw instead: |
| # http-request deny if { req.hdr_ip(x-forwarded-for,-1) -f /etc/haproxy/dibaguard-block.acl } |
| default_backend app |
| ``` |
|
|
| ```sh |
| touch /etc/haproxy/dibaguard-block.acl # must exist (may stay empty) when HAProxy starts |
| dibaguard-authz -listen "" -sidecar http://127.0.0.1:8787 \ |
| -haproxy-socket 127.0.0.1:9999 -haproxy-file /etc/haproxy/dibaguard-block.acl |
| ``` |
|
|
| * The ACL is **owned by the syncer**: entries it holds that are not in the sidecar's block set are deleted. Use a |
| dedicated file; put hand-made blocks in another ACL. |
| * Only the difference is sent (`add acl` / `del acl`; `set map` when a map value changes); additions go first. |
| * Deadlines are applied by the syncer: an entry leaves HAProxy when its deadline passes, even if the sidecar is down. |
| * On every (re)connect, every `-haproxy-resync`, and whenever the connection is found closed (stats timeout, |
| reload — checked before each sync without sending anything), it re-reads what HAProxy holds (`show acl`) and |
| diffs against that. A reload resets the ACL to the (empty) file on disk; it is repaired on the next sync. |
| * Keys are validated as IP addresses / CIDRs before they reach the socket (IPv4-mapped IPv6 is unmapped); nothing the |
| sidecar sends can inject a Runtime API command. With `-ipv6-block-prefix 64` (default) an IPv6 block is pushed as |
| its /64, one entry per network. |
| * Map variant, one table for all standing decisions: |
|
|
| ```haproxy |
| http-request deny deny_status 403 if { src,map_ip(/etc/haproxy/dibaguard.map) -m str block } |
| # e.g. send challenged / rate-limited sources through the verdict service (9b), let others skip it |
| ``` |
| `-haproxy-kind map -haproxy-file /etc/haproxy/dibaguard.map -haproxy-actions block,challenge,rate_limit` |
| (values: `block` > `challenge` > `rate_limit`). Match the value — `-m found` would also drop rate-limited sources. |
|
|
| ### 9b. Per-request verdict (lua auth-request) |
|
|
| With [haproxy-lua-http](https://github.com/haproxytech/haproxy-lua-http) and `auth-request.lua` |
| (TimWolla/haproxy-auth-request). The lua action calls the auth backend with `HEAD` and the current request headers, |
| so the original method/URI/client are set as headers first. |
|
|
| ```haproxy |
| global |
| lua-prepend-path /usr/share/haproxy/?/http.lua |
| lua-load /usr/share/haproxy/auth-request.lua |
| |
| frontend web |
| bind :80 |
| http-request set-header X-Original-Method %[method] |
| http-request set-header X-Original-URI %[url] |
| http-request set-header X-Forwarded-Host %[req.hdr(host)] |
| http-request set-header X-Forwarded-For %[src] # overwrite whatever the client sent |
| # fail open when no dibaguard-authz is healthy; otherwise deny everything the service did not allow |
| http-request lua.auth-request dibaguard_authz /forward if { nbsrv(dibaguard_authz) gt 0 } |
| http-request deny deny_status 403 if { nbsrv(dibaguard_authz) gt 0 } !{ var(txn.auth_response_successful) -m bool } !{ var(txn.auth_response_code) -m int 429 } |
| use_backend dibaguard_challenge if { var(txn.auth_response_code) -m int 429 } |
| default_backend app |
| |
| backend dibaguard_authz |
| timeout connect 50ms |
| timeout server 100ms |
| server authz1 127.0.0.1:9300 check inter 1s |
| |
| backend dibaguard_challenge # challenge page or 429 slow down, rendered for this client |
| http-request set-path /challenge |
| server authz1 127.0.0.1:9300 |
| |
| backend app |
| server app1 127.0.0.1:3000 |
| ``` |
|
|
| Deny on "not successful", not on a list of codes: any answer other than 2xx (a 400 for a malformed header included) |
| must not pass. The lua action does not return the service's body, hence the `dibaguard_challenge` backend, which |
| tells a challenge from a rate limit by the sidecar state. |
|
|
| --- |
|
|
| ## 10. Rolling out: report mode |
|
|
| `-mode report` answers **200 to everything**, even when fail-closed, and puts the would-be verdict in |
| `X-DibaGuard-Verdict` / `-Reason` (and the summaries the sidecar receives). Log `$dg_verdict` / `$dg_reason` (nginx), |
| `%[var(...)]` (HAProxy) or the upstream request headers (Envoy/Traefik/Caddy) for a few days, check the `block` and |
| `challenge` lines, then switch to `-mode enforce`. In report mode the rate-limit bucket is consumed as in enforce |
| (shadow evaluation), so `rate_limit` verdicts are what enforce would return. |
|
|
| --- |
|
|
| ## 11. Latency budget |
|
|
| Measured 2026-10-31 on an Intel i3-12100F (4C/8T), Windows 11, Go 1.23.4, with the test-fixture HTTP model, while |
| the machine carried ~65 % CPU load from other work — read these as upper bounds. |
|
|
| In process (`go test -bench`, one decision, no network): |
|
|
| | path | ns/op | allocs | |
| |---|---|---| |
| | `Decide` benign (`/product/42?page=2&sort=price`) | 24 000 – 29 700 | 37 (16.6 KB) | |
| | `Decide` benign Persian query | 21 800 – 24 400 | 35 | |
| | `Decide` SQLi (`1' union select ...`) | 92 000 – 125 000 | 34 | |
| | `Decide` Log4Shell header | 104 000 – 113 000 | 34 | |
| | `Decide`, sidecar-blocked source (no scoring) | 21 – 24 | 0 | |
| | ↳ of which `MatchSigs` benign / attack | 1 900 – 2 200 / 54 000 – 69 000 | | |
| | ↳ of which HTTP model `Logits` | 10 400 – 11 800 | | |
| | full `/nginx` handler, benign (identity, reconstruction, budgeted decision, headers) | 31 000 – 36 000 | 65 | |
| | full `/nginx` handler, SQLi | 113 000 – 124 000 | 73 | |
|
|
| An attack costs more than a benign request because the signature prefilter lets it through to all nine regex |
| families; a source the sidecar already blocked costs nothing. |
|
|
| Over the network (real nginx 1.30.6, Windows loopback, keep-alive to the service and the app, 500 sequential |
| requests per row, two rounds, client-side `time_total`): |
|
|
| | path | p50 | p90 | p99 | |
| |---|---|---|---| |
| | nginx → app, no DibaGuard | 116 – 126 µs | 174 – 180 µs | 232 – 257 µs | |
| | nginx + `auth_request` → app, benign | 242 – 246 µs | 320 – 329 µs | 426 – 602 µs | |
| | nginx + `auth_request`, SQLi refused (403) | 233 – 236 µs | 320 – 328 µs | 468 – 470 µs | |
| | `dibaguard-authz` alone over HTTP, benign / SQLi | 77 / 118 – 123 µs | 142 – 148 / 194 – 199 µs | 227 – 244 / 296 – 374 µs | |
|
|
| So `auth_request` adds **~120–130 µs at p50** and ~150 µs at p90 on this machine. Budget for the LB side: |
| connect timeout 50–100 ms, read timeout 100–200 ms, well above the service's own `-budget 25ms` so that the service |
| answers (with its fail mode) before the LB gives up. The HAProxy ACL path costs nothing per request; a new block |
| reaches HAProxy within poll + sync interval (≤ ~4 s with the defaults). |
|
|
| --- |
|
|
| ## 12. Failure modes |
|
|
| | failure | default (fail-open) | `-fail-closed` | |
| |---|---|---| |
| | model file missing / unreadable at start | signatures and sidecar state still enforced; everything else allowed, `X-DibaGuard-Degraded: model_unavailable` | every request denied, `fail_closed:model_unavailable` | |
| | sidecar down, or state older than `-state-stale` | last known state kept (its deadlines still apply), decisions continue, `Degraded: sidecar_unavailable` | every request denied | |
| | decision over `-budget` (body read included) | allowed, `fail_open:timeout`, answered at the budget | denied, `fail_closed:timeout` | |
| | no decision slot within the budget (`-max-inflight`) | allowed, `fail_open:overload` | denied | |
| | panic inside a decision | allowed, `fail_open:panic` | denied | |
| | report mode | never denies | never denies | |
| | service unreachable — nginx | 500 unless `error_page 500 = @dibaguard_failopen` **with the `$dg_status` guard** (§4) | drop the `error_page 500` line | |
| | service unreachable — Envoy | `failure_mode_allow: true` | `failure_mode_allow: false` | |
| | service unreachable — Traefik / Caddy / ingress-nginx | error to the client (no switch) | same | |
| | service unreachable — HAProxy lua | skipped while `nbsrv(dibaguard_authz)` is 0 (health check) | drop the `if { nbsrv ... }` conditions | |
| | HAProxy Runtime API unreachable | syncer retries every interval, logs each new error once; **HAProxy keeps its current ACL** (entries do not expire inside HAProxy while the syncer is away) | same | |
| | HAProxy reload | ACL back to the file on disk until the next sync detects the closed connection and resyncs | same | |
| |
| Fail-open trades availability for coverage: a flood that pushes decisions past the budget, or an outage, lets |
| requests through. Pick `-fail-closed` where an unchecked request is worse than a refused one. |
| |
| --- |
| |
| ## 13. Verified on real software (nginx, dibaguard-proxy) |
| |
| nginx 1.30.6 (`nginx.org` Windows zip, PGP signature checked), the §4 configuration (plus debug headers and extra |
| server blocks for report mode, a dead service and the naive fail-open), `dibaguard-authz` in enforce |
| and report mode, a fake sidecar (block 127.0.0.3, rate-limit 127.0.0.4, challenge 127.0.0.5) and clients bound to |
| distinct loopback addresses: |
| |
| | request | result | |
| |---|---| |
| | benign GET, Persian query, POST form | 200, verdict allow | |
| | SQLi, UNION SELECT, double-encoded SQLi, XSS, traversal, command injection, Log4Shell header | 403, `sig:A15/A19/A17/A18/A08` | |
| | `/.git/config` | 429 challenge page (via `error_page 401`); with its cookie 200; the same cookie from another address 429 | |
| | sidecar-blocked source; the same source sending `X-Forwarded-For: 1.2.3.4` | 403 `sidecar:block` both | |
| | another source sending `X-Forwarded-For: 127.0.0.3` (the blocked one) | 200 — identity not taken over; the sidecar summary is attributed to the real sender | |
| | sidecar-challenged source; with its cookie | 429 challenge; 200 | |
| | rate-limited source, 4 quick requests (1 r/s, burst 2) | 200, 200, 429 `slow down`, 429 | |
| | report mode: SQLi / blocked source / probe | 200 with verdict `block` / `block` / `challenge` | |
| | service down, guarded fail-open / no `error_page` | 200 from the app / 500 | |
| | SQLi with a `0x01` byte in `User-Agent` | guarded config 403; the naive `error_page 500 = @app` served the SQLi to a lenient app (200) | |
| |
| Second round, after the decoding and identity changes — through nginx + `dibaguard-authz` and directly against two |
| `dibaguard-proxy` instances (`-trusted 127.0.0.1/32`, one with `-proxy-protocol`): |
| |
| | request | result | |
| |---|---| |
| | `?id=%zz1'or 1=1-- -`, `?host=127.0.0.1;cat /etc/passwd`, `%u0027`-encoded SQLi, double encoding + bad escape (nginx) | 403 each | |
| | the same through the proxy, and a POST body `user=admin%27+or+1%3D1--+-&x=%zz` | 403 each | |
| | `?q=100%`, `?q=50%+off&sort=price;desc` (nginx and proxy) | 200 | |
| | proxy: sidecar-blocked 127.0.0.3 sending `X-Forwarded-For: 1.2.3.4` | 403 (cannot escape) | |
| | proxy: 127.0.0.2 sending `X-Forwarded-For: 127.0.0.3` | 200 (cannot frame; summary attributed to 127.0.0.2) | |
| | proxy: trusted 127.0.0.1 sending `127.0.0.3` / `127.0.0.3, 9.9.9.9` | 403 / 200 (client 9.9.9.9) | |
| | proxy + PROXY v1 (curl `--haproxy-protocol --haproxy-clientip`): client 127.0.0.3 / 203.0.113.7 / 203.0.113.7 with `X-Forwarded-For: 127.0.0.3` / 203.0.113.7 SQLi | 403 / 200 / 200 / 403 | |
| | proxy + PROXY: no header from the trusted peer; a header from untrusted 127.0.0.2 | connection closed without an answer, both | |
| |
| --- |
| |
| ## 14. Limits and notes |
| |
| * **Bodies.** nginx `auth_request`, Traefik, Caddy and HAProxy lua do not send the body, so body payloads are not |
| inspected through them (the URI, query and headers are). Envoy `with_request_body` and the inline proxy do inspect |
| the body prefix. |
| * **Upstream status.** An authorization check never sees the app's reply, so the summaries sent to the sidecar |
| carry status 0 (403 for content blocks). The sidecar's failure-share signals (credential stuffing, enumeration by |
| 4xx) need the inline proxy or the LB's access logs. |
| * **Challenge cookie.** Bound to the client address and the UTC day; all replicas must share `-challenge-secret`. It |
| is a JS check (stops clients that do not run JS), not a CAPTCHA. |
| * **What the app may trust.** Only `X-DibaGuard-*` headers the LB set (nginx `proxy_set_header`, Envoy |
| `allowed_upstream_headers`, Traefik `authResponseHeaders`, Caddy `copy_headers` all overwrite a client's value). |
| * **Header size.** The service accepts headers up to Go's 1 MB default, more than any of these LBs forwards by |
| default, so an oversized request gets a verdict rather than a 431. |
| * **Canonical text vs. the Python canonicaliser.** Percent decoding of malformed escapes now matches |
| `dibaguard/http/canon.py` (checked on 20 malformed / mixed / `%u` / double-encoded strings: identical). The Go fast |
| path still differs from `parse.py`'s `model_text` in ways that predate this work: it does not apply the HTML-entity, |
| `\xHH` / `\uXXXX`, NFKC, SQL-comment and whitespace layers; it keeps invalid UTF-8 bytes where Python writes |
| U+FFFD; `+` becomes a space in every decoding round (Python: the first only, so `%2B` ends as `+` there and as a |
| space here); the first decoding of a query pair is not counted in `deepest`; query keys are sorted; cookies are one |
| `h:cookie` field (Python: `c:` pairs); the header set differs (Go also reads `accept`, `authorization`, |
| `x-api-version`, `origin`; Python also `x-requested-with`). |
|
|
