E2E Tests & Benchmarks
End-to-end tests and benchmarks that run against a live OpenMeter stack over HTTP.
Both skip unless OPENMETER_ADDRESS is set.
Run the local stack
make env-local-up # build + start openmeter + infra (docker compose)
make env-local-down # tear down
Server listens on http://localhost:38888. config.yaml is bind-mounted, so config
changes need only make env-local-up (force-recreate), not a full down/up. Logs land in
e2e/logs/.
Benchmarks
BenchmarkGovernanceQuery measures POST /api/v3/openmeter/governance/query latency
across a customers × features grid. Seeds boolean entitlements (no usage events).
make bench-governance # 1x1 baseline + 10/50/100 diagonal
make bench-governance-matrix # full 3x3 customers x features matrix
Overridable vars (defaults shown):
| Var | Default | Purpose |
|---|---|---|
OPENMETER_ADDRESS |
http://localhost:38888 |
target server |
BENCHTIME |
20x |
iterations per sub-benchmark |
COUNT |
1 |
repeat count (use >1 for benchstat variance) |
For variance / before-after comparison:
make bench-governance COUNT=10 | tee baseline.txt
benchstat baseline.txt # mean ± %CV
benchstat baseline.txt after.txt # delta + p-value
ns/op= mean of sequential request latencies (no concurrency, no tail percentiles). Boolean entitlements skip the metered/ClickHouse balance path — a relative algorithmic baseline, not a production-latency oracle.
Traces (optional)
OpenMeter runs in a container, so OTLP must target the host, not container loopback.
Point the OTLP exporters in e2e/config.yaml at host.docker.internal:4317 and run a
collector on the host (e.g. grafana/otel-lgtm). Then query per-size latency percentiles
in Grafana (Tempo, TraceQL metrics):
{ name = "governance.QueryAccess" && span.customer_key_count = 100 && span.feature_key_count = 100 }
| quantile_over_time(duration, 0.5, 0.9, 0.95, 0.99)
High-fan-out requests emit thousands of SQL spans; bump Tempo's max_bytes_per_trace
(default 5MB) or traces get dropped at ingest.
Standard e2e tests
make test-local # full down → up → go test ./... → down