File size: 2,515 Bytes
b47934a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | # E2E Tests & Benchmarks
End-to-end tests and benchmarks that run against a **live OpenMeter stack** over HTTP.
Both skip unless `OPENMETER_ADDRESS` is set.
## Run the local stack
```sh
make env-local-up # build + start openmeter + infra (docker compose)
make env-local-down # tear down
```
Server listens on `http://localhost:38888`. `config.yaml` is bind-mounted, so config
changes need only `make env-local-up` (force-recreate), not a full down/up. Logs land in
`e2e/logs/`.
## Benchmarks
`BenchmarkGovernanceQuery` measures `POST /api/v3/openmeter/governance/query` latency
across a customers × features grid. Seeds boolean entitlements (no usage events).
```sh
make bench-governance # 1x1 baseline + 10/50/100 diagonal
make bench-governance-matrix # full 3x3 customers x features matrix
```
Overridable vars (defaults shown):
| Var | Default | Purpose |
|---------------------|--------------------------|----------------------------------------------|
| `OPENMETER_ADDRESS` | `http://localhost:38888` | target server |
| `BENCHTIME` | `20x` | iterations per sub-benchmark |
| `COUNT` | `1` | repeat count (use >1 for benchstat variance) |
For variance / before-after comparison:
```sh
make bench-governance COUNT=10 | tee baseline.txt
benchstat baseline.txt # mean ± %CV
benchstat baseline.txt after.txt # delta + p-value
```
> `ns/op` = mean of sequential request latencies (no concurrency, no tail percentiles).
> Boolean entitlements skip the metered/ClickHouse balance path — a relative algorithmic
> baseline, not a production-latency oracle.
## Traces (optional)
OpenMeter runs in a container, so OTLP must target the **host**, not container loopback.
Point the OTLP exporters in `e2e/config.yaml` at `host.docker.internal:4317` and run a
collector on the host (e.g. `grafana/otel-lgtm`). Then query per-size latency percentiles
in Grafana (Tempo, TraceQL metrics):
```
{ name = "governance.QueryAccess" && span.customer_key_count = 100 && span.feature_key_count = 100 }
| quantile_over_time(duration, 0.5, 0.9, 0.95, 0.99)
```
High-fan-out requests emit thousands of SQL spans; bump Tempo's `max_bytes_per_trace`
(default 5MB) or traces get dropped at ingest.
## Standard e2e tests
```sh
make test-local # full down → up → go test ./... → down
```
|