Jay Brown
477518e5eb
API instrumentation - metrics and logging * api instrumentation prometheus metrics and logging for all routes * Merge branch 'main' of bitbucket.org:aarete/query-orchestration into feature/prometheus * add log_level and sync the config initialize * docs * cleanup * add to compose:up:test * docs for prometheus testing * custom metrics * cleanup * merge main * add tests * cleanup docs * cleanup * lint * fix unit tests (attempt) * fix tests * resolvesetlogger * expose 8080 * compose changes * bugfixesformichael * don't build _test * merge in main * job_sync_runner
1.2 KiB
Prometheus docs
All instructions assume you are already in the devbox shell
To run various services and inspect prometheus metrics first run the
task compose:up:test to start the db and prometheus services.
Then you can run your service that exposed the metrics.
Testing the API service
Start the API server on the host like
DB_PORT=5430 go run ./cmd/queryService
Once started you will be able to see the raw metrics at http://localhost:8080/metrics
The prometheus console will be running at http://localhost:9090/
The docs to the api server will be available at http://localhost:8080/swagger/index.html
For example to run the query endpoint (to see query related metrics generated) go to http://localhost:8080/query
Sample prometheus queries for the console
Duration of the /query api call (from Echo metrics)
echo_request_duration_seconds_sum{code="200", host="localhost:8080", instance="host.docker.internal:8080", job="api", method="GET", url="/query"}
Memory allocation rate (from Go metrics)
rate(go_memstats_alloc_bytes_total[5m])
Time since last GC
time() - go_memstats_last_gc_time_seconds
Gc overhead
rate(go_gc_duration_seconds_sum[5m])
More examples TBD.