sdlc.dev

Backend engineer path · Running it in production · Topic 15 of 18

Actuator and observability

Not started

Spring Boot Actuator adds production features under /actuator: health checks, metrics, info about the build, the conditions report, thread dumps and more. Behind the metrics is Micrometer, a vendor-neutral facade: Boot instruments HTTP requests, the JVM, the connection pool and more, and a registry you choose ships the numbers to Prometheus, Datadog, an OTLP collector or something else. Since Boot 3, the same instrumentation also produces distributed traces through the Micrometer Observation API.

Why it matters: When something is slow at 3 a.m., the numbers Actuator and Micrometer already collect are what tell you which hop to blame.

  1. 1Endpoints, carefully exposed

    Actuator adds /actuator endpoints, but only health is exposed over HTTP by default; expose the rest on purpose.

  2. 2Liveness is not readiness

    Liveness says the process should be restarted; readiness says it should not get traffic right now.

  3. 3Micrometer is the facade

    Code records to Micrometer once; a registry ships the numbers to Prometheus, Datadog, OTLP or others.

  4. 4Observations give both

    One Observation produces a timer metric and, with tracing on, a span that carries the trace id across services.

1

How it works

The explanation, with examples you can run.

Expose with intent

management:  endpoints:    web:      exposure:        include: health,info,prometheus  endpoint:    health:      show-details: when-authorized   # default: never      probes:        enabled: true                 # on automatically when running on Kubernetes  server:    port: 8081                        # keep management traffic off the public port

Only health is exposed over HTTP by default. Add what you need, and protect or isolate the rest; env, heapdump and loggers can reveal secrets or change behaviour.

Health, liveness and readiness

/actuator/health aggregates HealthIndicators: the database, disk space, message brokers and your own. On Kubernetes, Boot also exposes two health groups:

  • /actuator/health/liveness: is the application's internal state broken beyond repair? If this fails, the container is restarted.
  • /actuator/health/readiness: can it take traffic now? If this fails, the pod is taken out of the load balancer until it recovers.

By default neither group includes external checks such as the database, and that is deliberate: if a shared database is down, restarting every instance helps nobody. You can add an indicator to readiness with management.endpoint.health.group.readiness.include=readinessState,db if traffic really should drain.

Metrics

Things you get without writing code include http.server.requests (a timer per method, URI template, status and outcome), jvm.memory.used, jvm.gc.pause, hikaricp.connections.*, tomcat.sessions.* and executor metrics. Your own are a few lines:

@Componentclass CheckoutMetrics {  private final Counter declined;  CheckoutMetrics(MeterRegistry registry) {    declined = Counter.builder("checkout.payments.declined")        .tag("provider", "stripe")        .register(registry);  }  void onDeclined() { declined.increment(); }}

Keep tag values bounded. A tag holding a user id or a raw URL creates a new time series per value and can overwhelm the metrics backend.

Observations and traces

An Observation wraps a piece of work once and feeds several handlers: a timer for metrics and, when a tracer such as Micrometer Tracing with an OpenTelemetry or Brave bridge is on the classpath, a span.

Observation.createNotStarted("inventory.reserve", registry)    .lowCardinalityKeyValue("warehouse", warehouseId)    .observe(() -> inventory.reserve(order));

Boot propagates the trace context on outgoing calls made with the auto-configured RestClient.Builder, RestTemplateBuilder or WebClient.Builder, and can put the trace id into every log line, so one slow checkout can be followed from the gateway through each service to the query that took the time.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.You add spring-boot-starter-actuator. Which endpoints can you call over HTTP without further configuration?
  2. 2.The database is down. Should the liveness probe fail?
  3. 3.What does Spring Boot record for every Spring MVC request with Actuator on the classpath?

0 of 3 right so far. Get all three to finish this topic.

Go deeper