Problem
How to detect that a running service instance is unable to handle requests?
Forces
- An alert should be generated when a service instance fails
- Requests should be routed to working service instances
Solution
A service has a health check API endpoint (e.g. HTTP /health) that returns the health of the service. The API endpoint handler performs
various checks, such as
- the status of the connections to the infrastructure services used by the service instance
- the status of the host, e.g. disk space
- application-specific logic
Occasionally, a service may be running but unable to handle requests. A newly started service instance may still be initializing and doing some sanity checks before it can handle requests.
It makes no sense for the deployment infrastructure to route HTTP requests to a service instance until it’s ready to process them.
It may also happen that the service instance fails without terminating, for example, all of the DB connections are used up and the database could not be accessed. The deployment infrastructure should not route requests to a service instance that failed and is still running; if the service instance fails to recover, it must be terminated and a new instance created.
A service instance must be able to tell the deployment infrastructure whether or not it is able to handle requests. You can use Spring Boot Actuator, which implements a health endpoint, to implement a health check endpoint for your service.
Resulting Context
This pattern has the following benefits:
- The health check endpoint enables the health of a service instance to be periodically tested
- The health check might not sufficiently comprehensive or the service instance might fail between health checks and so requests might still be routed to a failed service instance.
