Separate process health from readiness
One question is whether the application is functioning well enough to continue running. Another is whether this instance should receive new work. Those questions may need different checks and different responses.
Avoid making every downstream failure a reason to restart every application instance. If a shared dependency is unavailable, repeated restarts may add disruption without repairing the underlying condition. Decide which failures an instance can recover from and which require a different intervention.
Keep the probe meaningful and bounded
A health endpoint should respond predictably and avoid expensive work on each request. If it checks dependencies, define time limits and understand the extra load generated by repeated probes across all instances.
Do not confuse a lightweight probe with a complete user-journey test. A basic readiness check may say little about an email being delivered or an imported record becoming usable. Add separate checks for important workflows when those outcomes need independent verification.
Connect failure to an action
Document what the platform does when a check fails and what an operator should investigate. Review how an instance starts, warms up, receives traffic, and stops. The check should match those phases rather than relying on a fixed delay that happens to work in one environment.
Exercise a slow dependency and a partially initialized application in a test environment. Confirm that traffic handling and recovery behave as intended.
Health checks are operational interfaces. Their value comes from the decision they support, so choose a clear purpose for each check and avoid treating a green endpoint as evidence that every part of the product is working.
A practical example.
A process can answer a basic HTTP request while its background worker has stopped processing jobs. A useful health design separates whether the process is alive, whether it should receive traffic, and whether important workflows are actually completing.
For a request service, a simple endpoint might establish that the web process is responsive, while separate monitoring tracks database access and the age of queued work. Avoid making every transient external failure restart every application instance. Define the response to each check and test that response deliberately. The value of a check comes from the decision it supports; a green indicator without an operational interpretation can hide a failing service as easily as no indicator at all.
Put it into practice.
- State what each check proves and what it does not prove.
- Match failures to an appropriate routing, alerting, or recovery response.
- Observe important background work separately from the web process.