A new service can look like a clean ownership boundary in a design review and behave like another failure path at 2 a.m. For most teams, I would default to a modular monolith, not microservices, because the first production constraint is usually the ability to diagnose and reverse a bad change. Separate deployments earn their place when operational evidence shows that shared deployment is the bigger risk.
Microservices multiply failure paths before they isolate them
Microservices vs Monolith Choosing the Right Architecture frames the decision around application architecture. I would start with the incident path instead, because an on-call engineer has to restore a request that crosses deployments, networks, and data stores—not inspect a diagram of well-separated code.
Consider an API request that calls an account service, which calls a permissions service, while both use PostgreSQL. The services may have separate repositories and release schedules, but a saturated connection pool can still stop the whole request. A network hop also creates failure states that an in-process call does not have: the caller may time out after the callee commits, or retry while the first attempt is still running. Those states matter because the operator must decide whether to shed traffic, stop retries, roll back, or wait for a dependency to recover.
The isolation claim is especially weak when every service shares one Kubernetes cluster, one ingress path, and one database instance, because those components remain common failure domains. Kubernetes Deployments can restart failed pods, but a restarted caller will not repair an exhausted PostgreSQL connection limit. Istio or Envoy can enforce timeouts and expose upstream errors, but their configuration adds another place to look when a request fails. Neither tool is inherently the problem; the problem is treating an additional control plane as proof of isolation.
For an incident exercise, assume a request crosses four services; that is a scenario assumption, not a measured result. Draw each synchronous call and shared dependency, then mark where an HTTP 503, a timeout, or a successful-but-late response can occur. The drawing is useful because each mark represents a decision the on-call engineer may have to make without knowing whether work completed downstream.
I would not split a deployable merely because its code has distinct business modules, because a module boundary can prevent unwanted imports without introducing partial network failure. I would split it when that boundary also requires independent availability, scaling, or release control—and when the team can operate those properties during an incident.
A modular monolith makes production evidence cheaper to collect
Modular monolith versus microservices: the modular monolith wins when most requests share data and releases can move together, because one deployment and one in-process call graph leave fewer places for a request to disappear. It costs you independent scaling and requires discipline around module interfaces. Microservices win when a component has a genuinely different availability or scaling requirement and can be released without coordinating a shared schema; they cost you distributed failure handling, deployment coordination, and more telemetry to interpret.
That is a production comparison, not a vote against modular design. A monolith can keep account and permissions code behind separate packages, prohibit cross-module imports in CI, and publish explicit interfaces. Those checks matter because a shared process otherwise makes accidental coupling easy. PostgreSQL can keep a transaction local when both modules update related records; splitting the modules can turn the same operation into a workflow that needs idempotency and reconciliation.
Instrument the single deployment before arguing that it is too opaque. OpenTelemetry spans can identify expensive module operations without turning each operation into an HTTP service. The W3C traceparent header preserves context at the application boundary; inside the process, span attributes can retain module names. Prometheus request-duration histograms and Grafana dashboards can show latency and error rates, while PostgreSQL’s pg_stat_activity can help determine whether requests are waiting on database work. This instrumentation has a cost, but it is also needed after a split, when the question becomes harder to answer.
For a trial service-level objective, 99.9% availability over a 30-day window allows a calculated 43.2 minutes of unavailability. That arithmetic is a planning example, not a claim about your system’s performance. Use the error-budget definition that matches what users experience, because a healthy pod count can coexist with failed requests. If the team cannot establish the current request failure rate and a rollback time, adding deployments will make the effect of a split difficult to verify.
Microservices vs Monoliths Choosing the Right Architecture poses a useful choice, but the production test should be narrower: can the proposed boundary fail, recover, and roll back without making the existing request path less reliable? That question demands an exercise and evidence rather than confidence in the shape of the code.
Retries and rollouts turn boundaries into incident multipliers
A timeout does not cancel work that has already started downstream, so retrying without an idempotency rule can duplicate a write. A provisional 800 ms caller timeout is a value to tune against observed latency and the request’s total deadline, not a universal setting. Begin a retry experiment with a ceiling of one retry for an explicitly safe operation, then adjust it under load, because multiple callers retrying the same unhealthy dependency can increase the work it is already failing to finish.
HTTP 429 and 503 are useful only when callers respond appropriately: a caller that ignores Retry-After may send traffic back before capacity returns. Envoy retry policies and circuit breakers can bound some of that behavior, but operators still need to know which layer retries; an application retry followed by a proxy retry can produce more attempts than either owner intended. Record attempt counts in OpenTelemetry traces and compare them with incoming requests in Prometheus, because a rising ratio can reveal amplification before a dependency goes fully unavailable.
Rollbacks deserve the same skepticism as retries. Kubernetes publishes a default terminationGracePeriodSeconds of 30 seconds; that documented default is not evidence that an application finishes its in-flight work within the window. Set readiness probes to reflect whether the application can serve requests, and test termination while traffic is present, because a pod marked healthy before it can use its dependencies may receive requests it cannot complete.
The following shell script updates an existing Kubernetes Deployment and undoes the revision if rollout status fails. It requires kubectl access, a Deployment with a prior revision, a valid container name, and an image reference supplied as its fourth argument:
#!/bin/sh
set -eu
namespace="${1:-production}"
deployment="${2:-api}"
container="${3:-api}"
image="${4:?pass an image as fourth argument}"
kubectl -n "$namespace" set image "deployment/$deployment" "$container=$image"
if ! kubectl -n "$namespace" rollout status "deployment/$deployment" --timeout=180s; then
kubectl -n "$namespace" rollout undo "deployment/$deployment"
exit 1
fi
The 180-second timeout is an operational setting to tune for startup and probe behavior. This script detects a failed Kubernetes rollout, not a rise in customer-facing errors, because rollout status reports deployment progress rather than request outcomes. Pair it with an error-rate check and an operator-visible rollback procedure before relying on it in production. A single deployable makes that procedure simpler; several services require deciding which revision caused the failure and whether rolling it back is compatible with changes elsewhere.
A service split needs evidence of independent operation
The strongest case for a service is a component that can remain useful or recover independently when the rest of the application is impaired. Independence has to include its dependencies: moving code into another pod while retaining synchronous calls back to the monolith does not provide much failure isolation, because the new endpoint still waits on the old one. Similarly, a separate release is not independent if every deployment requires a coordinated PostgreSQL schema change.
Test a proposed boundary with a failure drill before extracting it. Disconnect its dependency, force slow responses, and stop its pods while sending representative traffic through the caller. Record the observed request error rate, the p95 latency, and time to restore service; these are measurements to collect in your environment, not numbers that an architecture pattern can supply. Use Jaeger traces to inspect where time was spent and Kubernetes events to see whether restarts or failed probes contributed, because either view alone can give an incomplete account of the incident.
Then rehearse a schema change in both release orders. If the old and new application versions cannot operate against the intermediate schema, the services do not have independent releases, because one deploy can break the other. Expand-and-contract migrations can make that overlap possible, but they require maintaining compatible reads and writes until the older version is gone. Keep the boundary inside the monolith if the team cannot afford that migration work; calling the component a service does not remove it.
A service may still be worth the cost when a load test shows that its workload needs separate capacity or an incident shows that its failures can be contained. In that case, specify the dependency deadlines, ownership of retries, idempotency behavior, dashboards, and rollback sequence before the first production cutover, because those details determine whether the separation helps the person carrying the pager.
Start with the request the pager cannot explain
Take one recent production alert and trace a failed request from ingress to its final dependency. Write down the missing observation and the action that would have restored service sooner. If the gap is a module boundary, enforce it in code first. If it is a failure that truly needs separate recovery, run that recovery drill before creating a new service.


