The release completed. Git and the cluster match. Argo CD shows Synced and Healthy.
Meanwhile the checkout endpoint is returning errors.
Which dashboard is lying? Neither. They're answering different questions.
Argo CD is excellent at reconciling desired infrastructure state. It is not automatically a test of business functionality.
What green really tells you
Synced means live state agrees with the desired state Argo CD tracks. Resource health is calculated from Kubernetes objects and resource-specific checks. A Deployment with the desired number of updated replicas can look healthy even when a downstream payment API, a feature flag, or a production-only authentication flow is broken.
This does not make Argo CD useless. It means that reconciliation and user experience must be measured separately.
A completely plausible failure
A new image deploys cleanly. Pods become Ready because /health only checks that the HTTP server responds. The application then calls a payment provider using a misconfigured secret. The request path fails, but the Deployment continues to meet its Kubernetes health conditions.
The GitOps tool has done exactly what it was configured to do.
Four checks I'd add after deployment
- Readiness that means "can serve traffic." Make the probe reflect the application's ability to handle requests without turning every transient downstream error into a cluster-wide readiness failure.
- Synthetic or end-to-end checks. Exercise a representative customer path from outside the pod and, where appropriate, outside the cluster.
- Release-aware service metrics. Compare error rate, request latency, and critical business transactions before and after the change.
- A rollback decision, not just a rollback command. Define who can stop promotion, what signal triggers it, and how a known-good version is restored.
For high-risk services, a progressive rollout with analysis of production metrics is often a better release gate than "the pods became Ready."
Keep the responsibilities separate
Use Argo CD to verify desired-state convergence and Kubernetes resource health. Use observability to judge user impact. Use your deployment process to decide whether the release should continue.
The official Argo CD resource health documentation describes the scope of its built-in assessments. That is a useful starting point when deciding what additional checks your team needs.
Discussion: Do you consider a Kubernetes deployment successful when Argo CD is green—or only when production traffic proves it?
If releases are technically successful but operationally risky, review your GitLab CI/CD setup or discuss a GitOps review.