TL;DR: This guide demonstrates how to replace fragile push-based CI/CD pipelines with a self-healing GitOps architecture on AWS EKS using ArgoCD and Kustomize. You will learn how to structure isolated manifest repositories, enforce cluster state using an ArgoCD
ApplicationCRD withselfHeal: true, and securely connect GitHub Actions to AWS ECR using OIDC authentication.
β‘ Key Takeaways
- Separate application source code from Kubernetes manifests using Kustomize directory structures (
base/andoverlays/) to prevent non-infrastructure commits from triggering deployments. - Deploy ArgoCD in High Availability mode on EKS via Helm by explicitly setting
server.replicas=2andrepoServer.replicas=2. - Prevent configuration drift and manual
kubectloverrides by enablingselfHeal: truein your ArgoCDApplicationmanifest. - Configure GitHub Actions to update the image tag directly in
kustomization.yamlafter pushing to ECR, allowing ArgoCD to pull the changes automatically. - Secure your CI/CD boundary by authenticating GitHub Actions to AWS using OpenID Connect (OIDC) rather than storing long-lived IAM user credentials.
Your continuous deployment pipeline is lying to you.
In a traditional push-based CI/CD setup, a GitHub Action or Jenkins job authenticates with your Kubernetes cluster and executes kubectl set image or helm upgrade. The pipeline turns green, and your team assumes the release was successful. Meanwhile, inside the cluster, your new pods are stuck in a CrashLoopBackOff state due to a missing environment variable.
Worse, when an engineer manually hotfixes the cluster via the CLI to resolve an outage, you introduce Configuration Drift. The cluster state no longer matches your Git repository. The next time the pipeline runs, it overwrites the hotfix, re-introducing the outage.
Push-based deployments are fragile, lack self-healing capabilities, and treat infrastructure state as an afterthought.
The solution is GitOps. By reversing the deployment flowβshifting from push to pullβwe can ensure that Git remains the single source of truth for your infrastructure. In this blueprint, we will architect a zero-downtime, self-healing deployment pipeline using AWS EKS, GitHub Actions, ArgoCD, and Argo Rollouts for automated canary deployments.
The Architecture of Pull-Based GitOps
In a GitOps model, your CI pipeline does not communicate with your Kubernetes cluster. It only interacts with your container registry (like AWS ECR) and a Git repository.
We separate our codebase into two distinct repositories (or deeply isolated directories):
- The Application Repository: Contains the source code (e.g., Node.js), Dockerfile, and unit tests.
- The Manifest Repository: Contains pure Kubernetes YAML manifests managed via Helm or Kustomize.
Production Note: Never store your Kubernetes manifests in the same repository as your application code if you are building enterprise applications. A single commit to a
README.mdshouldn't trigger an ArgoCD reconciliation loop.
Here is the architectural directory structure of our Manifest Repository using Kustomize:
infrastructure-manifests/
βββ overlays/
β βββ production/
β β βββ kustomization.yaml
β β βββ rollout-patch.yaml
β β βββ ingress.yaml
β βββ staging/
β βββ kustomization.yaml
β βββ ...
βββ base/
βββ kustomization.yaml
βββ rollout.yaml
βββ service.yaml
βββ configmap.yaml
When a developer merges a Pull Request, the CI pipeline builds the Docker image, pushes it to ECR, and commits the new image tag directly to the kustomization.yaml file in the Manifest Repository. ArgoCD, running inside EKS, continuously watches this repository. When it detects a new commit, it pulls the changes and reconciles the cluster state.
Provisioning ArgoCD on AWS EKS
To establish our GitOps controller, we need to install ArgoCD onto our EKS cluster. We will use Helm for the initial bootstrap.
First, create a dedicated namespace and deploy the ArgoCD Helm chart:
# Add the ArgoCD Helm repository
helm repo add argo https://argoproj.github.io/argo-helm
helm repo update
# Create namespace
kubectl create namespace argocd
# Install ArgoCD with High Availability enabled for production
helm install argocd argo/argo-cd \
--namespace argocd \
--set server.extraArgs=\{--insecure\} \
--set controller.replicas=1 \
--set server.replicas=2 \
--set repoServer.replicas=2 \
--set applicationSet.replicas=2
Once installed, ArgoCD needs to know which repository to monitor. We define this using ArgoCD's Application Custom Resource Definition (CRD). Apply the following manifest to your cluster to instruct ArgoCD to watch your production overlay:
# argocd-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: nodejs-microservice-production
namespace: argocd
spec:
project: default
source:
repoURL: 'git@github.com:your-org/infrastructure-manifests.git'
targetRevision: HEAD
path: overlays/production
destination:
server: 'https://kubernetes.default.svc'
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
The selfHeal: true flag is the magic bullet against configuration drift. If a rogue engineer manually deletes a deployment or alters a ConfigMap via kubectl, ArgoCD instantly reverts the change back to the state defined in Git.
Bridging the Gap: GitHub Actions to EKS
With ArgoCD actively polling our Manifest Repository, our CI pipeline's only responsibility is to build the application image and update the manifest repository.
When designing highly available cloud deployment pipelines for enterprise clients, securing the CI/CD boundary is paramount. We avoid long-lived IAM user credentials, opting instead for OpenID Connect (OIDC) to securely authenticate GitHub Actions with AWS.
Here is the GitHub Actions workflow (.github/workflows/deploy.yml) that builds the Docker image, pushes it to ECR, and securely updates the Kustomize manifests:
name: Build and Update Manifests
on:
push:
branches:
- main
permissions:
id-token: write
contents: read
jobs:
build-and-push:
runs-on: ubuntu-latest
steps:
- name: Checkout Application Code
uses: actions/checkout@v3
- name: Configure AWS Credentials via OIDC
uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsECRRole
aws-region: us-east-1
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1
- name: Build, tag, and push image to ECR
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: nodejs-microservice
IMAGE_TAG: ${{ github.sha }}
run: |
docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
- name: Checkout Manifest Repository
uses: actions/checkout@v3
with:
repository: your-org/infrastructure-manifests
token: ${{ secrets.GITOPS_PAT }}
path: manifests
- name: Update Kustomize Image Tag
run: |
cd manifests/overlays/production
kustomize edit set image backend-api=${{ steps.login-ecr.outputs.registry }}/nodejs-microservice:${{ github.sha }}
git config user.name "GitHub Actions"
git config user.email "actions@github.com"
git add kustomization.yaml
git commit -m "chore: update production image to ${{ github.sha }}"
git push
By utilizing kustomize edit set image, we programmatically patch the YAML file without relying on fragile sed replacements. Once this commit lands in the Manifest Repository, ArgoCD detects the change and automatically initiates the deployment.
Implementing Automated Canary Rollouts with Argo Rollouts
A standard Kubernetes Deployment uses a RollingUpdate strategy, replacing old pods with new pods incrementally. However, it lacks fine-grained control: you cannot pause the rollout midway, run automated integration tests against the new pods, or split traffic precisely at 5% and 95%.
To achieve true zero-downtime releases, we replace the native Deployment resource with Argo Rollouts, a Kubernetes controller designed specifically for advanced deployment strategies like Canary and Blue-Green.
First, install the Argo Rollouts controller on your EKS cluster:
kubectl create namespace argo-rollouts
kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
Next, in your manifest repository, change kind: Deployment to kind: Rollout. We will configure a Canary strategy that routes exactly 10% of traffic to the new version by integrating with the AWS Application Load Balancer (ALB) Ingress Controller.
# rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: nodejs-microservice
namespace: production
spec:
replicas: 5
revisionHistoryLimit: 2
selector:
matchLabels:
app: nodejs-microservice
template:
metadata:
labels:
app: nodejs-microservice
spec:
containers:
- name: backend-api
image: backend-api:latest # Managed dynamically by Kustomize
ports:
- containerPort: 3000
strategy:
canary:
# Traffic routing using AWS ALB
trafficRouting:
alb:
ingress: microservice-ingress
servicePort: 80
rootService: nodejs-service
steps:
- setWeight: 10
- pause: { duration: 15m } # Sit at 10% traffic for 15 mins
- setWeight: 50
- pause: { duration: 5m } # Move to 50% for 5 mins
- setWeight: 100
With this configuration, when ArgoCD syncs a new image tag, Argo Rollouts intercepts the process. It provisions a subset of new pods and reconfigures the AWS ALB Listener Rules to route exactly 10% of incoming HTTP requests to the new version. The rollout pauses for 15 minutes, giving your team (or your automated monitoring tools) time to verify application health.
Handling Node.js Graceful Shutdowns in Kubernetes
A sophisticated canary rollout strategy is entirely useless if your application drops active user connections when Kubernetes scales down the old pods.
When Kubernetes terminates a pod, it sends a SIGTERM signal to the container. If your Node.js application does not intercept this signal, the process dies instantly, severing all in-flight HTTP requests.
We implemented the following graceful shutdown pattern for a high-volume client in our recent production deployments to ensure financial transactions were never interrupted during aggressive auto-scaling events:
// server.js
const express = require('express');
const app = express();
app.get('/healthz', (req, res) => {
res.status(200).send('OK');
});
// ... your API routes ...
const server = app.listen(3000, () => {
console.log('Server listening on port 3000');
});
// Handle Kubernetes SIGTERM for graceful shutdown
process.on('SIGTERM', () => {
console.log('SIGTERM received. Initiating graceful shutdown.');
// server.close stops accepting new connections but keeps existing ones alive until they finish.
// Kubernetes will simultaneously remove this pod from the Service endpoint list.
server.close((err) => {
if (err) {
console.error(err);
process.exit(1);
}
console.log('All active connections gracefully closed. Exiting.');
process.exit(0);
});
});
Tip: Ensure your Kubernetes
readinessProbeandlivenessProbepoint to the/healthzendpoint. Argo Rollouts relies heavily on readiness probes to determine if a canary pod has successfully booted before routing ALB traffic to it.
Monitoring and Automated Rollbacks
The true power of Argo Rollouts lies in its ability to automatically evaluate metric data during the pause steps of a canary release. Instead of manually watching dashboards, we can automate rollbacks using an AnalysisTemplate.
By integrating Argo Rollouts with Prometheus, we can continuously monitor the new canary version. If it generates an abnormal amount of HTTP 500 errors, Argo Rollouts automatically aborts the deployment and shifts 100% of traffic back to the stable version.
Here is an AnalysisTemplate definition that queries Prometheus for error rates:
# analysis-template.yaml
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: error-rate-check
spec:
args:
- name: service-name
metrics:
- name: http-5xx-errors
interval: 1m
successCondition: result[0] < 0.05 # Fail if the error rate exceeds 5%
provider:
prometheus:
address: http://prometheus-server.monitoring.svc.cluster.local:9090
query: |
sum(rate(http_requests_total{status=~"5.*", service="{{args.service-name}}"}[1m]))
/
sum(rate(http_requests_total{service="{{args.service-name}}"}[1m]))
To bind this analysis to your Rollout, append the analysis step to your strategy:
steps:
- setWeight: 10
- analysis:
templates:
- templateName: error-rate-check
args:
- name: service-name
value: nodejs-microservice
- pause: { duration: 15m }
Now, your pipeline is entirely hands-off. A developer merges code to main. GitHub Actions builds the image and updates the manifest. ArgoCD applies the new manifest to EKS. Argo Rollouts spins up a 10% canary. Prometheus evaluates the error rate.
If the code is buggy, traffic immediately routes back to the stable version, and the developer is notified. If it's clean, the rollout smoothly proceeds to 100%.
Zero downtime, zero manual intervention, and zero configuration drift.
Work With Us
Need help building this in production? SoftwareCrafting is a full-stack dev agency β we ship React, Next.js, Node.js, React Native & Flutter apps for global clients.
Frequently Asked Questions
Why am I seeing [object Object] in my JavaScript output?
This occurs when a JavaScript object is implicitly converted to a string, usually through string concatenation or template literals. Because the default toString() method for plain objects returns the literal string [object Object], the underlying data properties are hidden from the output.
How can I properly log JavaScript objects instead of [object Object]?
To view the actual contents of the object, use console.log() with multiple arguments (e.g., console.log("Data:", obj)) rather than concatenating it with a plus sign. Alternatively, you can use console.table(obj) for a cleaner, tabular view of the object's properties in your developer console.
How do SoftwareCrafting services help resolve complex JavaScript data errors?
At SoftwareCrafting, our services include comprehensive code audits and debugging to eliminate silent type coercion errors before they reach production. We help development teams implement strict typing with TypeScript and robust logging mechanisms to ensure your application's data flow remains transparent and predictable.
Can JSON.stringify() prevent the [object Object] rendering issue?
Yes, wrapping your variable in JSON.stringify(obj, null, 2) converts the object into a readable JSON string rather than relying on the default string conversion. The additional parameters format the output with indentation, making nested properties much easier to read during debugging sessions.
How can I fix [object Object] showing up in my React or UI components?
In UI frameworks like React, this happens when you try to render a raw object directly inside your markup instead of a primitive value. You must map over the object or explicitly render its specific string or number properties, such as rendering user.name instead of attempting to render the entire user object.
Why should I utilize SoftwareCrafting services for persistent frontend bugs?
Persistent frontend bugs and improper data rendering can severely degrade user experience and slow down your release cycles. By leveraging SoftwareCrafting services, your team gains access to senior engineers who establish architectural best practices, implement proper state management, and prevent foundational JavaScript errors from occurring in the first place.
π Full Code on GitHub Gist: The complete
workflow-debug.jsfrom this post is available as a standalone GitHub Gist β copy, fork, or embed it directly.
