CI/CD Pipeline

Architecture

• Build Stage: Build & package stage: compile and test, build the image, and produce the deploy bundle.
• Deployment Environments: Staged ECS clusters: dev -> staging -> prod, each a blue/green target gated by the stage before it.
• GitHub: Source of truth. Push to main (via PR merge) fires a webhook that starts the pipeline; commit statuses are reported back.
• Slack: Receives stage results, prod approval requests, and rollback alerts.
• Release Pipeline: Orchestrates source -> build -> dev -> staging -> approval -> prod. One pipeline execution per commit to main; superseded executions are cancelled.
• Build & Test: Runs unit tests and lint, builds the container image tagged with the git SHA, and produces the deploy bundle (taskdef + appspec).
• ECR: Immutable image registry. Images are tagged :<git-sha>; nothing is ever retagged, so any previous version can be redeployed.
• Artifact Bucket: Versioned bucket holding build artifacts, task definitions, and appspec files per pipeline execution.
• Blue/Green Deployer: ECS blue/green deployments: launches the green task set, shifts canary traffic, watches alarms during the bake, auto-rolls back on breach.
• Dev Cluster: Auto-deployed on every green build. Smoke tests run here before anything else proceeds.
• Staging Cluster: Production mirror. Blue/green deploy plus integration and load checks gate the prod approval.
• Prod Cluster: Deployed only after manual approval. Canary 10% for 15 minutes, then full traffic shift.
• Canary Alarms: 5xx rate, p99 latency, and task health alarms scoped to the green task set. Any breach during the bake window triggers automatic rollback.

Workflow

• Push to main: PR merged; GitHub webhook starts a pipeline execution pinned to this commit SHA.
• Build + unit tests: Lint, unit tests, container build tagged :<git-sha>.
• Build green?
• Publish artifacts: Push image to ECR, upload taskdef/appspec bundle to the versioned artifact bucket.
• Deploy to dev: Rolling deploy to the dev cluster; no approval needed.
• Smoke tests: Hit health and critical-path endpoints against dev.
• Smoke passed?
• Blue/green to staging: CodeDeploy launches the green task set in the staging cluster and shifts traffic.
• Integration + load checks: Full integration suite plus a short load test against staging.
• Staging healthy?
• Await prod approval: Manual approval gate. Release notes and staging results posted to Slack; any release approver can approve or reject.
• Approved?
• Prod canary 10%: CodeDeploy launches the green task set in prod and shifts 10% of traffic to it.
• Bake period: Canary alarms (5xx rate, p99 latency, task health) watched against the green task set.
• Alarms healthy?
• Shift 100%, retire blue: All traffic to green; blue task set drained and terminated after the wait window.
• Released: Commit status set to released; release annotated in CloudWatch.
• Auto rollback: CodeDeploy reroutes 100% of traffic back to the blue task set and terminates green. Prod never runs the bad build at full traffic.
• Page on-call: Rollback alert to Slack with alarm details and diff link; commit marked failed.
• Rolled back
• Report failure: Fail the pipeline execution, set GitHub commit status, post the failing stage to Slack.
• Pipeline failed

Sequence

Participants: GitHub, CodePipeline, CodeBuild, ECR, Release Approver, CodeDeploy, Prod ECS Service, CloudWatch

• GitHub→CodePipelinepush webhook (main @ sha)
• CodePipeline→CodeBuildstart buildSource artifact pinned to the commit SHA
• CodeBuild→ECRpush image :sha
• CodeBuild→CodePipelinebuild succeeded + deploy bundleDev and staging stages run here (elided); staging results attach to the approval
• CodePipeline→Release Approverrequest prod approvalSlack message with release notes and staging results
• Release Approver→CodePipelineapprove release
• CodePipeline→CodeDeploycreate deployment (canary 10%)
• CodeDeploy→Prod ECS Servicelaunch green task set
• CodeDeploy→Prod ECS Serviceshift 10% traffic to green
• CodeDeploy→CloudWatchwatch canary alarms (15 min bake)
• CodeDeploy→Prod ECS Serviceshift 100%, terminate blue
• CodeDeploy→CodePipelinedeployment succeeded
• CodePipeline→GitHubcommit status: released
• CloudWatch→CodeDeployALARM: 5xx rate breach on green
• CodeDeploy→Prod ECS Servicereroute 100% to blue, terminate greenAutomatic; no human in the loop
• CodeDeploy→CodePipelinedeployment failed (rolled back)
• CodePipeline→Release Approverrollback alert with alarm details

About this design

About this design

What this is

A reference CI/CD pipeline for a containerized service: GitHub -> build/test -> immutable artifacts (ECR + S3) -> staged deploys through dev, staging, and prod with a manual approval gate, canary traffic shifting, and alarm-driven automatic rollback. The AWS pieces are CodePipeline, CodeBuild, and CodeDeploy driving ECS blue/green — but every stage maps 1:1 onto GitHub Actions + your deploy tool of choice, so treat the shape as the spec, not the vendor list.

Start with the workflow view — it is the whole pipeline as a state machine with every decision gate. The sequence view zooms into the most interesting part: a prod deploy, shown twice (canary healthy vs. alarm-triggered rollback).

How it works

One execution per commit. A merge to main fires a webhook; the pipeline pins everything to that git SHA. The image is tagged :<sha> and never retagged — immutable artifacts are what make rollback trivial.

Fail fast, cheaply. Unit tests gate the build; smoke tests gate promotion out of dev; a full integration + load pass gates the approval request. Each gate that fails reports to Slack and GitHub and ends the execution.

Humans approve prod, machines verify it. The approval gate (up to 24h) carries release notes and staging results. After approval, CodeDeploy launches the green task set, shifts 10% of traffic, and bakes for 15 minutes while CloudWatch alarms (5xx rate, p99 latency, task health) watch the green side only.

Rollback is automatic and boring. Any alarm breach during the bake reroutes 100% of traffic back to the blue task set — no human in the loop, and the bad build never took full traffic. Manual rollback later is just redeploying a previous :<sha> tag.

How to extend

Different runtime: the pattern is runtime-agnostic — swap ECS blue/green for Lambda alias traffic shifting or a Kubernetes progressive-delivery controller; keep the gates and the bake.

More environments: clone the staging stage (deploy -> verify -> gate). Resist adding environments that don't have a distinct verification job — every stage should earn its latency.

Tighter canaries: add synthetic canary traffic (CloudWatch Synthetics) so low-traffic services still exercise the green task set during the bake.

DB migrations: add an expand/contract migration step before the canary — forward-compatible migrations are what keep instant rollback safe.

Compliance: the approval gate is the natural place to attach change tickets; pipe approvals to your change-management system instead of (or alongside) Slack.