Designing a Modern CI/CD Pipeline from Scratch

Goals for Our Pipeline

When redesigning our CI/CD pipeline, we set clear goals:

1. Fast feedback — PR checks complete in under 5 minutes
2. Safe deployments — Progressive rollouts with automatic rollback
3. Developer experience — One-click deploys, clear failure messages

The Pipeline Architecture

Our pipeline has four stages:

Stage 1: Validate (parallel)

• Lint, type-check, and format verification
• Unit tests with coverage gates
• Security scanning (dependencies + SAST)

Stage 2: Build

• Docker multi-stage builds
• Layer caching for fast rebuilds
• Image signing for supply chain security

Stage 3: Test

• Integration tests against ephemeral environments
• E2E tests with Playwright
• Performance regression tests

Stage 4: Deploy

• Canary deployment (5% → 25% → 100%)
• Automated health checks at each stage
• Automatic rollback on error rate spike

Speed Optimizations

We cut our pipeline from 25 minutes to 4:

• Parallelization — Run independent jobs concurrently
• Caching — npm cache, Docker layers, test fixtures
• Selective testing — Only run affected tests based on changed files
• Larger runners — Sometimes throwing hardware at the problem is the right call

Monitoring Deployments

Every deployment is tracked with:

• Deploy frequency and lead time
• Change failure rate
• Mean time to recovery (MTTR)

These DORA metrics help us continuously improve our delivery process.