Test Automation Strategy | Coverage and CI Integration
Build test automation with the testing pyramid: 70% unit, 20% integration, 10% E2E. GitScrum integrates test results with quality visibility.
8 min read
Automated tests enable confidence and speed. GitScrum integrates with test results to show quality status alongside development progress.
Testing Strategy
Testing Pyramid
THE TESTING PYRAMID:
┌─────────────────────────────────────────────────────────────┐
│ │
│ /\ │
│ / \ │
│ / E2E\ ← Few (slow, expensive) │
│ / \ Manual + Automated │
│ /────────\ │
│ / \ │
│ / Integration\ ← Some (medium speed) │
│ / \ API, component tests │
│ /────────────────\ │
│ / \ │
│ / Unit \ ← Many (fast, cheap) │
│ / \ Function-level │
│ /────────────────────────\ │
│ │
│ ─────────────────────────────────────────────────────────── │
│ │
│ RECOMMENDED RATIO: │
│ • 70% Unit tests │
│ • 20% Integration tests │
│ • 10% E2E tests │
│ │
│ WHY THIS RATIO: │
│ ─────────────── │
│ UNIT TESTS: │
│ • Fast (milliseconds each) │
│ • Easy to maintain │
│ • Pinpoint failures exactly │
│ • Run on every commit │
│ │
│ INTEGRATION TESTS: │
│ • Test component interaction │
│ • Catch integration bugs │
│ • Slower than unit tests │
│ │
│ E2E TESTS: │
│ • Full user journey │
│ • Catch real-world issues │
│ • Slow and flaky │
│ • Expensive to maintain │
└─────────────────────────────────────────────────────────────┘
What to Automate
AUTOMATION DECISION:
┌─────────────────────────────────────────────────────────────┐
│ │
│ AUTOMATE IF: │
│ ──────────── │
│ ☑ Runs frequently (every commit, every deploy) │
│ ☑ Critical path (login, checkout, core features) │
│ ☑ Repetitive (same test, many data variations) │
│ ☑ Regression risk (previously broken, might break again) │
│ ☑ Time-consuming manually │
│ │
│ KEEP MANUAL IF: │
│ ─────────────── │
│ ☑ Exploratory (discovering unknown issues) │
│ ☑ Usability/UX (needs human judgment) │
│ ☑ Rare scenarios (one-time verification) │
│ ☑ Visual assessment (design quality) │
│ ☑ Complex setup with low ROI │
│ │
│ ─────────────────────────────────────────────────────────── │
│ │
│ AUTOMATION ROADMAP: │
│ ┌─────────────────────────────────────────────────────────┐│
│ │ PHASE 1: FOUNDATION ││
│ │ • Unit test framework setup ││
│ │ • CI pipeline running tests ││
│ │ • Critical path smoke tests ││
│ │ ││
│ │ PHASE 2: CORE COVERAGE ││
│ │ • Unit tests for business logic ││
│ │ • API integration tests ││
│ │ • Happy path E2E tests ││
│ │ ││
│ │ PHASE 3: COMPREHENSIVE ││
│ │ • Edge case coverage ││
│ │ • Performance tests ││
│ │ • Security tests ││
│ │ ││
│ │ PHASE 4: OPTIMIZATION ││
│ │ • Parallel test execution ││
│ │ • Test data management ││
│ │ • Flaky test elimination ││
│ └─────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────┘
Test Types
By Layer
TEST TYPES BY LAYER:
┌─────────────────────────────────────────────────────────────┐
│ │
│ UNIT TESTS: │
│ ─────────── │
│ Test individual functions/methods │
│ No external dependencies (mock them) │
│ Fast: thousands per minute │
│ │
│ EXAMPLE: │
│ ┌─────────────────────────────────────────────────────────┐│
│ │ test('calculates total with tax', () => { ││
│ │ const cart = { items: [{ price: 100 }] }; ││
│ │ expect(calculateTotal(cart, 0.1)).toBe(110); ││
│ │ }); ││
│ └─────────────────────────────────────────────────────────┘│
│ │
│ INTEGRATION TESTS: │
│ ────────────────── │
│ Test component interactions │
│ Real database, APIs (or realistic mocks) │
│ Medium speed: hundreds per minute │
│ │
│ EXAMPLE: │
│ ┌─────────────────────────────────────────────────────────┐│
│ │ test('creates order in database', async () => { ││
│ │ const order = await orderService.create(cartData); ││
│ │ const saved = await db.orders.findById(order.id); ││
│ │ expect(saved.status).toBe('pending'); ││
│ │ }); ││
│ └─────────────────────────────────────────────────────────┘│
│ │
│ END-TO-END TESTS: │
│ ───────────────── │
│ Full user journey through real UI │
│ Real browser, real backend │
│ Slow: tens per minute │
│ │
│ EXAMPLE: │
│ ┌─────────────────────────────────────────────────────────┐│
│ │ test('user can complete checkout', async () => { ││
│ │ await page.goto('/products'); ││
│ │ await page.click('[data-test="add-to-cart"]'); ││
│ │ await page.click('[data-test="checkout"]'); ││
│ │ await page.fill('#card-number', '4242...'); ││
│ │ await page.click('[data-test="pay"]'); ││
│ │ await expect(page).toHaveURL('/confirmation'); ││
│ │ }); ││
│ └─────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────┘
Test in CI/CD
Pipeline Integration
TESTS IN CI/CD PIPELINE:
┌─────────────────────────────────────────────────────────────┐
│ │
│ PIPELINE STAGES: │
│ │
│ COMMIT → UNIT → INTEGRATION → E2E → DEPLOY │
│ │ │ │ │ │ │
│ │ │ │ │ ▼ │
│ │ │ │ │ Production │
│ │ │ │ │ │
│ │ │ │ └─ 10 min (nightly/staging)│
│ │ │ │ │
│ │ │ └─ 5 min (every PR merge) │
│ │ │ │
│ │ └─ 2 min (every commit) │
│ │ │
│ └─ Triggered │
│ │
│ ─────────────────────────────────────────────────────────── │
│ │
│ FAST FEEDBACK LOOP: │
│ ┌─────────────────────────────────────────────────────────┐│
│ │ UNIT TESTS: Every push ││
│ │ • Run in < 5 minutes ││
│ │ • Block merge if failing ││
│ │ • Immediate developer feedback ││
│ │ ││
│ │ INTEGRATION: Every PR ││
│ │ • Run before merge ││
│ │ • API and service tests ││
│ │ • Block if critical failures ││
│ │ ││
│ │ E2E: Pre-deploy ││
│ │ • Run on staging ││
│ │ • Critical user journeys ││
│ │ • Gate for production deploy ││
│ └─────────────────────────────────────────────────────────┘│
│ │
│ QUALITY GATES: │
│ • Unit: 80%+ coverage, 0 failures │
│ • Integration: 0 critical failures │
│ • E2E: Happy path 100% pass │
└─────────────────────────────────────────────────────────────┘
Test Maintenance
Keeping Tests Healthy
TEST MAINTENANCE:
┌─────────────────────────────────────────────────────────────┐
│ │
│ FLAKY TEST MANAGEMENT: │
│ ────────────────────── │
│ Flaky test = sometimes passes, sometimes fails │
│ │
│ DANGER: Team ignores failures, misses real bugs │
│ │
│ RESPONSE: │
│ 1. Quarantine flaky test (don't delete) │
│ 2. Create ticket to fix │
│ 3. Fix root cause (timing, data, environment) │
│ 4. Return to suite when stable │
│ │
│ ─────────────────────────────────────────────────────────── │
│ │
│ TEST CODE QUALITY: │
│ ────────────────── │
│ • Tests are production code │
│ • Refactor for readability │
│ • DRY: Shared setup, helper functions │
│ • Clear naming: describes what's tested │
│ │
│ ─────────────────────────────────────────────────────────── │
│ │
│ REGULAR REVIEW: │
│ ┌─────────────────────────────────────────────────────────┐│
│ │ QUARTERLY TEST REVIEW ││
│ │ ││
│ │ METRICS: ││
│ │ • Test count: 2,450 (↑ from 2,100) ││
│ │ • Coverage: 78% (target: 80%) ││
│ │ • Suite runtime: 8 min (target: < 10) ││
│ │ • Flaky tests: 3 (target: 0) ││
│ │ ││
│ │ ACTIONS: ││
│ │ ☐ Fix 3 flaky tests ││
│ │ ☐ Add tests for auth module (gap identified) ││
│ │ ☐ Delete 15 obsolete tests ││
│ │ ☐ Parallelize E2E to reduce runtime ││
│ └─────────────────────────────────────────────────────────┘│
│ │
│ DELETE OBSOLETE TESTS: │
│ • Feature removed? Delete tests. │
│ • Covered by other tests? Delete. │
│ • Never catches bugs? Evaluate value. │
└─────────────────────────────────────────────────────────────┘
Coverage Strategy
Meaningful Coverage
TEST COVERAGE APPROACH:
┌─────────────────────────────────────────────────────────────┐
│ │
│ COVERAGE TARGETS: │
│ ───────────────── │
│ │
│ DON'T: Chase 100% coverage everywhere │
│ DO: Target high coverage where it matters │
│ │
│ HIGH COVERAGE (80%+): │
│ • Business logic │
│ • Payment processing │
│ • Security-critical code │
│ • Complex algorithms │
│ │
│ MEDIUM COVERAGE (60%+): │
│ • API endpoints │
│ • Data transformations │
│ • Validation logic │
│ │
│ LOWER COVERAGE OK: │
│ • Simple CRUD operations │
│ • Configuration code │
│ • UI scaffolding │
│ │
│ ─────────────────────────────────────────────────────────── │
│ │
│ COVERAGE REPORT: │
│ ┌─────────────────────────────────────────────────────────┐│
│ │ MODULE LINES BRANCHES FUNCTIONS ││
│ │ ────── ───── ──────── ───────── ││
│ │ auth/ 92% 88% 95% ││
│ │ payments/ 89% 85% 91% ││
│ │ api/handlers/ 76% 71% 80% ││
│ │ utils/ 68% 62% 75% ││
│ │ ui/components/ 45% 40% 52% ││
│ │ ──────────────────────────────────────────────────────││
│ │ OVERALL 72% 68% 78% ││
│ │ ││
│ │ TREND: ↗ Up 3% from last month ││
│ │ GAP: auth/ needs branch coverage improvement ││
│ └─────────────────────────────────────────────────────────┘│
│ │
│ Coverage shows what's tested, not test quality │
│ High coverage ≠ good tests │
│ Focus on testing behavior, not lines │
└─────────────────────────────────────────────────────────────┘