Microservices Communication | Patterns & Reliability
Implement microservices communication patterns: REST, gRPC, message queues, circuit breakers. GitScrum tracks service dependencies and failure modes.
10 min read
Microservices communication patterns enable reliable interaction between distributed services. GitScrum helps teams track service dependencies, communication failures, and inter-service coordination across complex microservices architectures.
Microservices communication fundamentals
Microservices architecture relies on effective communication patterns to maintain system reliability and performance. Poor communication design can lead to cascading failures, tight coupling, and maintenance nightmares.
Communication challenges in microservices
- Network reliability: Services communicate over networks prone to failures
- Latency: Network calls are slower than in-process communication
- Data consistency: Maintaining consistency across service boundaries
- Service discovery: Dynamic location of services in distributed environments
- Version compatibility: Managing API changes across service versions
Synchronous communication patterns
RESTful HTTP APIs
Request-response pattern:
Client Service ──► API Gateway ──► Service A ──► Service B
│ │ │ │
▼ ▼ ▼ ▼
HTTP Request Route & Auth Business Logic Data Access
(JSON/XML) & Rate Limiting & Response
Benefits:
- Simple and familiar HTTP semantics
- Human-readable payloads (JSON/XML)
- Built-in caching and browser support
- Easy testing and debugging
- Synchronous blocking calls
- Potential for cascading failures
- Tight coupling between services
- Network latency accumulation
gRPC communication
Protocol buffer-based RPC:
Client ──► Load Balancer ──► Service Instance
│ │ │
▼ ▼ ▼
Proto Service Discovery Business Logic
Request & Routing & Proto Response
Benefits:
- High performance with binary serialization
- Strongly typed contracts with proto files
- Built-in streaming support
- Language agnostic with code generation
- Less human-readable than JSON
- Requires proto file management
- Browser support limitations
- Steeper learning curve
Asynchronous communication patterns
Message queues and event-driven architecture
Publisher-subscriber pattern:
Publisher Service ──► Message Broker ──► Consumer Service 1
│ │ │
▼ ▼ ▼
Publish Event Queue/Topic Process Event
to Topic/Queue Management & Update State
Consumer Service 2
│
▼
Process Event
& Send Response
Benefits:
- Loose coupling between services
- Improved fault tolerance and scalability
- Eventual consistency model
- Natural fit for domain events
- Eventual consistency complexity
- Debugging distributed transactions
- Message ordering challenges
- Duplicate message handling
Event sourcing and CQRS
Command Query Responsibility Segregation:
Command Side ──► Event Store ──► Read Models
│ │ │
▼ ▼ ▼
Validate & Store Events Query Data
Generate as Sequence from Projections
Events
Benefits:
- Audit trail of all changes
- Temporal queries and time travel
- Scalable read and write models
- Event-driven architecture foundation
- Complex event schema evolution
- Learning curve for developers
- Additional infrastructure requirements
- Eventual consistency challenges
Service mesh patterns
Service mesh architecture
Sidecar proxy pattern:
Service A ──► Sidecar Proxy ──► Service Mesh ──► Sidecar Proxy ──► Service B
│ │ │ │ │
│ ▼ ▼ ▼ │
└────────► Application Code ──► Local Proxy ──► Control Plane ──► Application Code
Benefits:
- Transparent service communication
- Built-in observability and tracing
- Traffic management and security
- Language agnostic implementation
- Additional resource overhead
- Complex deployment and management
- Learning curve for operations teams
- Potential performance impact
Communication reliability patterns
Circuit breaker pattern
Failure handling mechanism:
Closed State ──► Open State ──► Half-Open State ──► Closed State
│ │ │ │
▼ ▼ ▼ ▼
Normal Operation Fast Fail Test Request Normal Operation
(Requests Flow) (Requests Fail) (Single Request) (Requests Flow)
Implementation:
- Monitor failure rates and response times
- Open circuit when failure threshold exceeded
- Allow limited requests in half-open state for testing
- Close circuit when service recovers
Retry and timeout patterns
Exponential backoff retry:
Request ──► Failure ──► Wait 1s ──► Retry ──► Failure ──► Wait 2s ──► Retry
│ │ │ │ │ │
▼ ▼ ▼ ▼ ▼ ▼
Initial Count Failure Backoff Attempt 2 Count Failure Backoff
Call & Track Calculation & Track Calculation
Timeout strategies:
- Connection timeouts for establishing connections
- Read timeouts for receiving responses
- Write timeouts for sending requests
- Total request timeouts as safety nets
Bulkhead pattern
Failure isolation:
Service A ──► Thread Pool 1 ──► External Service 1
Service B ──► Thread Pool 2 ──► External Service 2
Service C ──► Thread Pool 3 ──► External Service 3
Benefits:
- Prevent cascading failures
- Resource isolation between services
- Independent scaling and failure handling
- Improved overall system resilience
API gateway patterns
API gateway responsibilities
Request routing and composition:
Client ──► API Gateway ──► Service Registry ──► Service A, B, C
│ │ │ │
▼ ▼ ▼ ▼
Single Entry Authentication Load Balancing Business Logic
Point & Authorization & Circuit Aggregation
Breaking
Cross-cutting concerns:
- Authentication and authorization
- Rate limiting and throttling
- Request/response transformation
- Caching and optimization
- Monitoring and logging
Gateway routing patterns
Path-based routing:
/api/v1/users ──► User Service
/api/v1/orders ──► Order Service
/api/v1/products ──► Product Service
Header-based routing:
Accept-Version: v1 ──► V1 Services
Accept-Version: v2 ──► V2 Services
Service discovery patterns
Client-side discovery
Service registry interaction:
Client Service ──► Service Registry ──► Get Service Instances ──► Load Balance ──► Target Service
│ │ │ │ │
▼ ▼ ▼ ▼ ▼
Business Logic Query API Instance List Select Instance Make Request
Needs Service for Service with Endpoints via Algorithm to Endpoint
Instances
Benefits:
- Client control over load balancing
- No single point of failure
- Flexible routing algorithms
- Direct service communication
Server-side discovery
Load balancer integration:
Client ──► Load Balancer ──► Service Registry ──► Route to Instance ──► Target Service
│ │ │ │ │
▼ ▼ ▼ ▼ ▼
External Service Discovery Instance Health Request Routing Business Logic
Request & Registration Monitoring & Load Balance
Benefits:
- Simplified client implementation
- Centralized routing logic
- Better security controls
- Easier monitoring and debugging
Data consistency patterns
Saga pattern for distributed transactions
Choreography-based saga:
Service A ──► Event: Order Created ──► Service B ──► Event: Payment Processed ──► Service C
│ │ │
▼ ▼ ▼
Complete Process Payment & Update Inventory & Ship Order & Send
Order Send Success Event Send Success Event Confirmation Email
Orchestration-based saga:
Orchestrator ──► Service A: Create Order ──► Service B: Process Payment ──► Service C: Update Inventory
│ │ │ │
│ ▼ ▼ ▼
│ Order Created OK Payment Processed OK Inventory Updated OK
│ │ │ │
└─────────────► Service D: Ship Order ──► Complete Saga ──► Send Confirmation
Eventual consistency strategies
Conflict resolution:
- Last-write-wins (simple but lossy)
- Version vectors for conflict detection
- Business rule-based merging
- Manual conflict resolution workflows
- Define bounded contexts in domain-driven design
- Accept eventual consistency within boundaries
- Use strong consistency across boundaries when required
- Implement compensating actions for failures
Monitoring and observability
Distributed tracing
Request flow tracking:
Client ──► Service A (span 1) ──► Service B (span 2) ──► Service C (span 3)
│ │ │ │
▼ ▼ ▼ ▼
Trace ID: Start Span 1 Start Span 2 Start Span 3
12345 & Add Tags & Add Tags & Add Tags
Trace context propagation:
- Pass trace ID through all service calls
- Include span IDs for nested operations
- Add custom tags for business context
- Correlate logs with traces
Metrics and alerting
Key metrics to monitor:
- Request/response rates and latencies
- Error rates and types
- Circuit breaker states
- Queue depths and processing rates
- Resource utilization per service
- Service health checks and heartbeats
- Performance degradation alerts
- Error rate threshold alerts
- Dependency failure notifications
- Capacity planning alerts
Security in microservices communication
Authentication and authorization
JWT token propagation:
Client ──► Auth Service ──► JWT Token ──► Service A ──► Validate Token ──► Service B
│ │ │ │ │ │
▼ ▼ ▼ ▼ ▼ ▼
Login & Generate Token Include in Extract & Extract & Extract &
Get Token with Claims Request Header Validate Validate Validate
Token Token Token
Service-to-service authentication:
- Mutual TLS (mTLS) for service identity
- API keys for external service communication
- OAuth 2.0 client credentials flow
- Service mesh certificate management
Data protection
Encryption in transit:
- TLS 1.3 for all service communication
- Certificate rotation and management
- Perfect forward secrecy
- Secure service mesh communication
- Encrypt sensitive data in databases
- Secure secret management
- Key rotation policies
- Access control and auditing
Testing microservices communication
Contract testing
Consumer-driven contracts:
Consumer ──► Define Contract ──► Provider ──► Verify Contract ──► Generate Tests
│ │ │ │ │
▼ ▼ ▼ ▼ ▼
Write Tests Expected API Implement Run Tests Automated
for API Behavior & API to Match Against Regression
Expectations Response Format Contract Contract Tests
Integration testing strategies
Service virtualization:
- Mock external service dependencies
- Simulate network conditions and failures
- Test service interactions in isolation
- Enable parallel test execution
- Test complete user journeys
- Validate cross-service workflows
- Monitor performance under load
- Verify data consistency across services
GitScrum integration for microservices
Service dependency tracking
Dependency mapping:
- Track service-to-service relationships
- Monitor API contract changes
- Identify coupling and circular dependencies
- Plan service decomposition and refactoring
- Assess blast radius of service changes
- Identify affected downstream services
- Plan deployment sequences
- Coordinate cross-team releases
Communication failure management
Incident tracking:
- Log communication failures and timeouts
- Track circuit breaker state changes
- Monitor message queue backlogs
- Alert on service degradation
- Analyze failure patterns and root causes
- Document lessons learned
- Implement preventive measures
- Update communication patterns
Best practices for microservices communication
Design principles
API design:
- Design APIs around business capabilities
- Use RESTful conventions for HTTP APIs
- Version APIs explicitly and plan deprecation
- Document APIs with OpenAPI/Swagger
- Prefer asynchronous communication when possible
- Implement proper error handling and retries
- Use correlation IDs for request tracing
- Design for failure and implement graceful degradation
Operational practices
Deployment strategies:
- Use blue-green or canary deployments
- Implement proper service discovery
- Plan for zero-downtime deployments
- Automate rollback procedures
- Implement comprehensive observability
- Set up proper alerting and on-call rotation
- Regularly review and optimize communication patterns
- Plan for service scaling and capacity management
Common anti-patterns to avoid
Tight coupling pitfalls
- Synchronous hell: Avoid long chains of synchronous calls
- Shared databases: Don't share databases between services
- Direct service calls: Use service discovery instead of hardcoded URLs
- Large APIs: Keep service interfaces focused and minimal
Communication anti-patterns
- Chatty communication: Minimize round trips between services
- Fire-and-forget without tracking: Always track asynchronous messages
- Ignoring network failures: Implement proper retry and circuit breaker patterns
- Inconsistent error handling: Standardize error responses across services