[!NOTE] As your architecture grows from one service to dozens, you need infrastructure to handle routing, authentication, rate limiting, load balancing, and observability. This chapter covers the spectrum from simple proxies to full service meshes, and when to use each.
Forward Proxy vs Reverse Proxy
These are often confused but serve very different purposes:
Forward Proxy:
[Client] → [Forward Proxy] → [Internet] → [Server]
Client knows about the proxy. Server sees proxy''s IP.
Reverse Proxy:
[Client] → [Internet] → [Reverse Proxy] → [Backend Server]
Client doesn''t know about the proxy. Server is hidden.
| Aspect | Forward Proxy | Reverse Proxy |
|---|---|---|
| Sits in front of | Clients (protects clients) | Servers (protects servers) |
| Client awareness | Client must configure it | Client is unaware of it |
| Use cases | Privacy, caching, internet access control (corporate networks) | Load balancing, SSL termination, caching, security |
| Examples | Squid, corporate HTTP proxy | NGINX, HAProxy, Cloudflare |
API Gateway
An API Gateway is a reverse proxy with application-layer intelligence. It sits between clients and your microservices, handling cross-cutting concerns:
┌─→ User Service
Mobile App ──┐ │
Web App ───┤→ [API Gateway] ──┼─→ Order Service
3rd Party ───┘ │
├─→ Payment Service
│
└─→ Notification Service
Responsibilities:
- Request routing: Route
/users/*to User Service,/orders/*to Order Service. - Authentication/Authorization: Validate JWT tokens at the gateway; services don''t need to reimplement auth.
- Rate limiting: Apply per-client, per-route limits centrally.
- Request aggregation: Combine multiple backend calls into one client response.
- Protocol translation: Accept REST from clients, call gRPC backends.
- Caching: Cache common responses at the edge.
| Component | Layer | Intelligence | Use Case |
|---|---|---|---|
| Load Balancer | L4 (TCP) or L7 (HTTP) | Low (distributes traffic) | Distribute requests across backend instances |
| Reverse Proxy | L7 (HTTP) | Medium (routing, SSL, caching) | Single backend service needing SSL and caching |
| API Gateway | L7 (HTTP/gRPC) | High (auth, rate limit, transform) | Microservices requiring cross-cutting concerns |
Backend for Frontend (BFF) Pattern
Different clients (mobile, web, TV, IoT) often need different API shapes. Instead of one monolithic API gateway, create a dedicated backend per frontend type:
Mobile App ──→ [Mobile BFF] ──→ Backend Services
Web App ──→ [Web BFF] ──→ Backend Services
TV App ──→ [TV BFF] ──→ Backend Services
Each BFF aggregates, transforms, and optimizes responses for its specific client. Mobile BFF might return smaller payloads and fewer images; Web BFF might return richer data.
Real-world: Netflix uses BFFs for each platform (iOS, Android, TV, Web), each optimizing the API surface for its specific device constraints.
Service Mesh
In large microservices deployments (100+ services), managing service-to-service communication (mTLS, retries, circuit breaking, tracing) in every service becomes unsustainable. A service mesh extracts this logic into a dedicated infrastructure layer.
Without Service Mesh:
[Service A] ──direct HTTP──→ [Service B]
(A handles retries, circuit breaking, mTLS, tracing in its code)
With Service Mesh:
[Service A] ──→ [Sidecar Proxy A] ──mTLS──→ [Sidecar Proxy B] ──→ [Service B]
(All networking logic lives in the sidecar proxy, not in application code)
The service mesh has two planes:
- Data plane: Sidecar proxies (Envoy) deployed alongside every service instance, handling all network traffic.
- Control plane: Central component (Istio, Linkerd) that configures all the sidecar proxies with routing rules, security policies, etc.
Real-World Usage
- Kong: Open-source API gateway used by >50% of Fortune 500 companies. Plugin-based architecture for auth, rate limiting, logging.
- AWS API Gateway: Fully managed gateway with built-in Lambda integration, throttling, and API key management.
- Istio + Envoy: The dominant service mesh duo. Envoy is the sidecar proxy (created by Lyft); Istio is the control plane (created by Google).
- NGINX: Started as a reverse proxy/web server, now also functions as an API gateway and Ingress controller for Kubernetes.
Common Mistakes
- ❌ Making the API gateway a "god service" — keep business logic out of the gateway. It should handle only cross-cutting concerns.
- ❌ Adding a service mesh to a small system — service meshes add significant operational overhead. They are worth it at 50+ services, not at 5.
- ❌ Confusing forward and reverse proxies — forward proxies protect clients; reverse proxies protect servers. Know which you need.
- ❌ Single API gateway as SPOF — always deploy gateways in a cluster behind a load balancer with health checks.
API Gateway vs Service Mesh: When to Use Which
| Concern | API Gateway | Service Mesh |
|---|---|---|
| External traffic (client → service) | ✅ Primary use case | Not designed for this |
| Internal traffic (service → service) | Not ideal | ✅ Primary use case |
| Authentication | ✅ JWT, API keys | ✅ mTLS (mutual TLS) |
| Rate limiting | ✅ Per-client limits | ✅ Per-service limits |
| Observability | Basic metrics | ✅ Deep tracing per hop |
| Complexity | Low-Medium | High (sidecar per pod) |
Request Flow Through API Gateway
Client Request → DNS → Load Balancer → API Gateway Pipeline:
[TLS Termination] → Decrypt HTTPS
[Authentication] → Validate JWT / API key
[Rate Limiting] → Check per-client quotas
[Request Routing] → Route to correct microservice
[Load Balancing] → Pick healthy backend instance
[Request Transform] → Add headers, modify path
[Proxy] → Forward to backend
[Response Transform]→ Modify response, add CORS
[Logging/Metrics] → Record latency, status code
Total added latency: ~1-5ms (acceptable tradeoff)
[!TIP] Key Takeaways:
• Forward proxy: sits in front of clients (privacy, caching). Reverse proxy: sits in front of servers (LB, SSL, security).
• API Gateway: reverse proxy + auth + rate limiting + routing + aggregation. The entry point for microservices.
• BFF pattern: one backend per frontend type. Netflix does this for each device platform.
• Service mesh (Istio + Envoy): sidecar proxies for service-to-service traffic. Worth it only at scale (50+ services).
• Use API Gateway for north-south traffic (external), service mesh for east-west traffic (internal).