Load Balancers (LB)
Data centers handle millions of requests per second using thousands of servers. Load balancers distribute requests efficiently, preventing overload and ensuring stable performance.
- Global Server Load Balancing (GSLB): Distributes traffic across multiple geographical regions for redundancy, performance, and disaster recovery.
- Local Load Balancing: Manages traffic within a single data center to optimize resource utilization and efficiency.
DNS load balancing is an implementation of Global Server Load Balancing mechanism.
DNS LB has following limitations: small packet size limit IP selection, lack of real-time server health awareness, slow failover due to DNS caching (TTL), not being able to optimize routing based on user location.
Typical Load Balancer Responsibilities
| Responsibility | Explanation |
|---|---|
| Traffic Distribution | Distributes incoming requests across multiple servers to prevent overloading and optimize resource use. |
| Health Monitoring & High Availability | Ensures continuous service by redirecting traffic to healthy servers if one fails. Uses heartbeat and health checks. |
| SSL Termination | Offloads SSL/TLS encryption to reduce server processing overhead and improve performance. |
| Session Stickiness | Keeps user sessions directed to the same server to maintain stateful interactions. |
| Security & DDoS Protection | Blocks malicious traffic, enforces security policies, and mitigates Distributed Denial of Service (DDoS) attacks. |
| Content-Based Routing | Directs traffic based on request content, such as URL path, headers, or query parameters. |
| Scalability Management | Adjusts server allocation dynamically based on traffic demand to ensure efficiency. |
Distribution Algorithms
| Algorithm Name | Explanation |
|---|---|
| Round-robin | Requests are forwarded sequentially to each server in a repeating cycle. |
| Weighted round-robin | Assigns weights to servers based on their capacity, forwarding more requests to higher-weight servers. |
| Least connections | Directs new requests to servers with the fewest active connections to balance load dynamically. |
| Least response time | Chooses the server with the shortest response time to optimize performance. |
| IP hash | Uses a client’s IP address to consistently assign requests to the same server for specific service levels. |
| URL hash | Assigns requests to specific servers based on the requested URL, ensuring service consistency. |
There is no silver bullet while choosing distribution algorithm. Best sggestion is to "try & verify".
Distribution algorithm can be:
- Static - rely on fixed knowledge about server configurations (for ex. IP Hash, etc.). They are simpler and implemented on a single machine or router.
- Dynamic - monitor server states and adjust task assignments accordingly. They involve communication overhead and complexity, but offer improved forwarding decisions and better health monitoring of servers.
In addition, Load Balancers can be stateful (LB maintaining a state of the sessions established between clients and hosting servers) or stateless (rely consistent hashing to forward request to particular server)
Load Balancers in the OSI Model
Layer 3 Balancers
Balancing occurs on IP addresses (Layer 3 in the OSI model). They use techniques like NAT and IP routing to forward packets without inspecting transport or application data.
Layer 4 Load Balancers
Balance traffic based on transport protocols (TCP/UDP), maintain client sessions, and route traffic to the same backend server. Some support TLS termination.
Pros: Fast
Layer 7 Load Balancers
Make application-aware decisions using data like HTTP headers and cookies, support TLS termination, rate limiting, HTTP routing, and header rewriting.
Pros: Smart traffic inspection