Load Balancer, step by step.

1. Problem

Imagine you run an online store with one application server. Every customer’s request goes to that server: browsing products, searching, or placing an order. The application server reads and writes data in a database.

Traffic grows during a sale

With a small number of customers, the store works well. During a sale, thousands arrive together. The server has limited CPU, memory, and connections. If requests arrive faster than it can process them, work starts to pile up.

• Requests wait longer.
• Pages become slow.
• Some requests time out before customers receive a response.

One server also means one point of failure

If that application server stops working, customers cannot use the store—even if the database is still running. All requests depend on the same server.

What if we add another server?

You add a second application server running the same store application. Both servers can now handle customer requests, but we need a way to distribute incoming requests between them.

                    Application Server 1
Customers → ?
                    Application Server 2

2. Solving the distribution problem

Adding another server gives our store more capacity. We also need a way to decide which server handles each incoming request. This is where a load balancer helps.

We place the load balancer between the customers and the application servers. Customers connect to one website address. The load balancer receives their requests and forwards each one to a selected server.

Customers → Load Balancer ──→ Server 1
                         └─→ Server 2

How does it choose a server?

The load balancer follows a selection rule, called a load-balancing algorithm. A simple rule is round robin: take turns sending requests to each available server.

For this example, each request goes to one server. The server processes it and returns a response. Both servers share the incoming work, and customers do not need to choose a server themselves.

Round robin balances the number of requests, but requests can take different amounts of work. A product search may be quick while generating a report takes longer. Taking turns therefore does not guarantee that both servers have the same CPU usage or response time.

3. Two common types: Layer 4 and Layer 7

Load balancers can make their decision at different levels of the network. Layer 4 works with the network connection. Layer 7 understands the application request, such as an HTTP web request.

How a Layer 4 load balancer works

A Layer 4 load balancer receives a network connection and chooses a backend using connection information such as the destination port and the client or server IP addresses. It forwards the connection’s traffic to the selected server. It does not need to read the HTTP URL or ordinary HTTP headers to make that choice.

Client connection: IP address + port
             ↓
Layer 4 load balancer
             ↓ chooses a healthy server
Web Server 1 or Web Server 2

For example, if both servers run the same website, a Layer 4 load balancer can take turns assigning new connections to them without checking which page the customer requested.

How a Layer 7 load balancer works

A Layer 7 load balancer understands the application protocol. For a website, it can read the HTTP request’s Host (which site was requested), path (which URL was requested), and configured headers. It checks routing rules, chooses the matching server group, then uses a balancing rule such as round robin to pick a healthy server in that group.

Request: https://shop.example.com/images/cat.png
                         ↓
Layer 7 load balancer reads Host and path
                         ↓
/images/ → image servers   /orders/ → order servers

So, a Layer 7 load balancer can direct different parts of one application to different services. With HTTPS, it can inspect HTTP details when it terminates TLS (decrypts the connection); if it only passes encrypted traffic through, it cannot read the encrypted HTTP path or headers.

4. What if the load balancer itself needs backup?

A single load balancer can become a point of failure too. Real deployments can run multiple load-balancer instances or nodes. Clients still use the website’s domain name; a highly available front door gets their traffic to a working load-balancer node.

One common approach is DNS: when the browser looks up shop.example.com, DNS returns an address for the load-balancer service. Depending on the setup, that address may be a shared virtual IP handled by multiple nodes, or DNS may return one or more load-balancer addresses. A cloud service may manage this behind a single DNS name. Health monitoring and failover are configured by the platform or network design; DNS does not automatically detect every failure by itself.

Customer looks up shop.example.com
                 ↓
DNS / highly available front door
                 ↓ directs traffic to a working node
Load Balancer A  or  Load Balancer B
                 ↓ that load balancer selects a backend
         Application Server 1 or 2

This is two separate decisions: the front door gets the client to a working load balancer; that load balancer then chooses a healthy application server. DNS usually chooses at the level of an address or endpoint, not separately for every web request path. DNS answers can also be cached, so a change may not reach every client immediately.