In this tutorial, you will configure Nginx as a load balancer that distributes incoming HTTP requests across two or more backend application servers, choose between round robin, least_conn, and ip_hash balancing, tune how Nginx handles a backend that stops responding, and verify — rather than assume — that traffic is actually being distributed.
Prerequisites
An Ubuntu server with a sudo-enabled user
Nginx installed on that server (apt install -y nginx if not already present)
At least two backend application servers already running the same app and reachable from the Nginx host — this tutorial uses 192.0.2.10:3000 and 192.0.2.11:3000 as examples
A domain name pointed at the Nginx server's public IP (example.com is used throughout)
Basic familiarity with editing Nginx configuration files and using systemctl
Let an AI agent do this for you
Copy a ready-made prompt for an AI coding assistant with terminal access to your server (Claude Code, Cursor, or similar) — it can carry out the steps below for you. Review what it plans to run before it executes anything.
A single application server is a single point of failure and a hard ceiling on throughput. Put Nginx in front of two or more servers running the same application and it can spread incoming requests across them, so a traffic spike gets absorbed by the pool and one backend going down doesn't take the whole app offline. This covers configuring Nginx as an HTTP load balancer: grouping backend servers into a pool, choosing how requests are distributed across it, tuning how Nginx reacts when a backend stops responding, and — the step that's easy to skip — actually confirming the distribution is happening rather than trusting the defaults. TLS termination is a separate concern and isn't covered here; everything below is plain HTTP on port 80.
Step 1 — Confirm Nginx Is Installed and Backends Are Reachable
Check whether Nginx is already installed, and install it if not:
Before editing any load-balancing config, confirm from the Nginx host itself that both backends accept connections on the port your app listens on. This matters because a typo'd IP or an unreachable backend won't cause a config error later — Nginx will simply stop sending it traffic without telling you, which is the exact failure mode covered in Troubleshooting below.
Both should return an HTTP response header — even a 404 or 500 from the app confirms the TCP connection and HTTP layer work. If either hangs or refuses the connection, fix that first: confirm the app process is running and bound to that host and port, not just 127.0.0.1.
“If the backends run ufw, don't leave the application port open to the whole internet — restrict it to the Nginx host's IP rather than allowing it from anywhere. On each backend: sudo ufw allow from <nginx-server-ip> to any port 3000 proto tcp, substituting your Nginx server's actual IP address for <nginx-server-ip>. Only Nginx needs to reach the app port directly.”
Step 2 — Define the Upstream Server Pool
An upstream block groups backend servers under one name that you reference elsewhere in the config. It's valid either inside a site file under /etc/nginx/sites-available/ or in its own file under /etc/nginx/conf.d/ — both get pulled into Nginx's http context by the stock nginx.conf. This tutorial keeps it in the same site file, since it's only used by one server block.
bash
sudo nano /etc/nginx/sites-available/example.com
Tricknowtech VPS Hosting
Dedicated KVM resources and full root access — deployed in under 60 seconds, no ticket required.
upstream backend_pool {
server 192.0.2.10:3000;
server 192.0.2.11:3000;
}
backend_pool is an arbitrary name — use it consistently wherever you reference the pool. Each server line takes a host:port pair (a Unix socket or a hostname is also valid).
Step 3 — Proxy Requests to the Pool
Add a server block in the same file, below the upstream block, pointing proxy_pass at backend_pool instead of a single backend:
proxy_pass targets the pool, not a literal host — Nginx picks which member serves each request according to its balancing algorithm. The three proxy_set_header lines matter because by default a backend sees every request as coming from Nginx itself: Host restores the original hostname so the app can build correct URLs or route by vhost, and X-Real-IP / X-Forwarded-For pass along the actual client IP, which the backend would otherwise lose entirely for logging or rate limiting.
Enable the site (skip if it's already symlinked), then move on:
With no directive specified, Nginx uses round robin: requests go to each server in the pool in turn, splitting traffic roughly evenly over time. That's a sensible default for stateless backends of similar capacity. Two common alternatives, set as the first line inside the upstream block:
least_conn; — send each new request to whichever backend currently has the fewest active connections. Useful when request handling time varies noticeably, since round robin's even split of requests doesn't guarantee an even split of load.
ip_hash; — always route a given client IP to the same backend. Useful for apps that keep session state in memory on one server instead of shared storage (Redis, a database) — without it, a client can land on a different backend each request and appear logged out.
nginx
upstream backend_pool {
least_conn;
server 192.0.2.10:3000;
server 192.0.2.11:3000;
}
Only one algorithm directive is active at a time; omit it entirely to use round robin.
Step 5 — Configure Failure Handling
Nginx tracks failed connection attempts per backend and can pull one out of rotation automatically. This is controlled by max_fails and fail_timeout, set as parameters on a server line. By default, a backend is marked unavailable after a single failed attempt (max_fails=1) and stays out of rotation for 10 seconds (fail_timeout=10s) before Nginx tries it again. If occasional slow responses shouldn't take a backend out after one hiccup, raise both explicitly:
nginx
upstream backend_pool {
server 192.0.2.10:3000 max_fails=3 fail_timeout=30s;
server 192.0.2.11:3000 max_fails=3 fail_timeout=30s;
}
With this, a backend has to fail three consecutive attempts before Nginx stops routing to it, and it's retried after 30 seconds. This is what keeps a truly degraded backend from being hammered forever, but it's also the same mechanism that silently hides a misconfigured or unreachable backend — covered in Troubleshooting below.
Step 6 — Test and Apply the Configuration
Always validate syntax before reloading — nginx -t catches typos and invalid directives without touching the running config:
bash
sudo nginx -t
sudo systemctl reload nginx
reload, not restart — it applies the new config without dropping connections that are already in flight.
Step 7 — Verify Requests Are Actually Being Distributed
nginx -t passing and the reload succeeding only confirm the config is syntactically valid — neither tells you whether both backends are actually receiving traffic. Confirming that requires being able to tell, per request, which backend answered, and two identical instances of the same app return indistinguishable responses by default.
“Before testing, give each backend a way to identify itself in its response — a temporary distinct header (for example X-Backend-Id: 1 returned by 192.0.2.10 and X-Backend-Id: 2 by 192.0.2.11) or distinguishing text on the page. Without that, round robin can be working correctly and still look, from the client side, like nothing is being distributed.”
Then send several requests to the domain and watch which backend answers each one:
bash
for i in {1..6}; do curl -s -I http://example.com/; done
With round robin (the default), the identifying marker should alternate roughly evenly between the two backends across those six requests. With least_conn in effect, expect responses to cluster on whichever backend currently has fewer active connections rather than strictly alternating every other request.
Troubleshooting: All Traffic Goes to One Backend
If every request in Step 7 comes back identifying the same backend, despite round robin being in effect and no algorithm directive favoring one server, the most common cause is that only one backend is actually reachable from the Nginx host. Nginx does not error loudly when a backend can't be reached — it applies max_fails/fail_timeout, quietly drops it from rotation, and keeps serving everything from whichever backend still responds. Both nginx -t passing and reload succeeding tell you nothing about backend reachability, only config syntax.
Confirm directly, from the Nginx host, the same way as Step 1:
If one of these hangs or is refused, check on that backend: that the app process is running and bound to the server's actual network interface rather than only 127.0.0.1 (which refuses connections from other hosts, including the Nginx host), and that its firewall allows inbound traffic from the Nginx host's IP on that port. Also re-check the IP:port pair in the upstream block against the backend's real address — a transposed digit in a 192.0.2.x address is a common cause and produces no error until you go looking for one.
