Skip to navigation

The routing algorithm is a rule that decides which backend gets the next request (Layer 7) or the next connection (Layer 4). Every algorithm only chooses among backends that pass their health check, and every algorithm respects each backend’s weight.

When to use each algorithm

AlgorithmAPI valueHow it choosesUse it when
Round robin (default)round_robinTakes the backends in turnYour backends are the same size and requests take about the same time. Start here.
Least requestleast_requestPicks the backend with the fewest requests in progressSome requests are slow (reports, uploads) and you don’t want them to pile up on one backend
RandomrandomPicks a backend at randomYou have many backends and don’t need an even turn order
Consistent hashconsistent_hashHashes each request so the same request key keeps going to the same backendYou run caches on the backends and want repeat requests to land on the same one

If you’re not sure, keep Round robin.

Sticky sessions (the Sticky header name field, default X-User-ID) are saved with the load balancer but are not applied to traffic yet, so the API’s sticky_header doesn’t choose the backend either. Don’t rely on a client staying on one backend. Keep session data somewhere every backend can read, such as a database or a shared cache.

How it works

  1. The load balancer keeps a list of backends that currently pass their health check.
  2. It applies the weights: a backend with weight 200 is chosen twice as often as one with weight 100.
  3. The algorithm picks one backend from that weighted list for the request or connection.
  4. If Automatic retries are on and the attempt fails with a 5xx response, a reset or a connection failure, the load balancer tries again, up to the number of Attempts you set.

Weights

Every backend has a Weight from 1 to 1000 (default 100). A backend’s share of traffic is its weight divided by the total of all healthy backends’ weights.

BackendWeightShare of traffic
203.0.113.10 (8 vCPUs)200200 of 400, half
203.0.113.11 (4 vCPUs)100100 of 400, a quarter
203.0.113.12 (4 vCPUs)100100 of 400, a quarter

Common uses:

  • Bigger servers get more traffic. Give a backend with twice the CPU twice the weight.
  • Try a new version on a little traffic (a “canary”): weight 90 on the current version and 10 on the new one.
  • Take a backend out gently before maintenance: lower its weight to 1 with the API so it gets almost no new traffic, then remove it from the list.

If a backend fails its health check, its share is split among the others in proportion to their weights.

Before you begin

  • A load balancer, or the details to create one. See Create a load balancer.
  • To change the algorithm or weights later, an API token with load-balancer.write. The portal can’t edit a load balancer.

Set the algorithm in the portal

1

Open the create form

Under Network & Security, click Load Balancer, then Create Load Balancer.

2

Set the weights

In Backends, enter a Weight for each row. Leave all weights at 100 for an even split.

3

Choose the algorithm

Under Routing and health, choose a Routing algorithm: Round robin, Least request, Random or Consistent hash.

4

Turn on retries (optional)

Under Advanced behavior, turn on Automatic retries and set Attempts and Per-retry timeout (ms).

5

Create

Click Create Load Balancer. The Configuration summary shows the algorithm you picked under Algorithm.

Change the algorithm or weights with the API

Send only the part you want to change. This request switches to least request:

curl -X PATCH "https://api.ibee.ai/v1/networking/load-balancers/l7/3f9c2b7e-5d1a-4c8e-9b6f-0a2d4e6f8a1c?workspace_id=710995" \
-H "Authorization: Bearer $IBEE_TOKEN" \
-H "Content-Type: application/json" \
-d '{"routing": {"algorithm": "least_request"}}'

To change weights, send the whole backend list with the new weights. Backends you leave out are removed.

curl -X PATCH "https://api.ibee.ai/v1/networking/load-balancers/l7/3f9c2b7e-5d1a-4c8e-9b6f-0a2d4e6f8a1c?workspace_id=710995" \
-H "Authorization: Bearer $IBEE_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"backends": [
{"type": "ip", "target": "203.0.113.10", "port": 8080, "weight": 90},
{"type": "ip", "target": "203.0.113.11", "port": 8080, "weight": 10}
]
}'

For an L4 load balancer, use /networking/load-balancers/l4/LB-ID instead.

Verify

Send a batch of requests and count where they land. If each backend returns its own hostname (for example on a /whoami page), this shows the split:

for i in $(seq 1 20); do curl -s https://LB-HOST/whoami; echo; done | sort | uniq -c

With round robin and equal weights, the counts are about equal. With weights 90 and 10, about 18 of 20 go to the first backend. Small samples vary; use 100 or more requests for a clearer picture.

Limits

SettingAllowedDefault
Weight1–1000100
Retry Attempts1–103
Per-retry timeout100–120,000 ms5,000 ms
Request timeout (Layer 7)100–300,000 ms30,000 ms (30 seconds)
Retry triggers5xx responses, reset connections, connection failuresAll three

Retries and the request timeout apply to HTTP and HTTPS load balancers. A Layer 4 load balancer forwards the TCP connection as it is.

Troubleshooting

You seeWhat it meansWhat to do
One backend gets almost all the trafficOther backends are failing their health check, or their weights are lowCheck each backend’s health endpoint and the weights on the detail page’s Backends section.
Users are logged out when they move between backendsSticky sessions aren’t applied yetStore sessions in a shared database or cache instead of in each backend’s memory.
Slow requests pile up on one backendRound robin ignores how busy a backend isSwitch to Least request with the API.
”Backend weight must be between 1 and 1000”A weight is outside the rangeUse a whole number from 1 to 1000.
A PATCH with backends removed some backendsThe list you send replaces the whole listSend every backend you want to keep.