Load balancer versus API gateway

01Responsibilities are more useful than product names

A shop exposes order and search APIs through several application servers. Someone must choose a server for each request, and someone must verify credentials, limit abusive traffic and route to the right API.

These responsibilities lead to two common roles:

• A load balancer distributes connections or requests among servers configured for that service and passing its readiness checks. A transport-layer balancer uses network addresses and ports. An application-layer balancer can also inspect HTTP hostnames, paths and headers.
• An API gateway commonly checks credentials, applies API limits, routes versions, transforms requests or assembles responses from services.

The roles overlap. A gateway may balance requests; a load balancer may understand HTTP and terminate TLS. Draw the work you need first, then decide whether one component or two should do it. Neither product name guarantees every feature or the correctness of an order.

02Place policy at the right boundary

Follow an order request from Green, a customer organization whose users share one request allowance. Such an organization is called a tenant.

1. The public entry point receives the TLS-protected request. A balancer selects a healthy gateway instance if a separate gateway fleet is needed.
2. The gateway verifies the caller's credential. This establishes identity. It also checks request size and the allowance shared by Green's users across all gateways.
3. The gateway routes /orders to an order server. Choosing a route does not establish permission to change an order.
4. The order service checks the actual record. Does the verified caller belong to Green, own the order and meet its cancellation rules? This operation-specific decision is authorization.
5. Each service uses the time remaining before the caller's deadline. Choose one component to retry temporary connection failures within a maximum attempt count. Before retrying a write, it must reuse the request key whose saved result prevents a repeated order. Independent retry loops can repeat the same backend work many times.

03Worked example: order and search APIs

Assume 2,000 incoming requests/s, three gateway instances and a shared Green allowance of 100 requests/s.

1. Give each gateway an independent 100/s counter. Green could then send 100/s through each: about 300/s in total. Matching tenant names do not make separate counters shared.
2. For an exact 100/s total, coordinate spending. A shared store checks and deducts remaining allowance in one indivisible operation, or each gateway spends a reserved portion whose combined total stays within the limit. If local limits intentionally approximate the total, state the accepted overshoot.
3. Check nested retries. Three attempts at the outer balancer, each causing three gateway attempts, can produce nine backend attempts. That is nine attempts, not proof of nine committed orders.
4. Protect uncertain writes. Save the caller's stable request key and order result with the order change. On a retry, look up that key; return its saved result instead of creating another order. Reject reuse with different request content.

Now check health and placement:

• A listening-port check shows the process responds, often called liveness. Readiness asks whether it can do the required work, such as reaching the order database. Match the check to that route.
• An optional recommendation failure should not remove all otherwise usable order servers from routing.
• If the gateway already provides required balancing, another balancer needs a specific additional benefit. Every extra component adds a call, capacity limits and configuration to maintain.

04Evaluate the whole request path

Allocate responsibilities explicitly:

• Distribute traffic where equivalent healthy backends need selection.
• Centralize common API rules where this reduces repeated credential, version or quota work across services.
• Keep order-specific rules in the order service. Two implementations of cancellation eligibility can drift apart.
• Block or equally protect bypass routes. A direct backend route must not silently skip a mandatory request limit or access check.
• Release common policy gradually. Several gateway instances can all reject requests after the same bad rule update; retain rollback.
• Measure different outcomes separately. A valid quota denial, invalid credential, saturated gateway and failed order database need different responses.

In the interview, trace one request and one retry. Name who selects a server, checks identity, spends allowance and changes the order record. A justified division of work is more useful than a mandatory pair of product names.