Design complete systems / CHAPTER 61
Design an Uber-style ride service
Find nearby drivers quickly, then make one durable assignment despite retries, stale locations and racing acceptances.
Builds on Design nearby places and friends (Yelp), Leader and follower replication, Synchronous versus asynchronous communication.
- Use recent locations to propose drivers without treating the map as a reservation.
- Make trip and driver assignment one coordinated decision.
- Recover retries and regional handoffs without assigning a driver or trip twice.
01Functional requirements
A ride service first finds possible drivers, then asks one of them to accept. Finding a driver on the map does not reserve that driver. We will keep fast, approximate location search separate from the final assignment decision.
FR1: Driver availability. Let an authenticated driver report available, assigned, or offline status subject to server-side trip ownership. An assigned driver cannot become available merely by sending a new client flag.
FR2: Driver location reports. Accept driver locations from verified sessions, using increasing sequence numbers to reject reports that arrive out of order. Use those reports to find possible drivers and update the active-trip map.
FR3: Fare and arrival quote. Return a versioned basic fare quote and estimated pickup time for a pickup, destination, and service class. The server saves the quoted price; the browser references that quote rather than supplying its own charge amount.
FR4: Trip requests. Create one recoverable trip process for each rider request identity. After reconnecting or retrying, the rider can retrieve that same trip’s matching, assigned or ended state.
FR5: Driver offers. Find eligible nearby candidates and send expiring offers. If the bounded matching attempt finds no accepting eligible driver, return an explicit unmatched outcome rather than leaving the rider waiting indefinitely.
FR6: Driver acceptance. Accept an offer only if the trip is still matching, the driver is still available and the offer has not expired. Only the server currently permitted to write assignment records may make that decision. A replaced server cannot commit a late assignment.
FR7: Cancellation. When cancellation policy allows it, let the trip handler save the cancellation and any driver release together in the trip database. A cancellation that wins before acceptance must prevent a stale offer from reviving the trip.
FR8: Trip status. Keep the rider and assigned driver informed with versioned lifecycle status and necessary live location updates. Notifications may be retried; both participants can read current trip state.
Out of scope
- Ride pooling and a production route or dynamic-pricing model. Basic fare and arrival estimates remain in scope.
- Using the approximate nearby-driver map as an assignment authority; only the coordinated write path can claim a driver.
Example. Mina requests T1 while another rider requests T2. Both searches may discover D1. Discovery makes D1 a candidate for both; it grants neither trip ownership of D1.
02Nonfunctional requirements
These are independently chosen exercise assumptions to validate, not measured production guarantees.
NFR1: Initial-offer latency. Publish an initial offer within five seconds for at least 95% of admitted matching requests in the stated exercise workload, measured from durable trip creation. Record requests with no eligible driver separately and count their matching outcomes; this is not a promise that a driver accepts.
NFR2: Candidate-location freshness. At least 95% of candidate positions inspected during matching should have a valid report younger than ten seconds. Measure age before removing stale candidates, then reject reports outside the cutoff before offering; map animation rate is not the measurement.
NFR3: Telemetry capacity. Plan for 200,000 driver reports/second under the stated assumptions. Size this frequent location-report path separately from the smaller set of transactions that claim drivers and trips.
NFR4: Trip-request capacity. Plan for about 230 trip requests/second at the assumed peak. Include offer waves, acceptance contention, and notification work in the capacity test.
NFR5: One driver per trip. Allow zero active trips with more than one assigned driver. The assignment transaction checks the current trip state, valid offer and current assignment owner before committing.
NFR6: One active trip per driver. Allow zero drivers with more than one active trip. Claim the driver and trip together; two independent best-effort record updates do not establish this guarantee.
NFR7: Transition recovery. After a trip-state change reports success, a process crash must not lose that saved state or the work to notify its participants. The trip handler saves both together under the agreed database durability settings. A lost reply or notification recovers the same trip, operation and version. If the service cannot establish which server may currently write, temporarily reject assignment.
03Back-of-the-envelope Estimates
The app sends far more location reports than trip requests. Giving both the same write path would make the correctness-critical assignment work compete with much larger map traffic.
- Location reports. One million online drivers sending every five seconds create
1,000,000 ÷5 =200,000 reports/second. At 100 bytes each, raw ingress is 20 MB/second. Give this path its own capacity budget. - Position retention. Saving every report would add about
20 MB ×86,400 =1.73 TB/daybefore indexes and copies. Saving only one latest 100-byte report per driver is about 100 MB raw. Keep history only for a defined audit or analytics purpose. - Trip requests. Two million trips/day average
2,000,000 ÷86,400 ≈23/second. A chosen tenfold peak is 230/second. If matching tries three rounds and inspects five drivers per round, peak work is230 ×3 ×5 =3,450candidate evaluations/second. Limit those rounds; more location reports do not justify unlimited matching. - Trip records. At 5 KB for each trip and its lifecycle events, two million trips add 10 GB/day raw. Routes, maps and verbose logs need separate byte estimates.
- Connected apps. Assume two million simultaneous rider/driver connections and 20 KB of server state per connection. That is 40 GB across the gateway fleet before actual encryption, buffers and runtime costs. Measure those costs and test reconnect bursts.
- Busy regions. A stadium exit can overwhelm one region while the global average remains low. Measure local report rate, matching attempts and competing assignment transactions, keeping spare capacity for a failed server.
04API Design and Retry Contracts
Mina’s app first asks for a quote. The server saves its price, route inputs and expiry as Q17. Mina then references that quote when starting her trip; T1 is the saved trip ID returned to her.
POST /trips
Idempotency-Key: mina-trip-17
Content-Type: application/json
{"quoteId":"Q17"}
{"tripId":"T1","state":"matching","version":1}
- Quote the trip.
POST /quotestakes pickup, destination and service class. It returnsquoteId, estimated fare, currency, expiry and estimated pickup seconds. The server sets the price; a browser-provided amount is not a fare decision. - Create one matching process.
POST /tripstakes the quote ID and rider idempotency key. Save the key’s result withT1, then return202andmatching. A retry with that key retrievesT1instead of starting another trip. The 202 does not mean a driver has accepted. - Accept a specific offer.
POST /offers/{offerId}/acceptidentifies the saved trip/driver invitation. The server returns its committed assignment or 409 if the offer expired, the trip ended or someone else won. The driver identity comes from authentication. - Report driver state.
POST /drivers/me/locationsandPATCH /drivers/me/availabilityverify the driver’s current session and increasing report sequence. The server rejects attempts to declare an already assigned driver available without a valid trip transition. - Cancel or recover.
POST /trips/T1/canceltakes a stable command key and expected trip version.GET /trips/T1returns current versioned state only to authorized participants, including an explicit unmatched result when matching ends without a driver. Both apps use it after a lost reply or notification.
05Data Model and Access Paths
The trip database makes the final assignment decision. The map only helps matching code find drivers to ask. For the example, T1 is Mina’s trip, T2 is another rider’s trip, and D1/D2 identify two drivers.
| Record / key | Important fields and index | Purpose |
|---|---|---|
Trip tripId |
Rider, pickup/destination, quote, state, driverId, version, ownerEpoch; rider/time index |
Authoritative lifecycle |
Driver availability driverId |
Available/assigned/offline, activeTripId, version, ownerEpoch |
Prevent double assignment |
Offer offerId |
Trip, driver, expiry, state | Bounded invitation, not ownership |
Latest location driverId |
Session, sequence, acceptedAt, coordinates, cell |
Candidate discovery |
Outbox eventId |
Trip, transition version, notification type | Retry notifications after commit |
- A trip row records whether that trip is matching, assigned, in progress, completed, cancelled or unmatched. An offer row records which driver was invited and until when; sending it does not reserve the driver.
- A driver row records whether the driver already has an active trip. The acceptance transaction changes this row and the trip row together, so neither can claim an assignment without the other.
- An owner-permission record says which regional assignment process may write. Its number is stored as
ownerEpoch. For example, a failover controller can replace permission 12 with 13. The database write path rejects an assignment carrying 12 once 13 is current. This checked permission number is an epoch; it is not a timestamp or a number a driver may choose. - An outbox row records “send this committed trip transition” in the same database transaction as the transition. A relay program reads it later and retries delivery if a gateway or phone was unavailable.
- The latest-location row stores one verified session’s increasing report number and accepted coordinates. It supplies approximate discovery, not proof of assignment or physical presence.
Payment after completion has its own saved operation and provider reference. A trip’s assignment ID alone does not prove what a payment provider did.
06Baseline: Start with a regional matching authority
Start with one regional app, a spatial index and a transactional trip database. First follow Mina’s request to an offer:
- The quote handler saves
Q17. The trip handler validates it and savesT1asmatchingwith Mina’s creation-key result. - Matching code asks the spatial index for nearby apparently available drivers. It rejects old reports and wrong service classes; the index entry is only a candidate to inspect.
- It uses straight-line distance to narrow the list, then compares road arrival estimates for a small finalist set. A river or one-way road can make the nearer coordinate slower by road.
- It saves an offer row linking
T1andD1with a server-time expiry, then the app’s connection code sends that invitation. Do not hold database locks while waiting for the driver’s phone.
When D1 accepts, the acceptance handler performs this short transaction:
- Lock or conditionally claim the trip and driver records in a consistent order, reducing circular waits between competing transactions.
- Check that the caller is
D1, the offer belongs toT1/D1and is unexpired,T1is still matching andD1is still available. Check the current regional write-permission number too. - Set
T1.driverId=D1andT1’s state toassigned. SetD1.activeTripId=T1andD1’s state toassigned. Enforce unique active assignment so another transaction cannot claim either one simultaneously. - Save the accepted offer/result and pending notification work in that same commit. Return success only after the required records are durable.
A lock only inside one matching process is insufficient because another process can accept concurrently. The database must protect the paired decision. A conditional update is useful only if the whole trip/driver decision remains coordinated.
Sequential offers are simple but wait on declines. Small parallel waves improve the chance of a quick acceptance but create legitimate losing acceptances; they still use this transaction. Bound rounds and total matching time. If no driver accepts, the matching code commits an unmatched result. That terminal transition uses the same current-trip check, so a late offer cannot revive it.
Two discoveries, one assignment authority
Follow the numbered arrows
- Trip T1 sees D1 → Assignment transactionaccept offer for T1
- Trip T2 sees D1 → Assignment transactionaccept offer for T2
- Assignment transaction → T1 → D1 commitsfirst valid commit wins
- T1 → D1 commits → T2 receives conflictlater check sees claim
Trace the diagram in words
- Matching code can return D1 to both trip searches.
- Each acceptance handler asks the database to claim the trip and driver together.
- After T1 claims D1, the T2 check sees D1 assigned and returns conflict.
BUILD THE DESIGN, ONE CHANGE AT A TIME
Watch the architecture evolve.
Advance through the stages in this same diagram. Existing components keep their places; additions fill the available space, then add rows. Highlights show what changed from the previous stage, and numbered arrows match the connection key below.
Stage 1 of 3
Start here. First let two matching attempts discover D1 without allowing both to claim that driver.
Regional matching
Follow the numbered arrows
- Trip request → Matching servicecreate or recover matching trip
- Driver app → Spatial candidate indexlatest location/availability report
- Matching service → Spatial candidate indexnearby candidate IDs
- Matching service → Driver appshort-lived offer
- Driver app → Matching serviceaccept offer identity
- Matching service → Trip/driver authorityvalidate and commit trip-driver pair
Trace the diagram in words
- The trip handler saves Mina’s T1 and creation result.
- Matching code offers T1 to a recently reported eligible driver without keeping locks while waiting.
- Acceptance verifies current records, expiry and write permission, then claims both trip and driver.
- The app finds nearby candidates and sends a saved expiring offer.
- It waits for the phone outside database locks.
- Acceptance checks and changes the current trip and driver together in one transaction.
The cost: Sequential offers can delay matching; small parallel waves require explicit conflict replies. One region still has a tested capacity limit.
Why change it? Large numbers of connected apps need independent connection capacity, and a lost assignment reply must recover the same saved trip.
Recoverable app connections
Follow the numbered arrows
- Driver app → Spatial candidate indexlatest location/availability report
- Matching service → Spatial candidate indexnearby candidate IDs
- Matching service → Trip/driver authorityvalidate and commit trip-driver pair
- Trip request → Connection gatewaysauthenticated trip command/reconnect
- Driver app → Connection gatewaysaccept current offer
- Connection gateways → Matching servicevalidated command
- Matching service → Connection gatewaysoffer/status payload
- Connection gateways → Driver appversioned notification
- Trip/driver authority → Notification relaydurable transition intent
- Notification relay → Connection gatewaysretry committed notification
- Connection gateways → Trip requestparticipant trip status + resume
Trace the diagram in words
- The app reconnects and identifies the last trip version it applied.
- Gateways deliver offers without holding the trip transaction open.
- A repeated accepted offer or current-state read recovers the committed assignment after a lost reply.
- Connection gateways keep authenticated app channels independently of matching workers.
- The relay sends committed trip notifications from saved outbox work.
- Apps reconnect with their applied version and read current trip state; duplicate offers or status messages use the same IDs.
Inspect the connection changes
- Now: Trip request → Connection gateways: authenticated trip command/reconnect
- Now: Driver app → Connection gateways: accept current offer
- Now: Connection gateways → Matching service: validated command
- Now: Matching service → Connection gateways: offer/status payload
- Now: Connection gateways → Driver app: versioned notification
- Now: Trip/driver authority → Notification relay: durable transition intent
- Now: Notification relay → Connection gateways: retry committed notification
- Now: Connection gateways → Trip request: participant trip status + resume
- Previously: Trip request → Matching service: create or recover matching trip
- Previously: Matching service → Driver app: short-lived offer
- Previously: Driver app → Matching service: accept offer identity
All other connections are retained. Response paths are implicit unless drawn.
The cost: Gateways add memory limits, reconnect handling and repeated-message detection. A delivered notification still does not grant ownership.
Why change it? Driver reports dominate traffic. When a driver crosses a regional boundary or a writer is replaced, the old process may still be running; assignment writes must reject its old permission number before the replacement can claim drivers.
Partitioned telemetry ownership
Follow the numbered arrows
- Matching service → Spatial candidate indexnearby candidate IDs
- Matching service → Trip/driver authorityvalidate and commit trip-driver pair
- Trip request → Connection gatewaysauthenticated trip command/reconnect
- Driver app → Connection gatewaysaccept current offer
- Connection gateways → Matching servicevalidated command
- Matching service → Connection gatewaysoffer/status payload
- Connection gateways → Driver appversioned notification
- Trip/driver authority → Notification relaydurable transition intent
- Notification relay → Connection gatewaysretry committed notification
- Driver app → Telemetry ingestionhigh-rate valid location reports
- Telemetry ingestion → Spatial candidate indexlatest position + derived membership
- Ownership handoff → Trip/driver authorityreject writes with the old permission number
- Connection gateways → Trip requestparticipant trip status + resume
Trace the diagram in words
- Ingest code stores only a newer report for the driver’s active session.
- Matching code removes duplicate IDs and checks current positions from possible stale cells.
- The database rejects an old writer’s permission number rather than accepting a second assignment owner.
- Location-ingest workers process high-rate reports separately from trip transactions.
- The spatial index divides regions and may temporarily repeat or omit a driver; matching code checks latest reports.
- The controller updates the database’s write-permission number during handoff, and assignment writes check that current number.
Inspect the connection changes
- Now: Driver app → Telemetry ingestion: high-rate valid location reports
- Now: Telemetry ingestion → Spatial candidate index: latest position + derived membership
- Now: Ownership handoff → Trip/driver authority: reject writes with the old permission number
- Previously: Driver app → Spatial candidate index: latest location/availability report
All other connections are retained. Response paths are implicit unless drawn.
The cost: Distributed cell updates and region handoffs need repair and admission limits. They never relax one active trip per driver or one driver per trip.
07Evolution: Scale matching without weakening ownership
First test a larger regional server or database for the measured resource limit. Neither extra CPU nor extra location workers lets two writers assign one driver independently.
Move connections away from matching work
- Run connection gateways to authenticate apps and keep their live channels. They deliver saved offers and status messages; matching workers focus on candidate selection and assignment attempts.
- The relay reads a committed outbox row and sends its trip ID/version through the gateway. If the send succeeded but progress recording failed, it may send again. The app recognizes the same offer ID or trip version instead of treating it as a new assignment.
- On reconnect, the app supplies the last trip version it applied and reads current
T1state. A missed notification must not erase the database’s assignment.
Divide location work without dividing ownership incorrectly
- Location-ingest code accepts only a newer report number for the current driver session. Partition reports by region/cell and retain the latest position separately from optional history.
- Background cell updates may temporarily list
D1twice or omitD1’s new cell. Matching code removes repeated IDs and checks latest accepted coordinates and age before offering. - Keep the trip and driver claim within one regional assignment authority. If proposed storage puts them on independent shards, co-locate the decision or show a real cross-store coordination protocol. Two unrelated “set assigned” calls are not the transaction above.
- During a region handoff, the failover controller installs a new write-permission number. The database checks that number on every accepted assignment write. An old matcher carrying 12 cannot commit after 13 is current, even if its phone connections still work. If current permission cannot be established, pause assignments rather than allow two owners.
| Choice | Benefit | Cost or limit |
|---|---|---|
| Sequential offer | Fewer simultaneous acceptances | More waiting after a decline |
| Small parallel wave | Earlier chance of acceptance | More notifications and explicit conflict responses |
| Separate gateways | Connection capacity grows independently | Reconnect limits and duplicate message handling |
| Geographic partitions | Local candidate searches and divided report load | Busy regions, delayed memberships and safe owner handoff |
Supply/demand aggregates may improve pricing, but a good fare or arrival estimate does not claim a driver. Quote publication, discovery and exclusive assignment retain different records and responsibilities.
See stage 2: Recoverable app connections ↗See stage 3: Partitioned telemetry ownership ↗08Failure and Recovery Scenarios
Test each race against the saved trip and driver rows:
- D1 accepts T1 and T2. Both searches may have found
D1. IfT1’s transaction claimsD1first,T2’s transaction reads assigned state and returns conflict, then searches elsewhere. - D2 also accepts T1.
T1’s transaction allows one winning driver. A legal ordering isT1→D2andT2→D1;D1owning both trips orT1owning both drivers is forbidden. - Success is saved but the reply is lost. Repeating the same accepted offer recovers its saved assignment instead of making another one. Read current trip state too if it has since been cancelled or completed. The outbox relay separately retries the missed notification.
- Cancellation wins the race. A cancellation handler checks the same trip state and expected version. If it commits before acceptance, the old offer loses because
T1is no longer matching. An allowed assigned-trip cancellation releases the driver in the coordinated transition. Ending an in-progress ride follows its own fare/payment policy. - An old regional process remains alive. It still carries permission 12 after the controller installed 13. The database rejects 12, even if the old process’s map looks fresh. Changing routing alone would not stop that stale process from writing through another path.
- The phone shows an expired offer. The server checks expiry, not the phone’s displayed screen or clock. If the current writer or required state cannot be verified, return a temporary failure instead of assigning from the map.
Authenticate each status read before returning precise pickup or location data. Notifications help people notice the result; only the saved coordinated transition decides it.
09Trade-offs and Validation
Use separate measurements for discovery, assignment and delivery:
- Discovery: report age, time before positions reach the index, repeated or missing candidates and busiest-region load.
- Assignment: time to first offer and final assignment, rejected-offer reasons, transaction waits, stuck matching attempts and any duplicate claim.
- Delivery: gateway memory, reconnect bursts, notification delay and the version recovered after a missed message.
- Privacy and abuse: trip-creation limits, location-query limits, access to exact coordinates and retention. A rider needs the assigned driver’s necessary status, not a history of every driver.
- Payment recovery: if completion charging is uncertain, reconcile the same saved payment operation rather than submit another charge ID.
Close by tracing Mina’s quote, saved T1, nearby candidates, expiring offer, paired acceptance transaction and reconnect read. The fast map proposes a driver; the main database checks and changes the two ownership records. Scaling the map must not remove that final check.
OPTIONAL DESIGN EXTENSIONWhere AI Fits in This Design
An optional model improves arrival estimates for drivers already selected by geographic matching. It does not decide who owns a trip.
- Input and output. Send the road-routing duration, time of day, coarse traffic features and valid recent pickup/driver positions. The model returns an estimated arrival time with a version and diagnostic confidence. Uber’s DeepETA example combines a routing estimate with learned correction.
- Integration. Matching code applies the estimate after cheap distance filtering. It may rank a farther driver first, but acceptance still checks current trip, driver, offer expiry and database write permission. A low predicted time cannot make busy
D1available. - Fallback and budget. Assume at most 50 ms for finalist scoring within the five-second offer target. On timeout, missing features, a negative predicted time or a value outside the configured pickup-time bounds, use the road-routing estimate. Matching can continue without waiting indefinitely for the model.
- Evaluation and privacy. Measure arrival error by region, traffic and trip segment, plus slow calls, cost/match, cancellation changes and unequal service quality. Minimize retained coordinates and restrict access to journey features; avoid exporting full rider histories unnecessarily.
Quote prices, payment rules and exclusive assignment remain explicit server decisions.
DRAW. CALCULATE. EXPLAIN.
Prove both assignment invariants
75 minutes · guided practice
Draw two trips T1/T2 and two drivers D1/D2. Trace D1 accepting both trips while D2 also accepts T1. Write the conditions checked in the transaction and list legal outcomes. Then lose the winning transaction response and its first notification. Explain cancellation arriving during the same race and how a region handoff changes the write checks.
Compare the worked solution
- Write the paired claim. The acceptance handler locks or conditionally claims the current trip and driver rows, checks the valid offer/expiry and current write-permission number, then changes both rows and saves notification work in one transaction.
- List legal winners. If
D1winsT1, neitherT2can also claimD1norD2also claimT1. Another legal order isD2winningT1whileD1winsT2. The invariant protects both sides, not only one driver field in each trip. - Lose the reply. Repeat the same offer acceptance and recover its stored assignment. The relay retries saved notification work, while both apps can fetch the trip’s current version. A lost trip-creation reply similarly reuses its original key.
- Race cancellation. The cancellation handler checks the same current trip record. It either wins before acceptance or follows the allowed assigned-trip cancellation rule, releasing the driver together. A late offer cannot restore matching.
- Replace a regional writer. The controller changes the stored permission number from 12 to 13. The database rejects assignment writes carrying 12 after that switch. An old process staying alive or holding a phone connection does not authorize another commit.
- Keep ranking separate. A driver across a river may have the shortest straight-line distance but a slower road arrival time. That changes offer preference, not the transaction required to claim the driver.
The checkboxes record your own assessment. The worked solution gives you a way to check the reasoning.
RETRIEVE BEFORE YOU REVEAL
Keep the important ideas.
Say the answer aloud, then turn the card over.
CARD 01Why is the nearby-driver map not the assignment authority?
The map supplies possible drivers and may be old. Acceptance reads the current trip, driver and unexpired offer, then changes trip and driver ownership together in one database transaction. The currently permitted regional writer must perform that operation.
CARD 02What happens if an acceptance response is lost?
A lost reply can follow a successful assignment. Retry the same acceptance and recover its saved result; read current trip status after reconnect. The outbox retains notification work, so delivery can retry without inventing a second assignment.
CARD 03What makes geographic scale-out difficult?
The controller gives the current writer a stored permission number, such as 13 replacing 12. The database rejects writes with 12 after the change. Checking that number on the write path stops an old process; changing a route or logging a number does not.
CHECK THE REASONING
Can you explain the choice?
Answer every question, then finish for an immediate score and an explanation for each answer. Aim for 80%, then revisit any gaps.
CLOSE THE LOOP
Explain it without the page.
Summarise the mechanism, one trade-off, and one failure mode. Then mark this chapter complete.