LearnastraTHE DESIGN COURSE

Make the right trade-off / CHAPTER 34

Serverless versus server-based execution

Compare provider-managed execution with your own worker fleet using traffic, processing time, cost and recovery needs.

Intermediate1h of deliberate study10 questions

Builds on Back-of-the-envelope estimation, Stateful versus stateless architecture.

By the end, you can
  • Distinguish operational responsibility from physical infrastructure.
  • Explain startup, concurrency, and dependency limits.
  • Compare cost with explicit assumptions.

01Servers still exist in both models

A thumbnail job reads an uploaded image, produces a smaller image and saves the result. Someone must arrange where the code runs.

  1. Serverless execution: a provider starts or reuses execution environments when requests or events arrive, within configured limits. One run is an invocation. Your team supplies code and configuration rather than directly operating every machine.
  2. Server-based fleet: your team chooses machine sizes, keeps worker processes running and controls their scheduling, replacement and scaling. Managed server platforms share these tasks in different ways.

Physical servers exist in both models. Serverless changes operating responsibilities; it does not remove the team's responsibility for permissions, saved jobs, retries, data protection and recovery. A server fleet can also scale automatically.

Compare the actual service limits and workload. “Serverless scales” does not establish that the database can handle the resulting connections, or that an interrupted job can recover.

02Elastic compute can expose fixed dependencies

Follow one thumbnail job in either model:

  1. Save accepted work before running it. A durable queue or job table records the image reference, task and stable job identity so a worker crash does not erase the request.
  2. Start a worker within configured capacity. A provider may prepare an environment; a fleet may assign an existing process. Initialization is a cold start and can occur after failure in either model.
  3. Read the image and produce the thumbnail. Limit simultaneous database or storage work separately from the number of running jobs.
  4. Save the result under that job's identity. Look up the job’s saved completion record. Reuse an output only when that record refers to a fully saved thumbnail for the same input image version and requested size; a partial file or a thumbnail from another edit is not completion.
  5. Acknowledge completion only when the output is recoverable. A crash before acknowledgment can cause redelivery; the new worker must inspect the saved result.
Question Managed execution Long-lived worker fleet
What happens during idle time? The service may avoid charging for a full idle worker allocation. Reserved machines may remain unused.
What limits execution? Runtime, duration, hardware and concurrency limits depend on the service. Worker size, count and scheduler settings determine available resources.
What starts slowly? New environments initialize packages and connections. New or restarted processes also initialize.
Can local memory preserve a promised result? Environment reuse is not guaranteed. Processes still fail; local-only results are unsafe here too.
What work remains for the team? Code, limits, data and recovery remain. Those tasks plus more machine and fleet management remain.

Reusing parsed configuration or a connection in a warm environment is an optimization. Correctness must not depend on that environment surviving.

03Worked comparison: irregular image processing

Assume 200 thumbnail jobs/s and mean job duration 0.5 s under stable load.

  1. Calculate concurrent work: 200 × 0.5 = 100 jobs on average.
  2. The database slows and mean duration becomes 2 s. The same rate would require about 400 concurrent jobs, even without more arrivals.
  3. The database safely supports 150 connections. Letting every job open its own connection can overwhelm it. Share a fixed connection pool or limit the number of jobs doing the database stage at once.
  4. Keep excess jobs saved while they wait. Measure the oldest job's age. More compute environments do not supply more database capacity.
  5. A worker saves a thumbnail and crashes before queue acknowledgment. A later attempt uses the same job identity to verify or return the saved thumbnail. It must not assume the first attempt did nothing.

For cost, assume 50,000 jobs/day at 0.5 s each: 25,000 useful execution-seconds/day, before startup and retries. This is workload, not a complete bill.

  • A fleet bill depends on reserved machine hours and how much of them are used.
  • A managed-execution bill depends on actual allocation and provider pricing dimensions.
  • Both comparisons need transfer, storage, redundancy, startup, retries and team operating effort.

Measure these quantities rather than inserting a universal price or cold-start delay.

Serverless versus server-based execution — decision flow

Serverless versus server-based execution — decision flowSave the thumbnail job before starting temporary processing. Limit simultaneous execution and database work so new workers cannot exceed database capacity. Save the result under its stable job identity before acknowledgment so a retry can inspect it. Numbered arrows correspond to the connection key below. All connections are directional.Image-jobrequestsArrivals may come inburstsSaved job queueAccepted jobs surviveworker failureManaged workersLimit jobs running atthe same timeResult databaseHas fixed connectionand write limits1234Serverless versus server-based execution — decision flowSave the thumbnail job before starting temporary processing. Limit simultaneous execution and database work so new workers cannot exceed database capacity. Save the result under its stable job identity before acknowledgment so a retry can inspect it. Numbered arrows correspond to the connection key below. All connections are directional.Image-jobrequestsArrivals may come inburstsSaved job queueAccepted jobs surviveworker failureManaged workersLimit jobs running atthe same timeResult databaseHas fixed connectionand write limits1234Serverless versus server-based execution — decision flowSave the thumbnail job before starting temporary processing. Limit simultaneous execution and database work so new workers cannot exceed database capacity. Save the result under its stable job identity before acknowledgment so a retry can inspect it. Numbered arrows correspond to the connection key below. All connections are directional.Image-jobrequestsArrivals may come inburstsSaved job queueAccepted jobs surviveworker failureManaged workersLimit jobs running atthe same timeResult databaseHas fixed connectionand write limits1234

Follow the numbered arrows

  1. Image-job requests → Saved job queuesave the accepted thumbnail job
  2. Saved job queue → Managed workersstart only permitted simultaneous jobs
  3. Managed workers → Result databasesave or reuse the job’s existing result
  4. Managed workers → Saved job queueacknowledge after the result is saved
Saved thumbnail jobs survive temporary workers. Limit simultaneous database work, save the result and then acknowledge completion.
Trace the diagram in words
  1. Save the thumbnail job before starting temporary processing.
  2. Limit simultaneous execution and database work so new workers cannot exceed database capacity.
  3. Save the result under its stable job identity before acknowledgment so a retry can inspect it.

04Choose boundaries before a platform

Choose using the job's actual needs:

  • Bursty bounded jobs can fit managed event execution and reduce idle machine management.
  • Consistently busy jobs, special hardware or long-running processes may justify a fleet with more direct resource control.
  • Mixed workloads can combine them, such as servers for live connections and managed background image jobs.
  • Scale-down must stop new work and finish active jobs or leave them recoverable for redelivery. Never acknowledge the only result in memory and then terminate it.
  • Monitoring should include startup time, job age, repeated attempts, worker saturation and dependency connections.

State arrival pattern, duration, completion target, simultaneous jobs and downstream capacity before choosing the platform. The successful design completes saved work reliably at acceptable total cost; fast compute expansion alone is not success.

DRAW. CALCULATE. EXPLAIN.

Bound a thumbnail pipeline

20 minutes · guided practice

Jobs arrive at 500/s and take 0.8 s on average. The database supports 150 active job connections. Explain concurrency, overload handling, job acknowledgment and the information needed for a cost comparison.

Your self-check
Compare the worked solution
  1. At 500 jobs/s and 0.8 s mean duration, stable average execution demand is 400 jobs.
  2. One database connection per job would exceed the 150-connection limit. Cap simultaneous database work or use a fixed pool of at most 150 active database connections, then measure whether jobs still finish on time.
  3. Preserve waiting jobs and track oldest age rather than opening unlimited connections.
  4. Save outputs under stable job IDs before acknowledging completion. A retry checks that saved output rather than creating another logical thumbnail.
  5. Compare allocated resources, idle time, startup, retries, transfer, redundancy and team effort using the actual platform prices.

The checkboxes record your own assessment. The worked solution gives you a way to check the reasoning.

RETRIEVE BEFORE YOU REVEAL

Keep the important ideas.

Say the answer aloud, then turn the card over.

CARD 01What responsibility does serverless change?
  1. The provider manages more execution startup and machine provisioning.
  2. The application still saves accepted jobs, checks permission and handles retries.
  3. Protect dependency capacity and expose actual completion.
CARD 02How do job duration and concurrency interact?
  1. Multiply stable completions per second by mean job seconds.
  2. The result estimates average active jobs.
  3. Slower database work raises this demand even if arrival rate is unchanged.
CARD 03How should execution costs be compared?
  1. Measure job duration and traffic shape.
  2. Count allocation, idle capacity, startup and retries.
  3. Add transfer, storage, redundancy and team work.

Useful execution time alone is not a full bill.

Practise this topic in the revision deck ↗

CHECK THE REASONING

Can you explain the choice?

Answer every question, then finish for an immediate score and an explanation for each answer. Aim for 80%, then revisit any gaps.

1 / 10200 jobs/s at 0.5 s mean duration implies what mean concurrency?
2 / 10The same rate now takes 2 s on average. What is demand under stable assumptions?
3 / 10The database safely supports 150 connections. What should 400 active jobs avoid?
4 / 10A warm environment caches a parsed static configuration. What is the safe interpretation?
5 / 10A worker writes the output then crashes before queue acknowledgment. What should a retry do?
6 / 1050,000 jobs/day at 0.5 s each represents how much useful execution time?
7 / 10Which need can favor a long-lived fleet?
8 / 10Which statement about cold starts is appropriate?
9 / 10What should a safe scale-down preserve?
10 / 10Why include operating effort in the comparison?
0 of 10 answered

CLOSE THE LOOP

Explain it without the page.

Summarise the mechanism, one trade-off, and one failure mode. Then mark this chapter complete.

Go to the source