Design Pastebin

01Share text without turning it into executable content

A developer wants to send a diagnostic log to a teammate without copying it into a chat message. They paste the text, receive /p/p7Hk2Lm9, and share the link. Unlike a URL shortener, our service owns the content being returned. It must preserve the saved text, control who may retrieve it and avoid executing anything the author pasted.

Support immutable text, optional expiry, owner deletion, custom aliases and owner listings. Distinguish public, unlisted and private pastes. Public content may be discovered; unlisted content is available to anybody with the address; private content requires an authenticated permission check. An obscure URL alone does not make a paste private.

Assume authenticated creators, a 10 MB maximum and a 10 KB average. Editing, images, collaborative cursors and a public full-text search engine are outside this exercise. A successful creation means the complete intended text is stored, not that an upload merely started. An owner can see a clear failed or processing state when publication cannot finish. Read authorization applies to each new request; bytes a reader already copied cannot be recalled.

Clarify whether a forwarded link should grant access, whether text can change after publication, and whether completion must wait for the entire body. The following requirements choose immutable text with explicit public, unlisted and private access.

02Functional requirements

Agree on these supported actions before selecting components.

1. Publish immutable text. Authenticated authors create a paste with text, title, visibility, optional custom alias and expiry; publication returns one stable identity.

2. Retrieve safely. Readers can request a formatted page or raw text. Public and unlisted pastes are accessible under their stated rules; private pastes require a current grant. Pasted markup is displayed as text.

3. Manage owned pastes. Owners list their pastes, inspect pending/ready/failed status and delete content to prevent new reads.

4. Recover interrupted creation. A repeated creation key with unchanged content resumes or returns the same paste, rather than publishing another body.

03Non-functional requirements

Use these as illustrative interview assumptions to agree with the interviewer. Numerical targets require measurement; they are not claims about an existing product or a proven implementation. p95 (the 95th percentile) means 95% of measured operations finish within the stated time.

1. Workload and limits. Use one million creations/day, approximately 116 creates/s and 579 reads/s at peak, a 10 KB average and a hard 10 MB text limit. Ten-year logical content is approximately 36.5 TB before copies.

2. Read latency. Target regional p95 time to first byte of 200 ms for authorized reads at the planning peak under normal operation. Whole-body transfer time depends on size and client bandwidth; the 10 MB maximum is not promised within 200 ms.

3. Publication integrity and durability. Success means the complete intended text and its metadata are durable. They must survive an application restart or one storage-node failure within the region; a pending upload is not a published paste.

4. Privacy and expiry. Every new private read must pass current authorization, including cache hits. Expired or deleted pastes cannot begin a new authorized read; already released bytes cannot be recalled.

5. Bounded resource use. Enforce byte quotas, upload concurrency and bounded buffering as well as request limits. Slow maximum-size uploads must not exhaust all read capacity.

6. Retry retention. Keep owner-scoped creation outcomes for at least 24 hours. After that window, clients need explicit status recovery or a new intentional operation rather than an indefinite replay promise.

04Save a log and read it back through one database

The first implementation has an application and a relational database. A creation request contains text, visibility and an owner-scoped request key. Validate the encoding and size, choose a random paste ID, then commit the text, metadata and request result together. After commit, return the paste URL. The database unique constraint prevents two pastes from claiming the same ID or custom alias.

A reader requests p7Hk2Lm9. The app finds its row, checks expiry and deletion, checks the reader's grant if it is private, and sends the stored text. A raw endpoint uses text/plain; a formatted browser page escapes HTML characters before showing them. Pasting <script> must display that text, not run a program in another reader's browser.

The single transaction gives a simple publication rule: either the complete paste exists or none of it is visible. A lost creation response is recovered by looking up the same request key. For modest traffic, storing text in SQL is reasonable. Large retained bodies and slow transfers may eventually make it expensive, but the product name does not decide the database technology.

05Separate request rate from retained bytes

Assume one million new pastes per day and five reads per paste. That is about 11.6 creations and 57.9 reads per second on average. A tenfold peak is approximately 116 creations and 579 reads per second. These are not extreme request rates; the storage and large-object cases deserve more attention.

The final row is a stress case, not the expected average. At five seconds per upload, 116 arrivals/s imply about 580 concurrent uploads in a stable system. Buffering every maximum-size body would consume about 5.8 GB before application overhead. This motivates streaming, bounded concurrency and separate byte quotas. It does not require hundreds of independent services.

06Give clients an identity they can safely retry

Use an idempotency key to identify a creation request.

Creation request

POST /v1/pastes
Idempotency-Key: paste-204
Content-Type: application/json

Example request body

{
  "text": "Connection timed out while contacting inventory.",
  "title": "Inventory timeout",
  "visibility": "private",
  "expiresAt": "2030-12-31T23:59:59Z"
}

Store a fingerprint of the validated input so that repeating the key with different text is rejected. The publication outcome determines the response:

Reject oversized text with 413 and exhausted quotas with 429. Scope owner listing and status reads to authenticated identity. Avoid returning different private error details that reveal whether a guessed paste exists. Specify how long creation-request identities remain recoverable; a documented retry window is more useful than an indefinite promise unsupported by retained records.

Custom aliases are claimed atomically and retained as used identities after deletion. Otherwise an old incident ticket could silently lead to another person's content years later.

07Keep content, metadata and permission roles distinct

The records separate saved content, retry recovery and private access.

Metadata describes the content; it is not a substitute for storing the content itself.

The common read is an exact paste-ID lookup followed by a permission check. Owner listing needs an index such as (ownerId, createdAt, pasteId). Expiry cleanup needs an index ordered by deadline, so a worker can fetch due batches rather than scan the whole database. Reads still compare the deadline independently: delayed cleanup should waste storage, not extend access.

When body storage grows costly, move text into a private object store and keep its immutable object identity, byte length and checksum in the metadata row. An object store retrieves bytes by key and can scale bulk storage separately from database indexes. This separation introduces an important new question: what should readers see if one store succeeds and the other fails? The answer is a publication state, rather than assuming the two writes are one transaction.

08Publish only after the complete body exists

Publish a body stored outside the database in this order:

1. Reserve p7Hk2Lm9 in UPLOADING state, with an immutable body name tied to this upload attempt.
2. Stream the text to object storage using a bounded buffer.
3. Verify the stored length and checksum.
4. Change the metadata to READY in a database transaction.

Readers require READY before retrieving the object.

Why this order? If metadata were ready first, a crash before uploading would expose a broken paste. Uploading first can leave an unused object, but that object remains invisible and recoverable. An orphan is storage work to clean up; a ready link pointing to absent bytes is a user-visible correctness failure.

The upload request identity points to the reserved paste. A retry inspects that same attempt and either finishes publication or returns the stored result. It must not silently replace the content under a published cache key. Immutable object names prevent one upload from changing bytes readers already associate with another version.

Cleanup also needs a rule. Before deleting an abandoned attempt's bytes, atomically mark that attempt canceled so it can no longer publish. Publication requires the still-active upload state. The shared state check lets either publication or cancellation win, never both. Detailed lease renewal and multipart recovery are follow-ups; a timer followed by an unguarded object delete is not safe even in the simple design.

09Move transfer work out of the metadata bottleneck

A large upload can hold a connection much longer than an indexed metadata read. Give uploads bounded streaming buffers and concurrency limits so slow clients do not consume every reader worker. For larger workloads, separate upload and read capacity. Direct uploads can remove byte transfer from application servers, but the application must still authenticate completion and verify the exact accepted object.

Cache immutable public bodies when repeated reads justify it. Measure distinct hot bytes and byte-hit ratio, not merely the percentage of requests hitting cache. A 10 MB paste can displace many small logs, so impose entry-size or admission limits. Coalesce concurrent misses for the same body and cap object-store requests during cache failure.

Private content may be cached internally, but every delivery still checks current permission. Do not publish a permanent public object URL and expect a private metadata page to protect it. Short-lived signed download links trade reduced authorization traffic for a period during which the link remains usable; negotiate that behavior rather than claiming immediate revocation.

Replicate authoritative metadata and protect object bytes under their own storage policy. Partition metadata only after measured limits justify it. Distributing different paste IDs improves aggregate capacity; one viral paste is primarily a delivery-cache problem.

Meet the one-node-loss target with durable majority commits across three metadata replicas in independent regional failure domains, and object storage whose acknowledged writes survive one storage-node loss. Verify both policies before READY publication; three-copy arithmetic alone is not evidence of either guarantee.

10Recover publication without exposing partial data

Suppose upload U stores the body, then the application crashes before READY commits. The owner sees no success, and readers still see no published paste. A retry locates U, verifies its stored bytes and asks the database to change the still-active upload to READY. If a cleanup worker already canceled U, the transition fails instead of publishing an object that may be deleted.

If the application crashes after READY commits but before returning, the request result retrieves the same paste. If an object read fails later, return a temporary error; do not report an empty paste as the saved content. A checksum helps identify corruption, while backups and redundant storage provide recovery paths.

Delete metadata before scheduling physical body removal. Store the removal intention in the same transaction as deletion, often as an outbox row: a durable work record that a background worker can retry. Repeated deletion of the exact obsolete object is harmless. Do not delete shared deduplicated bytes merely because one paste no longer uses them; shared storage would require reference accounting.

During a permission-store outage, fail private reads rather than trusting an old grant. Public-content availability can use a separately declared caching policy. These outcomes distinguish unavailable data from missing data and prevent an availability workaround from becoming a privacy leak.

11Watch publication, privacy and byte consumption

Monitor upload-to-ready latency, failed uploads, oldest pending attempts and any READY record whose body cannot be found. Separate metadata latency, time to first byte and whole-transfer duration. A 10 MB response over a 10 Mb/s connection already needs roughly eight seconds before overhead, so one small latency target cannot describe every stage.

Escape rendered text, disable content sniffing for raw delivery and keep object storage private. Rate-limit both requests and bytes. Avoid logging paste contents: diagnostic logs can contain credentials or personal data even when the uploader does not recognize them. Titles, object keys and checksums also need sensible access and retention policies.

Approximate view counts can flow through a bounded asynchronous queue instead of updating the paste row on every read. Measure dropped events and label delayed counts honestly. Cost comes mainly from retained bodies, replicated copies, delivery bytes and object operations. Compression can reduce text storage, but enforce decompressed-size limits.

Test a slow maximum-size uploader, a popular paste after cache loss, a denied private read with a warm cache, and a restore containing both metadata and body objects. Restoring only one side does not restore the service.

12Check the design against the requirements

Use the agreed lists to check the finished design. The tests below still need to establish the targets; a proposed mechanism is not a measured result. FR refers to the numbered functional requirements above; NFR refers to the numbered non-functional requirements.

13Rapid revision

Remember: Store and verify the bytes before READY; hidden unfinished storage is safer than a visible broken paste.

In a spoken summary, begin with saving and retrieving one diagnostic log. Use the estimates to explain why the request rate is manageable while retained bytes may justify object storage. Then show the one crash between byte storage and publication, followed by the private-cache check. That covers the design's central differences from a URL shortener. Editing and strict multi-region revocation add separate requirements rather than following automatically from this architecture.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.