Design a personalized news feed

01Separate finding stories from deciding their order

Design a personalized feed of posts from followed people, pages and groups. Leo opens the application and receives twenty eligible stories, including Maya’s trail report p882. Publication stores Maya’s post; candidate generation finds possible stories for Leo; ranking orders those candidates; delivery returns the selected content. These are separate responsibilities even if one application initially performs them all.

Include text and references to already uploaded media, follow/unfollow, block rules, group membership, reply filtering, refresh and older pages. Ads, recommendation-model training and video processing are outside scope. A few seconds of propagation delay is acceptable for newly published stories, but current eligibility controls what each response may disclose.

A follow expresses distribution interest. It does not automatically grant access to friends-only or private-group content. Friends-only stories require the product’s approved friendship relation; private-group stories require current membership. Keep those relationship types explicit instead of using one generic edge as every kind of permission.

Choose a bounded ranked browsing session: its ordering stays fixed while the user pages, except that newly ineligible stories can disappear. Refresh starts a new session and includes newer posts. Already delivered content cannot be recalled after a later block or deletion.

Clarify which relationships make a story eligible, whether the feed is ranked, how fresh it should be and whether page two must preserve an earlier order. These choices determine both the candidate pipeline and its browsing-session contract.

02Functional requirements

Agree on these supported actions before selecting components.

1. Publish and manage stories. Authenticated authors publish/delete posts referencing verified media. Retried creation returns the same durable source post.

2. Manage eligible sources. Support follows/unfollows, blocks, approved friendships and private-group membership, with each relation retaining its distinct access meaning.

3. Read a ranked feed. Return up to twenty eligible stories from followed people, pages and groups; apply reply filtering and a declared ranking policy.

4. Refresh and continue. Refresh creates a new browsing session; older pages continue a bounded stored order while rechecking current eligibility.

03Non-functional requirements

Use these as illustrative interview assumptions to agree with the interviewer. Numerical targets require measurement; they are not claims about an existing product or a proven implementation. p95 (the 95th percentile) means 95% of measured operations finish within the stated time.

1. Workload. Plan for approximately 86,805 feed requests/s at the fivefold peak, 500 followed entities per reader and skewed authors with millions of active recipients. Candidate reads/writes and media bytes need separate budgets.

2. Response latency. Target regional p95 feed-metadata latency of 300 ms at admitted peak load in normal operation, including candidate merge, ranking and eligibility checks. Media transfer has a separate delivery measurement.

3. Freshness. Target p95 propagation from source commit to the candidate lists or author histories used by eligible active readers’ refreshes within five seconds during normal fanout processing. Bounded retrieval and ranking can omit an available story; a retained session intentionally keeps its prior order.

4. Source durability and recovery. Accepted posts, relationships and publication events must survive a process restart or one database-node failure within the region. Candidate lists and session caches may be lost and rebuilt or restarted without losing source posts.

5. Access and paging consistency. Check current eligibility for the exact returned content version, even on cached candidates and older pages. Sessions expire after five minutes; a new session may rank differently. Already released content cannot be recalled.

6. Bounded ranking and degradation. Rank a bounded retrieved pool and disclose that it is not a global optimum. If ranking fails, use eligible chronological results; if permission cannot be established, omit or fail rather than expose content.

04Build a useful feed with one application

Start with a SQL database containing posts, author timelines and typed relationships. Maya sends a create request with key k91. One transaction saves p882, its author-history entry, the request result and a publication event. The API acknowledges commit. A retry of k91 returns the same p882; success does not claim that every follower already has a prepared feed.

For Leo’s first feed, the application reads his followed sources, retrieves bounded recent posts from their indexed timelines, checks current eligibility and reply rules, sorts by time and returns twenty. This is fanout on read: the application gathers several author histories when the reader asks. The baseline needs no background copies to produce a correct page.

Store the bounded candidate order for the browsing session before returning its cursor. New posts appear on refresh rather than shifting older page positions. Later pages still recheck deletion, blocks and membership. Media bytes stay in the media service; the feed contains authorized references and summaries.

Chronological ordering is a complete initial ranking policy. Personalization can be introduced once candidate retrieval and permission behavior are clear. The first scaling problem is repeatedly collecting and merging similar author histories for millions of readers.

05Compare repeated reads with recipient writes

Assume 300 million daily active readers, five feed opens per day and 500 followed entities per reader. Use a fivefold peak and twenty stories per page.

Preparing references can reduce repeated reads, but publication then creates recipient writes. If an ordinary author has 500 followers and 40% are active, one post produces about 200 candidate writes. A twenty-million-follower author with the same active fraction produces eight million. At 40 bytes per candidate entry, that is 320 MB of logical mutations for one publication.

The useful comparison is publication frequency × active recipients × write cost versus actual reader requests × author-merge cost. Follower count alone is a starting heuristic. Audience activity and posting rate can change whether precomputation saves work.

06Define publication, refresh and continuation

First-page request

GET /feed?limit=20&excludeReplies=true

Continuation request

GET /feed?cursor=token

Here token stands for the opaque cursor returned for the browsing session.

Derive author and viewer identities from authentication. The request key is scoped to the author and payload; changing the payload while reusing it conflicts. Media references must belong to uploads the author is permitted to attach.

Choose a five-minute session lifetime for this exercise. Its opaque cursor binds the viewer, filters, session and next position; another user cannot reuse it. Expiry requires refresh rather than inventing a continuation from newly ranked data. Restrict page size and total session depth.

A new-stories notification is optional and lightweight. Pull-to-refresh remains the complete delivery path if notifications are lost. Precomputing a server-side candidate list while Leo is offline is different from transmitting unseen stories to his phone.

07Make candidate lists disposable, not authoritative

Source records and durable publication work

If the application crashes between committing p882 and sending a queue message, the outbox relay still discovers the pending event.

A candidate stores a reference, not a duplicate body or permission grant.

Make (viewerId, postId) unique so repeated fanout cannot create duplicate visible slots. Partition candidates by viewer for local retrieval, while author timelines are indexed by author and creation-time/post-ID order.

Maintain both following and follower access paths. Feed reads ask which sources Leo follows; publication asks which active viewers follow Maya. Scanning every viewer on each publication would defeat the intended optimization.

Derived serving state

Source posts, relationships and durable events support recovery; candidate caches and session lists cannot replace them.

For the chosen one-node-loss target, authoritative post, relationship and publication-event stores acknowledge changes only after durable majority commits across three replicas in independent regional failure domains. Each store must preserve its own acknowledged decisions on failover; this does not create a global transaction across the social graph. Candidate and session views retain their stated rebuild/restart behavior.

08Push ordinary candidates and pull expensive audiences

For ordinary authors, use publication events to insert post references into active followers’ candidate lists. This is fanout on write: the work happens after publication, before those readers request a page. Reads usually need one bounded candidate lookup instead of hundreds of author queries. Keep only a useful recent window, such as 200–500 candidates, and avoid filling unlimited feeds for inactive accounts.

Popular authors whose fanout would dominate writes remain on a pull path. At read time, merge their bounded recent histories with Leo’s prepared ordinary candidates. Deduplicate post IDs because a policy change or overlapping path can temporarily supply the same story twice. This chosen hybrid trades two retrieval paths for manageable celebrity publication cost.

A returning inactive reader may need a bounded rebuild from current relationships and recent author timelines. Let concurrent requests for the same viewer share one rebuild, so ten browser retries do not trigger ten independent scans. Make the slower cold-start behavior visible rather than promising every abandoned cache can be reconstructed instantly.

Fanout workers process one recipient page at a time, write unique candidate entries and save progress only after those writes can be recovered. A crash can replay the page. Indexing an event as consumed before its candidate writes survive can create missing stories. Replica and cache capacity should follow measured candidate writes per consumed story, not only total user count.

09Rank a bounded eligible pool

Collect, for example, 300 ordinary candidates and 100 from pull-only authors. Remove duplicates, filter replies under the request policy and check eligibility. These are illustrative budgets to evaluate. They bound work; they do not guarantee that the globally best story exists inside this retrieved set.

Start with a lightweight score using declared signals such as recency, prior interaction with the author and topic interest. A more advanced ranker can predict outcomes.

Illustrative scoring rule

score = 2 × P(meaningful interaction)
      + 0.5 × P(save)
      + 0.1 × freshness
      − P(hide)

P means the predicted probability of that outcome, and freshness is normalized between zero and one. The weights are an illustrative product policy.

P(interaction) in the table means P(meaningful interaction) from the rule. The first post ranks higher despite being less fresh.

Break ties deterministically and apply simple diversity rules, such as avoiding many consecutive posts from one author. Freeze the resulting bounded order for the session. Evaluate usefulness, unwanted exposure and whether predicted probabilities match observed outcomes; raw engagement alone can reward the wrong behavior.

10Resolve the late-fanout privacy race

A worker reads that Leo follows Maya at relationship version six, then pauses. Leo unfollows her, committing version seven. The worker resumes and inserts p882 into his candidate cache. That insertion is stale but recoverable: the feed service checks current relationships while serving and excludes Maya’s story from this followed-content feed.

The same principle applies to deleted posts, blocks and restricted groups, with their actual permission rules. Removing stale candidate entries asynchronously reduces wasted work; it is not the decisive access check. An unfollow does not make Maya’s otherwise public profile secret, while losing private-group membership can remove permission to read its content at all.

Authorize the exact content version returned. If a check approved public version one, do not fetch and return a later private version two under that old decision. Fetch the approved immutable version or recheck the changed version within a bounded deadline. If current access cannot be established, omit the story or fail rather than return cached text optimistically.

Authorization is a point in the request: a response already authorized before a subsequent revocation may finish. The next serving check observes the change. A stronger barrier that stops every in-flight response is a different contract, not a property supplied by asynchronous invalidation.

11Recover source-backed views and degrade ranking

A lost candidate cache is not an empty product history. Reconstruct a recent window from durable source posts and current relationships. To avoid missing publications during the scan, record an event-log boundary before rebuilding, replay changes from that boundary and deduplicate before switching to the new list. Detailed multi-partition handoff belongs in the Advanced version, but the need for overlap must be explicit.

If a fanout worker fails halfway through a recipient page, replay its unique viewer/post insertions and then checkpoint. If event processing is behind, show measured freshness degradation, prioritize active readers and bound the queue. A five-minute periodic rebuild by itself cannot meet a five-second new-story target.

If ranking times out, return a bounded chronological page of eligible candidates and identify the fallback. If permission checks fail, that fallback cannot safely reveal uncertain private stories. Lost notifications merely remove the new-stories hint; refresh can still read current state.

Deliver private media through an authenticated media edge that checks access before serving cached bytes. Already downloaded bytes and transfers admitted before a later revocation cannot be recalled. A public long-lived object URL would undermine a correct feed-body permission check.

12Check the design against the requirements

Use the agreed lists to check the finished design. The tests below still need to establish the targets; a proposed mechanism is not a measured result. FR refers to the numbered functional requirements above; NFR refers to the numbered non-functional requirements.

13Rapid revision

Remember: Prepare candidates to save repeated reads, then check current access; an old candidate list cannot grant permission.

Measure publication-to-eligible-feed delay separately from response latency. Track candidate writes per post, active-recipient fraction, candidates scored per page, duplicate/empty pages, permission-filter rate and cache rebuild cost. At the peak estimate, scoring 500 candidates per request means approximately 43.4 million candidate-viewer scores/s, so bounded pools and staged scoring matter.

For a final explanation, follow p882 through committed source state, ordinary fanout or celebrity pull, bounded candidate merge, eligibility and ranking. State that candidate caches are rebuildable views and the product chooses a few seconds of propagation delay. The next tuning decision follows measured audience activity and consumed-story cost, while ranking changes are judged by usefulness and safety rather than engagement alone.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.