Design a file synchronization service

01Problem and scope

A file synchronization service propagates saved file revisions across devices while preserving edits made offline. The server must record which revision is current, retain committed changes and decide what happens when edits conflict. Copying bytes alone does not solve those problems. For example, clients A and B both edit budget.xlsx from revision 12. If client B commits revision 13 first, client A’s later upload must preserve its conflicting work rather than silently overwrite revision 13.

A revision describes one immutable saved version. A chunk is a piece of file bytes; a revision's manifest lists chunks in order. A 9 MiB file can contain two 4 MiB chunks and a final 1 MiB chunk. If the middle chunk changes, the other two can be reused. A stable file ID survives renames; its path does not. These definitions separate identity, history and storage.

A workspace is the shared collection of files, folders and membership permissions governed by one metadata authority in this design. Choosing it as the transaction boundary lets a file change, its access checks and the corresponding history entry succeed together.

Scope the service to general files up to 1 GiB, shared folders, offline editing and version history. Preserve conflicting binary edits as conflict copies; automatic merging requires file-format-specific semantics. Publication is atomic within one workspace. A cross-workspace move is an explicit copy/delete workflow rather than a global transaction. Character-level simultaneous editing is excluded, and clients expose pending and conflict states when synchronization cannot immediately complete.

02Functional requirements

The server records committed changes in an ordered change log. A device's cursor identifies how far it has safely applied that history. Deletion adds a tombstone, a retained record that tells offline devices to remove a file when they catch up; simply erasing the server row would lose that instruction.

1. Commit file revision: All referenced chunks exist durably before the revision becomes current.
2. Handle concurrent edits: An unseen current revision is not silently overwritten.
3. Rename/move: Stable file identity and an atomic workspace metadata change.
4. Delete: A tombstone reaches devices; retained history follows stated retention.
5. Catch up: Apply every committed change after the device's cursor or request a fresh snapshot.
6. Share/revoke: Server checks current grants at commit and before issuing downloads.

Devices, folders and offline work

Users can create folders, upload, download, rename, move within a workspace, delete, restore retained revisions and share with readers or writers. Desktop clients automatically watch selected folders; mobile clients may list metadata immediately and download bytes on demand. Each device remembers its applied change cursor and unsent edits so reconnecting does not depend on receiving every live notification.

Local versus server completion

“Saved locally,” “uploading” and “synchronized” are distinct statuses. Client A can close the device while work is pending; the local journal must survive restart. A successful server commit returns a revision and a change sequence. Downloading to another device may still be pending. An online hint accelerates discovery but is not the only record of a change. A snapshot for an expired cursor must identify its corresponding log position so concurrent changes are neither skipped nor lost.

03Non-functional requirements

1. Latency: Metadata commit p95 below 300 ms inside a region; ordinary connected-device discovery within five seconds.
2. Transfer time: File size and bandwidth govern byte completion. A 1 GiB upload over 20 Mb/s takes over seven minutes even without overhead.
3. Availability: 99.9% eligible metadata availability. Preserve correctness during conflicting writes or loss of the workspace's authoritative majority.
4. Durability: An acknowledged revision, including metadata and referenced chunks, survives one node or availability-zone failure. Replica placement must match this promise.
5. Regional recovery: Use a separate, initially asynchronous disaster-recovery policy with a tested recovery point and restore procedure.
6. Retention: Keep old revisions and the change log for an illustrative 30 days. Longer legal/product retention has a separate cost; devices beyond the log horizon must resnapshot.
7. Authorization: Recheck grants at final commit and before issuing downloads. Revocation during a long upload prevents publication into the workspace.

Commit and recovery invariants

These limitations belong in the product contract. Calling the database ACID does not establish cross-device completion, current permission checks, or the lifetime of already issued download capabilities.

04Capacity estimates

Workload assumptions and arithmetic

Assume 500 million accounts, 100 million daily active users, three devices per account and 200 files/account averaging 100 KB. That gives 500M × 200 = 100B files and 100B × 100 KB = 10 PB of current logical content. Versions, replicas and deduplication change the physical total. At an assumed 1 KB of metadata per file, file metadata alone is 100 TB before indexes.

Add five committed changes per active user/day: 100M × 5 / 86,400 = 5,787 commits/s, or about 28,935/s at fivefold peak. At 200 KB of changed bytes per commit, mean upload ingress is about 5,787 × 200 KB = 1.16 GB/s. If each change reaches two other devices, mean change deliveries approach 11,574/s before larger shared folders. Downloads depend on active-device behavior, not merely registered-device count.

Worked estimates

Capacity implications and limits

A connections-per-minute figure measures establishment rate, not the number of concurrently active sockets; model these quantities separately. Chunk reuse offers substantial savings for a changed region of a large file, but many 100 KB files fit in one chunk, and compressed spreadsheets may change widely. Measure transferred bytes per committed revision before claiming a fixed deduplication percentage.

05APIs and contracts

Request and response example

Client A starts POST /workspaces/w7/files/f42/uploads with {"baseRevision":12,"requestId":"edit-77","size":9437184}. The server returns session up8, expiry and a list of chunk upload targets or already available workspace-scoped chunks. A manifest commit names ordered chunk identities, expected lengths and total checksum. An uploaded chunk alone never changes the visible file.

Interface contracts

Validation and response semantics

A conflict returns 409 with the stable conflict-copy identity and current revision; a retry of edit-77 returns the same result. A missing/expired session returns an explicit retry-required error, not permission to publish unprotected chunks. A cursor older than retention returns a resnapshot requirement. Pagination advances only through a committed prefix: allocating sequence 881 before commit and letting 882 become a returned cursor could permanently skip 881. Our workspace owner allocates and publishes sequence positions in serialized metadata transactions.

06Data model and access patterns

Keep the stable file identity separate from its immutable revision history. Each revision names chunks that may also belong to other retained revisions, so the server needs records for both the bytes and their continued use. Upload sessions protect pending work; change-log entries let other devices discover a committed result.

The same workspace metadata group controls the rows needed to publish and protect chunks. A session pin is a record preventing a chunk from being deleted while an active upload may publish it; a retained reference protects it while a saved revision still uses it. Device cursors acknowledge how far each device has applied the change log; they are not the only history.

Index folder children by (workspaceId,parentId,name) and changes by (workspaceId,sequence). A rename changes the directory entry while preserving f42. Manifests are immutable; restoration creates a new current revision rather than editing old history. The query WHERE workspaceId='w7' AND sequence>880 ORDER BY sequence LIMIT 1000 reads the change log, with a consistent upper committed watermark.

Chunks live in private object storage keyed by workspace and verified identity. Deduplication within that boundary can reuse equal chunks; cross-tenant deduplication is deliberately deferred because a global “does this hash exist?” endpoint can disclose private content. Metadata is authoritative for references and permissions; caches and notification directories are derived. If a giant workspace outgrows one metadata owner, splitting its transactions requires an explicit new protocol. Hashing file IDs across databases without preserving workspace invariants is not a free scale improvement.

A namespace mutation also enforces a unique directory entry (workspaceId,parentId,normalizedName) under the chosen case/Unicode policy. Within the serialized workspace transaction, verify that the target is a folder, the actor can modify both source and destination, and a folder is not moved beneath itself or a descendant. Concurrent rename/move validation must share that serialization; checking the tree before taking the namespace lock leaves a cycle race. File IDs remain stable while directory entries change.

07Basic working design

Whole-file revision commit

A minimal product uses one API, one transactional metadata database and durable file storage. Client A uploads the full 9 MiB file under an immutable temporary revision key. The API verifies it, then transactionally checks base revision 12, writes revision 13, points f42 to 13 and appends change 881. Only after commit does it report that the server saved the revision. If the file transfer fails, no new revision is visible; if commit response is lost, request edit-77 retrieves the saved result.

Device catch-up and local replacement

Client B polls changes after cursor 880, downloads revision 13 to a temporary file, verifies it and replaces the local copy after preserving any unsent edits. It updates its local journal/cursor so a crash can replay safely. The filesystem and local metadata database do not share one atomic transaction: journal the intended replacement, perform the verified rename, then finalize metadata, checking on restart whether that revision is already applied.

Why this baseline is useful

This baseline transfers whole files and polls periodically. It can be useful for a small team and already handles the essential revision conflict. Backups include both metadata and immutable files. Adding chunking or WebSockets later should improve efficiency, not be required to rescue an unclear correctness model. The initial commit and retry record remain the anchor for the rest of the interview.

08Find the baseline flaws

Whole-file transfer is the baseline's efficiency limit. The other examples show why two tempting shortcuts—accepting the last upload and using the largest allocated sequence as a cursor—would break the conflict and recovery guarantees already established. Scaling must retain those guarantees.

09Improve the design, step by step

1. Add fixed-size chunk transfer and a local client index. The trigger is repeated whole-file retransmission. A chunker splits large files, an indexer compares manifests, a watcher reports filesystem events, and a local database remembers revisions and pending operations. Only changed chunks move; retries repeat at most a chunk. Costs are client CPU, metadata and shifted boundaries after insertions. Whole-file transfer remains simpler for small files; content-defined boundaries become worthwhile if large shifted files dominate measured traffic.

2. Separate byte endpoints from metadata and add protected upload sessions. Long transfers trigger separate block-serving capacity with health-aware balancing, bounded buffers and resumable sessions. Metadata commits stay responsive while bytes move. Cleanup could now delete a chunk just before publication. In one metadata transaction, replace the upload session’s protection with the saved manifest’s references. A single pool is preferable until concurrency measurements justify isolation.

3. Use a durable change log plus notification gateways. Empty polls from millions of devices trigger long polling or persistent connections. Gateways send a lightweight “changes available” hint, and clients catch up using their cursors. This lowers discovery delay without storing infinite private queues. It costs connection memory, heartbeats and reconnect management. Per-device durable response queues are a valid architectural alternative, but require bounded retention and cleanup; the log offers shared recovery history.

4. Partition by workspace and replicate its authority. The 100 TB metadata estimate and 29,000 peak commits/s trigger many logical workspace partitions. Each has a replicated owner; different workspaces proceed independently. This preserves local transactions but makes an exceptionally large workspace a hot owner. Alternatives include directory/file partitions with an explicit shared-log and transaction protocol, or a distributed transactional database. Neither “consistent hashing” nor a NoSQL label automatically fixes one hot workspace or supplies global ACID behavior.

Fixed-size chunking places boundaries at fixed byte offsets, so inserting bytes near the start can change many later chunks. Content-defined chunking chooses boundaries from patterns in the content instead; unchanged regions can then keep matching even after their offsets shift. It can reduce retransmission for that workload, but requires more boundary-detection work and measurement.

Add chunk and manifest caches only where reuse is measured. A byte cache should not make a current permission decision, and many cold small files may be cheaper to serve directly.

10Detailed architecture

Desktop client responsibilities

The desktop client contains four responsibilities, not four mandatory backend services: watcher detects local changes, chunker computes/reconstructs pieces, indexer schedules uploads/downloads, and the local database records manifests, revisions and durable pending work. The client also merges remote metadata with unsent local edits and shows conflicts. Mobile clients use the same revision protocol while choosing lazy byte download.

Workspace metadata and chunk authority

On the server, metadata APIs route workspace w7 to its current owning replica group. That group owns file heads, manifests, chunk-reference/pin metadata, membership, request results and the ordered change log. Byte gateways authorize exact chunk operations and serve private object storage through an optional cache. Object durability and metadata durability have separate implementations but meet the same acknowledged-failure contract.

Notification hints and session directory

An outbox or committed-change relay sends hints to notification gateways, whose directory maps devices to live connections. Hints may be duplicated or missed. Devices recover from the shared log, so an offline device does not require an unbounded gateway queue. Cleanup workers act through the metadata authority before deleting unreferenced chunk generations. Background deduplication, if used, likewise changes references through authority rather than silently swapping arbitrary bytes.

Commit boundary and independent recovery

The server replies after one transaction saves the manifest, current revision, change entry and request result. Other devices catch up, caches fill and unused bytes are removed afterward. A workspace migration copies and catches up its log, fences the old ownership epoch, and then switches routing; merely changing a configuration pointer risks two concurrent owners.

Capture a stable local byte version

Before hashing or uploading, the client must capture one stable local byte version. Use an immutable staging snapshot or copy made under an appropriate filesystem/application lock; read every chunk from that captured version. Hashing a live file while an application rewrites it can otherwise produce a manifest mixing two saves even when each chunk checksum is valid. When the platform cannot provide a reliable capture, detect concurrent modification and retry or show a pending/conflict state rather than promising an application-consistent snapshot from a watcher event alone.

11Write path and acknowledgement

Before making a revision current, the server checks permission and the expected base revision, protects its durable chunks, and saves a change entry in the same commit. The successful trace below uses file f42 in workspace w7, request edit-77, and current base revision 12; the later conflict example considers an intervening commit.

1. Client A's watcher reports a change. Its local database records that f42 began at revision 12 with [cA,cB,cC]. The chunker produces [cA,cD,cC], and the indexer journals edit-77 before sending network work.
2. The server authorizes client A and reserves session up8 with base 12, a finite lease and pins for reusable chunks cA/cC. It rejects chunk claims outside w7's authorized namespace.
3. Client A uploads only cD. The byte endpoint verifies length/checksum and records the immutable object's durable availability. The session pins protect the chunk while publication remains allowed.
4. Commit locks the session, f42, workspace sequence state and referenced chunk metadata in a deterministic order. It verifies the active session, current grant, all chunk states and currentRevision=12.
5. In one transaction, create revision 13, transfer protection to retained manifest references, point f42 to 13, append committed change 881, and save edit-77's successful result. The workspace sequence advances with this commit.
6. After required replication acknowledges, return revision 13/sequence 881. Lost responses recover by edit-77. A conflict records a stable conflict-copy result instead of modifying the winner's head.
7. A change hint is emitted. Chunk reclamation happens later after no active pin or retained manifest references the object. Upload success alone never means that the file became current.

For a 1 GiB file, at most 256 fixed 4 MiB chunks bound manifest validation. Batch operations where safe, but do not remove the checks that prevent a partial manifest from being accepted.

12Read and delivery path

Device synchronization recovers from the durable change log, independently of live notification delivery. This trace advances a second device from cursor 880 through committed change 881 and applies revision 13 safely.

1. Client B receives a hint or reconnects and requests changes after 880. The server verifies current workspace membership and returns committed change 881 with an upper watermark that cannot skip an earlier uncommitted publication.
2. Its indexer journals that f42 revision 13 is to be applied. If it has unsent local changes, preserve them and follow the conflict protocol before replacing local bytes.
3. Fetch revision 13's immutable manifest [cA,cD,cC]. Existing local chunks cA/cC are verified and reused; request only cD with a short-lived workspace-scoped download grant.
4. The byte gateway validates the grant before a cache hit or origin read. A miss fetches private storage, verifies transfer integrity and streams bounded chunks. A hot-file cache changes latency, not permission ownership.
5. Reconstruct to a temporary location, verify the whole intended file, and replace the local file. The recovery journal handles a crash after replacement but before the local metadata update by recognizing the already-applied revision.
6. Advance the durable local cursor only after the change is safely applied or the local recovery journal durably records everything needed to finish applying it after a crash. A repeated page or hint therefore does not create another user-visible revision.

13Correctness deep dive

Concurrent revision outcome

First consider client A and client B competing from base 12. Both may upload valid chunks. The workspace owner serializes their commit transactions. Client B wins and sets the head to 13. Client A's transaction then observes current 13, records its immutable manifest as conflict copy f42-conflict-edit77, and returns that identity without changing f42's head. Retry edit-77 returns the same conflict. The design preserves both byte sets rather than relying on timestamp order.

Chunk collection uses the same authority

The second race is publication versus a chunk collector. Keep chunk-state and protection metadata under the same workspace transaction boundary as manifest publication.

Publication versus collection proof

Retained-history references

Retained historical revisions count as references, not just the current file head. Otherwise restoring revision 12 could fail after its old middle chunk cB was reclaimed. The protocol is workspace-local; a later cross-workspace deduplication scheme needs its own reference authority rather than assuming this transaction spans every tenant.

Conflict copies are real publications

14Failure and recovery

15Operations, security, and cost

Commit, conflict and integrity signals

Monitor commit p95/p99, conflict rate by file format, unsynchronized-device age, chunk retry bytes, pending-session age, DELETING backlog and integrity failures. Verify log cursor monotonicity against committed watermarks. An accepted revision with an absent chunk is a critical integrity incident; a delayed notification is a different, recoverable latency incident. Track both instead of aggregating them into one sync-success count.

Chunk-reuse economics

Deduplication economics depend on reuse. For a 9 MiB file whose middle 4 MiB changes, uploading one chunk saves 5/9 ≈ 56% of that upload compared with full retransmission. For an average 100 KB file that changes entirely, fixed chunking saves no bytes and adds bookkeeping. Inline deduplication avoids transfer but adds lookup latency and privacy controls; post-process deduplication keeps ingestion simpler but temporarily stores and transfers duplicates. Hashes locate candidate equality; verify length and, where required, bytes rather than presenting hashing as mathematical uniqueness.

Scoped transfer and authorization

Encrypt transport and storage, scope chunk tokens to object, operation and expiry, and recheck grants at publication. Exclude secret filenames and contents from telemetry. Quotas include historical revisions, not just current file size, or repeated edits can bypass storage budgeting.

Recovery and migration drills

Recovery tests include restoring revision 12 after revision 13 and deletion, restarting after local rename but before cursor persistence, reconnecting beyond log retention, and racing session expiry with commit. Migration tests copy a workspace, replay committed changes, compare directory listings/manifests, fence the old owner and verify no stale epoch can publish. Repartitioning a hot workspace is a product-level consistency change requiring explicit design review, not routine modulo arithmetic.

Delta encoding inside a changed chunk

A further bandwidth optimization is delta encoding inside a changed chunk: upload a patch against an explicitly identified retained base instead of the whole chunk. The receiver reconstructs the full new immutable chunk and verifies its checksum before it becomes eligible for a manifest. This differs from reusing an unchanged chunk. Patches add CPU, base-retention dependencies and retry complexity, so use them only when measured small edits save enough bytes.

16Decision ledger and limitations

Functional partitioning into separate user, file and chunk stores simplifies ownership but can make joins and atomic checks cross databases. Alphabetical pathname ranges support some ordered scans but skew as popular prefixes grow and renames move keys. Hashing file IDs spreads independent records but requires directory indexes and may break workspace-local transactions. Consistent hashing reduces movement when owners change; it does not solve one hot file or workspace by itself.

Caches can retain manifests and popular chunks with size-aware admission and least-recently-used (LRU) eviction, but a 4 MiB chunk consumes far more memory than a small metadata row. Health checks and load-aware admission, not round-robin alone, protect overloaded byte endpoints. No chosen store is exempt from its actual transaction and replication contract; modern key-value systems may provide conditional writes or transactions, and partitioned SQL remains viable.

17Interview closing

“I designed a shared file workspace that preserves offline edits and makes every accepted revision recoverable. Each commit checks its expected base revision. An intervening edit creates a recoverable conflict result rather than silently overwriting unseen work. The client has a watcher, chunker, indexer and durable local journal, while the server separates byte transfer from workspace metadata authority.

“Chunks upload first under protected sessions. A workspace transaction validates membership, base revision and protected chunks, then commits the manifest, current head, request result and ordered change entry. Device hints only accelerate discovery; cursors over a committed prefix recover missed notifications. Collection cannot delete a chunk that publication is still allowed to reference.

“The workload is around 5,800 average commits per second and ten petabytes of current logical bytes, so I partition independent workspaces and measure chunk reuse. The main remaining bottleneck is a very large workspace, not the number of WebSocket servers. My next tests are concurrent binary edits and restoration after garbage collection.”

If the interviewer demands automatic spreadsheet merging, ask which format semantics and conflict rules are acceptable. That requirement needs a merge-aware document service; last-write-wins plus a sync transport does not satisfy it.

Recall the key ideas

Answer from memory before opening each card. Explain why the choice works and what it costs. Revisit missed cards tomorrow.