Distributed Systems

A database migration becomes a distributed-systems problem as soon as the application must keep accepting changes. A bulk copy can move records, but it cannot determine which writer is authoritative, whether a delayed event is still valid, or what happens when a successful write loses its response. Those questions define the project more precisely than the choice of destination database. Consider a multi-tenant service whose read path assembles frequently requested objects from several document collections. Background imports compete with interactive requests, and a small engineering team operates the storage stack. The redesign separates authoritative records from query projections, moves the primary access path from MongoDB to DynamoDB, and uses Redis selectively for reusable computation. ...

An application can run out of database capacity while most of its connections are idle. Every independently scaled worker brings a connection pool, every pool reserves some headroom, and a deployment temporarily runs both generations. The database sees the sum of those decisions. No individual service needs to be unusually busy for the aggregate to become expensive. Consider a shared data access service used by short-lived workers and long-running applications. It exposes bounded record operations over MySQL and caches reusable reads in Redis. The engineering problem is to absorb changing client concurrency without allowing connection growth, serialization work, and cache misses to amplify one another. ...

Consistent Hashing for Distributed Systems Many distributed systems need a deterministic answer to a simple question: given a key, which node should own it? A stateless load balancer can choose the least-loaded backend for every request. A cache, shard map, or stateful service cannot. Sending the same key to different nodes destroys locality, while moving too many keys during a resize can overload storage and downstream databases. Consistent hashing is a family of techniques for balancing four goals: ...