Distributed Systems Basics
A distributed system is one where parts run on different machines and cooperate by sending messages over a network. A web application with a browser, a backend, a database and a few third-party APIs is already a small distributed system.
Why a network call is not a function call
The “fallacies of distributed computing” list assumptions that turn out to be false, including that the network is reliable, latency is zero, bandwidth is unlimited and the network is secure. In practice:
- A call over a network takes milliseconds or more (a local function call takes nanoseconds), and the time varies.
- One part can be down or slow while the rest works. A call may time out, and you do not know whether the other side did the work.
- Each machine has its own state and its own idea of time, and there is no shared memory (see Dates, Times, and Time Zones).
Common problems and responses
| Problem | Typical response |
|---|---|
| Call fails or times out | Set timeouts and retry with a growing delay |
| Retry runs the action twice | Make operations idempotent, so repeating them is safe |
| Dependency is down | Fail gracefully, use a cache or fallback, stop calling it for a while (circuit breaker) |
| Caller is faster than receiver | Buffer work in a message queue |
| Data copies disagree | Accept eventual consistency, or use a transaction where one is available |
| Hard to find what happened | Correlation ids and shared logging |
Trade-offs to know by name
- Consistency versus availability: when the network between nodes breaks, a system must choose between answering with possibly stale data and refusing to answer. This is the idea behind the CAP theorem.
- Synchronous versus asynchronous: waiting for a reply is simple but couples the caller to the callee. Messages and events decouple them but are harder to follow.
- Monolith versus microservices: independent services give team and scaling freedom, but turn every internal call into a network call. Many teams start with one deployable application and split later (see Modularity, Coupling, and Cohesion).
You meet all this when calling any web API (HTTP Fundamentals), using a cloud database or following a request across services.
Try it
Write a small script that calls a URL with a short timeout and retries up to three times with a doubling delay. If the request were a payment, what would happen when the first attempt succeeded but the reply was lost?
Common mistakes
- Treating a remote call like a local one: no timeout, no failure handling.
- Retrying a non-idempotent action, such as a payment, and running it twice.
- Splitting into microservices early, turning every internal call into a network call.
Check yourself
- Why is retrying a payment request risky, and what makes it safe?
- What does a message queue help with?
- What choice does the CAP theorem describe?