The Fallacies of Distributed Computing

distributedsystem systemdesign softwarearchitecture booknotes fundamentalsofsa

‘Monolithic Versus Distributed Architectures’
—Mark Richards, “Fundamentals of Software Architecture”

Introduction

Distributed architecture(DA) is more powerful than monolithic, but it have significant trade-off. The first group of issues facing all distributed architectures are described in the fallacies of distributed computing,

A fallacy is the use of invalid or otherwise faulty reasoning in the construction of an argument that may appear to be well-reasoned if unnoticed.

1. Reliable Network
2. Zero Latency
3. Infinite Bandwidth
4. Secure Network
5. Fixed Topology
6. Single Administrator
7. Zero Transport Cost
8. Homogeneous Network

---

1. The Network Is Reliable

All distributed architecture styles rely on the network for communication to and from services, as well as between services.
In the DA we assume that the network is reliable but it’s not.

This is why things like timeouts and circuit breakers exist between services.

The more a system relies on the network (such as microservices architecture), the potentially less reliable it becomes.

👉 Split Brain Condition

2. Latency Is Zero

t_remote will always be greater that t_local.

Latency in any distributed system architecture is not zero. yet most architects ignore this fallacy, insisting that they have fast networks.

When using any distributed architecture, architects must know this latency average.

Assuming an average of 100 milliseconds of latency per request, chaining together 10 service calls to perform a particular business function adds 1,000 milliseconds to the request! Knowing the average latency is important, but even more important is also knowing the 95th to 99th percentile. While an average latency might yield only 60 milliseconds (which is good), the 95th percentile might be 400 milliseconds! It’s usually this “long tail” latency that will kill performance in a distributed architecture.
Tail Latency

3. Bandwidth is Infinite

Bandwidth usually not a concern in monolithic architectures.

Stamp Coupling

https://wiki.c2.com/?StampCoupling

Two modules are stamp coupled if they communicate via a passed data structure that contains more information than necessary for them to perform their functions

Stamp coupling can be resolved in the following ways:

• Create private RESTful API endpoints
• Use field selectors in the contract
• Use GraphQL to decouple contracts
• Use value-driven contracts with consumer-driven contracts (CDCs)
• Use internal messaging endpoints

4. Network Is Secure

Most architects and developers get so comfortable using virtual private networks (VPNs), trusted networks, and firewalls that they tend to forget about this fallacy of distributed computing: the network is not secure.

5. Topology Never Changes

This fallacy assumes that network topology is fixed and unchanging, but in reality, it constantly evolves. When architects fail to account for network changes, unexpected issues can arise, such as service timeouts due to altered latency. A minor network upgrade can disrupt assumptions, leading to cascading failures. To prevent this, architects must maintain ongoing communication with operations and network administrators to stay informed about changes and adjust systems accordingly.

this fallacy leads directly to the next fallacy.

6. There Is Only One Administrator

Architects all the time fall into this fallacy, assuming they only need to collaborate and communicate with one administrator.

7. Transport Cost is Zero

This fallacy assumes that making a simple RESTful call has no financial cost, but in reality, distributed architectures are significantly more expensive than monolithic ones. Increased costs stem from additional infrastructure needs, such as hardware, servers, gateways, and security measures. Architects must analyze server and network capacity, bandwidth, latency, and security zones before transitioning to a distributed system to avoid unexpected expenses.

8. Network is Homogeneous

Most architects and developers assume a network is ¹homogeneous—made up by only one network hardware vendor. Nothing could be farther from the truth. Most companies have multiple network hardware vendors in their infrastructure, if not more.

Footnotes

1. of the same kind; alike. ↩