distributedsystem/books/think-distributed-systems
Mental Models
Knowing vs Understanding
Mental Models
Quote
A mental model is a simplified explanation of how something works.
Any idea, belief, or concept can be boiled down to its essence. Like a map, mental models highlight key information while ignoring irrelevant details. They’re tools for compressing complexity into manageable chunks.
⎯ Mental Models: The Best Way to Make Intelligent Decisions (~100 Models Explained)
Model is a representation of a target system. A mental models is the internal representation of the target system and the basis of comprehension. (ability to Understand) and communication (ability to convey Understanding)
- System -> A system can be understood as a set of facts that constitutes the ground truth.
- Mental Model -> A mental model can be understood as a set of facts that constitutes out perceived truth.
A good mental model is a mental model that is correct and complete:
- Correctness -> In a correct mental model, every fact of the model is a fact of the system. In other words, a correct mental model does not contain falsehoods.
- Completeness -> In a complete mental model, every relevant fact of the system is a fact of the model. In other words, a complete mental model does not contain omissions.
No model captures every fact of a system. Therefore, we define completeness as capturing every relevant fact of the system. Relevance is application-specific. In other words, relevance lies in the eye of the beholder.
Problem of the Analogy
Different types of Models
There are many models to explain one system.
This diversity adds complexity but also provides depth, with each mental model providing a unique perspective. Studying multiple mental models helps us gain a holistic
understanding and allows us to identify omissions or inconsistencies in our thinking.
- Different models describing the same aspects of a system
- Different models describing different aspects of a system
If you encounter different models of a distributed system, that doesn’t mean that some of them are incorrect. When you read posts, articles, or papers, try to understand not only the model chosen by the author but also the reasons behind their choice.
Thinking about Distributed Systems
Every modern application is a distributed application.
A distributed system is a collection of collaborating, concurrent components that communicate with one another by sending and receiving messages over a network. Each component has exclusive access to its own local state, which other components cannot access. The network has exclusive access to its own local state, including messages that are in flight.
- The behavior of a distributed system emerges from the behavior of its components and their interactions.
- The complexity of a distributed system emerges from the complexity of its components and the intricacy of their interactions.
An accurate, concise way to think about the behavior of a distributed system is as a state machine.
In the most general terms, the system proceeds in discrete steps. Each step is taken by either a component or the network. These steps can be categorized as follows:
- External steps -> Actions such as receiving a message or sending a message
- Internal steps -> Actions such as performing local computations or accessing the local state
Correctness of the System
Lamport defines the correctness of a system in terms of its safety and liveness properties: (proving.pdf)
- Safety -> Informally, a safety property guarantees that something bad will never happen.
- Liveness -> Informally, a liveness property guarantees that something good will eventually happen.
Scalability and Reliability
- Scalability of a system is defined as its ability to be responsive in the presence of load.
- Reliability of a system is defined as its ability to be responsive in the presence of failure.
02 - Scalability, DS12.1 - Fault Tolerance > Reliability and Availability
Responsiveness
Responsiveness of a system as the ability of the system to keep its error rate below its error budget.
- Service-level indicator -> A quantitative observation about the behavior of a system.
- Service-level objective -> A predicate (a function that yields true or false) on a servicelevel indicator that determines whether the behavior of a system meets an objective.
- Error rate -> The ratio of the number of observations that do not meet their objectives to the total number of observations for a fixed time interval.
- Error budget -> An acceptable (to us) upper limit on the error rate.
Two Big Ideas
1. Systems of systems
A distributed system consists of concurrent, communicating components.
However, components are often not simple atomic units. Instead, they are usually subsystems composed of smaller interacting parts.
2. Global view vs Local view
Global View
An external observer can see:
- Every component
- Every state
- The entire system
This complete knowledge is called the global view.
Local View
Each component can only see:
- Its own state
- Its network connection
A component cannot directly know the state of the entire system.
Important Consequence
To understand the overall system state, components must exchange information through communication.
Distributed Systems Incorporated (Mental Model)
The book introduces a corporation analogy.
Mapping
| Corporation | Distributed System |
|---|---|
| Building | Entire system |
| Employee | Component |
| Room | Local state |
| Pneumatic tubes | Network |
| Mailbox | External interface |
| This analogy helps visualize abstract distributed-system concepts. |
Failure Models
Crash Semantics
Employee absences represent failures:
| Employee Situation | System Failure |
|---|---|
| Short break | Transient failure |
| Vacation | Intermittent failure |
| Leaving company | Permanent failure |
Message Delivery Semantics
Mailroom mistakes represent network behavior:
| Mailroom Behavior | Network Behavior |
|---|---|
| Loses message | Message loss |
| Sends twice | Duplicate delivery |
| Changes order | Reordered delivery |
Navigating Complexity
1. Simple Components Can Create Complex Systems
Even if each component follows simple rules, their interactions can create highly complex behavior.
2. Emergent Behavior
Important properties do not come from individual components.
Examples:
- Reliability
- Scalability
- Fault tolerance
These emerge from the interactions among components.
Key Idea: Reliable systems can be built from unreliable components.
3. Changing Resolution vs. Changing Perspective
Black-Box View -> Treat the system as a single entity.
White-Box View -> Examine the internal components.
Both are still global viewpoints.
True Perspective Change
Moving from: Observer’s global view to Component’s local view This is the fundamental shift in distributed systems thinking.
Think Globally, Act Locally
A distributed system must achieve global goals even though each component only has local information.