systemdesign

Introduction

API Servers: Stateless

  • API servers do not maintain state between requests.
  • If one goes down, it’s non-critical β€” a new instance can spin up automatically (auto-scaling).
  • Statelessness enables horizontal scalability and resilience.

Databases: Stateful

  • Databases maintain state (data).
  • If a database goes down, it can lead to critical outages.
  • Worst-case scenario: disk crash resulting in data loss.

Building Resilience: Data Redundancy

The only way to protect against data loss is to create multiple copies of it.

Types of Redundancy

  • Row/document-level
  • Table-level
  • Database-level

Where Redundant Data Can Be Stored

  • Different tables
  • Different databases
  • Different geographical regions

Backup and Restore Strategy

Backup is a basic and essential form of redundancy.

Best Practices

  • Daily incremental backup
  • Weekly full backup
  • Store backups across regions (offsite backups) <β€” For Disaster recovery
  • In the event of failure: restore from the most recent backup

This is often the easiest and most widely used approach.

Continuous Redundancy (Replication)

Real-time or near-real-time duplication of data across databases.

Types of Replication

  1. Synchronous Replication
    • API writes to both primary and replica at the same time.
    • Ensures strong consistency.
    • Higher write latency.
  2. Asynchronous Replication
    • API writes only to the primary.
    • Changes are later copied to the replica.
    • Lower latency, but potential data lag.

How It Works

  • A replica (standby database) is kept updated in real-time or near-real-time.
  • If the primary database fails, the replica can take over almost instantly (failover).
  • Replica may not handle production traffic until a switchover occurs.

🌟 Leader Election for Auto Recovery