database/redis distributedsystem systemdesign

TOC

  1. Redis Explained
  2. Redis Underlying Data Structures
    1. Title Unavailable | Site Unreachable
  3. Redis Serialization Protocol(RESP)
    1. Redis serialization protocol specification | Docs
  4. Redis PubSub and Implementation
  5. Redis Queue
  6. Redis Reliable Queue Pattern
  7. Redis Data Persistence
  8. Redis HSET, EXPIRE & TTL
  9. Redis Streams
  10. Redis Geospatial
  11. Redis Sorted Sets
  12. Redis Sentinal
  13. Redis Cluster
  14. Redis Hyperloglog

Introduction

Redis (β€œREmote DIctionary Service”) is an open-source key-value database server.

Transclude of row-based-and-column-based-database#memory-vs-disk-based-databases


Queues, Pub/sub, and Streams

FeatureRedis Queue aka Redis ListRedis Pub/SubRedis Streams
PurposeTask/job scheduling and background processing.Real-time messaging between publishers and subscribers.Real-time data processing with persistence and consumer groups.
Data ModelFIFO (First In, First Out) queue.Publish-subscribe messaging system.Append-only log of key-value pairs.
PersistenceNo inherent persistence; tasks are lost if not processed before a crash.No persistence; messages are lost if subscribers are offline.Persistent; entries remain in the stream until explicitly trimmed.
ScalabilityCan distribute jobs across multiple workers for horizontal scaling.Scales well with a large number of publishers and subscribers.Supports multiple consumers via consumer groups for distributed processing.
ReliabilitySupports job retries, prioritization, and dependencies for reliable task execution.Fire-and-forget; no guarantees on message delivery or retries.Provides at-least-once delivery with acknowledgment mechanisms for reliability.
Use Cases- Background job processing
- Task scheduling
- Distributed workloads
- Job prioritization and dependencies.
- Real-time notifications
- Chat applications
- Live updates
- Event broadcasting.
- IoT data ingestion
- Real-time analytics
- Message brokering
- Logging/audit trails.
- Activity feeds.
Consumer ModelWorkers explicitly pull tasks from the queue (pull-based).Subscribers passively receive messages when published (push-based).Consumers actively read entries from the stream; consumer groups enable parallel processing (pull-based).
Message RetentionTasks remain in the queue until processed or manually deleted.Messages are ephemeral and only delivered to active subscribers at the time of publishing.Messages persist in the stream until explicitly trimmed or expired based on retention policies.

Redis High Availability

High Availability (HA) in Redis ensures that the database remains operational even if some nodes fail. It is crucial for applications that require minimal downtime and data consistency.
08 - ESB & ASG > Scalability & High Availability

Redis provides two primary HA solutions:

  1. Redis Sentinal – Provides automatic failover but does NOT support Database Sharding.
  2. Redis Cluster – Provides both failover and data sharding for scalability.

Redis Sentinel Vs. Redis Cluster

Redis Sentinel vs Clustering | Baeldung

FeatureRedis SentinelRedis Cluster
PurposeAutomatic FailoverFailover + Data Sharding
Data Sharding❌ Noβœ… Yes
Replicationβœ… Master-Replicaβœ… Master-Replica
Failover Handlingβœ… Yesβœ… Yes
Minimum NodesAt least 3 SentinelsAt least 3 Masters + Replicas
Best ForFailover & HA onlyScalability + HA