Nimendra's Notes 🪴

Time Series Data & TSDB

3 min read

kubernetes database devops/observability/prometheus kubernetes/extras

Core Idea

Time series data is time-ordered points, and a TSDB exists to hold it: fast high-ingest writes, time-range queries, compression, and retention policies that a general database does not give you.

  • TSDB features: bulk ingestion, aggregations over windows (min, max, average, percentiles), compression for repeating patterns, downsampling, and a tag-based data model.
  • Why not relational: performance at millions of points per second, built-in roll-ups and window functions, and irregular intervals spanning years.
  • Examples in the note: InfluxDB (InfluxQL), Prometheus (monitoring and alerting), and TimescaleDB (PostgreSQL-based).

What is Time Series Data?

Time series data is a sequence of data points collected, recorded, or measured at successive time intervals. The primary characteristic of time series data is that it is time-ordered, meaning each data point is associated with a specific timestamp.

Characteristics of Time Series Data

  1. Temporal Order: Data is captured in chronological order (e.g., hourly, daily, monthly).
  2. Continuous or Discrete: Can be collected at regular intervals (e.g., every second) or irregular intervals (e.g., event-driven data).
  3. Examples:
    • Stock prices recorded every minute.
    • Weather data such as temperature and humidity measured hourly.
    • IoT sensor readings (e.g., electricity usage, heart rate).
    • Application performance metrics (e.g., CPU usage over time).

What is a Time Series Database (TSDB)?

A time series database is a specialized database optimized for storing, retrieving, and analyzing time series data. Unlike traditional relational databases, TSDBs are built to handle high-ingest rates, large volumes of data, and queries focused on time ranges.

Key Features of a TSDB

  1. Efficient Data Ingestion:
    • Handles high write speeds as time series data is typically generated in real-time.
    • Optimized for bulk insertion of data points.
  2. Time-Optimized Queries:
    • Allows fast querying of time ranges, such as “last 24 hours” or “monthly averages.”
    • Supports aggregations like min, max, average, and percentiles over time windows.
  3. Compression and Storage:
    • Uses compression techniques to reduce storage costs since time series data often has repeating patterns.
    • Can store long-term historical data efficiently.
  4. Retention Policies:
    • Enables automatic deletion or downsampling of older data to manage storage.
  5. Data Model:
    • Often uses a tag-based model where data points are associated with metadata (tags) for easier querying (e.g., device ID, location).

Why Use a Time Series Database?

  1. Performance: Designed for high-speed writes and efficient time-based queries.
  2. Scalability: Can handle millions of data points generated per second.
  3. Specialized Functions: Built-in features for time-series analysis (e.g., roll-ups, interpolations, window functions).
  4. Flexibility: Supports irregularly spaced time intervals and large datasets spanning years or decades.

Examples of Time Series Databases

  1. InfluxDB:
    • Open-source and widely used TSDB.
    • Designed for metrics, events, and logs.
    • Query language: InfluxQL (similar to SQL).
  2. Prometheus:
    • Open-source monitoring system and TSDB.
    • Optimized for real-time system and application monitoring.
    • Works well with alerting systems.
  3. TimeScaleDB:
    • Built on PostgreSQL, providing relational features combined with time-series optimization.
    • Easy to integrate with existing systems using SQL.

Backlinks

Graph View