Hybrid Logical Clocks: Spanner Without Atomic Clocks
TrueTime requires Google-scale infrastructure, the kind of setup most distributed systems teams will never have. CockroachDB and YugabyteDB achieve similar guarantees using Hybrid Logical Clocks (HLC): a combination of physical wall-clock time and a logical counter. The physical component tracks real time. The logical counter tracks causality within the same millisecond. The HLC on each node advances to max(local HLC, received HLC) + 1 on every message exchange, ensuring causal consistency even when clocks drift. CockroachDB adds an uncertainty interval: when a transaction reads data, if a conflicting write occurred within the uncertainty window (50-150ms), CockroachDB either restarts the transaction with a later timestamp or waits for the uncertainty interval to expire. The result: serializable isolation across a geo-distributed cluster without atomic hardware.
NTP clock skew on commodity servers: typically 10-100ms. Spanner's TrueTime: under 7ms with atomic clocks.
HLC physical component: wall clock time. Advances monotonically, resets on restart only if clock is moved backward.
HLC logical component: increments when two events share the same physical timestamp, preserving causality.
Uncertainty interval: the range of time within which a write could have happened. CockroachDB reads must see all writes within this window.
CockroachDB vs YugabyteDB: both use HLC, both offer PostgreSQL compatibility. CockroachDB optimizes for global distribution, YugabyteDB for lower latency regional deployments.
Choosing a Time-Series Database: A Practical Decision Matrix
Five major TSDB options serve different primary use cases. InfluxDB is purpose-built for metrics with a Flux query language, strong in IoT and monitoring. TimescaleDB extends PostgreSQL: you get SQL, joins, and a mature ecosystem with time-series-optimized storage via hypertables and automatic partitioning by time. Prometheus is a pull-based monitoring system not a general-purpose TSDB: it stores metrics for Kubernetes and application monitoring, with PromQL for alerting and Grafana for visualization. ClickHouse is a columnar OLAP database rather than a pure TSDB, but handles analytical queries over billions of time-stamped rows in seconds. OpenTSDB sits on HBase and scales to billions of metrics per second but requires operating an HBase cluster.
InfluxDB: best for IoT sensor data and custom application metrics. Good client libraries. IOx (V3) uses Apache Arrow/Parquet.
TimescaleDB: best when you want SQL compatibility and are already PostgreSQL-fluent. Continuous aggregates handle downsampling automatically.
Prometheus + Thanos/Cortex: best for Kubernetes monitoring. Pull model, service discovery, PromQL for alerting.
ClickHouse: best for analytical queries over historical time-series data. Not suited for per-metric write volumes above ~100K/sec.
Retention policies: all TSDBs support automatic downsampling (1-second resolution → 1-minute → hourly) to control storage growth.
What Interviewers Test About Advanced Data Systems
Questions about Spanner, Spark, and TSDBs appear in senior and staff interviews for data-intensive systems. They test whether you know the right tool for each problem class and can justify trade-offs quantitatively.
Know: why Spark is better than MapReduce for iterative algorithms, and when you would still prefer Flink over Spark (true streaming vs micro-batch)
Know: the difference between OLTP (many concurrent short transactions) and OLAP (few long analytical queries) and why they need different storage engines
Know: when to use a TSDB vs a general-purpose database, and what the cost of NOT using a TSDB is at scale
Know: how Spanner's external consistency differs from serializable isolation in a single-node database
Know: why NewSQL (Spanner, CockroachDB) is not just 'distributed SQL': it requires rethinking transaction costs across geographic distances
Hybrid Logical Clocks: Spanner Without Atomic Clocks
TrueTime requires Google-scale infrastructure. CockroachDB and YugabyteDB achieve similar guarantees using Hybrid Logical Clocks (HLC): a combination of physical wall-clock time and a logical counter. The physical component tracks real time. The logical counter tracks causality within the same millisecond. The HLC on each node advances to max(local HLC, received HLC) + 1 on every message exchange, ensuring causal consistency even when clocks drift. CockroachDB adds an uncertainty interval: when a transaction reads data, if a conflicting write occurred within the uncertainty window (50-150ms), CockroachDB either restarts the transaction with a later timestamp or waits for the uncertainty interval to expire. The result: serializable isolation across a geo-distributed cluster without atomic hardware.
NTP clock skew on commodity servers: typically 10-100ms. Spanner's TrueTime: under 7ms with atomic clocks.
HLC physical component: wall clock time. Advances monotonically, resets on restart only if clock is moved backward.
HLC logical component: increments when two events share the same physical timestamp, preserving causality.
Uncertainty interval: the range of time within which a write could have happened. CockroachDB reads must see all writes within this window.
CockroachDB vs YugabyteDB: both use HLC, both offer PostgreSQL compatibility. CockroachDB optimizes for global distribution, YugabyteDB for lower latency regional deployments.
Choosing a Time-Series Database: A Practical Decision Matrix
Five major TSDB options serve different primary use cases. InfluxDB is purpose-built for metrics with a Flux query language, strong in IoT and monitoring. TimescaleDB extends PostgreSQL: you get SQL, joins, and a mature ecosystem with time-series-optimized storage via hypertables and automatic partitioning by time. Prometheus is a pull-based monitoring system not a general-purpose TSDB: it stores metrics for Kubernetes and application monitoring, with PromQL for alerting and Grafana for visualization. ClickHouse is a columnar OLAP database rather than a pure TSDB, but handles analytical queries over billions of time-stamped rows in seconds. OpenTSDB sits on HBase and scales to billions of metrics per second but requires operating an HBase cluster.
InfluxDB: best for IoT sensor data and custom application metrics. Good client libraries. IOx (V3) uses Apache Arrow/Parquet.
TimescaleDB: best when you want SQL compatibility and are already PostgreSQL-fluent. Continuous aggregates handle downsampling automatically.
Prometheus + Thanos/Cortex: best for Kubernetes monitoring. Pull model, service discovery, PromQL for alerting.
ClickHouse: best for analytical queries over historical time-series data. Not suited for per-metric write volumes above ~100K/sec.
Retention policies: all TSDBs support automatic downsampling (1-second resolution → 1-minute → hourly) to control storage growth.
What Interviewers Test About Advanced Data Systems
Questions about Spanner, Spark, and TSDBs appear in senior and staff interviews for data-intensive systems. They test whether you know the right tool for each problem class and can justify trade-offs quantitatively.
Know: why Spark is better than MapReduce for iterative algorithms, and when you would still prefer Flink over Spark (true streaming vs micro-batch)
Know: the difference between OLTP (many concurrent short transactions) and OLAP (few long analytical queries) and why they need different storage engines
Know: when to use a TSDB vs a general-purpose database, and what the cost of NOT using a TSDB is at scale
Know: how Spanner's external consistency differs from serializable isolation in a single-node database
Know: why NewSQL (Spanner, CockroachDB) is not just 'distributed SQL': it requires rethinking transaction costs across geographic distances
Related reading
Hybrid Logical Clocks: Spanner Without Atomic Clocks
TrueTime requires Google-scale infrastructure. CockroachDB and YugabyteDB achieve similar guarantees using Hybrid Logical Clocks (HLC): a combination of physical wall-clock time and a logical counter. The physical component tracks real time. The logical counter tracks causality within the same millisecond. The HLC on each node advances to max(local HLC, received HLC) + 1 on every message exchange, ensuring causal consistency even when clocks drift. CockroachDB adds an uncertainty interval: when a transaction reads data, if a conflicting write occurred within the uncertainty window (50-150ms), CockroachDB either restarts the transaction with a later timestamp or waits for the uncertainty interval to expire. The result: serializable isolation across a geo-distributed cluster without atomic hardware.
NTP clock skew on commodity servers: typically 10-100ms. Spanner's TrueTime: under 7ms with atomic clocks.
HLC physical component: wall clock time. Advances monotonically, resets on restart only if clock is moved backward.
HLC logical component: increments when two events share the same physical timestamp, preserving causality.
Uncertainty interval: the range of time within which a write could have happened. CockroachDB reads must see all writes within this window.
CockroachDB vs YugabyteDB: both use HLC, both offer PostgreSQL compatibility. CockroachDB optimizes for global distribution, YugabyteDB for lower latency regional deployments.
Choosing a Time-Series Database: A Practical Decision Matrix
Five major TSDB options serve different primary use cases. InfluxDB is purpose-built for metrics with a Flux query language, strong in IoT and monitoring. TimescaleDB extends PostgreSQL: you get SQL, joins, and a mature ecosystem with time-series-optimized storage via hypertables and automatic partitioning by time. Prometheus is a pull-based monitoring system not a general-purpose TSDB: it stores metrics for Kubernetes and application monitoring, with PromQL for alerting and Grafana for visualization. ClickHouse is a columnar OLAP database rather than a pure TSDB, but handles analytical queries over billions of time-stamped rows in seconds. OpenTSDB sits on HBase and scales to billions of metrics per second but requires operating an HBase cluster.
InfluxDB: best for IoT sensor data and custom application metrics. Good client libraries. IOx (V3) uses Apache Arrow/Parquet.
TimescaleDB: best when you want SQL compatibility and are already PostgreSQL-fluent. Continuous aggregates handle downsampling automatically.
Prometheus + Thanos/Cortex: best for Kubernetes monitoring. Pull model, service discovery, PromQL for alerting.
ClickHouse: best for analytical queries over historical time-series data. Not suited for per-metric write volumes above ~100K/sec.
Retention policies: all TSDBs support automatic downsampling (1-second resolution → 1-minute → hourly) to control storage growth.
What Interviewers Test About Advanced Data Systems
Questions about Spanner, Spark, and TSDBs appear in senior and staff interviews for data-intensive systems. They test whether you know the right tool for each problem class and can justify trade-offs quantitatively.
Know: why Spark is better than MapReduce for iterative algorithms, and when you would still prefer Flink over Spark (true streaming vs micro-batch)
Know: the difference between OLTP (many concurrent short transactions) and OLAP (few long analytical queries) and why they need different storage engines
Know: when to use a TSDB vs a general-purpose database, and what the cost of NOT using a TSDB is at scale
Know: how Spanner's external consistency differs from serializable isolation in a single-node database
Know: why NewSQL (Spanner, CockroachDB) is not just 'distributed SQL': it requires rethinking transaction costs across geographic distances
Hybrid Logical Clocks: Spanner Without Atomic Clocks
TrueTime requires Google-scale infrastructure. CockroachDB and YugabyteDB achieve similar guarantees using Hybrid Logical Clocks (HLC): a combination of physical wall-clock time and a logical counter. The physical component tracks real time. The logical counter tracks causality within the same millisecond. The HLC on each node advances to max(local HLC, received HLC) + 1 on every message exchange, ensuring causal consistency even when clocks drift. CockroachDB adds an uncertainty interval: when a transaction reads data, if a conflicting write occurred within the uncertainty window (50-150ms), CockroachDB either restarts the transaction with a later timestamp or waits for the uncertainty interval to expire. The result: serializable isolation across a geo-distributed cluster without atomic hardware.
- NTP clock skew on commodity servers: typically 10-100ms. Spanner's TrueTime: under 7ms with atomic clocks.
- HLC physical component: wall clock time. Advances monotonically, resets on restart only if clock is moved backward.
- HLC logical component: increments when two events share the same physical timestamp, preserving causality.
- Uncertainty interval: the range of time within which a write could have happened. CockroachDB reads must see all writes within this window.
- CockroachDB vs YugabyteDB: both use HLC, both offer PostgreSQL compatibility. CockroachDB optimizes for global distribution, YugabyteDB for lower latency regional deployments.
Choosing a Time-Series Database: A Practical Decision Matrix
Five major TSDB options serve different primary use cases. InfluxDB is purpose-built for metrics with a Flux query language, strong in IoT and monitoring. TimescaleDB extends PostgreSQL: you get SQL, joins, and a mature ecosystem with time-series-optimized storage via hypertables and automatic partitioning by time. Prometheus is a pull-based monitoring system not a general-purpose TSDB: it stores metrics for Kubernetes and application monitoring, with PromQL for alerting and Grafana for visualization. ClickHouse is a columnar OLAP database rather than a pure TSDB, but handles analytical queries over billions of time-stamped rows in seconds. OpenTSDB sits on HBase and scales to billions of metrics per second but requires operating an HBase cluster.
- InfluxDB: best for IoT sensor data and custom application metrics. Good client libraries. IOx (V3) uses Apache Arrow/Parquet.
- TimescaleDB: best when you want SQL compatibility and are already PostgreSQL-fluent. Continuous aggregates handle downsampling automatically.
- Prometheus + Thanos/Cortex: best for Kubernetes monitoring. Pull model, service discovery, PromQL for alerting.
- ClickHouse: best for analytical queries over historical time-series data. Not suited for per-metric write volumes above ~100K/sec.
- Retention policies: all TSDBs support automatic downsampling (1-second resolution → 1-minute → hourly) to control storage growth.
What Interviewers Test About Advanced Data Systems
Questions about Spanner, Spark, and TSDBs appear in senior and staff interviews for data-intensive systems. They test whether you know the right tool for each problem class and can justify trade-offs quantitatively.
- Know: why Spark is better than MapReduce for iterative algorithms, and when you would still prefer Flink over Spark (true streaming vs micro-batch)
- Know: the difference between OLTP (many concurrent short transactions) and OLAP (few long analytical queries) and why they need different storage engines
- Know: when to use a TSDB vs a general-purpose database, and what the cost of NOT using a TSDB is at scale
- Know: how Spanner's external consistency differs from serializable isolation in a single-node database
- Know: why NewSQL (Spanner, CockroachDB) is not just 'distributed SQL': it requires rethinking transaction costs across geographic distances
