Temporal Data, Star Schema & Slowly Changing Dimensions
You'll learn to
- -Design a time-series schema for high-volume market/event data with efficient range queries
- -Build a star schema with fact and dimension tables, and implement a Slowly Changing Dimension (Type 2) for reporting
This closing chapter of the module shifts from transactional schema design (optimized for individual reads/writes) to analytical schema design (optimized for aggregating and reporting over large volumes of historical data) - a genuinely different set of trade-offs, both of which show up constantly in real financial systems.
Time-Series Data: A Market Data Feed
A market data feed generates a continuous stream of price ticks, and the dominant query pattern is range-based - "give me every tick for AAPL between 9:30am and 10:00am." The schema itself is simple; what matters is designing it (and its indexes) around that range-query access pattern specifically.
CREATE TABLE market_ticks (
id BIGINT PRIMARY KEY,
symbol TEXT NOT NULL,
price DECIMAL(12, 4) NOT NULL,
tick_at TIMESTAMP NOT NULL
);
CREATE INDEX idx_symbol_time ON market_ticks (symbol, tick_at);
-- (symbol, tick_at) as a composite index directly serves the dominant query:
SELECT * FROM market_ticks WHERE symbol = 'AAPL' AND tick_at BETWEEN :start AND :end;At real production volume, a single unpartitioned table becomes unwieldy - time-based partitioning (splitting the table by day or month behind the scenes) is the natural next step, previewed here and covered in full depth in the planet-scale-schema module's "Sharding & Partitioning Strategies" chapter.
Star Schema: Fact and Dimension Tables
A star schema separates a central fact table (the measurable events - trades, with quantitative values like amount and quantity) from surrounding dimension tables (the descriptive context - which client, which security, which date) that the fact table references. This shape is optimized specifically for aggregation queries ("total trade volume by client, by month"), which is the dominant access pattern for regulatory reporting.
CREATE TABLE dim_client ( -- dimension: descriptive context
id INTEGER PRIMARY KEY,
client_name TEXT NOT NULL,
risk_tier TEXT NOT NULL
);
CREATE TABLE dim_date ( -- dimension: pre-computed date attributes
id INTEGER PRIMARY KEY,
full_date DATE NOT NULL,
year INTEGER NOT NULL,
quarter INTEGER NOT NULL
);
CREATE TABLE fact_trades ( -- fact: the measurable events
id INTEGER PRIMARY KEY,
client_id INTEGER REFERENCES dim_client(id),
date_id INTEGER REFERENCES dim_date(id),
trade_amount DECIMAL(15, 2) NOT NULL,
quantity INTEGER NOT NULL
);
-- Reporting query: total trade volume per risk tier, per quarter - a simple join+aggregate
SELECT c.risk_tier, d.quarter, SUM(f.trade_amount)
FROM fact_trades f
JOIN dim_client c ON f.client_id = c.id
JOIN dim_date d ON f.date_id = d.id
GROUP BY c.risk_tier, d.quarter;Slowly Changing Dimensions (SCD) Type 2: Preserving Dimension History
A client's risk_tier changes over time - but if dim_client is updated in place, every historical fact_trades row now appears to have been made under the client's current risk tier, which is factually wrong for regulatory reporting on past periods. SCD Type 2 fixes this by never updating a dimension row: instead, it inserts a new row with a new surrogate key and marks the old row as no-longer-current, so historical facts keep pointing at the dimension version that was actually true at the time.
CREATE TABLE dim_client (
id INTEGER PRIMARY KEY, -- surrogate key: a new row per version
client_natural_id INTEGER NOT NULL, -- the real, stable client identity
risk_tier TEXT NOT NULL,
valid_from DATE NOT NULL,
valid_to DATE, -- NULL means "current version"
is_current BOOLEAN NOT NULL DEFAULT TRUE
);
-- when risk_tier changes: close the old row, insert a new one
-- UPDATE dim_client SET valid_to = CURRENT_DATE, is_current = FALSE WHERE client_natural_id = 42 AND is_current;
-- INSERT INTO dim_client (client_natural_id, risk_tier, valid_from, is_current) VALUES (42, 'high', CURRENT_DATE, TRUE);
-- fact_trades.client_id keeps pointing at whichever dim_client.id version was current
-- at the time the trade happened, so historical reports stay accurateThe distinction worth naming explicitly: the transactional schema (Wolf of Wall Street's earlier levels) is optimized for correctly recording individual events; a star schema is optimized for aggregating across large volumes of them. Real systems typically maintain both, with an ETL process periodically populating the star schema from the transactional one.
Interview Signal is part of Pro
See a real weak answer next to a real strong one for this exact topic.
Quiz is part of Pro
Test what you just read with a short quiz, and bank the XP.
Design the Market Data Feed, Regulatory Reporting, and the Stratton Oakmont capstone schema in the LLD Lab's Wolf of Wall Street act.