Capstone: An ML Platform for ETA Prediction

A ride and delivery app shows customers a promised arrival time. It is the number people plan around, and it appears on every order. Design the machine learning system behind it. This is a classic ML design question, and it uses every idea from the module: framing, labels, features, skew, batch versus online, serving, evaluation, and monitoring.

• -Prediction: the trip duration, shown before the trip and updated during it.
• -Latency: the estimate must return within about 50 ms at the 99th percentile, thousands of times a second.
• -Freshness: traffic changes in minutes, so some features must be a minute or two old at most.
• -Quality: accurate on average, and not systematically wrong in any city or at rush hour.
• -Constraints: a late arrival annoys customers more than an early one, which will shape the target.

The business goal is fewer complaints and cancellations. The prediction task is the duration of a trip. The label is the actual trip time, which arrives only after the trip ends: a label delay of the length of the trip itself. Because lateness costs more than earliness, the model should not predict an average.

• -Batch features, computed daily: historical average speed by road segment and hour, by day of week.
• -Streaming features, fresh within minutes: current traffic speed on the route's segments, active incidents, nearby driver supply.
• -Request features, known at call time: distance, number of stops, time, weather.
• -A feature store serves the same definitions to training and serving, and training rows use point-in-time joins.

Start with a simple baseline: the historical average for that route and hour. Then a gradient boosted tree model on the features above, which is strong on tabular data and fast to serve. Move to richer models only if the gap justifies the cost.

Now the target. If lateness costs three times as much as earliness, predicting the average is the wrong goal. The block below simulates 20,000 trips on one kind of route, with a long tail of slow trips, and finds the promise that costs customers the least. Press Run, and compare the three promises, then read the slice table.

Promising the mean, 20.8 minutes, makes 44% of customers late. Promising the median is worse, at 50% late and a higher expected cost. Promising the 75th percentile, 24.3 minutes, cuts lateness to 25% and has the lowest cost, 8.16, and a search over all promises finds the same answer. That is no accident: when lateness costs three times earliness, the best promise is the point where 3 ÷ (3 + 1) = 75% of trips finish earlier. So the model should be trained with a quantile objective for that percentile, or predict the whole distribution and let the product choose. In practice, many apps show a range.

• -Online scoring, since traffic is fresh and every order is different. Batch alone would be hours stale.
• -A latency budget: a few milliseconds each for the gateway and the model, the rest for feature lookups, with timeouts.
• -A fallback: if streaming features are missing, use the historical average for that route and hour, and mark the estimate as lower confidence.
• -Staged rollout: shadow first, then a canary in one city, then wider.

The overall mean absolute error is 3.4 minutes, which looks fine. The slice table shows the real picture. Rush hour in a big city has an error of 6.8 minutes, twice the average and more than three times the best slice, and it is the situation customers care most about. Report the metric by city, by time of day, by trip length, and by new versus old cities, with a target for each.

• -Online experiment: compare against the current ETA on cancellations, customer contacts, and lateness complaints, with guardrails on latency and crashes, and a sample ratio check.
• -Monitoring: speed features for drift, with extra attention after holidays and big events, plus error by slice once trips finish and labels arrive.
• -Retraining: daily or weekly on a time-based split, released in stages as a challenger.
• -Failure modes to plan for: a new city with no history, an outage in the traffic feed, and a sudden event such as a storm.