Design a Notification System: Delivering Billions of Alerts Reliably

Facebook: 1 Trillion Notifications Per Year

Facebook sends over 1 trillion push notifications per year across its family of apps. Each notification must be delivered to the right device, via the right channel, respecting user preferences and quiet hours, with retry logic for failures. A single misfire - like sending a notification to a user who muted that discussion - erodes trust and causes uninstalls. Notification systems are deceptively complex.

Notification Architecture

Notification Pipeline:

  [Event Source]  (order shipped, new follower, etc.)
       |
  [Notification Service]
       |
  1. Lookup user preferences:
     - Which channels enabled? (push, email, SMS)
     - Quiet hours? (10pm-8am: no push)
     - Frequency caps? (max 5 push/hour)
     - Unsubscribed from this event type?
       |
  2. Template + personalize:
     'Hey {name}, your order #{id} shipped!'
       |
  3. Route to per-channel queues:
       |
  +--------+--------+--------+--------+
  |        |        |        |        |
  [Push]   [Email]  [SMS]    [In-App] [Webhook]
  Queue    Queue    Queue    Queue    Queue
  |        |        |        |        |
  [Worker] [Worker] [Worker] [Worker] [Worker]
  |        |        |        |        |
  APNs/    SES/     Twilio/  WebSocket HTTP
  FCM      SendGrid Vonage   push     POST
       |
  4. Update delivery status:
     QUEUED -> SENT -> DELIVERED -> READ

Channel Comparison

Notification Channels:

  Channel     Latency     Cost/msg    Delivery Rate  Use For
  ──────────  ──────────  ──────────  ────────────  ─────────────
  Push        Instant     Free*       60-80%         Engagement
  In-app      Instant     Free        100% (if open) Context-aware
  Email       Seconds     $0.0001     95%+ delivery  Receipts, digest
  SMS         1-5 sec     $0.0075     98%+           OTP, critical
  Webhook     Instant     Free        99%+           Developer API

  * Apple/Google charge nothing for push delivery
    but you pay for the infrastructure to send them

  Multi-channel strategy: try push first.
  If not delivered in 5 min, fallback to email.
  For critical (OTP): SMS (highest delivery rate).

Failure Handling and Rate Limiting

Failure Handling per Channel:

  Error Type           Action
  ───────────────────  ───────────────────────────────
  APNs token invalid   Remove device token, stop retrying
  FCM rate limited     Retry with exponential backoff
  SES bounce           Mark email as undeliverable
  Twilio timeout       Retry up to 3 times
  5 retries exhausted  Move to Dead Letter Queue (DLQ)

  Rate Limiting (prevent notification spam):

  Per-user limits:
  - Max 5 push notifications per hour
  - Max 1 email per event type per day
  - If exceeded: aggregate into digest
    '50 new likes' instead of 50 separate pushes

  Global limits:
  - Twilio: 100 SMS/sec (account limit)
  - APNs: token-based (virtually unlimited)
  - SES: 500 emails/sec (can increase)

Interview Tip

When designing a notification system, cover: (1) per-channel queues with dedicated workers (decouple sending from event generation), (2) user preference lookup BEFORE queuing (do not waste resources on blocked notifications), (3) retry with exponential backoff and dead letter queues, (4) per-user rate limiting with digest aggregation. The key trade-off: sending too many notifications causes uninstalls, sending too few causes disengagement.

<

/blockquote>

Key Takeaway

Notification systems route events through per-channel delivery queues, respect user preferences and quiet hours, retry failed deliveries with backoff, and aggregate high volumes into digests. The architecture must handle third-party API failures gracefully without losing notifications.