Block vs File vs Object Storage: Choosing the Right Storage Abstraction

The $500K AWS Bill That Exposed a Storage Mistake

A startup stored 50TB of video files on EBS (block storage) because their developers mounted it like a local disk and it 'just worked.' Their monthly AWS bill was $5,000/TB for EBS gp3 volumes - $250,000 per month. When they migrated to S3 (object storage) at $23/TB, the same 50TB cost $1,150/month - a 99.5% cost reduction. The videos were write-once, read-many content that never needed random writes or POSIX locking. They had been paying for block storage features they never used. This mistake happens constantly because engineers default to what feels familiar (a disk) instead of choosing the right storage abstraction for their access pattern. Understanding when to use block, file, or object storage is one of the highest-ROI decisions in system architecture.

Three Storage Abstractions

Block vs File vs Object Storage:

  BLOCK STORAGE (raw disk blocks)
  +------------------------------------------+
  | [Block 0][Block 1][Block 2]...[Block N]  |
  | Your OS sees it as a raw disk.            |
  | YOU manage the filesystem on top (ext4).  |
  +------------------------------------------+
  | Access: /dev/sda1 (device-level)          |
  | Latency: <1ms (sub-millisecond)           |
  | Use for: Databases, VMs, boot volumes     |
  | Can modify: YES (random read/write)       |
  | Shared access: NO (one server at a time)  |
  | AWS: EBS (Elastic Block Store)            |
  | Cost: $$$ (highest per GB)                |
  +------------------------------------------+

  FILE STORAGE (shared network filesystem)
  +------------------------------------------+
  | /data/                                    |
  |   /uploads/photo1.jpg                    |
  |   /configs/app.yaml                      |
  | Multiple servers mount the SAME fs.       |
  +------------------------------------------+
  | Access: NFS/SMB mount (file-level)        |
  | Latency: 1-10ms (network overhead)        |
  | Use for: Shared configs, media editing    |
  | Can modify: YES (in-place edits)          |
  | Shared access: YES (multi-server mount)   |
  | AWS: EFS (Elastic File System)            |
  | Cost: $$ (medium per GB)                  |
  +------------------------------------------+

  OBJECT STORAGE (immutable blobs via HTTP)
  +------------------------------------------+
  | PUT /bucket/photos/vacation.jpg           |
  | GET /bucket/photos/vacation.jpg           |
  | DELETE /bucket/photos/vacation.jpg        |
  | Flat namespace. No directories. No edits. |
  +------------------------------------------+
  | Access: HTTP REST API                     |
  | Latency: 50-200ms (first byte)            |
  | Use for: Images, videos, backups, logs    |
  | Can modify: NO (overwrite = new version)  |
  | Shared access: YES (any client via HTTP)  |
  | AWS: S3                                   |
  | Cost: $ (cheapest per GB)                 |
  +------------------------------------------+

Decision Matrix

Choosing the Right Storage:

  Question                              Answer -> Use
  ------------------------------------  ------------------
  Need sub-ms latency?                  YES -> Block (EBS)
  Need random read/write?               YES -> Block (EBS)
  Running a database?                   YES -> Block (EBS)
  VM boot disk?                         YES -> Block (EBS)
  Multiple servers need same files?     YES -> File (EFS)
  Need POSIX file system interface?     YES -> File (EFS)
  In-place file editing?                YES -> File (EFS)
  Write-once, read-many?                YES -> Object (S3)
  Images, videos, user uploads?         YES -> Object (S3)
  Backups and archives?                 YES -> Object (S3)
  Logs and analytics data?              YES -> Object (S3)
  Static website assets?                YES -> Object (S3)
  ML training datasets?                 YES -> Object (S3)
  Need unlimited scale?                 YES -> Object (S3)

Cost Comparison (50 TB stored for 1 year):

  Storage Type   AWS Service     Monthly Cost    Annual Cost
  -------------- --------------- --------------- ----------
  Block (gp3)    EBS             $250,000        $3,000,000
  File (general) EFS             $15,000         $180,000
  Object (std)   S3 Standard     $1,150          $13,800
  Object (cold)  S3 Glacier Deep $50             $600

  The difference is 200-5000x.
  Choose wrong = burn money.

Real-World Architecture: Polyglot Storage

Typical Production Architecture (uses all three):

  [Application Server]
       |
       +-- PostgreSQL database
       |   Data: user records, orders, transactions
       |   Storage: EBS gp3 (block) - needs IOPS, random writes
       |
       +-- Shared config / ML models
       |   Data: config files, model weights
       |   Storage: EFS (file) - multiple servers read same files
       |
       +-- User uploads / static assets
       |   Data: profile photos, PDFs, videos
       |   Storage: S3 (object) - write-once, serve via CDN
       |
       +-- Application logs
       |   Data: JSON log lines, metrics
       |   Storage: S3 (object) - write-once, query with Athena
       |
       +-- Database backups
           Data: pg_dump files
           Storage: S3 Glacier (object-cold) - rarely accessed

  This is polyglot storage:
  Use the right tool for each access pattern.
  Do NOT put everything on one storage type.

Interview Tip

When choosing storage in system design, say: 'I would use polyglot storage based on access patterns. Block storage (EBS) for databases that need sub-millisecond random read/write. Object storage (S3) for user uploads, static assets, logs, and backups - anything write-once-read-many. File storage (EFS) only when multiple servers need to share the same filesystem. The cost difference is dramatic: block storage costs 200x more per GB than object storage, so storing media files or backups on EBS is a common and expensive mistake. I would serve S3 objects through a CDN for global low-latency access.'

<

/blockquote>

Key Takeaway

Block storage is for databases and VMs (sub-ms, random I/O, expensive). File storage is for shared access across servers (network filesystem). Object storage is for everything else (cheapest, unlimited scale, HTTP API). The cost difference is 200-5000x between block and object storage. Use polyglot storage - the right abstraction for each workload - to optimize both performance and cost.