← All comparisons
Open Amazon S3 (Object Storage) →Open Instagram Feed →

Amazon S3 (Object Storage) vs Instagram Feed

Amazon S3 (Object Storage)

Amazon S3 (Object Storage)

Strongly consistent object storage: a metadata plane that locates objects, and a replicated (or erasure-coded) data plane spread across availability zones.

Components (9)

  • Client
  • API Service
  • Identity & Access Mgmt
  • Metadata Service
  • Data Routing Service
  • Placement Service
  • Primary Data Node · AZ1
  • Secondary · AZ2
  • Secondary · AZ3

Headline numbers

  • Erasure-coding overhead vs replication~50% less
  • Expected object losses / yr per 10B objects~0.1
  • Per-prefix QPS ceiling~5,500 GET / 3,500 PUT
Instagram Feed

Instagram Feed

Fan-out on write vs read, ranking, CDN.

Components (10)

  • Mobile Client
  • CDN
  • API Gateway
  • Feed Service
  • Post Service
  • Kafka
  • Fan-out Worker
  • Redis
  • Postgres
  • S3

Headline numbers

  • Posts / sec (avg)~3,000/sec
  • Feed read QPS~58,000/sec
  • Fan-out writes / sec~600,000/sec

Key differences

Only in Amazon S3 (Object Storage)
None.
In both
  • Client
  • Api Gateway
  • Service
  • Database
  • Storage
Only in Instagram Feed
  • CDN
  • Kafka
  • Redis

Flow shape

Amazon S3 (Object Storage) flows
  • PUT an object (replication)8 steps
  • GET an object5 steps
  • ⚠ Alternative: erasure coding + AZ loss5 steps
  • ⚠ Primary data node fails3 steps
Instagram Feed flows
  • Post a photo (fan-out on write)7 steps
  • Open feed (precomputed timeline)5 steps
  • ⚠ Fan-out Worker is down7 steps
  • ⚠ Timeline cache flushed (Redis restart)4 steps