Reqflow
← All comparisons

Distributed Job Scheduler vs URL Shortener

Distributed Job Scheduler

Distributed Job Scheduler

Cron-at-scale: exactly-once execution, missed-run recovery, at-least-once delivery, priority queues.

Components (8)

  • Client / Admin
  • API Gateway
  • Scheduler Service
  • Job Queue
  • Worker Pool
  • Job Store
  • Run History
  • Dead Letter Queue

Headline numbers

  • Scheduler poll load~3,000 DB reads/sec
  • Workers needed at peak~830 concurrent workers
  • Run history write rate~5,000 writes/sec
URL Shortener

URL Shortener

Hashing, key generation, read-heavy caching.

Components (6)

  • Client
  • API Gateway
  • Write Service
  • Read Service
  • Redis
  • Postgres

Headline numbers

  • Write QPS (avg)~1,200/sec
  • Read QPS (avg)~120,000/sec
  • Storage per year~5 TB

Key differences

Only in Distributed Job Scheduler
  • Job Queue
In both
  • Client
  • Api Gateway
  • Service
  • Database
Only in URL Shortener
  • Redis

Flow shape

Distributed Job Scheduler flows
  • Scheduled job fires5 steps
  • Worker crashes mid-execution4 steps
  • Scheduler leader fails3 steps
URL Shortener flows
  • Shorten a URL3 steps
  • Resolve short URL (cache hit)3 steps
  • Resolve short URL (cache miss)4 steps
  • Redis is down4 steps