Skip to content
Pidgely Docs

Platform

Scaling

How Pidgely grows from one node to independent fleets by configuration, not rewrites - roles, per-kind workers, read replicas, partitions and an external broker.

View as Markdown

Pidgely is designed to grow by configuration, not rewrites. Every stage below runs the same code; the next stage is adopted when a metric calls for it. This is a summary for the curious: as a customer of the hosted service, you don't need to do anything.

Stage Layout Move on when
1. Single node pidgely all + PostgreSQL on one server CPU above 60%, delivery queue lag above a minute at peak, or zero-downtime API deploys needed
2. Split roles api replicas behind a load balancer; worker, inbound and smtp on servers with clean IPs and port 25 One kind of work starves another (a large broadcast delays transactional mail)
3. Per-kind worker fleets Separate fleets for delivery, webhooks and broadcast fan-out, each scaled on its own queue lag The primary database is busy with reads
4. Read replica List endpoints read from a replica; single-object reads stay on the primary Monthly partitions grow very large
5. Retention and partitions Monthly partitions dropped by retention instead of row deletes The job table becomes the bottleneck (thousands of jobs per second)
6. External broker Jobs still written transactionally (outbox), relayed to a message broker A context needs independent scaling
7. Extracted services Webhooks, marketing or inbound run as their own services —

What makes it possible

  • Stateless roles with health, readiness and metrics endpoints.
  • Transactional outbox: jobs are written in the same transaction as the change that causes them, so nothing is lost or duplicated by a crash.
  • Idempotent job handlers: every job tolerates running twice, late, or after its subject changed. For you this means webhook events can arrive more than once; de-duplicate on svix-id.
  • Bounded contexts: sending, feedback, integrations, marketing, templates and accounts each own their schema and talk through small interfaces, so any of them can be extracted.
  • Read-your-writes: GET /emails/{id} always reads the primary, even when lists are served by replicas.