Skip to content
AntFabric Corev1.0.0local

Scaling

An instance runs as N identical API replicas and M worker replicas behind any load balancer, with no sticky sessions. Any replica can serve any request and can be stopped at any moment. Scaling up means adding replicas; nothing else changes.

Store Holds If it is lost
Postgres Everything that must survive: users, sessions, plugin data, outboxes, job claims, idempotency results It is the source of truth: back it up
Redis Short-lived shared state: rate limits, presence, SSO state, cache-bus and socket pub/sub. Every key has a TTL Presence resets, an SSO sign-in in progress restarts; no data lost
Temporal Long-running work: workflows, schedules, retries Its own persistence (on the instance’s Postgres)
S3 File bytes; clients move them directly on signed URLs Back it up
Process Small caches only, each with a TTL and cleared on every replica by the cache bus Nothing: rebuilt on demand
  • Sessions and roles: every request checks its session in Postgres. A sign-out or role change clears the short check cache on every replica and closes or rechecks that person’s sockets everywhere.
  • Rate limits: one counter per client in Redis, for the whole instance. TRUST_PROXY makes the client’s address the real one behind the balancer.
  • Outboxes: workflow starts and events are written in the business change’s transaction. Every replica dispatches, and rows are leased (SKIP LOCKED), so each goes out once.
  • Sockets: WebSocket only, so no stickiness. The Redis adapter carries events to sockets on every replica.
  • Periodic jobs: each job is claimed in Postgres once per period, whatever the replica count.
  • Server-side events: one delivery row per subscriber, claimed by worker replicas, each retried on its own.
  • Shutdown: on SIGTERM a replica answers not-ready, moves its sockets off, lets requests and Activities finish within a grace period, then closes its connections. Rolling updates lose nothing.
  1. Nothing that matters in process memory.
  2. Every in-process cache has a short TTL (core’s are also cleared over the cache bus).
  3. Redis is only for short-lived state, always with a TTL.
  4. No timers for periodic work: declare a schedule.
  5. Every side effect is idempotent (keys, workflow ids, event ids, @Idempotent()).
  6. Your code can be stopped at any moment: long work belongs in workflows.
  7. Configuration only from the environment (plugin configuration and secrets come from core).

The review checklist checks these on every plugin merge request.

  • Backend e2e: each mechanism is tested against real Postgres, Redis and Temporal. Some suites run two app instances in one process. The whole suite also runs through PgBouncer.
  • pnpm --filter backend scale:test: the production image as 3 API and 2 worker replicas behind nginx, tested from the outside.
  • pnpm --filter backend scale:chaos: the same tests in rounds while a random API replica restarts every 12 s. Last run: 120 of 120 tests passed across 7 restarts.

The full reference, with settings, metrics and current limits, is How the backend scales in the developer docs (apps/documentation/docs/backend/scaling.md).