Skip to content

Connect readiness wait ignores a replacement process generation #2262

Description

@hsusul

Bug

When a caller enters Realtime.Tenants.Connect.get_status/1 while a tenant Connect process is initializing, the readiness wait pins the pid returned by the first :syn.lookup/2.

If :syn resolves a cluster conflict or reconnect handoff by replacing that Connect process before initialization completes, the subscriber can observe the replacement in its second lookup but still accepts ready and connect_down events only from the old pid. A ready event from the authoritative replacement is ignored and the call times out with {:error, :initializing}.

Deterministic reproduction

  1. Make the first :syn.lookup(Realtime.Tenants.Connect, tenant_id) return generation A with %{conn: nil}.
  2. After the readiness subscriber is installed, make the next lookup return generation B with %{conn: nil}.
  3. Register a database connection for generation B and broadcast its ready event on connect:<tenant_id>.
  4. Call Realtime.Tenants.Connect.get_status/1.

Expected

The call follows the current :syn registration and returns {:ok, db_conn} when generation B becomes ready.

Actual

The generation B event does not match the pid pinned from the first lookup. The call waits until connect_connection_ready_timeout and returns {:error, :initializing}.

Root cause

wait_for_connection/2 performs a second authoritative registry lookup, but its receive clauses continue matching only the pid from the first lookup. Registry ownership can legitimately change during multi-node conflict resolution.

Impact

Concurrent websocket cold starts during a Connect generation handoff can receive a transient tenant-database-unavailable result even though the replacement connection is ready. This adds avoidable channel join failures during cluster conflict resolution or reconnection.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions