Skip to content

A2A task state is a process-local dict, so a background-initiated run loses its only handle on restart #2257

Description

@KonstantinMirin

Summary

AdCPRequestHandler keeps A2A task state in a process-local dict:

self.tasks: dict[str, Task] = {}  # In-memory task storage

src/a2a_server/adcp_a2a_server.py:182, read by _get_task_or_raise (:336) and so by
tasks/get (:370), tasks/cancel (:382) and tasks/list. Meanwhile the durable
record of long-running work already exists: workflow_steps plus
object_workflow_mapping, reached through WorkflowRepository, tenant-scoped by a
DBContext join.

This is not a regression — the dict is on main and on every branch; it predates the
boundary work. It is filed because the consequences land squarely on background-initiated
runs, and those are getting more common.

What breaks

A task id is the buyer's only handle on an operation that did not finish inside the
request. Holding that handle in a process-local dict means:

  1. A restart loses every in-flight task. tasks/get then answers TaskNotFoundError
    for work that is still running, or that completed and was recorded in
    workflow_steps. The buyer cannot distinguish "never existed" from "I forgot".
  2. More than one worker means a coin flip. Any replicated deployment answers
    tasks/get from whichever process the request landed on, so a task created on worker A
    is not found on worker B. Nothing in the handler makes the dict sticky to a connection.
  3. tasks/cancel cancels a copy, not the work. It mutates the stored Task's status
    to TASK_STATE_CANCELED and returns it. Nothing downstream reads that flag, so the
    adapter call or approval the task represents proceeds. The buyer is told the operation
    was cancelled and it was not.
  4. The store is not tenant-scoped. Every other read of durable state in this codebase
    goes through a repository that joins tenant. A plain dict keyed by task id has no such
    join; only the id's unguessability separates tenants.
  5. The two records can disagree. A workflow step can be completed in the database
    while the dict still reports working, because nothing keeps them in step.

Why background-initiated runs make it worse

A run the seller starts without a buyer request in flight — an approval that resolves
later, a webhook-driven continuation, a deferred adapter call — has no request to carry
its result back on. The task id IS the mechanism. Holding it in memory means the one
handle to a background-initiated run is the least durable thing in the system, and the
buyer's only recovery is to stop polling and guess.

Proposed change

Answer tasks/get, tasks/cancel and tasks/list from the durable workflow store, and
delete the dict rather than syncing it. Two records that must agree is the defect; one
record cannot disagree with itself.

  • a repository method mapping an A2A task id to its workflow step, tenant-scoped like
    every other read;
  • tasks/cancel requesting cancellation of the underlying work and reporting what the
    store says, rather than stamping a field nothing reads;
  • TaskNotFoundError reserved for an id the store does not hold, so it stops meaning
    "this process restarted".

Grounding needed before implementing

Per CLAUDE.md's spec-grounding gate, the request/response contract for these three
methods must be cited against the pinned AdCP release (3.1.1 via adcp==6.6.0) before any
code changes, together with the conformance storyboard step that grades each — or a note
that it is ungraded. src/core/tools/task_management.py:69 already cites
protocol/calling-an-agent.mdx:95-99 for the normative tasks/get response shape, which
is the place to start. The A2A surface is graded by the storyboard runner, so check
whether the task-lifecycle storyboards exercise a restart at all; if they do not, the
gap is invisible to CI today and a scenario is part of the work.

Notes

  • #1670 covers why an unknown task id still reaches the wire as -32603 rather than the
    spec's -32001; that is a separate defect on the same handlers and the two should be
    read together.
  • The in-memory dict is also why the A2A surface currently passes a restart-free CI run
    without complaint. Any fix should come with a scenario that survives a restart, or the
    next regression is equally invisible.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions