Skip to content

fix(server): record executor failures through the event consumer - #1366

Merged
Iwaniukooo11 merged 3 commits into
a2aproject:mainfrom
Yatsuiii:fix/producer-failure-write-race
Oct 23, 2026
Merged

Iwaniukooo11 merged 3 commits into
a2aproject:mainfrom
Yatsuiii:fix/producer-failure-write-race

Conversation

@Yatsuiii

@Yatsuiii Yatsuiii commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Description

When an executor enqueues its Task and raises before that Task is saved, _run_producer and the consumer both insert the same row through the shared TaskManager. With DatabaseTaskStore one insert fails, and the request either returns the SUBMITTED task as a success or raises IntegrityError instead of the executor's error.

This moves the failure write to EventConsumer, as suggested on the issue:

  • _run_producer no longer saves the failure. It only enqueues the exception, as before.
  • The consumer records the failure when it dequeues the exception, so FAILED is written after every event the executor enqueued before raising.
  • _write_failed_status creates the fallback task through ensure_task_id, with the context id from _task_manager.context_id, only when no task exists, which keeps the case fix: persist early producer failure as FAILED with originating message #1106 fixed. FAILED is stamped with the failure time, so last_updated is set. The write goes through _handle_task_modification_event rather than _handle_task_event, so it still lands when the agent answered with a Message.

Visible differences from the old producer write:

  • FAILED carries the time of the failure. The old write kept the previous status time, or none, which left last_updated NULL.
  • The agent's last status message moves from status.message into history, as on any status update.
  • When a resumed task fails, the follow-up user message is now recorded in history. The old write dropped it.

Three tests are added. One holds the store's insert of the agent's Task until the executor has raised, which is the window a database round trip leaves open. It fails on main with UNIQUE constraint failed: tasks.id and passes with this change. The other two check that a task which fails after reporting progress is stamped with the failure time, and that a follow-up answered with a Message still leaves the task FAILED.

Checks on current main (79a6f45), Python 3.10: the full suite passes with Postgres 18 behind the database tests (2396 passed, 92 skipped; the 4 cross-version tests timed out starting their servers in that run and pass on their own). I did not run the MySQL tests. ruff check, ruff format --check and ty check show nothing new.

Fixes #1351

When an executor enqueued its Task and raised before that Task was saved,
the producer's failure write and the consumer's save both inserted the same
row. With DatabaseTaskStore one insert failed: the request returned the
SUBMITTED task as a success, or raised IntegrityError in place of the
executor's error.

The producer now enqueues a _RequestFailed event and the consumer writes
FAILED when it dequeues it, after the events the executor enqueued first.
The fallback task is still created when the executor never made one, FAILED
is stamped with the failure time, and the write still lands when the agent
answered with a Message.

Fixes a2aproject#1351
@Yatsuiii
Yatsuiii requested a review from a team as a code owner October 22, 2026 07:56
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

🧪 Code Coverage (vs main)

⬇️ Download Full Report

Base PR Delta
src/a2a/server/agent_execution/active_task.py 95.40% 95.46% 🟢 +0.07%
src/a2a/server/tasks/database_task_store.py 92.47% 95.16% 🟢 +2.69%
Total 92.90% 92.96% 🟢 +0.06%

Generated by coverage-comment.yml

Comment thread src/a2a/server/agent_execution/active_task.py Outdated
Comment thread src/a2a/server/agent_execution/active_task.py Outdated
Comment thread src/a2a/server/agent_execution/active_task.py
The consumer now notes a producer failure when it dequeues the exception,
since only the producer enqueues exceptions, and takes the context id for
the fallback task from the task manager. The fallback path returns the task
it created, and _write_failed_status documents its three cases.

@Iwaniukooo11 Iwaniukooo11 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Iwaniukooo11
Iwaniukooo11 merged commit ad85806 into a2aproject:main Oct 23, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Executor failure right after its Task is enqueued leaves the task SUBMITTED on DatabaseTaskStore

2 participants