Skip to content

fix: serialize progress callbacks and drain them before the request completes - #3753

Closed
Jihaochang1 wants to merge 1 commit into
modelcontextprotocol:mainfrom
Jihaochang1:fix/progress-callback-ordering-3747
Closed

Jihaochang1 wants to merge 1 commit into
modelcontextprotocol:mainfrom
Jihaochang1:fix/progress-callback-ordering-3747

Conversation

@Jihaochang1

@Jihaochang1 Jihaochang1 commented Oct 24, 2026 •

Copy link
Copy Markdown

AI disclosure: prepared with AI assistance under my direction; I have read the change and verified the reproduction and test results myself.

Fixes #3747.

Problem

Over any transport that goes through JSONRPCDispatcher (stdio, SSE, streamable HTTP, in-memory streams), each notifications/progress spawned its own task for the request's progress_callback, and nothing waited for it. As soon as the callback awaited anything:/n/n- callbacks for the same request ran concurrently and could finish in any order

  • call_tool() / send_request() could return before earlier progress callbacks had run

The reporter's example printed ['call_tool returned', 'progress 3.0', 'progress 2.0', 'progress 1.0'] on a real transport, while the in-process DirectDispatcher (which awaits the callback inline) printed the expected order. tests/interaction/lowlevel/test_progress.py did not catch it because its callback never awaits.

Fix

Each notification's task now awaits its predecessor's done-event before invoking the callback, so the callbacks for a request run one at a time, in the order the read loop saw the notifications. send_raw_request drains the tail of the chain before the request completes.

Design notes:/n/n- The read loop never blocks on a callback: notification handling still just spawns, so a slow callback cannot stall other messages on the connection (awaiting the callback inline in the read loop, the other option the issue mentions, would do exactly that).

  • The callback stays shielded and exception-contained via the existing _shielded_progress boundary; the done-event is set in a finally, so the chain and the drain release even when the dispatcher is cancelled.
  • The drain is bounded: the pending entry is removed (and the chain tail captured) before waiting, so no further callbacks can be spawned for the request. Progress arriving after the response is still dropped, as before.
  • The drain is shielded, so a caller cancellation cannot strand the chain mid-drain. On the caller-cancelled path the request propagates without waiting, as before.
  • The same dispatcher serves client call_tool(progress_callback=...) and server send_request(progress_callback=...), so both directions are covered.

Verification

  • New regression test test_progress_callbacks_are_serialized_and_finish_before_the_result uses an awaiting callback with reversed sleep durations; it fails on 5 of 6 matrix cells before the change (the modern in-memory cell uses DirectDispatcher, which was already correct) and passes on all 6 after.
  • Full tests/shared/ suite (incl. dispatcher unit tests): 1025 passed.
  • tests/interaction/, tests/client/, tests/server/: 3127 passed, 10 skipped, 1 xfailed.
  • ruff check, ruff format --check, and pyright are clean on the changed files.

@github-actions github-actions Bot added the missing-issue-link Auto-closed: PR needs a linked issue assigned to its author (see CONTRIBUTING.md) label Oct 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR has been closed automatically. This repo only keeps pull requests open when they come from a maintainer, or from a contributor a maintainer has assigned to the linked issue, and you aren't currently assigned to #3747.

If a maintainer assigns you to #3747, this PR reopens on its own and there's nothing more you need to do here. Assignment is a maintainer call based on capacity; comments that only ask to be assigned don't factor in. What does help is engaging on the issue itself by confirming the repro, explaining why it matters for your use case, or describing the approach you'd take.

You're welcome to keep pushing commits here (just avoid force-pushing, since GitHub can't reopen a rewritten branch), but that on its own won't get the PR reviewed or the issue assigned, and realistically most auto-closed PRs stay closed. There's no need to open a new PR either way.

CONTRIBUTING.md has the full reasoning, but in short:

  • We're a small team with very little capacity to review community PRs right now.
  • Many recent PRs are AI-generated with little human review, and reviewing one carefully still costs a maintainer as much time as it ever did. A well-described issue is usually more useful to us than the code.

Maintainers: reopen, remove missing-issue-link, or add bypass-issue-check to override.

…ompletes

Each `notifications/progress` used to spawn its own task with nothing waiting
for it, so callbacks for the same request could run concurrently, complete in
any order, and still be running after `send_raw_request()` handed back the
result (modelcontextprotocol#3747). A callback that awaits anything (updating a UI, writing a log)
also delayed the callbacks behind it arbitrarily.

Each notification's task now awaits its predecessor's done-event before
invoking the callback, so the callbacks for a request run one at a time in the
order the notifications arrived, and `send_raw_request` drains the tail of the
chain before the request completes - on the success and error-response paths
alike. The callback stays shielded and exception-contained, the read loop
never blocks on it, and the drain is bounded because the pending entry is
removed first (progress arriving after the response is still dropped, as
before). The same dispatcher serves client `call_tool(progress_callback=...)`
and server `send_request(progress_callback=...)`, so both directions are fixed.

Fixes modelcontextprotocol#3747
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

missing-issue-link Auto-closed: PR needs a linked issue assigned to its author (see CONTRIBUTING.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Progress callbacks run concurrently, out of order, and after call_tool() returns

1 participant