Skip to content

fix(client): stop cancelled batched streams from running forever on the server - #2161

Open
dinwwwh wants to merge 1 commit into
middleapi:mainfrom
dinwwwh:claude/batched-streams-cancel-fix-75242b
Open

dinwwwh wants to merge 1 commit into
middleapi:mainfrom
dinwwwh:claude/batched-streams-cancel-fix-75242b

Conversation

@dinwwwh

@dinwwwh dinwwwh commented Sep 30, 2026

Copy link
Copy Markdown
Member

Cancelling a batched stream now stops it on the server. Before, the batch request was aborted only once every call in it had been cancelled, so if another call in the same batch had already finished, the connection stayed open and the cancelled procedure kept running. An endless generator never stopped.

Fixes

  • Cancelling a batched event iterator or stream aborts the batch request as soon as only cancelled calls are still running on the server, which stops them.
  • Batches whose calls all finish normally still end on their own, so the connection can be reused as before.
  • A call cancelled before the batch is sent still travels inside the batch and does not abort it.

Testing

  • New unit tests cover aborting once only the cancelled stream is left, staying open while another stream is still sending, and not aborting when the cancel arrives after the server finished that call, when the cancelled stream finishes on its own, or when the cancel is sent before the batch.
  • New end-to-end test on node-http, hono-fetch and both compression setups batches JSON, Blob, ReadableStream, an event iterator and a cancelled endless generator. The live streams arrive in full and the generator stops. It fails on main.

…he server

Cancelling a batched stream now aborts the batch request once only
cancelled subrequests could still be running on the server. Before, the
batch was aborted only when every subrequest sent a cancel, so a sibling
that had already finished kept the connection open and the server
generator running.
@codecov

codecov Bot commented Sep 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.18919% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
packages/client/src/plugins/batch.ts 89.18% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • Cancellation gating rewritten in BatchLinkPlugin — the old activeCount counter (decremented per cancel, abort at zero) is replaced by openRequestIds / cancelledRunningRequestIds sets, so the batch aborts as soon as only cancelled subrequests are still running rather than waiting for every subrequest to be cancelled.
  • Server-message completion tracking — a new isLastServerMessage helper classifies terminal peer messages (response with body/binary or no content-type + standard-server hint, event-stream close/error, octet-stream close: true, cancel), and decodeLengthPrefixedStream now takes a receive callback that removes finished ids and re-checks the abort condition.
  • Tests — new startStreamingBatch helper with 5 unit tests covering abort-on-only-cancelled, keep-open-while-other-stream-runs, cancel-after-server-finished, cancelled-stream-finishes-first, and pre-batch cancel; plus an e2e test across node-http, hono-fetch and both compression setups asserting the cancelled endless generator stops and live streams arrive in full.

I verified the wire shapes against @standard-server/peer v0.10.0: every server terminal message kind is recognized, and stream/cancel (the one kind that returns false) is only emitted for client request-body streams, which the batch plugin never batches. I also confirmed peer.close uses Queue.close() (which preserves buffered items) rather than abort(), so aborting the batch to stop a cancelled stream cannot discard already-received buffered data for the other subrequests. The unit and e2e suites pass, and type:check and eslint are clean on all three files.

Pullfrog  | View workflow run | Using DeepSeek Flash (free via Pullfrog for OSS) | 𝕏

@codspeed

codspeed Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will improve performance by 12.74%

⚡ 1 improved benchmark
✅ 29 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ octet stream 726.6 µs 644.4 µs +12.74%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing dinwwwh:claude/batched-streams-cancel-fix-75242b (ff130b2) with main (0d42e65)

Open in CodSpeed

@pkg-pr-new

pkg-pr-new Bot commented Sep 30, 2026

Copy link
Copy Markdown
More templates

@orpc/ai-sdk

npm i https://pkg.pr.new/@orpc/ai-sdk@2161

@orpc/arktype

npm i https://pkg.pr.new/@orpc/arktype@2161

@orpc/bun

npm i https://pkg.pr.new/@orpc/bun@2161

@orpc/client

npm i https://pkg.pr.new/@orpc/client@2161

@orpc/cloudflare

npm i https://pkg.pr.new/@orpc/cloudflare@2161

@orpc/contract

npm i https://pkg.pr.new/@orpc/contract@2161

@orpc/experimental-effect

npm i https://pkg.pr.new/@orpc/experimental-effect@2161

@orpc/evlog

npm i https://pkg.pr.new/@orpc/evlog@2161

@orpc/hibernation

npm i https://pkg.pr.new/@orpc/hibernation@2161

@orpc/json-schema

npm i https://pkg.pr.new/@orpc/json-schema@2161

@orpc/experimental-lock

npm i https://pkg.pr.new/@orpc/experimental-lock@2161

@orpc/experimental-msw

npm i https://pkg.pr.new/@orpc/experimental-msw@2161

@orpc/nest

npm i https://pkg.pr.new/@orpc/nest@2161

@orpc/next

npm i https://pkg.pr.new/@orpc/next@2161

@orpc/node

npm i https://pkg.pr.new/@orpc/node@2161

@orpc/openapi

npm i https://pkg.pr.new/@orpc/openapi@2161

@orpc/opentelemetry

npm i https://pkg.pr.new/@orpc/opentelemetry@2161

@orpc/pinia-colada

npm i https://pkg.pr.new/@orpc/pinia-colada@2161

@orpc/pino

npm i https://pkg.pr.new/@orpc/pino@2161

@orpc/publisher

npm i https://pkg.pr.new/@orpc/publisher@2161

@orpc/ratelimit

npm i https://pkg.pr.new/@orpc/ratelimit@2161

@orpc/server

npm i https://pkg.pr.new/@orpc/server@2161

@orpc/shared

npm i https://pkg.pr.new/@orpc/shared@2161

@orpc/swr

npm i https://pkg.pr.new/@orpc/swr@2161

@orpc/tanstack-query

npm i https://pkg.pr.new/@orpc/tanstack-query@2161

@orpc/trpc

npm i https://pkg.pr.new/@orpc/trpc@2161

@orpc/valibot

npm i https://pkg.pr.new/@orpc/valibot@2161

@orpc/zod

npm i https://pkg.pr.new/@orpc/zod@2161

commit: ff130b2

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant