Skip to content

Fix bthread_id lock leak in ProcessNsheadMcpackResponse - #3574

Merged
chenBright merged 3 commits into
apache:masterfrom
wwbmmm:oncall/req-20261001-014506
Oct 1, 2026
Merged

chenBright merged 3 commits into
apache:masterfrom
wwbmmm:oncall/req-20261001-014506

Conversation

@wwbmmm

@wwbmmm wwbmmm commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: resolve #N/A

Problem Summary:

In ProcessNsheadMcpackResponse (brpc/policy/nshead_mcpack_protocol.cpp), after bthread_id_lock(cid, &cntl) succeeds, there were two early-return paths that skipped accessor.OnResponse(cid, saved_error), which is the only place that unlocks the correlation_id:

  1. cntl->response() == nullptr (no response object)
  2. handler.parse_from_iobuf(...) fails (malformed mcpack body), which did return cntl->CloseConnection(...)

Once the lock leaks, the synchronous RPC caller hangs forever in Join(correlation_id), async calls never run done->Run(), and even the RPC timeout cannot rescue the call: HandleTimeout only enqueues the error into pending_q, which is consumed exclusively inside bthread_id_unlock. A server that returns a valid nshead header plus a body that cannot be parsed as mcpack reliably triggers this, permanently hanging one client bthread per affected call.

What is changed and the side effects?

Changed:

  • ProcessNsheadMcpackResponse now follows the same do { ... break ... } while (0) pattern used by ProcessRpcResponse and other protocol handlers: all paths (missing response object, mcpack parse failure, success) fall through to msg.reset() and accessor.OnResponse(cid, saved_error), so the bthread_id lock is always released.
  • Added test/brpc_nshead_mcpack_protocol_unittest.cpp covering: malformed mcpack body, unregistered message handler, missing response object, and the success path. Each error-path case asserts that the correlation_id is no longer locked (a leaked lock would make brpc::Join hang forever). Without the fix these three cases fail; with the fix all four pass.

Side effects:

  • Performance effects: none; the change only restructures control flow after response parsing.

  • Breaking backward compatibility: none. RPCs whose response body cannot be parsed now fail the controller with the existing ECLOSE error (from CloseConnection) and return to the caller instead of hanging forever.

Check List:

  • Compilable: verified with cmake -S . -B build -DBUILD_UNIT_TESTS=ON + make -j6.
  • Tests: brpc_nshead_mcpack_protocol_unittest (new, 4 cases, verified failing before the fix and passing after); regression-checked brpc_mcpack2pb_unittest, brpc_nova_pbrpc_protocol_unittest, brpc_esp_protocol_unittest, brpc_sofa_pbrpc_protocol_unittest — all pass.

🤖 This PR was automatically created by brpc-oncall

After ProcessNsheadMcpackResponse acquires the bthread_id lock, its two
early-return paths (response object missing, mcpack parse failure) skip
accessor.OnResponse(), the only unlock entry. The leaked lock makes
Join(correlation_id) hang forever and the RPC timeout cannot rescue it
since the timeout error is only consumed during unlocking. A server
returning a valid nshead header plus a malformed mcpack body reliably
triggers this.

Adopt the do { ... break ... } while(0) pattern (consistent with
ProcessRpcResponse) so that OnResponse() is always reached.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The production fix is sound and well covered; only minor test-fixture descriptor cleanup remains.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Prevents nshead-mcpack response handling from leaking correlation-ID locks on error paths.

Changes:

  • Ensures all post-lock paths call OnResponse.
  • Adds coverage for malformed, unregistered, missing-response, and successful cases.
File Description
src/​brpc/​policy/​nshead_mcpack_protocol.cpp Consolidates response completion and lock release.
test/​brpc_nshead_mcpack_protocol_unittest.cpp Tests lock release across response paths.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread test/brpc_nshead_mcpack_protocol_unittest.cpp Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The control-flow fix is correct and the regression tests cover every affected path.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)

Descriptor::full_name() returns absl::string_view since protobuf 35.x,
which cannot be implicitly converted to the const std::string& parameter
of mcpack2pb::register_message_handler_or_die. Wrap it into std::string
explicitly.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The control-flow fix is correct, focused, and covered by targeted regression tests.

Review effort: Balanced
Findings: None

Resolved since last review (1)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The control-flow fix is correct and the regression tests cover every affected path.

Review effort: Balanced
Findings: None

@chenBright chenBright left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chenBright
chenBright merged commit 2621a26 into apache:master Oct 1, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants