Skip to content

Deliver and release WebSocket frames pipelined with the server upgrade request - #6476

Open
Sanne wants to merge 1 commit into
eclipse-vertx:masterfrom
Sanne:ws-pipelined-frame-leak
Open

Sanne wants to merge 1 commit into
eclipse-vertx:masterfrom
Sanne:ws-pipelined-frame-leak

Conversation

@Sanne

@Sanne Sanne commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #6475

A client sending a WebSocket frame in the same write as the upgrade request (before the 101) has that frame decoded when the handshaker replaces the HTTP decoder, before the WebSocket VertxHandler replaces the HTTP connection handler. The frame reached Http1ServerConnection#handleOther which silently dropped it without releasing its buffer, leaking pooled memory under client control.

Http1ServerConnection now buffers such frames until the upgrade completes and forwards them to the WebSocket handler (or releases them when the upgrade fails or the connection closes), and releases any other unexpected message.

Assisted-by: Anthropic Claude (claude-opus-5-5)

Copilot AI balanced review requested due to automatic review settings October 30, 2026 11:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@Sanne

Sanne commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

My agents claim that the current patch is not fully complete, I'll see if I can update it.

…e request

A client sending a WebSocket frame in the same write as the upgrade request
(before the 101) has that frame decoded when the handshaker replaces the HTTP
decoder, before the WebSocket VertxHandler replaces the HTTP connection handler.
The frame reached Http1ServerConnection#handleOther which silently dropped it
without releasing its buffer, leaking pooled memory under client control.

Http1ServerConnection now buffers such frames until the upgrade completes and
forwards them to the WebSocket handler, or releases them when the upgrade fails
or the connection closes. Since a frame decoded during the upgrade usually sits
in the connection inbound queue until the handshake listener has run, a frame
dequeued after the upgrade completed is forwarded directly and a frame dequeued
after the upgrade failed is released directly. Any other unexpected message is
released as well.

Reading the channel is paused while frames are buffered, so a client flooding
the connection before it receives the 101 cannot grow the buffer beyond the
bytes already read, and resumed once the 101 is written, before the WebSocket
is handed over to the application so an application pause is not overridden.

A pipelined frame with RSV bits set was decoded before any extension decoder
was installed (they are added once the 101 is written) and cannot be
interpreted: it is rejected as a protocol violation instead of being delivered.

Known limitation: a forwarded frame is read by both the HTTP connection and
the WebSocket connection, so its bytes are reported twice to the bytesRead
network metrics. This only affects clients sending frames before the 101,
which RFC 6455 forbids, and is deemed acceptable.

Assisted-by: Anthropic Claude (claude-opus-5-5)
Signed-off-by: Sanne Grinovero <sanne@commonhaus.dev>
@Sanne
Sanne force-pushed the ws-pipelined-frame-leak branch from ac6b156 to 2b59c7a Compare October 30, 2026 15:50
@Sanne

Sanne commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Updated now - the main differences:

  • handling of the case of the upgrade fails w/o leaking the buffer
  • that temporary buffer of frames is no longer unbound, we pause consumption aggressively during the transition

@vietj vietj self-assigned this Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Memory leak and dropped buffer during pipelined Websockets upgrade

3 participants