Repository navigation
Conversation
Slow HTTP DoS scanners (issue #183) hold connections open by dribbling request bytes, and two inherited gaps let them: 1. deadlineconn re-extended the read deadline on every partial read, converting net/http's absolute ReadHeaderTimeout into an inter-byte activity timeout that a trickle bypasses entirely. 2. the --read-header-timeout flag (default 30s) was never copied into the server context, so the server actually ran with header timeout disabled (upstream shares this bug). Keep deadlineconn on the write side only so response writes stay activity-based, and leave ReadTimeout at zero now that native read deadlines are honored, since an absolute cap would terminate large uploads. Request bodies keep an activity-based bound instead via a per-read deadline refresh in xhttp, and the flag is wired through common-main. With this, a slow-header connection is cut at ReadHeaderTimeout (verified live: 33s kill vs. unlimited before), while slow progressing uploads (32s for 2 MiB), stalled bodies (30s cleanup), slow readers (~30s write cutoff) and plain S3 traffic are unaffected. Fixes #183 Signed-off-by: Feng Ruohang <rh@vonng.com>
This was referenced Sep 15, 2026
Member
Author
|
The request-header timeout fix has landed through PR #196, merged as The merged implementation preserves the absolute header deadline, wires As of September 16, these changes are on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #183
Root cause
A slow HTTP DoS scanner holds connections open by dribbling request bytes (slowloris). Two inherited gaps allowed exactly that:
httpListener.Accept()wrapped every accepted connection withdeadlineconn.WithReadDeadline(idle), which re-extends the read deadline on every partial read.net/httpsetsReadHeaderTimeoutas an absolute deadline (headers must complete within N seconds in total), but the wrapper converted it into an inter-byte activity timeout — a client sending 1 byte every 25 s keeps the connection (goroutine + fd) alive forever.--read-header-timeoutflag (default 30 s) is defined but never copied intoglobalServerCtxtincommon-main.go, so the server ran withReadHeaderTimeout = 0, i.e. disabled. Upstream MinIO master shares both defects (verified againstminio/minio@master).Fix
internal/http/listener.go: keepdeadlineconnon the write side only. Write deadlines stay activity-based (big downloads must not be capped in total), and native read deadlines are honored again.cmd/common-main.go: wire--read-header-timeoutinto the server context.cmd/server-main.go: dropUseReadTimeout. With native read deadlines honored, an absoluteReadTimeoutwould cap the whole request including large uploads; request bodies are bounded per-read instead (next point).WriteTimeoutstays: it only resets the activity-based write deadline per request.internal/http/server.go: wrap request bodies inidleTimeoutBody, which refreshes the connection read deadline ahead of every body read viahttp.NewResponseController. Stalled uploads are cut off after the idle timeout while progressing ones extend indefinitely. Under HTTP/2 the controller is supported natively per stream; on hijacked connections it is a no-op.HTTP/2 audit:
x/net/http2only sets zero (infinite) read deadlines on the conn, andServer.IdleTimeoutis wired into the http2 idle timer, so idle h2 connections are still cleaned up after 30 s — no regression from dropping the read-side wrapper.Verification
Unit tests (
internal/http/server-deadline_test.go, full xhttp stack incl. listener and TCPOptions wiring):ReadHeaderTimeout— verified the test fails on the old listener code (connection survives) and passes with the fixLive single-node run (default flags):
Regression:
go build ./...,go vet,./internal/http ./internal/deadlineconnsuites, and the full./cmdsuite (720 s) all pass.Residual: deliberate slow-body/slow-read trickle (inter-byte gap < idle) still holds a connection, as before — an object store cannot cap total transfer time without breaking legitimate large/slow uploads. Perimeter timeouts (LB/reverse proxy) remain the recommendation for strict environments; this change closes the unbounded slow-header case the scanner flagged.