Skip to content

fix(sync): fail a run whose stored host POST /repos refuses - #907

Merged
ThomasK33 merged 7 commits into
mainfrom
sync-stored-host-check
Nov 17, 2026
Merged

ThomasK33 merged 7 commits into
mainfrom
sync-stored-host-check

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Since #903, POST /repos refuses an upstream host that a sync run cannot use as given. A link stored before that keeps such a host, and every pull and push run still handed it to git. The result depended on the host, and so did the failure class. The new test on main (allow-private egress policy, the test fixture's default) shows it:

Stored host Pull and push run on main
bücher.example, https://bücher.example failed to clone repository / failed to push to upstream: this host's curl refused the URL (URL using bad/illegal format)
http://2130706433:<port>, http://0x7f.1:<port>, http://127.0.0.1:<port>?, http://127.0.0.1:<port>#x the request reached the loopback upstream: curl dialed 127.0.0.1
<<<.com, http://127.0.0.1:<port>/a b, forge.example:65536 curl refused the URL
HTTP://127.0.0.1:<port> git refused the transport (transport 'HTTP' not allowed)
https://[fe80::1%25eth0] upstream repository unreachable: the egress dial failed

Now each run checks the stored host with the rule of POST /repos before git starts. A host that fails it fails the run as repository configuration error. The server's WARN line for the run gives the cause repository configuration error: upstream host is not a valid host, with the reason that POST /repos gives, and does not repeat the host.

A stored link that synced before can now fail every run. With the allow-private policy, a http://2130706433:<port> link reached a git server on 127.0.0.1 and could sync from it; a bücher.example link could sync where the server's curl and locale convert the name (I did not verify this; this host's curl refuses it). This is the intended change, and it goes into the release notes with #903's. The docs tell operators to create a sync repository again with the host in a form that POST /repos takes, such as the ASCII form of a name.

Changes

  • ValidUpstreamHost and its helpers (dialableAuthority, dialableHostname, resolverName, curlIPv4Name, curlIPv4Part) move unchanged from internal/api/handlers_sync.go to internal/sync/upstreamhost.go, because api imports sync and not the reverse. InvalidUpstreamHostReason (is not a valid host) is the reason both packages give. TestValidUpstreamHost moves with them.
  • upstreamURLFailure, which both lanes call before git starts, checks the host before the URL length, as POST /repos does. A file:// host is exempt: only the tests' fixtures store one, POST /repos never has, and GIT_ALLOW_PROTOCOL refuses it in production.
  • Docs: docs/reference/http-api.md ("Upstream sync") says how such a run fails, that no route changes the host of a stored link, and that a new sync repository with the host in a form POST /repos takes syncs the upstream again; the READMEs of internal/api and internal/sync.
  • fix(sync): send an upper-case http(s) scheme through the egress proxy #905's two lane tests (TestWorkerUpperCaseSchemeUpstreamNeverDialsAroundProxy, TestPushRunUpperCaseSchemeUpstreamNeverDialsAroundProxy, renamed from ...DialsThroughProxy) now stop at the host check, before git starts. They still fail only when both guards break: with the host check off they pass on the egress proxy, with the proxy decision back to lower case only they pass on the host check, and with both they fail. TestViaEgressProxy pins the proxy decision by itself.
  • worker.go and handlers_sync.go are covered by the lane-schedule model, so the fingerprints are re-recorded. The host rule's functions leave the record with handlers_sync.go: the model describes lanes, not hosts.

Evidence

  • Tests: TestRunRefusesStoredInvalidHost (internal/sync) stores the eleven hosts above, at least one per class that POST /repos refuses, and runs each through the pull lane and the push lane. Each run must return exactly repository configuration error: upstream host is not a valid host, emit a failed event with repository configuration error, send no request to the loopback upstream, and log that cause without the host.
    TestRunChecksStoredHostBeforeURLLength stores a non-ASCII base URL whose upstream URL is also over the length limit: the run gives the host's reason.
  • Red on main: with main's worker.go, all 22 cases fail as in the table, and the order test gets the URL-length cause.
  • Lanes: go test -race -tags=openstorage_viewcheck of ./internal/sync/... and ./internal/api/... without pg passes at d75a879 (91 s, 373 s) and at 72ffbda, after the merge with fix(sync): send an upper-case http(s) scheme through the egress proxy #905 (72 s, 384 s). PostgreSQL and S3-WAL rely on CI.
  • lint, vet (linux and darwin), gofmt, the docs gate and both fingerprint gates pass.

Reviews

  • Round 1 (d75a879): the Codex review and the Codex security review found nothing. A fresh read-only review: PASS. It diffed the moved functions against main (code-for-code identical), traced both lanes (the check runs before every git command and dial), found no host that POST /repos stores and a run now refuses (provider defaults and every host in the tests), and checked the table above and the "could sync from it" claim against a real git http-backend with main's worker.go. Its mutants (no check, one lane only, every non-http scheme exempt, the host in the cause, another class or reason) all failed the test, except the host checked after the URL length. Round 2 adds that test, the <<<.com and #x hosts, and two docs points: no route changes a stored host, and an older server during an upgrade can store one. Left open (P3): ADR 044 still names validUpstreamHost (a historical record), and a stored file:// row, which no release ever wrote, would fail as a clone or push error instead.
  • Round 2 (c20ef1f): 40e4416 and 9b4b519 merge main (fix(sync): send an upper-case http(s) scheme through the egress proxy #905, fix(githttp): cut the unknown ref policy op that a push refusal quotes #906); each conflicted only on the lane-schedule sync commit, which c20ef1f re-records on the merged tree. 72ffbda renames and rewords fix(sync): send an upper-case http(s) scheme through the egress proxy #905's lane tests (above). The Codex review and the Codex security review found nothing. A fresh read-only review found the code, tests, docs, merges and gates correct: git diff origin/main touches only this PR's files, the fingerprints hold all three PRs' entries, the two-guard claim and the order test hold under mutants, and the new table rows reproduce with main's worker.go. Its one P2 was this PR body, still the round-1 text at the new head; it checked this text claim by claim. Left open (P3): http-api.md says the WARN line gives the cause upstream host is not a valid host, which the logged cause contains after repository configuration error: .

Found by the reviews of #903 (desk follow-up).


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: xhigh

as given: a name with letters outside ASCII, a number that curl reads as
an IPv4 address, a port outside 1-65535, an IPv4 literal with a zone, or
a base URL with `?`, `#`, a space or an upper-case scheme. A link stored
before that keeps its host, and every run handed it to git. Depending on
the host, git or curl refused the URL, or curl dialed an address that the
registration check never saw (2130706433 is 127.0.0.1), and the failure
class varied.

Each pull and push run now checks the stored host with the rule of
POST /repos before git starts. A host that fails it fails the run as
`repository configuration error`, and the cause is the reason that
POST /repos gives, `upstream host is not a valid host`, without the
host.

ValidUpstreamHost and its helpers move from internal/api to
internal/sync, because api imports sync and not the reverse, and
InvalidUpstreamHostReason is the shared reason. A file:// host stays
exempt: only the tests' fixtures store one, and GIT_ALLOW_PROTOCOL
refuses it in production. The docs tell operators to link such an
upstream again with the host in its ASCII form.

Signed-off-by: Thomas Kosiewski <tk@coder.com>

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_

Change-Id: I4bd9c08939f72be8e5b3a5c90bda11244103b1ae
Signed-off-by: Thomas Kosiewski <tk@coder.com>
worker.go and handlers_sync.go are covered code: upstreamURLFailure checks
the stored host, and the host rule moves to internal/sync.

Signed-off-by: Thomas Kosiewski <tk@coder.com>

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_

Change-Id: I52a372967181736ba269cb6f75dbde8513212b8b
Signed-off-by: Thomas Kosiewski <tk@coder.com>
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-17T03:40:13.109088Z c20ef1f Manual request
🔒 Security Review ✅ Completed 2026-11-17T03:40:59.451998Z c20ef1f Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: d75a8793f5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: d75a8793f5

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

The run now has cases for a resolver-refused ASCII name and a base URL
with a fragment, and a case that the host is checked before the URL
length. The docs say that no route changes the host of a stored link,
and that an older server during an upgrade can store such a host too.

Signed-off-by: Thomas Kosiewski <tk@coder.com>

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_

Change-Id: I11956a6d92baf4cdc08b52ed816caf13101db5df
Signed-off-by: Thomas Kosiewski <tk@coder.com>
Main gained #905 (both sync lanes decide on the egress proxy with
viaEgressProxy). The only conflict is the sync commit of the
lane-schedule model; it is re-recorded on the merged tree.

Signed-off-by: Thomas Kosiewski <tk@coder.com>

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_

Change-Id: I592ce6e29585d166e4b341c338bb234bdc0d0255
Signed-off-by: Thomas Kosiewski <tk@coder.com>
With the run's host check, #905's lane tests no longer reach the egress
proxy: the run refuses an `HTTP://` host before git starts. The tests
still pass only while one of the two guards holds, so they now say so in
their names and comments. TestViaEgressProxy pins the proxy decision by
itself.

Signed-off-by: Thomas Kosiewski <tk@coder.com>

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_

Change-Id: I9b3211586de1bf4cb092471eb629f0e63e886072
Signed-off-by: Thomas Kosiewski <tk@coder.com>
Main gained #906 (the hook quotes an unknown ref policy op with
errtext.Quote). The only conflict is the sync commit of the
lane-schedule model; the next commit re-records it on the merged tree.

Signed-off-by: Thomas Kosiewski <tk@coder.com>

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_

Change-Id: Idc3d6c16d4e6288a5a66f32fbe057badec2ba825
Signed-off-by: Thomas Kosiewski <tk@coder.com>
The merges brought #905's and #906's fingerprints; TLC passes on the
merged tree, so the sync commit moves to it. No fingerprint changes.

Signed-off-by: Thomas Kosiewski <tk@coder.com>

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `xhigh`_

Change-Id: I1e93add5a92183dda736b91f90b89090e7e46ff4
Signed-off-by: Thomas Kosiewski <tk@coder.com>
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: c20ef1facc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: c20ef1facc

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 3385593 Nov 17, 2026
16 checks passed
@ThomasK33
ThomasK33 deleted the sync-stored-host-check branch November 17, 2026 04:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant