Skip to content

fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini - #11449

Merged
lpcox merged 5 commits into
mainfrom
copilot/fix-budget-notices-prompt-cache
Nov 11, 2026
Merged

lpcox merged 5 commits into
mainfrom
copilot/fix-budget-notices-prompt-cache

Conversation

Copilot AI commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Budget notices ([AWF AI CREDIT WARNING] / [AWF TIME WARNING]) on Messages, Chat Completions and Gemini were placed near the start of the prompt, so the provider cache missed for everything after them. On Messages that meant everything after the tool definitions; on Chat Completions and Gemini, the whole conversation. This PR appends the notice after the last tool result on every API, as #9884 already does for Responses, and leaves the top-level system prompt and instructions unchanged.

Placement (injectSteeringMessage, containers/api-proxy/body-transform.js)

API Before After
Anthropic / Copilot Messages Appended to system {type:"text"} block after the tool_result blocks in the final user message. String content becomes [text, notice].
Chat Completions system message after the existing system messages Final {role:"user"} message
Gemini Appended to systemInstruction.parts {text} part after the function responses in the final user turn. A final function-role turn is followed by a new user turn.
Responses Final developer item Unchanged
// Messages, final user turn after injection
{ "role": "user", "content": [
  { "type": "tool_result", "tool_use_id": "toolu-1", "content": "…", "cache_control": { "type": "ephemeral" } },
  { "type": "text", "text": "[AWF AI CREDIT WARNING] …" }
]}

Behaviour notes

  • No injection on empty or assistant-ending requests. If the conversation is empty or ends in an assistant or model turn (for example a prefill), the request is left alone. The warning stays pending until a request ends in a user or tool turn; previously these requests got the notice in system.
  • Cache breakpoints: the last breakpoint before the notice stays where it was. This is the harness's own breakpoint, or AWF's tail breakpoint when anthropicAutoCache is on. The notice block itself carries no cache_control.
  • Wire translation: a Chat Completions request translated to Responses keeps the notice as the last input item. It is a user message, so translateChatMessages doesn't move it into instructions. Responses → Chat already kept it last.
  • Vertex: requests with a messages array now get a final user message. Previously a system message was inserted into messages, which Anthropic rejects. Native Vertex Gemini bodies are still not steered.

Tests and docs

  • Byte-identical prefix: for each API, everything up to the end of the last tool result matches the same request without a notice.
  • Cache breakpoint: with anthropicAutoCache on and off, the breakpoint stays on the block just before the notice.
  • Translation: Chat → Responses and Responses → Chat keep the notice last and leave instructions unchanged.
  • Docs: updated docs/awf-config-spec.md §10.1 and docs/api-proxy-sidecar.md ("Runtime and AI-credit steering").

…every API

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix budget notices breaking prompt cache on messages and completions Put budget notices at the end of the conversation on Messages, Chat Completions and Gemini Nov 11, 2026
Copilot AI requested a review from rabeckett November 11, 2026 05:19
@rabeckett
rabeckett requested a max review from Copilot November 11, 2026 05:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The warning can still invalidate the cached message prefix on older extended-thinking models by triggering a new assistant loop that strips prior thinking blocks.

1 open issue
What changed in this PR

Moves AWF budget and time warnings to the conversation tail for Messages, Chat Completions, and Gemini, extending the existing Responses approach to preserve cached prompt prefixes.

Changes:

  • Append notices after tool results without changing top-level prompts; defer injection for empty or assistant/model-ending conversations on the changed APIs.
  • Expand regression coverage for prefix preservation, cache breakpoints, one-shot delivery, and wire translation.
  • Update provider-specific steering documentation.
File Description
docs/​awf-config-spec.md Documents notice placement and injection eligibility.
docs/​api-proxy-sidecar.md Updates the provider-specific steering reference.
containers/​api-proxy/​server.token-steering.test.js Updates request-level assertions for tail notices.
containers/​api-proxy/​body-transform.test.js Adds prefix-preservation and cache-breakpoint checks.
containers/​api-proxy/​body-transform.js Implements conversation-tail injection and eligibility checks.
containers/​api-proxy/​body-handler.test.js Expands end-to-end steering and translation checks.

Review effort: Max

Comment thread containers/api-proxy/body-transform.js Resolved
… notices

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The documented skip rule incorrectly includes Responses, which still appends a developer notice and consumes the pending warning for empty or assistant-ending input arrays.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Low severity Responses warnings are consumed for empty or assistant-ending inputs

docs/​awf-config-spec.md:1375

Responses still appends a developer notice for empty or assistant-ending input arrays and consumes the pending warning. Scope the skip rule to Messages, Chat Completions, and Gemini so it does not promise that Responses warnings stay pending. Same issue in docs/api-proxy-sidecar.md:2096 and containers/api-proxy/body-transform.js:93–94.

1 resolved since last review

Review effort: Max

…mpletions and Gemini

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human review is needed to weigh unpublished, optional observations about test-assertion precision and documentation accuracy.

No open issues 🎉

Review effort: Max

@github-actions

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@lpcox
lpcox marked this pull request as ready for review November 11, 2026 16:03
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Documentation Preview

Documentation has been built for this PR.

Download preview artifact

To view locally:

  1. Download the docs-preview-pr-11449 artifact from the workflow run
  2. Unzip and open index.html in your browser

Built from commit 599f531

@github-actions

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 93.87% 93.88% ➡️ +0.01%
Statements 92.42% 92.42% ➡️ +0.00%
Functions 93.11% 93.11% ➡️ +0.00%
Branches 86.80% 86.81% 📈 +0.01%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/nvx/one-shot-process.ts 81.5% → 84.0% (+2.47%) 74.7% → 76.8% (+2.11%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@rabeckett rabeckett changed the title Put budget notices at the end of the conversation on Messages, Chat Completions and Gemini fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini Nov 11, 2026
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Claude passed

Generated by Smoke Claude for #11449

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

Smoke Copilot Network Isolation

Outcome: reported_action

The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Smoke Copilot Network Isolation · auto (Claude Sonnet 5.5) · ~16 AIC · ⊞ 9.8K · ◷

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

✅ Security Guard completed successfully!

Security Guard

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Security Guard · auto (Claude Sonnet 5.5) · ~13 AIC · ⊞ 9.0K · ◷

Generated by Security Guard for #11449

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

✅ Build Test Suite completed successfully!

Generated by Build Test Suite for #11449

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Smoke Codex

Outcome: reported_action

The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Smoke Codex · gpt54 · ~16 AIC · ⊞ 22.5K · ◷

🔮 The oracle has spoken through Smoke Codex

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

@github-actions

Copy link
Copy Markdown
Contributor

Smoke (Copilot): PR "feat: send each budget notice once to every conversation in the run"

  • ✅ GitHub MCP
  • ✅ GitHub.com (200)
  • ✅ File write/read

Overall: PASS — @Copilot @rabeckett

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Cloud Hypervisor + Copilot smoke test

  1. Merged PR lookup: PASS
  2. github.com connectivity: PASS (HTTP 200)
  3. File write/read: PASS
  4. example.com blocked: PASS (HTTP 000)

Overall: PASS

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine

  • API status: ✅ PASS
  • GitHub check: ✅ PASS
  • File status: ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #11449 · claude · haiku45 · ~14 AIC · ⊞ 6.3K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct) Mode ✅

Test Result
MCP Connectivity ✅
GitHub.com HTTP 200 ✅
File Write/Read ✅
BYOK Inference ✅

Mode: Direct BYOK (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com

Status: PASS

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke services: Redis PING ✅, pg_isready ✅, SELECT 1 ✅ — PASS

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

OTEL smoke test

  • ✅ S1 Module loading: otel.js exports startRequestSpan, setTokenAttributes, endSpan, shutdown, isEnabled, etc.
  • ✅ S2 Tests: 3 suites, 75/75 passed
  • ✅ S3 Env forwarding: trace context in env-passthrough.ts; OTEL vars in api-proxy-env-config.ts
  • ✅ S4 Token tracker: onUsage hook present in token-tracker-http.js
  • ✅ S5 Diagnostics: otel.jsonl present (1 line; the post-step output was not available to me)

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

PRs: Use published AWF artifacts for refactoring scanner; fix: preserve mounted hosts inode during enclave gateway setup
Merged PR read ✅
PR detail read ✅
Playwright ❌
File write/read ✅
Discussion comment ❌
Build ✅
Overall: FAIL

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

  • ✅ Allowed domain (api.github.com → 200)
  • ✅ Blocked domain (example.com blocked)

Overall: PASS — @Copilot

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison

Runtime Host Version Chroot Version Match?
Python Python 3.12.16 Python 3.12.16 ✅
Node.js v24.22.2 v22.23.2 ❌
Go go1.22.12 go1.22.12 ✅

Result: Not all tests passed (Node.js version mismatch), so the smoke-chroot label was not added.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia ✅ 1/1 passed ✅ PASS
Bun hono ✅ 1/1 passed ✅ PASS
C++ fmt ✅ N/A ✅ PASS
C++ json ✅ N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world ✅ N/A ✅ PASS
.NET json-parse ✅ N/A ✅ PASS
Go color ✅ pass ✅ PASS
Go env ✅ pass ✅ PASS
Go uuid ✅ pass ✅ PASS
Java gson ❌ 0/0 ❌ FAIL
Java caffeine ❌ 0/0 ❌ FAIL
Node.js clsx ✅ pass ✅ PASS
Node.js execa ✅ pass ✅ PASS
Node.js p-limit ✅ pass ✅ PASS
Rust fd ✅ 1/1 passed ✅ PASS
Rust zoxide ✅ 1/1 passed ✅ PASS

Overall: 7/8 ecosystems passed — FAIL

Java failure details: Maven could not resolve plugins from Maven Central: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty (the JVM truststore has no CA certificates). With the default ~/.m2 repository, Maven also failed with LocalRepositoryNotAccessibleException. Retrying with a disposable local repo hit the truststore error above.

.NET note: The first run crashed because /run/user/1001 was missing (NuGet mutex). The runs passed after setting XDG_RUNTIME_DIR to a temp directory.

Generated by Build Test Suite for #11449 · copilot · auto (Claude Sonnet 5.5) · ~20 AIC · ⊞ 5.9K · ◷
Add label ready-for-aw to run again

@lpcox
lpcox merged commit 4bb4b27 into main Nov 11, 2026
150 of 155 checks passed
@lpcox
lpcox deleted the copilot/fix-budget-notices-prompt-cache branch November 11, 2026 20:14
lpcox pushed a commit that referenced this pull request Nov 11, 2026
…hat Completions and Gemini (#11449)

* Initial plan

* fix(api-proxy): add budget notices at the end of the conversation on every API

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): note extended-thinking cache limitation for Messages notices

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): scope pending-warning skip rule to Messages, Chat Completions and Gemini

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
lpcox added a commit that referenced this pull request Nov 12, 2026
…11479)

* fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini (#11449)

* Initial plan

* fix(api-proxy): add budget notices at the end of the conversation on every API

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): note extended-thinking cache limitation for Messages notices

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): scope pending-warning skip rule to Messages, Chat Completions and Gemini

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* feat: send each budget notice once to every conversation in the run

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: steer Claude on Vertex, default Responses roles, skip token counting

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: steer Claude on Bedrock and give stored-response requests their own notice entry

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* perf: skip steering conversation inspection until a budget threshold is crossed

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: never steer Responses compaction requests

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix: key steering conversations with a per-process HMAC secret

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* test: assert steering conversation key properties instead of exact derivation

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: persist and log the opaque steering conversation prefix directly

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs: describe both cases where steering_delivery.conversation is null

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
lpcox added a commit that referenced this pull request Nov 12, 2026
…11479)

* fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini (#11449)

* Initial plan

* fix(api-proxy): add budget notices at the end of the conversation on every API

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): note extended-thinking cache limitation for Messages notices

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): scope pending-warning skip rule to Messages, Chat Completions and Gemini

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* feat: send each budget notice once to every conversation in the run

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: steer Claude on Vertex, default Responses roles, skip token counting

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: steer Claude on Bedrock and give stored-response requests their own notice entry

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* perf: skip steering conversation inspection until a budget threshold is crossed

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: never steer Responses compaction requests

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix: key steering conversations with a per-process HMAC secret

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* test: assert steering conversation key properties instead of exact derivation

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: persist and log the opaque steering conversation prefix directly

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs: describe both cases where steering_delivery.conversation is null

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
lpcox added a commit that referenced this pull request Nov 12, 2026
* feat: send each budget notice once to every conversation in the run (#11479)

* fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini (#11449)

* Initial plan

* fix(api-proxy): add budget notices at the end of the conversation on every API

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): note extended-thinking cache limitation for Messages notices

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): scope pending-warning skip rule to Messages, Chat Completions and Gemini

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* feat: send each budget notice once to every conversation in the run

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: steer Claude on Vertex, default Responses roles, skip token counting

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: steer Claude on Bedrock and give stored-response requests their own notice entry

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* perf: skip steering conversation inspection until a budget threshold is crossed

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: never steer Responses compaction requests

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix: key steering conversations with a per-process HMAC secret

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* test: assert steering conversation key properties instead of exact derivation

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: persist and log the opaque steering conversation prefix directly

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs: describe both cases where steering_delivery.conversation is null

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* feat: make budget notices state what is left and what to do

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: record superseded threshold on little-left delivery and round credit notices conservatively

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
lpcox added a commit that referenced this pull request Nov 12, 2026
* fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini (#11449)

* Initial plan

* fix(api-proxy): add budget notices at the end of the conversation on every API

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): note extended-thinking cache limitation for Messages notices

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): scope pending-warning skip rule to Messages, Chat Completions and Gemini

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* feat: send each budget notice once to every conversation in the run

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: steer Claude on Vertex, default Responses roles, skip token counting

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: steer Claude on Bedrock and give stored-response requests their own notice entry

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* perf: skip steering conversation inspection until a budget threshold is crossed

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: never steer Responses compaction requests

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix: key steering conversations with a per-process HMAC secret

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* test: assert steering conversation key properties instead of exact derivation

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* fix: persist and log the opaque steering conversation prefix directly

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs: describe both cases where steering_delivery.conversation is null

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* Initial plan

* feat: turn budget notices on by default

* docs: name budget steering as the notice switch

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

---------

Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Budget notices break the prompt cache on Messages, Chat Completions and Gemini

4 participants