You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini - #11449
Budget notices ([AWF AI CREDIT WARNING] / [AWF TIME WARNING]) on Messages, Chat Completions and Gemini were placed near the start of the prompt, so the provider cache missed for everything after them. On Messages that meant everything after the tool definitions; on Chat Completions and Gemini, the whole conversation. This PR appends the notice after the last tool result on every API, as #9884 already does for Responses, and leaves the top-level system prompt and instructions unchanged.
{type:"text"} block after the tool_result blocks in the final user message. String content becomes [text, notice].
Chat Completions
system message after the existing system messages
Final {role:"user"} message
Gemini
Appended to systemInstruction.parts
{text} part after the function responses in the final user turn. A final function-role turn is followed by a new user turn.
Responses
Final developer item
Unchanged
// Messages, final user turn after injection
{ "role": "user", "content": [
{ "type": "tool_result", "tool_use_id": "toolu-1", "content": "…", "cache_control": { "type": "ephemeral" } },
{ "type": "text", "text": "[AWF AI CREDIT WARNING] …" }
]}
Behaviour notes
No injection on empty or assistant-ending requests. If the conversation is empty or ends in an assistant or model turn (for example a prefill), the request is left alone. The warning stays pending until a request ends in a user or tool turn; previously these requests got the notice in system.
Cache breakpoints: the last breakpoint before the notice stays where it was. This is the harness's own breakpoint, or AWF's tail breakpoint when anthropicAutoCache is on. The notice block itself carries no cache_control.
Wire translation: a Chat Completions request translated to Responses keeps the notice as the last input item. It is a user message, so translateChatMessages doesn't move it into instructions. Responses → Chat already kept it last.
Vertex: requests with a messages array now get a final user message. Previously a system message was inserted into messages, which Anthropic rejects. Native Vertex Gemini bodies are still not steered.
Tests and docs
Byte-identical prefix: for each API, everything up to the end of the last tool result matches the same request without a notice.
Cache breakpoint: with anthropicAutoCache on and off, the breakpoint stays on the block just before the notice.
Translation: Chat → Responses and Responses → Chat keep the notice last and leave instructions unchanged.
Docs: updated docs/awf-config-spec.md §10.1 and docs/api-proxy-sidecar.md ("Runtime and AI-credit steering").
…every API
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
CopilotAI
changed the title
[WIP] Fix budget notices breaking prompt cache on messages and completions
Put budget notices at the end of the conversation on Messages, Chat Completions and Gemini
Nov 11, 2026
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
The warning can still invalidate the cached message prefix on older extended-thinking models by triggering a new assistant loop that strips prior thinking blocks.
Moves AWF budget and time warnings to the conversation tail for Messages, Chat Completions, and Gemini, extending the existing Responses approach to preserve cached prompt prefixes.
Changes:
Append notices after tool results without changing top-level prompts; defer injection for empty or assistant/model-ending conversations on the changed APIs.
Expand regression coverage for prefix preservation, cache breakpoints, one-shot delivery, and wire translation.
Update provider-specific steering documentation.
File
Description
docs/awf-config-spec.md
Documents notice placement and injection eligibility.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
The documented skip rule incorrectly includes Responses, which still appends a developer notice and consumes the pending warning for empty or assistant-ending input arrays.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Responses warnings are consumed for empty or assistant-ending inputs
docs/awf-config-spec.md:1375
Responses still appends a developer notice for empty or assistant-ending input arrays and consumes the pending warning. Scope the skip rule to Messages, Chat Completions, and Gemini so it does not promise that Responses warnings stay pending. Same issue in docs/api-proxy-sidecar.md:2096 and containers/api-proxy/body-transform.js:93–94.
Coverage comparison generated by scripts/ci/compare-coverage.ts
rabeckett
changed the title
Put budget notices at the end of the conversation on Messages, Chat Completions and Gemini
fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini
Nov 11, 2026
The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.
Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
Generated from Security Guard · auto (Claude Sonnet 5.5) · ~13 AIC · ⊞ 9.0K · ◷
Java failure details: Maven could not resolve plugins from Maven Central: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty (the JVM truststore has no CA certificates). With the default ~/.m2 repository, Maven also failed with LocalRepositoryNotAccessibleException. Retrying with a disposable local repo hit the truststore error above.
.NET note: The first run crashed because /run/user/1001 was missing (NuGet mutex). The runs passed after setting XDG_RUNTIME_DIR to a temp directory.
Generated by Build Test Suite for #11449 · copilot · auto (Claude Sonnet 5.5) · ~20 AIC · ⊞ 5.9K · ◷ Add label ready-for-aw to run again
…hat Completions and Gemini (#11449)
* Initial plan
* fix(api-proxy): add budget notices at the end of the conversation on every API
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs(api-proxy): note extended-thinking cache limitation for Messages notices
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs(api-proxy): scope pending-warning skip rule to Messages, Chat Completions and Gemini
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
…11479)
* fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini (#11449)
* Initial plan
* fix(api-proxy): add budget notices at the end of the conversation on every API
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs(api-proxy): note extended-thinking cache limitation for Messages notices
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs(api-proxy): scope pending-warning skip rule to Messages, Chat Completions and Gemini
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* feat: send each budget notice once to every conversation in the run
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: steer Claude on Vertex, default Responses roles, skip token counting
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: steer Claude on Bedrock and give stored-response requests their own notice entry
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* perf: skip steering conversation inspection until a budget threshold is crossed
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: never steer Responses compaction requests
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* fix: key steering conversations with a per-process HMAC secret
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* test: assert steering conversation key properties instead of exact derivation
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: persist and log the opaque steering conversation prefix directly
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs: describe both cases where steering_delivery.conversation is null
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…11479)
* fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini (#11449)
* Initial plan
* fix(api-proxy): add budget notices at the end of the conversation on every API
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs(api-proxy): note extended-thinking cache limitation for Messages notices
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs(api-proxy): scope pending-warning skip rule to Messages, Chat Completions and Gemini
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* feat: send each budget notice once to every conversation in the run
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: steer Claude on Vertex, default Responses roles, skip token counting
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: steer Claude on Bedrock and give stored-response requests their own notice entry
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* perf: skip steering conversation inspection until a budget threshold is crossed
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: never steer Responses compaction requests
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* fix: key steering conversations with a per-process HMAC secret
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* test: assert steering conversation key properties instead of exact derivation
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: persist and log the opaque steering conversation prefix directly
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs: describe both cases where steering_delivery.conversation is null
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* feat: send each budget notice once to every conversation in the run (#11479)
* fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini (#11449)
* Initial plan
* fix(api-proxy): add budget notices at the end of the conversation on every API
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs(api-proxy): note extended-thinking cache limitation for Messages notices
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs(api-proxy): scope pending-warning skip rule to Messages, Chat Completions and Gemini
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* feat: send each budget notice once to every conversation in the run
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: steer Claude on Vertex, default Responses roles, skip token counting
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: steer Claude on Bedrock and give stored-response requests their own notice entry
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* perf: skip steering conversation inspection until a budget threshold is crossed
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: never steer Responses compaction requests
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* fix: key steering conversations with a per-process HMAC secret
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* test: assert steering conversation key properties instead of exact derivation
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: persist and log the opaque steering conversation prefix directly
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs: describe both cases where steering_delivery.conversation is null
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* feat: make budget notices state what is left and what to do
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: record superseded threshold on little-left delivery and round credit notices conservatively
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* fix: put budget notices at the end of the conversation on Messages, Chat Completions and Gemini (#11449)
* Initial plan
* fix(api-proxy): add budget notices at the end of the conversation on every API
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs(api-proxy): note extended-thinking cache limitation for Messages notices
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs(api-proxy): scope pending-warning skip rule to Messages, Chat Completions and Gemini
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* feat: send each budget notice once to every conversation in the run
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: steer Claude on Vertex, default Responses roles, skip token counting
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: steer Claude on Bedrock and give stored-response requests their own notice entry
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* perf: skip steering conversation inspection until a budget threshold is crossed
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: never steer Responses compaction requests
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* fix: key steering conversations with a per-process HMAC secret
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* test: assert steering conversation key properties instead of exact derivation
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* fix: persist and log the opaque steering conversation prefix directly
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* docs: describe both cases where steering_delivery.conversation is null
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
* Initial plan
* feat: turn budget notices on by default
* docs: name budget steering as the notice switch
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Landon Cox <landon.cox@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Budget notices (
[AWF AI CREDIT WARNING]/[AWF TIME WARNING]) on Messages, Chat Completions and Gemini were placed near the start of the prompt, so the provider cache missed for everything after them. On Messages that meant everything after the tool definitions; on Chat Completions and Gemini, the whole conversation. This PR appends the notice after the last tool result on every API, as #9884 already does for Responses, and leaves the top-level system prompt and instructions unchanged.Placement (
injectSteeringMessage,containers/api-proxy/body-transform.js)system{type:"text"}block after thetool_resultblocks in the final user message. String content becomes[text, notice].systemmessage after the existing system messages{role:"user"}messagesystemInstruction.parts{text}part after the function responses in the final user turn. A finalfunction-role turn is followed by a new user turn.developeritemBehaviour notes
system.anthropicAutoCacheis on. The notice block itself carries nocache_control.inputitem. It is a user message, sotranslateChatMessagesdoesn't move it intoinstructions. Responses → Chat already kept it last.messagesarray now get a final user message. Previously asystemmessage was inserted intomessages, which Anthropic rejects. Native Vertex Gemini bodies are still not steered.Tests and docs
anthropicAutoCacheon and off, the breakpoint stays on the block just before the notice.instructionsunchanged.docs/awf-config-spec.md§10.1 anddocs/api-proxy-sidecar.md("Runtime and AI-credit steering").