Skip to content

feat: turn budget notices on by default - #11486

Merged
lpcox merged 17 commits into
mainfrom
copilot/turn-budget-notices-on-by-default
Nov 12, 2026
Merged

lpcox merged 17 commits into
mainfrom
copilot/turn-budget-notices-on-by-default

Conversation

Copilot AI commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Makes apiProxy.steering.budget (alias enableTokenSteering / --enable-token-steering) default to true when neither the config file nor the CLI sets it. gh-aw already turns notices on for every workflow with a credit budget, so this mainly changes runs outside gh-aw.

Stacked on #11479 (#11445). It should not merge before #11443 and #11445 have shipped.

Resolution

  • resolveSteeringBudget (src/parsers/rate-limit-parsers.ts) now falls back to true instead of false.
  • --enable-token-steering no longer has a built-in false default. With that default, an unset alias would read as an explicit "off" and the new default would never apply.
  • Unchanged:
    • The explicit off forms: steering.budget: false, enableTokenSteering: false and --no-steering-budget.
    • The error when the alias and the canonical setting conflict.
    • The API-proxy requirement. It doesn't block the default because the proxy is always forced on before that check runs.
  • WrapperConfig.enableTokenSteering still means off when unset. Only the CLI path turns it on, so callers that build the config directly keep their current behaviour.
  • containers/api-proxy is not touched (Make budget notices say how much is left and what to do #11446 is changing it in parallel).
Config file CLI Notices
none / apiProxy: {} none on
none --no-steering-budget off
steering.budget: false or enableTokenSteering: false none off
steering.budget: false --steering-budget on (CLI wins)
any --enable-token-steering --no-steering-budget error

To turn notices off:

apiProxy:
  steering:
    budget: false

Docs and help text

  • CLI help for --steering-budget and --no-steering-budget.
  • Schema descriptions for steering.budget and enableTokenSteering (docs/awf-config.schema.json, regenerated into src/awf-config-schema.json).
  • docs/awf-config-spec.md: the option list and §10.1 no longer call budget steering opt-in.
  • docs/api-proxy-sidecar.md, "Runtime and AI-credit steering".
  • The docs-site CLI reference, which also called it opt-in.

Tests

  • Updated for the new default, the explicit off forms, and the alias flag being unset rather than false.

Copilot AI and others added 14 commits November 11, 2026 12:14
…hat Completions and Gemini (#11449)

* Initial plan

* fix(api-proxy): add budget notices at the end of the conversation on every API

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): note extended-thinking cache limitation for Messages notices

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

* docs(api-proxy): scope pending-warning skip rule to Messages, Chat Completions and Gemini

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
…ting

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
…own notice entry

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
…is crossed

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…rivation

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
…otice-once-per-conversation

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Copilot AI linked an issue Nov 11, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Turn budget notices on by default feat: turn budget notices on by default Nov 11, 2026
Copilot AI requested a review from rabeckett November 11, 2026 23:50
@rabeckett
rabeckett requested a max review from Copilot November 11, 2026 23:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The documentation misleadingly implies that maxAiCredits enables both notice types, although notices are enabled independently and runtime notices require container.agentTimeout.

1 open issue
What changed in this PR

This PR enables AI-credit and time-budget notices by default for AWF CLI runs. Explicit opt-outs remain supported, while callers that construct WrapperConfig directly retain their existing behavior.

Changes:

  • Default budget steering to on and remove the alias flag’s implicit false default.
  • Update tests for the new default, explicit disabling, and unset alias behavior.
  • Align CLI help, schema descriptions, and documentation with the new default.
File Description
src/​types/​rate-limit-options.ts Clarifies CLI versus direct-config behavior in documentation.
src/​parsers/​rate-limit-parsers.ts Changes the steering fallback to true.
src/​parsers/​rate-limit-parsers.test.ts Tests the enabled default and explicit alias opt-out.
src/​commands/​build-config.test.ts Updates configuration expectations for the new default.
src/​cli-options.ts Removes the alias’s false default and updates help text.
src/​cli-options.test.ts Expects an unset alias to remain undefined.
src/​awf-config-schema.json Updates generated steering descriptions.
docs/​awf-config.schema.json Documents the enabled default and opt-out forms.
docs/​awf-config-spec.md Revises budget-steering configuration guidance.
docs/​api-proxy-sidecar.md Updates default behavior and usage examples.
docs-site/​src/​content/​docs/​reference/​cli-reference.md Aligns CLI reference defaults and flag descriptions.

Review effort: Max

Comment thread docs/api-proxy-sidecar.md Resolved
Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Base automatically changed from copilot/send-budget-notice-once-per-conversation to main November 12, 2026 01:11
@lpcox
lpcox marked this pull request as ready for review November 12, 2026 01:16
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Smoke Chroot

Outcome: reported_action

The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Smoke Chroot · auto (Claude Sonnet 5.5) · ~13 AIC · ⊞ 8.2K · ◷

Tested by Smoke Chroot

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

Smoke OTel Tracing

Outcome: reported_action

The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Smoke OTel Tracing · auto (Claude Sonnet 5.5) · ~17 AIC · ⊞ 5.5K · ◷

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

✅ Build Test Suite completed successfully!

Generated by Build Test Suite for #11486

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

Smoke Copilot Network Isolation

Outcome: reported_action

The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Smoke Copilot Network Isolation · auto (Claude Sonnet 5.5) · ~15 AIC · ⊞ 9.8K · ◷

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Smoke Codex

Outcome: reported_action

The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Smoke Codex · gpt54 · ~12 AIC · ⊞ 22.5K · ◷

🔮 The oracle has spoken through Smoke Codex

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

Copy link
Copy Markdown
Contributor

Cloud Hypervisor + Copilot smoke test

  1. Merged PR lookup: PASS
  2. github.com connectivity: PASS (HTTP 200)
  3. File write/read: PASS
  4. example.com blocked: PASS (HTTP 000)

Overall: PASS

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia ✅ 1/1 passed ✅ PASS
Bun hono ✅ 1/1 passed ✅ PASS
C++ fmt ✅ N/A ✅ PASS
C++ json ✅ N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world ❌ N/A ❌ FAIL
.NET json-parse ❌ N/A ❌ FAIL
Go color ✅ ok (pass) ✅ PASS
Go env ✅ ok (pass) ✅ PASS
Go uuid ✅ ok (pass) ✅ PASS
Java gson ❌ 0/0 ❌ FAIL
Java caffeine ❌ 0/0 ❌ FAIL
Node.js clsx ✅ all passed ✅ PASS
Node.js execa ✅ all passed ✅ PASS
Node.js p-limit ✅ all passed ✅ PASS
Rust fd ✅ 1/1 passed ✅ PASS
Rust zoxide ✅ 1/1 passed ✅ PASS

Overall: 6/8 ecosystems passed — FAIL

Failures

  • .NET: dotnet restore crashed in first-time-use setup: System.IO.IOException ... NuGet-Migrations ... lstat("/run/user/1001") ENOENT. This is a sandbox limitation (missing /run/user/<uid>); setting DOTNET_CLI_HOME/TMPDIR did not help.
  • Java: Maven could not resolve plugins from Maven Central: InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty. The Java trust store is empty in this environment. The default ~/.m2 was also not accessible (LocalRepositoryNotAccessibleException), so a disposable local repo was used.

The build-test label was not added because of these failures.

Generated by Build Test Suite for #11486 · copilot · auto (Claude Sonnet 5.5) · ~28 AIC · ⊞ 5.9K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine

Check Result
API status ✅ PASS
GitHub check ✅ PASS
File status ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #11486 · claude · haiku45 · ~36 AIC · ⊞ 6.3K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

OTEL smoke test

  • ✅ S1 Module loading: otel.js loads; exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, routerCallTracing, shutdown, isEnabled
  • ✅ S2 Tests: 3 suites, 75/75 passed
  • ✅ S3 Env forwarding: trace ID/parent span ID forwarded in env-passthrough.ts and api-proxy-env-config.ts (OTLP vars too)
  • ✅ S4 Token tracker: onUsage hook present in token-tracker-http.js
  • ✅ S5 Diagnostics: local otel.jsonl has ~3.4KB of span output (file exporter). I did not check the OTLP export to a remote endpoint.

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Services smoke test: PASS

  • ✅ Redis PING → PONG
  • ✅ pg_isready → accepting connections
  • ✅ psql SELECT 1 → 1

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Copilot: PR "feat: turn budget notices on by default"

  • GitHub MCP ✅
  • github.com connectivity ✅
  • File write/read ✅

Overall: PASS — @Copilot @rabeckett

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison

Runtime Host Version Chroot Version Match?
Python 3.12.16 3.12.16 ✅ YES
Node.js v24.22.2 v22.23.2 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Result: ❌ Not all tests passed. Node.js differs between host and chroot, so the smoke-chroot label was not added.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct Mode) ✅ PASS

  • ✅ GitHub MCP: PR connectivity verified
  • ✅ GitHub.com: HTTP 200
  • ✅ File I/O: smoke-test-copilot-byok.txt readable
  • ✅ BYOK Inference: Direct mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com

Running in direct BYOK mode. All tests passed.

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions github-actions Bot added smoke-copilot-network-isolation Copilot network-isolation egress smoke test smoke-copilot-byok labels Nov 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

  • ✅ Allowed domain (api.github.com → 200)
  • ✅ Blocked domain (example.com blocked)
  • Overall: PASS — cc @lpcox

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke test

  • feat: make budget notices state what is left and what to do
  • feat: send each budget notice once to every conversation in the run
  • Merged PR read: ✅
  • PR detail read: ✅
  • Playwright title check: ❌
  • File write/read: ✅
  • Discussion lookup/comment: ✅
  • Build: ✅
  • Overall: FAIL

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@lpcox
lpcox merged commit 72336a0 into main Nov 12, 2026
148 of 149 checks passed
@lpcox
lpcox deleted the copilot/turn-budget-notices-on-by-default branch November 12, 2026 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Turn budget notices on by default

4 participants