Skip to content

fix(proxy): try configured fallback providers after primary failures - #333

Open
liangzhechen655 wants to merge 1 commit into
algorithmicsuperintelligence:mainfrom
liangzhechen655:fix/proxy-fallback-providers-20261009
Open

liangzhechen655 wants to merge 1 commit into
algorithmicsuperintelligence:mainfrom
liangzhechen655:fix/proxy-fallback-providers-20261009

Conversation

@liangzhechen655

Copy link
Copy Markdown

When a healthy primary provider fails during a request, ProxyClient exhausts its initial primary-provider list and immediately switches to the server's default client. Configured fallback_only providers are only considered if no primary is healthy before the request starts, so they are skipped in the failure path they are meant to handle.

Try unattempted configured fallback providers after the primary candidates are exhausted, before using the default client. The existing router, model mapping, timeout handling, health updates, and semaphore cleanup apply to these requests too. Each provider is attempted at most once per request.

Regression coverage includes all three routing strategies, upstream failures/timeouts, exhausted primary capacity, multiple primary and backup providers, default-client ordering, failure reporting, and released concurrency slots. An additional test uses the real OpenAI SDK with httpx.MockTransport to exercise HTTP 503 followed by a successful mapped-model request to the backup.

Validation on Windows, Python 3.13:

  • The initial new regressions fail on unmodified main, including the backup being skipped with all three routing strategies.
  • python -m pytest tests/test_proxy_failover.py tests/test_proxy_plugin.py tests/test_ssl_config.py tests/test_conversation_logger.py tests/test_conversation_logging_approaches.py -q: 45 passed, 6 subtests passed.
  • Ruff lint and formatting checks for the new test module, and git diff --check: passed.

Tests use the proxy-only dependency set plus test dependencies and Beautiful Soup for the existing SSL tests. Local-model inference and live-provider integration suites were not run.

This PR was generated with Codex (GPT-6); the regression tests and checks above were executed locally.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants