Conversation
Signed-off-by: Pawel Rein <pawel.rein@prezi.com>
Contributor
|
✅ DCO Check Passed Thanks @prein, all your commits are properly signed off. 🎉 |
Contributor
Merge Protections🟢 Merge protection satisfied — ready to merge. Show 1 satisfied protection🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
Signed-off-by: Paweł Rein <pawel.rein@prezi.com>
Author
|
We are running it in our preprod environment and it works as expected. It would be amazing if you could find time to review it and hopefully merge and release so we don't have to maintain it in a fork. |
|
any news on this? |
k-sparrow
pushed a commit
to k-sparrow/artemis
that referenced
this pull request
Aug 24, 2026
…or -> 429, not unhandled 500 QueueLimitExceededError (raised by RayOrchestrator.enqueue when DOCLING_SERVE_ENG_RAY_ENABLE_QUEUE_LIMIT_REJECTION rejects a full queue) has no registered FastAPI exception handler in stock docling-serve as of v1.29.0 through the current v1.31.0 -- it propagates as an unhandled 500 Internal Server Error, not the documented 429. Confirmed live against the real patched image: at MAX_QUEUED_TASKS=3, 17/20 concurrent requests all came back 500 "Internal Server Error" with a raw traceback in the logs, zero 429s. Without this, submit_parse/submit_chunk's 429-retry policy (src/backend/controller/worker/tasks.py) never actually engages against real backpressure -- it falls into the 5xx branch instead (wrong, short exponential-backoff cadence) and counts toward parsing_breaker's fail_max, tripping the breaker on ordinary backpressure. Filed upstream as docling-project/docling-serve#581; a fix exists (docling-project/docling-serve#583) but is unmerged, unreviewed, and additionally depends on a SystemCapacity class that doesn't exist in any released docling-jobkit -- too large/unstable a surface to backport wholesale, so app.py here mirrors just the handler shape from #583 (itself mirroring the existing RedisBackpressureError -> 503 handler already in stock docling-serve). The patch targets /opt/app-root/src/docling_serve (not site-packages, unlike the sibling models.py patch): docling_serve is installed editable (PEP 660), so Python's import resolves there regardless of what's overlaid at site-packages -- confirmed empirically inside the built image, same failure class as the lib/lib64 symlink gotcha models.py's own writeup describes, just via a different mechanism. Re-verified end-to-end against Artemis's real compose stack after fixing the target path: 70/70 concurrent requests past the queue limit now correctly return 429 with the expected body and Retry-After header. Mirrored into the docling-serve-ray sister repo, which independently patches the same base image. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on docling-project/docling-jobkit#126 (
SystemCapacity+get_capacity()).See #581 for full problem descriptio
Summary
GET /v1/capacity- returns system-level capacity snapshot (queue depth, active jobs, worker count); cached with configurable TTL (default 2s) to avoid per-request Redis scansQueueLimitExceededError→ 429 - fixes unhandled 500 when Ray queue limit rejection is enabled; returns capacity snapshot in response bodyadmission_max_queue_sizesetting; when set, rejects submissions with 429 beforeenqueue()if queue is fullIssue resolved by this Pull Request:
Resolves #581