Skip to content

Keep each turn's outcome through engine finalization and use it for workflow and cron completion - #481

Merged
pufit merged 1 commit into
ClickHouse:mainfrom
groeneai:groeneai/turn-outcome
Oct 20, 2026
Merged

pufit merged 1 commit into
ClickHouse:mainfrom
groeneai:groeneai/turn-outcome

Conversation

@groeneai

@groeneai groeneai commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Closes #292 (the remaining scope; #428 fixed the Claude translator).

What breaks today

AgentEngine.run() returns normally for a turn that hit max turns, got an API error, was aborted or crashed after streaming. Cron then logs success, and a workflow run ends done because it reads sessions.status, which such a turn leaves (or restores to) active. An aborted turn gets no inline notice, and abnormal backend results are not logged.

Change

  • The engine records a TurnOutcome (completed / failed / interrupted + reason) where each turn ends. A normally finished turn publishes it only after it is saved; a failed save publishes failed.
  • run_cron / run_persistent_cron return TurnResult(text, outcome) for their own turn; cron logs error with the reason. run() still returns str.
  • engine.watch_turns(session_id) reports the latest outcome of a session's turns, background continuations included. A workflow run watches its own session through the existing quiescence wait and ends failed on an abnormal outcome (a missing one fails closed). The session-status check is gone, so turn failure stays separate from session health.
  • A failed or interrupted turn reported by the backend (max turns, an API error, an abort) gets an inline Turn <status>[: reason] notice and one WARNING line. The engine's own crash, stop and timeout paths keep their existing notices and log lines.

Scope is option A as agreed in #292 (#292 (comment)). Unchanged limits: a parked isolated cron run is logged from its launching turn; a workflow follow-up turn that ends without content does not change the result; the pre-query drain desync is untouched.

Behaviour changes

  • Failed or interrupted cron runs (a user stop included) are logged error, so they do not anchor the interval and are caught up once at restart. I can give interrupted runs their own status if you prefer.
  • A workflow run whose last turn was interrupted ends failed; a stop through Nerve is still recorded killed by the stop listener.
  • A review-loop leg that hits max turns is retried, then escalated, instead of going to the verifier.

Validation

tests/test_turn_outcomes.py: 26 integration tests (real engine and database) over the engine, cron and workflow paths; each of 13 mutations of the change fails them. Full suite: 6438 passed (main: 6412).

🤖 Generated with Claude Code

@groeneai

groeneai commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author
Internal second-model review: adjudication log (click to expand)

Pre-publication review by an independent model (engine: codex; 3 findings) plus my own review.

# Sev Finding Verdict Evidence / action
1 ⚠️ Crash, hard-cancel and background stream-end paths do not get the new Turn <status> notice and WARNING DISAGREE #292 asked for the notice and logging on backend results; those paths already show Agent error: ... / [Stopped by user] / [Background turn interrupted ...] and log at ERROR / INFO / WARNING, and keep doing so. The PR text now says the notice is for backend-reported results.
2 ⚠️ A forced cancel publishes interrupted before the best-effort save of the partial text DISAGREE interrupted is how the turn ended whether or not the partial row is saved; cron and workflow treat any non-completed outcome the same and only read the outcome after the turn has returned.
3 ⚠️ Worker onboarding still logs "setup session completed" whatever the outcome DISAGREE Unchanged from main, and it writes no cron log or workflow row, so it is outside cron/workflow completion. I can add the branch here if you want it.

Severity: ❌ blocker / ⚠️ major / 💡 nit. DISAGREE verdicts carry recorded evidence and are terminal per finding.

Session id: cron:clickhouse-review-slot-8:20261001-135100

@pufit

pufit commented Oct 20, 2026

Copy link
Copy Markdown
Member

@groeneai is it still relevant? If so - resolve the conflicts

…orkflow and cron completion

AgentEngine.run() returns normally for every turn that does not raise out
of the engine, so its callers could not tell how a turn ended. Cron logged
"success" for any normal return, and workflow runs inferred failure from
sessions.status, which a failed turn deliberately leaves (or restores to)
"active". A max-turns, API-error, aborted or crashed turn was therefore
reported as successful automation.

The engine now records a TurnOutcome (completed / failed / interrupted,
plus the reason) where each persisted turn ends:

- run_cron / run_persistent_cron return TurnResult(text, outcome) for the
  invocation's own turn (via _run_turn, which holds run()'s old body;
  run() still returns str), and cron logs "error" with the reason when
  the outcome is not completed.
- watch_turns(session_id) reports the latest outcome of a session's
  persisted turns, including autonomous background-continuation turns.
  A workflow run watches its own session over the launch and the existing
  quiescence loop and ends "failed" when that outcome is not completed;
  a missing outcome fails closed. The session-status inference is gone,
  so turn failure no longer depends on session health.
- The outcome is published only after the turn is persisted
  (_finalize_turn wraps the old body, now _persist_turn); a persistence
  failure or a cancel while saving publishes failed / interrupted.
- The autonomous drain exits that persist a partial turn (idle timeout,
  stream end, cancel) set the matching status instead of "completed".
- Every abnormal backend end gets an inline "Turn <status>[: reason]"
  notice (previously only failed turns with an error did) and one
  WARNING log line.

Scope follows the owner's choice of option A on ClickHouse#292: a parked isolated
cron run is logged from its launching turn, content-less autonomous turns
are not tracked, and the pre-query drain desync is left alone.

Closes ClickHouse#292

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@groeneai
groeneai force-pushed the groeneai/turn-outcome branch from c1db878 to 3bd7f48 Compare October 20, 2026 01:17
@groeneai

Copy link
Copy Markdown
Contributor Author

Yes, it is still relevant: on current main (5133895) a cron run whose turn hits max turns, is aborted or crashes is still logged success, and a workflow run with such a turn still ends done (I re-ran the repro against main).

I rebased the branch onto main and resolved the conflicts; it is still one commit (3bd7f48). The main adaptation: _run_turn now wraps main's queued run() body, and its duplicate-message and cancelled-while-queued returns report an interrupted outcome. Full suite passes.

@pufit
pufit merged commit 5274987 into ClickHouse:main Oct 20, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claude backend silently swallows abnormal turn terminations (max_turns, API errors, aborts)

2 participants