Skip to content

Add join-tail execution state - #997

Draft
ezrosent wants to merge 1 commit into
codex/packed-trie-split-04-probesfrom
codex/packed-trie-split-05-join-tail
Draft

ezrosent wants to merge 1 commit into
codex/packed-trie-split-04-probesfrom
codex/packed-trie-split-05-join-tail

Conversation

@ezrosent

@ezrosent ezrosent commented Aug 14, 2026 •

Copy link
Copy Markdown
Contributor

This PR is part of an eight-PR stack that jointly replaces #959.

Stack base: #969

Replacement stack:

  1. Add small residual join indexes #993 — small residual indexes
  2. Add packed join root caches #994 — packed root caches
  3. Add prepared join index state #995 — prepared index state
  4. Add packed join probe abstractions #996 — packed probe abstractions
  5. Add join-tail execution state #997 — join-tail execution state
  6. Use packed arena tries for generic joins #998 — executor integration
  7. Compact buffered free-join updates #999 — compact frame updates
  8. Lazily prepare packed join state #1000 — lazy preparation

Validation on the complete stack:

  • cargo fmt --check
  • cargo nextest run -p egglog-core-relations
  • 113 tests passed

Layer 5 of 8.

Adds join-tail analysis, binding-state projection, ordering helpers, and worker-local state in a dedicated implementation file. The executor wiring is intentionally deferred to #998.

Non-test diff: +633 / -0.

@codecov-commenter

codecov-commenter commented Aug 14, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.37%. Comparing base (5d69f4c) to head (3c94cc2).

Additional details and impacted files
@@                          Coverage Diff                          @@
##           codex/packed-trie-split-04-probes     #997      +/-   ##
=====================================================================
+ Coverage                              87.34%   87.37%   +0.03%     
=====================================================================
  Files                                     96       96              
  Lines                                  32369    32369              
=====================================================================
+ Hits                                   28273    28284      +11     
+ Misses                                  4096     4085      -11     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed

codspeed Bot commented Aug 14, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 37 untouched benchmarks
⏩ 227 skipped benchmarks1


Comparing codex/packed-trie-split-05-join-tail (3c94cc2) with codex/packed-trie-split-04-probes (5d69f4c)

Open in CodSpeed

Footnotes

  1. 227 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@ezrosent
ezrosent force-pushed the codex/packed-trie-split-05-join-tail branch from 4b27e6a to fc12fe5 Compare August 17, 2026 05:41
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-05-join-tail branch from fc12fe5 to 9059f3e Compare August 17, 2026 05:47
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-05-join-tail branch from 9059f3e to 9ccc399 Compare August 17, 2026 05:52
Comment thread core-relations/src/free_join/join_tail.rs
Comment thread core-relations/src/free_join/join_tail.rs
Comment thread core-relations/src/free_join/join_tail.rs Outdated
Comment thread core-relations/src/free_join/join_tail.rs Outdated
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-05-join-tail branch 2 times, most recently from 8fd0e9c to c6a0f67 Compare August 17, 2026 06:01

@yihozhang yihozhang left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two thoughts:

  • materialization_is_live_in_tail may repeat the same analysis even when the variable ordering is unchanged
  • Sibling branches do not share analysis results, even though they may share the same (updated) variable ordering from DVO.

The first one is a low-hanging fruit IMO. We can follow up on them and benchmark the actual cost before deciding whether optimization is worthwhile.

Comment thread core-relations/src/free_join/join_tail.rs
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-05-join-tail branch from c6a0f67 to 0478405 Compare September 14, 2026 05:01
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-05-join-tail branch from 0478405 to 802a93c Compare September 14, 2026 05:12
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-05-join-tail branch 2 times, most recently from c4b9bb4 to c818bc2 Compare September 14, 2026 05:32
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-05-join-tail branch from c818bc2 to 199d107 Compare September 16, 2026 00:09
@ezrosent

Copy link
Copy Markdown
Contributor Author

+1... I think we should look at sharing some of that in a follow-up PR. Definite potential low-hanging fruit.

@ezrosent
ezrosent force-pushed the codex/packed-trie-split-05-join-tail branch from 199d107 to ec540ae Compare September 16, 2026 03:21
@ezrosent
ezrosent removed this pull request from stack #1001 September 16, 2026 04:35
@ezrosent
ezrosent added this pull request to stack #1025 September 16, 2026 04:36
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-05-join-tail branch from ec540ae to 3c94cc2 Compare September 18, 2026 05:24

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants