Skip to content

Use packed arena tries for generic joins - #998

Draft
ezrosent wants to merge 1 commit into
codex/packed-trie-split-05-join-tailfrom
codex/packed-trie-split-06-executor
Draft

ezrosent wants to merge 1 commit into
codex/packed-trie-split-05-join-tailfrom
codex/packed-trie-split-06-executor

Conversation

@ezrosent

@ezrosent ezrosent commented Aug 14, 2026 •

Copy link
Copy Markdown
Contributor

This PR is part of an eight-PR stack that jointly replaces #959.

Stack base: #969

Replacement stack:

  1. Add small residual join indexes #993 — small residual indexes
  2. Add packed join root caches #994 — packed root caches
  3. Add prepared join index state #995 — prepared index state
  4. Add packed join probe abstractions #996 — packed probe abstractions
  5. Add join-tail execution state #997 — join-tail execution state
  6. Use packed arena tries for generic joins #998 — executor integration
  7. Compact buffered free-join updates #999 — compact frame updates
  8. Lazily prepare packed join state #1000 — lazy preparation

Validation on the complete stack:

  • cargo fmt --check
  • cargo nextest run -p egglog-core-relations
  • 113 tests passed

Layer 6 of 8.

Wires the supporting abstractions into generic-join execution and switches the executor to packed arena tries. Unit tests live in a separate execute_tests.rs file; end-to-end coverage remains in the crate test module.

Diff (including tests): +3130 / -2276, net +854.

@codecov-commenter

codecov-commenter commented Aug 14, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.90210% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.28%. Comparing base (3c94cc2) to head (eff0c6c).

Files with missing lines Patch % Lines
core-relations/src/free_join/residual_index.rs 81.81% 2 Missing ⚠️
core-relations/src/free_join/frame_update.rs 83.33% 1 Missing ⚠️
Additional details and impacted files
@@                           Coverage Diff                            @@
##           codex/packed-trie-split-05-join-tail     #998      +/-   ##
========================================================================
- Coverage                                 87.36%   87.28%   -0.09%     
========================================================================
  Files                                        96      101       +5     
  Lines                                     32369    33305     +936     
========================================================================
+ Hits                                      28280    29070     +790     
- Misses                                     4089     4235     +146     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed

codspeed Bot commented Aug 14, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 8.68%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 2 regressed benchmarks
✅ 35 untouched benchmarks
⏩ 227 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ tests[python_array_optimize] 1.1 s 1.3 s -10.39%
❌ tests[typeinfer] 115.8 ms 124.5 ms -6.95%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing codex/packed-trie-split-06-executor (eff0c6c) with codex/packed-trie-split-05-join-tail (3c94cc2)

Open in CodSpeed

Footnotes

  1. 227 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch from ce72f92 to f5e2783 Compare August 17, 2026 05:41
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch from f5e2783 to e002afb Compare August 17, 2026 05:47
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch from e002afb to feb28fb Compare August 17, 2026 05:52
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch from feb28fb to f2f3dc1 Compare August 17, 2026 05:57
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch from f2f3dc1 to f55562c Compare August 17, 2026 06:01
Comment thread core-relations/src/free_join/execute.rs
Comment thread core-relations/src/free_join/execute.rs Outdated
Comment thread core-relations/src/free_join/execute.rs Outdated
Comment thread core-relations/src/free_join/execute.rs
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch from f55562c to 0594652 Compare August 17, 2026 06:09
let can_use_catalog = root_range.is_some()
&& all_cacheable
&& constraints.is_empty()
&& !info.table.has_stale_rows()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this condition? We used to not need info.table.has_stale_rows() and instead do a filter_live() when scanning the subset.

Similarly for constraints.is_empty()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some tests and confirmed that this is required, as is. I agree it's a bit gross.

Comment thread core-relations/src/free_join/execute.rs Outdated
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch from 0594652 to 266d491 Compare September 14, 2026 05:01
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch from 266d491 to 0bdbc87 Compare September 14, 2026 05:12
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch from 0bdbc87 to 49ec3a0 Compare September 14, 2026 05:24
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch 2 times, most recently from fc2b24e to 7e0ebe8 Compare September 16, 2026 00:09
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch 2 times, most recently from c250c01 to 0364148 Compare September 16, 2026 03:52
@ezrosent
ezrosent removed this pull request from stack #1001 September 16, 2026 04:35
@ezrosent
ezrosent added this pull request to stack #1025 September 16, 2026 04:36
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-06-executor branch from 0364148 to eff0c6c Compare September 18, 2026 05:24

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants