Skip to content

Add prepared join index state - #995

Draft
ezrosent wants to merge 1 commit into
codex/packed-trie-split-02-packed-cachefrom
codex/packed-trie-split-03-prepared-index
Draft

ezrosent wants to merge 1 commit into
codex/packed-trie-split-02-packed-cachefrom
codex/packed-trie-split-03-prepared-index

Conversation

@ezrosent

@ezrosent ezrosent commented Aug 14, 2026 •

Copy link
Copy Markdown
Contributor

This PR is part of an eight-PR stack that jointly replaces #959.

Stack base: #969

Replacement stack:

  1. Add small residual join indexes #993 — small residual indexes
  2. Add packed join root caches #994 — packed root caches
  3. Add prepared join index state #995 — prepared index state
  4. Add packed join probe abstractions #996 — packed probe abstractions
  5. Add join-tail execution state #997 — join-tail execution state
  6. Use packed arena tries for generic joins #998 — executor integration
  7. Compact buffered free-join updates #999 — compact frame updates
  8. Lazily prepare packed join state #1000 — lazy preparation

Validation on the complete stack:

  • cargo fmt --check
  • cargo nextest run -p egglog-core-relations
  • 113 tests passed

Layer 3 of 8.

Adds execution-scoped prepared-index state, continuation caches, and tail-mask preparation in a dedicated implementation file. The executor wiring is intentionally deferred to #998.

Non-test diff: +473 / -0.

@codecov-commenter

codecov-commenter commented Aug 14, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.36%. Comparing base (b949610) to head (8046fd0).

Additional details and impacted files
@@                           Coverage Diff                            @@
##           codex/packed-trie-split-02-packed-cache     #995   +/-   ##
========================================================================
  Coverage                                    87.36%   87.36%           
========================================================================
  Files                                           96       96           
  Lines                                        32389    32389           
========================================================================
+ Hits                                         28296    28297    +1     
+ Misses                                        4093     4092    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed

codspeed Bot commented Aug 14, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 37 untouched benchmarks
⏩ 227 skipped benchmarks1


Comparing codex/packed-trie-split-03-prepared-index (8046fd0) with codex/packed-trie-split-02-packed-cache (b949610)

Open in CodSpeed

Footnotes

  1. 227 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@ezrosent
ezrosent force-pushed the codex/packed-trie-split-03-prepared-index branch from 3dea2ab to 892e543 Compare August 17, 2026 05:41
Comment thread core-relations/src/free_join/prepared_index.rs Outdated
Comment thread core-relations/src/free_join/prepared_index.rs
Comment thread core-relations/src/free_join/prepared_index.rs
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-03-prepared-index branch 2 times, most recently from 9b36649 to b896bf9 Compare August 17, 2026 05:52
}

impl PreparedJoinIndexes {
fn new(db: &Database, atoms: &Arc<DenseIdMap<AtomId, Atom>>, stages: &JoinStages) -> Self {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize I should have read this function first, which explains the high-level structure of this file. Maybe leave a note in the top-level doc that someone new to this file should start with this function

Uncacheable,
}

struct PreparedIndexSlot {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this struct hard to understand, probably because all of its uses are in execute.rs (in a later PR) via direct field access, and it lacks any associated methods to encapsulate operations over PreparedIndexSlot.

@ezrosent
ezrosent force-pushed the codex/packed-trie-split-03-prepared-index branch from b896bf9 to 397f205 Compare September 14, 2026 05:01
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-03-prepared-index branch 3 times, most recently from 8cfec0f to 05470c4 Compare September 14, 2026 05:32
@ezrosent
ezrosent removed this pull request from stack #1001 September 16, 2026 04:35
@ezrosent
ezrosent added this pull request to stack #1025 September 16, 2026 04:36
@ezrosent
ezrosent force-pushed the codex/packed-trie-split-03-prepared-index branch from 05470c4 to 8046fd0 Compare September 18, 2026 05:24

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants