Repository navigation
Fix the empty-level assert and wrong level labels in compaction resume - #15568
Open
omkarhgawde wants to merge 1 commit into
Open
omkarhgawde wants to merge 1 commit into
omkarhgawde wants to merge 1 commit into
Conversation
Summary: Why the change is required: When a resumable remote compaction (`OpenAndCompactOptions::allow_resumption`) resumes, `MaybeResumeSubcompactionProgressOnInputIterator()` restores the saved output files of the output level and, with per-key placement, of the proximal output level. It reads their table properties with `CompactionJob::ReadOutputFilesTableProperties()`. With per-key placement, a debug build aborted on resume when one of the two levels had no output files, and a failure on a proximal level file was logged as an output level failure. Fix: allow a level with no output files. A level can be empty when the progress is saved, for example when every key so far goes to the last level. `ReadOutputFilesTableProperties()` then failed `assert(!output_files.empty())`. Release builds compile the assert out, and the rest of the restore path already handles an empty list: there are no properties to read, no files or output records to restore, and no file numbers to skip. The assert is removed, with a comment that the list can be empty. Fix: label the level correctly. `level_type` was a function-level `static`, so the first call in the process fixed the label for every later call. The only caller also did not pass `is_proximal_level`, so it always took its default of `false`. The caller's own log printed "output levelfiles", with a missing space. `level_type` is now a normal local variable, the caller passes `is_proximal_level`, and the parameter default is removed so that a later caller cannot omit it. Safety and limitations: Release behavior does not change, except for the log and status text. The empty-level case needs per-key placement (the compaction outputs to the last level and `preclude_last_level_data_seconds` is set) and saved progress with files at only one of the two levels. Progress is not saved while per-key placement writes blob files. A related case, where the other level still has an open file when the progress is saved, is fixed separately in D128238210. Differential Revision: D128228840
|
@omkarhgawde has exported this pull request. If you are a Meta employee, you can view the originating Diff in D128228840. |
✅ Claude Code ReviewAuto-triggered after CI passed — reviewing commit ccdae60 SummaryClean, well-scoped bugfix that corrects two real defects in resumable remote compaction: (1) a debug-only assertion failure when one output level is empty, and (2) a High-severity findings (0): Full review (click to expand)Findings🟡 MEDIUMM1.
|
| Context | Affected? | Safe? | Notes |
|---|---|---|---|
| Per-key placement | Yes | Yes | Triggering context; fix correctly handles it |
| WritePreparedTxnDB | No | N/A | Resume doesn't interact with transaction visibility |
| ReadOnly DB | No | N/A | No compaction in read-only mode |
| DB mutex | No | N/A | ReadOutputFilesTableProperties does file I/O, not under DB mutex |
| Concurrent writers | No | N/A | Resume happens before compaction iteration starts |
Assumption stress-test:
- "empty output_files is safe": Verified all downstream consumers —
reserve(0)is harmless,RestoreCompactionOutputsloop is a no-op, file number skip loop is a no-op. No counterexample found. - "early-return guard doesn't prevent reaching the empty case": Confirmed — guard triggers only when BOTH levels have 0 records, but in the empty-proximal scenario the output level has records > 0.
Positive Observations
- Correct identification of the
staticlocal variable bug — a subtle C++ pitfall where the value is set only on the first call - Removal of default parameter follows CLAUDE.md guidance and prevents future omission bugs
- Well-constructed test using sync points to force the exact scenario
- Minimal, focused scope — changes only what's necessary
ℹ️ About this response
Generated by Claude Code.
Review methodology: claude_md/ci_review_prompt.md
Limitations:
- Claude may miss context from files not in the diff
- Large PRs may be truncated
- Always apply human judgment to AI suggestions
Commands:
/claude-review [context]— Request a code review/claude-query <question>— Ask about the PR or codebase
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Why the change is required:
When a resumable remote compaction (
OpenAndCompactOptions::allow_resumption) resumes,MaybeResumeSubcompactionProgressOnInputIterator()restores the saved output files of the output level and, with per-key placement, of the proximal output level. It reads their table properties withCompactionJob::ReadOutputFilesTableProperties(). With per-key placement, a debug build aborted on resume when one of the two levels had no output files, and a failure on a proximal level file was logged as an output level failure.Fix: allow a level with no output files. A level can be empty when the progress is saved, for example when every key so far goes to the last level.
ReadOutputFilesTableProperties()then failedassert(!output_files.empty()). Release builds compile the assert out, and the rest of the restore path already handles an empty list: there are no properties to read, no files or output records to restore, and no file numbers to skip. The assert is removed, with a comment that the list can be empty.Fix: label the level correctly.
level_typewas a function-levelstatic, so the first call in the process fixed the label for every later call. The only caller also did not passis_proximal_level, so it always took its default offalse. The caller's own log printed "output levelfiles", with a missing space.level_typeis now a normal local variable, the caller passesis_proximal_level, and the parameter default is removed so that a later caller cannot omit it.Safety and limitations:
Release behavior does not change, except for the log and status text. The empty-level case needs per-key placement (the compaction outputs to the last level and
preclude_last_level_data_secondsis set) and saved progress with files at only one of the two levels. Progress is not saved while per-key placement writes blob files. A related case, where the other level still has an open file when the progress is saved, is fixed separately in D128238210.Differential Revision: D128228840