Repository navigation
HBASE-30480 Backup code mixes up region server casing - #8784
Open
DieterDP-ng wants to merge 1 commit into
Open
DieterDP-ng wants to merge 1 commit into
DieterDP-ng wants to merge 1 commit into
Conversation
|
@hgromer This fixes the casing issue I mentioned in the HBASE-29776 PR. |
|
Test failure is related to |
|
Adding @rmdmattingly and @ndimiduk as reviewers - If any of you are available. |
hgromer
reviewed
Oct 22, 2026
The backup code keys per-server state by host:port. WAL file names contain the lowercased hostname, while ServerName#getAddress keeps the hostname's original case. For a region server whose hostname contains a capital letter, the backup code therefore tracked the server under two keys, and lookups between them missed. Incremental backups then fell back to the oldest boundary of any server, so they could include WALs that an earlier backup already covered. Replaying those WALs could bring back deleted data, or fail the restore when the WALs held edits for a column family that was dropped since. Boundaries were also stored under the wrong key for later backups. The log cleaner missed the host's boundary and kept WALs longer than needed. Various tests were broken when running on a host with a capital letter. BackupUtils#toHostKey now lowercases the key. The WAL path parsers, the log roll (both variants) and the live-server check of the full backup use it. Reads from the backup system table and BackupBoundaries also normalize, because existing tables can hold mixed-case keys. When a host has a roll result under both spellings, the newest one is kept. Mixed-case keys that are already stored are phased out as follows: - The per-table boundaries (trslm rows) are rewritten with lowercase keys by the next backup that includes the table. A table that is not backed up again keeps its old row. Backups with continuous backup enabled do not rewrite this row. The HBASE-29776 code, which is not in any release, could store a host under both spellings in one row. The read keeps only one of those entries, so clusters that ran that code should take a full backup of all tables in each backup root after upgrading (mentioned in the migration notes for this ticket). - The roll results (rslogts rows) are never phased out, because nothing deletes these rows. Once the host rolls its WALs on the new code, its lowercase row holds the larger value, and the read keeps that value. - The boundaries copied into each backup's BackupInfo keep their mixed-case keys until that backup is deleted. BackupBoundaries normalizes them. The legacy log roll procedure looked up the previous roll result by hostname without port, so that lookup never matched. It now uses the full key. TestBackupOfflineRS now runs with a mixed-case region server hostname, so it covers this on hosts with lowercase names too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DieterDP-ng
force-pushed
the
HBASE-30480_backup-region-server-casing
branch
from
November 6, 2026 19:49
e2ecbe0 to
67b9de8
Compare
|
@hgromer I've addressed your point, could you review & approve? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The backup code keys per-server state by host:port. WAL file names contain the lowercased hostname, while ServerName#getAddress keeps the hostname's original case. For a region server whose hostname contains a capital letter, the backup code therefore tracked the server under two keys, and lookups between them missed. Incremental backups then fell back to the oldest boundary of any server, so they could include WALs that an earlier backup already covered. Replaying those WALs could bring back deleted data, or fail the restore when the WALs held edits for a column family that was dropped since. Boundaries were also stored under the wrong key for later backups. The log cleaner missed the host's boundary and kept WALs longer than needed. Various tests were broken when running on a host with a capital letter.
BackupUtils#toHostKey now lowercases the key. The WAL path parsers, the log roll (both variants) and the live-server check of the full backup use it. Reads from the backup system table and BackupBoundaries also normalize, because existing tables can hold mixed-case keys. On a collision, the newest roll result and the oldest boundary are kept.
Mixed-case keys that are already stored are phased out as follows:
The legacy log roll procedure looked up the previous roll result by hostname without port, so that lookup never matched. It now uses the full key.
TestBackupOfflineRS now runs with a mixed-case region server hostname, so it covers this on hosts with lowercase names too.