Repository navigation
Conversation
… than the prefix HBASE-28621 removed the row length check from PrefixFilter.filterRowKey, so the filter compares prefix.length bytes starting at the row offset even when the row is shorter than the prefix. - When the row is followed by other bytes of the cell (family length, family, ...), those bytes are compared as if they were part of the row. A shorter row can be included by mistake, or can end the scan before the matching rows are reached. - When the backing array holds nothing but the row, as in the fake cells built from the scan start row, the comparison throws ArrayIndexOutOfBoundsException. Before HBASE-27691 a scan with explicit columns handed such a cell to the filter. Compare no more than the row itself. A row shorter than the prefix that matches the prefix as far as it goes is before the prefix, so a forward scan seeks to the prefix and a reversed scan is done.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes
PrefixFilter#filterRowKeycomparing bytes past the end of a row that is shorter than the prefix.HBASE-28621 removed the row length check from
filterRowKey, so the filter comparesprefix.lengthbytes starting at the row offset even when the row is shorter than the prefix.fand rowsab,ab\x00x,ab\x01f1,PrefixFilter("ab\x01f")returns[ab, ab\x01f1]instead of[ab\x01f1], andPrefixFilter("ab\x00")returns[]instead of[ab\x00x].ArrayIndexOutOfBoundsException. Before HBASE-27691 a scan with explicit columns and a start row shorter than the prefix handed such a cell to the filter.filterRowKeynow compares no more than the row itself. A row shorter than the prefix that matches the prefix as far as it goes is before the prefix, so a forward scan seeks to the prefix with the hint added by HBASE-28621 and a reversed scan is done.Tests:
TestPrefixFilter: shorter rows in a row-only cell, aKeyValueand aByteBufferKeyValue. Five of the six new tests fail without the fix (two withArrayIndexOutOfBoundsException).TestPrefixFilterShorterRow(new, mini cluster): the two scans above fail without the fix. The scans with explicit columns and a shorter start row pass on master with or without the fix because of HBASE-27691; on a 2.6.6 based branch without HBASE-27691 the forward ones fail withArrayIndexOutOfBoundsExceptionbefore the fix and pass after it.Details: https://issues.apache.org/jira/browse/HBASE-30537