Skip to content

HBASE-30537 PrefixFilter compares bytes past the end of a row shorter than the prefix - #8868

Open
eomiks wants to merge 1 commit into
apache:masterfrom
eomiks:PrefixFilter-short-row-AIOOBE
Open

eomiks wants to merge 1 commit into
apache:masterfrom
eomiks:PrefixFilter-short-row-AIOOBE

Conversation

@eomiks

@eomiks eomiks commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

Fixes PrefixFilter#filterRowKey comparing bytes past the end of a row that is shorter than the prefix.

HBASE-28621 removed the row length check from filterRowKey, so the filter compares prefix.length bytes starting at the row offset even when the row is shorter than the prefix.

  • When the row is followed by other bytes of the cell (family length, family, ...), those bytes are compared as if they were part of the row. With family f and rows ab, ab\x00x, ab\x01f1, PrefixFilter("ab\x01f") returns [ab, ab\x01f1] instead of [ab\x01f1], and PrefixFilter("ab\x00") returns [] instead of [ab\x00x].
  • When the backing array holds nothing but the row, as in the fake cells built from the scan start row, the comparison throws ArrayIndexOutOfBoundsException. Before HBASE-27691 a scan with explicit columns and a start row shorter than the prefix handed such a cell to the filter.

filterRowKey now compares no more than the row itself. A row shorter than the prefix that matches the prefix as far as it goes is before the prefix, so a forward scan seeks to the prefix with the hint added by HBASE-28621 and a reversed scan is done.

Tests:

  • TestPrefixFilter: shorter rows in a row-only cell, a KeyValue and a ByteBufferKeyValue. Five of the six new tests fail without the fix (two with ArrayIndexOutOfBoundsException).
  • TestPrefixFilterShorterRow (new, mini cluster): the two scans above fail without the fix. The scans with explicit columns and a shorter start row pass on master with or without the fix because of HBASE-27691; on a 2.6.6 based branch without HBASE-27691 the forward ones fail with ArrayIndexOutOfBoundsException before the fix and pass after it.

Details: https://issues.apache.org/jira/browse/HBASE-30537

… than the prefix

HBASE-28621 removed the row length check from PrefixFilter.filterRowKey, so
the filter compares prefix.length bytes starting at the row offset even when
the row is shorter than the prefix.

- When the row is followed by other bytes of the cell (family length, family,
  ...), those bytes are compared as if they were part of the row. A shorter
  row can be included by mistake, or can end the scan before the matching
  rows are reached.
- When the backing array holds nothing but the row, as in the fake cells
  built from the scan start row, the comparison throws
  ArrayIndexOutOfBoundsException. Before HBASE-27691 a scan with explicit
  columns handed such a cell to the filter.

Compare no more than the row itself. A row shorter than the prefix that
matches the prefix as far as it goes is before the prefix, so a forward scan
seeks to the prefix and a reversed scan is done.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant