You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Add batch decode for RLE (FLOAT/DOUBLE) and GORILLA_V1 encodings #1071
PR #1068 introduced C++ FLOAT/DOUBLE RLE (FloatRLEDecoder / DoubleRLEDecoder in cpp/src/encoding/float_rle_decoder.h) and GORILLA_V1 (FloatGorillaV1Decoder / DoubleGorillaV1Decoder in cpp/src/encoding/gorilla_v1_decoder.h). Both decode one value at a time and never override the read_batch_* methods, so Decoder::read_batch_double / read_batch_int64 / ... (cpp/src/encoding/decoder.h) fall back to a per-value loop over read_double / read_int64.
Existing codecs already received batch decode: PLAIN in #823, GORILLA and TS_2DIFF in #898. The newly added encodings were missed.
Impact
Profiling a wide-table read (5214 columns, RLE-encoded) shows decode dominating the read path: DoubleRLEDecoder::read_double ~43% and Int64RleDecoder::read_int64 (time column) ~38%, essentially all inside the per-value loop.
Proposed changes
Int64RleDecoder / Int32RleDecoder (cpp/src/encoding/int64_rle_decoder.h / int32_rle_decoder.h): add read_batch_int64 / read_batch_int32. read_int has two natural batch cases:
RLE run (is_rle_run_): the next current_count_ values are all the same rle_value_, so fill the output range in one pass.
bit-packing: current_buffer_ already holds the fully unpacked group; memcpy the remaining current_count_ values.
This benefits both the time column (which uses Int64RleDecoder directly) and the FLOAT/DOUBLE value columns (which wrap it).
FloatRLEDecoder / DoubleRLEDecoder: add read_batch_float / read_batch_double overrides that batch-decode the inner int values and then apply the per-value value_at scaling (which depends on page_pos_, so the scaling itself stays per-value).
GorillaV1DecoderBase: add read_batch_* overrides following the pattern already in GorillaDecoder (cpp/src/encoding/gorilla_decoder.h).
The RLE half of this (items 1–2: Int64RleDecoder/Int32RleDecoderread_batch_int64/read_batch_int32 and FloatRLEDecoder/DoubleRLEDecoderread_batch_float/read_batch_double) is implemented in #1072. Item 3 (GORILLA_V1 batch decode) is still open.
Activity
ColinLeeo commentedon Nov 5, 2026
The RLE half of this (items 1–2:
Int64RleDecoder/Int32RleDecoderread_batch_int64/read_batch_int32andFloatRLEDecoder/DoubleRLEDecoderread_batch_float/read_batch_double) is implemented in #1072. Item 3 (GORILLA_V1 batch decode) is still open.