Skip to content

Add batch decode for RLE (FLOAT/DOUBLE) and GORILLA_V1 encodings #1071

Open

Description

@ColinLeeo

PR #1068 introduced C++ FLOAT/DOUBLE RLE (FloatRLEDecoder / DoubleRLEDecoder in cpp/src/encoding/float_rle_decoder.h) and GORILLA_V1 (FloatGorillaV1Decoder / DoubleGorillaV1Decoder in cpp/src/encoding/gorilla_v1_decoder.h). Both decode one value at a time and never override the read_batch_* methods, so Decoder::read_batch_double / read_batch_int64 / ... (cpp/src/encoding/decoder.h) fall back to a per-value loop over read_double / read_int64.

Existing codecs already received batch decode: PLAIN in #823, GORILLA and TS_2DIFF in #898. The newly added encodings were missed.

Impact

Profiling a wide-table read (5214 columns, RLE-encoded) shows decode dominating the read path: DoubleRLEDecoder::read_double ~43% and Int64RleDecoder::read_int64 (time column) ~38%, essentially all inside the per-value loop.

Proposed changes

  1. Int64RleDecoder / Int32RleDecoder (cpp/src/encoding/int64_rle_decoder.h / int32_rle_decoder.h): add read_batch_int64 / read_batch_int32. read_int has two natural batch cases:

    • RLE run (is_rle_run_): the next current_count_ values are all the same rle_value_, so fill the output range in one pass.
    • bit-packing: current_buffer_ already holds the fully unpacked group; memcpy the remaining current_count_ values.
      This benefits both the time column (which uses Int64RleDecoder directly) and the FLOAT/DOUBLE value columns (which wrap it).
  2. FloatRLEDecoder / DoubleRLEDecoder: add read_batch_float / read_batch_double overrides that batch-decode the inner int values and then apply the per-value value_at scaling (which depends on page_pos_, so the scaling itself stays per-value).

  3. GorillaV1DecoderBase: add read_batch_* overrides following the pattern already in GorillaDecoder (cpp/src/encoding/gorilla_decoder.h).

Activity

  1. added this to the 2.5.1 milestone on Nov 5, 2026
  2. ColinLeeo commented on Nov 5, 2026

    @ColinLeeo
    ContributorAuthor

    The RLE half of this (items 1–2: Int64RleDecoder/Int32RleDecoder read_batch_int64/read_batch_int32 and FloatRLEDecoder/DoubleRLEDecoder read_batch_float/read_batch_double) is implemented in #1072. Item 3 (GORILLA_V1 batch decode) is still open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions