Skip to content

GH-3820: Exercise explicit binary statistics truncation - #3821

Open
dhruv-15-03 wants to merge 1 commit into
apache:masterfrom
dhruv-15-03:gh-stats-truncation-test
Open

dhruv-15-03 wants to merge 1 commit into
apache:masterfrom
dhruv-15-03:gh-stats-truncation-test

Conversation

@dhruv-15-03

Copy link
Copy Markdown

Rationale for this change

The binary-statistics truncation test calls a static default overload through a configured converter instance. The constructor's truncation length is ignored, and the existing ordering-only assertions let untruncated bounds pass.

What changes are included in this PR?

  • Call the explicit toParquetStatistics(stats, truncateLen) overload.
  • Use deterministic byte arrays and check exact truncated bytes and lengths, including the conservative increment of the maximum.
  • Check that combined bounds of 4095 bytes are accepted and 4096 bytes are omitted, with both default and finite truncation.
  • Preserve invalid-length constructor checks and current null-count omission behavior.

Only TestParquetMetadataConverter.java changes. The personal-fork validation workflow is not included.

Are these changes tested?

Yes. On the contribution commit a9940be5cfee6962dc0f3e064b4c619a07b2d3b7 (Temurin 17), the full TestParquetMetadataConverter class ran 78 tests with 0 failures, 0 errors and 0 skipped, including testBinaryStatsWithTruncation and testBinaryStatsWithTruncationAtSizeLimit. mvn -pl parquet-hadoop spotless:check reported 262 files clean. Run: https://github.com/dhruv-15-03/parquet-java/actions/runs/36249258685

Are there any user-facing changes?

None. Production code and output semantics are unchanged. #3819 addresses null-count serialization; this PR repairs a separate test-coverage gap and does not fix or close #3574.

Closes #3820

Use the explicit truncation overload and deterministic byte assertions. Cover the strict combined statistics-size boundary while preserving null-count omission and constructor validation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings September 26, 2026 14:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Binary statistics truncation test ignores its configured truncation length null_count is omitted for large columns in parquet files

2 participants