Skip to content

Handle short reads from InputStream - #1061

Merged
JackieTien97 merged 5 commits into
developfrom
fix/read-fully-input-stream
Nov 3, 2026
Merged

JackieTien97 merged 5 commits into
developfrom
fix/read-fully-input-stream

Conversation

@shuwenwei

Copy link
Copy Markdown
Member

Problem

ReadWriteIOUtils assumed that a single InputStream.read() call would fill the requested buffer. The InputStream contract permits partial reads, so valid input could fail with Intend to read N bytes but M are actually returned.

This was exposed when a ConfigNode snapshot schema was read through a buffered stream: a schema larger than 8 KB crossed a buffer boundary and produced Intend to read 4 bytes but 3 are actually returned.

Change

  • Read fixed-width values and length-prefixed strings until the requested bytes are available or EOF is reached.
  • Treat single-byte reader EOF as EOFException.
  • Avoid zero-read infinite loops.
  • Remove an extra read in readAsPossible(TsFileInput, long, ByteBuffer) that silently consumed data.
  • Make BytesUtils.safeReadInputStreamToBytes fail on truncated input instead of returning zero-filled data.

Validation

Regression tests cover fragmented streams that return zero bytes once and then short reads. Tests and compilation were not run locally, per task constraints.

@codecov-commenter

codecov-commenter commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.52542% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.49%. Comparing base (931a5d3) to head (a72c55e).
⚠️ Report is 1 commits behind head on develop.

Files with missing lines Patch % Lines
...java/org/apache/tsfile/utils/ReadWriteIOUtils.java 91.83% 4 Missing ⚠️
.../main/java/org/apache/tsfile/utils/BytesUtils.java 90.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1061      +/-   ##
===========================================
+ Coverage    66.47%   66.49%   +0.02%     
===========================================
  Files          773      773              
  Lines        55802    55838      +36     
  Branches      8948     8955       +7     
===========================================
+ Hits         37092    37131      +39     
+ Misses       17053    17051       -2     
+ Partials      1657     1656       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JackieTien97
JackieTien97 merged commit 510ac39 into develop Nov 3, 2026
19 checks passed
@JackieTien97
JackieTien97 deleted the fix/read-fully-input-stream branch November 3, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants