Skip to content

fix(jenkins): reject an out buffer of the wrong size - #895

Closed
Duo-ee wants to merge 1 commit into
zarr-developers:mainfrom
Duo-ee:fix/jenkins-out-size
Closed

Duo-ee wants to merge 1 commit into
zarr-developers:mainfrom
Duo-ee:fix/jenkins-out-size

Conversation

@Duo-ee

@Duo-ee Duo-ee commented Nov 12, 2026

Copy link
Copy Markdown

JenkinsLookup3.decode writes the payload with out.view("uint8")[:] = b[:-4], so an output buffer larger than the decoded payload silently broadcasts the single byte across the whole buffer:

codec = JenkinsLookup3()
out = np.zeros(4, dtype="u1")
codec.decode(codec.encode(b"x"), out=out)
out.tobytes()  # b'xxxx'

A buffer with the same byte count but another dtype/shape raises a broadcasting error, and a bytearray raises AttributeError.

This compares memoryview(out).nbytes with the payload before writing anything, and fills the buffer through a flat uint8 view:

j = JenkinsLookup3()
enc = j.encode(b"x")
out = np.zeros(4, dtype="u1")
j.decode(enc, out=out)  # ValueError: Output buffer has 4 bytes, but the decoded payload has 1 bytes.

The buffer is left untouched when the sizes differ, and a same-size bytearray or a multi-dimensional array now decodes as expected.

Fixes #894.

Tests: tests/test_jenkins.py gains test_out_too_large_is_rejected, test_out_with_other_dtype_is_filled and test_out_bytearray_is_filled.

@codecov

codecov Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.74%. Comparing base (4afa628) to head (f6473ba).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #895      +/-   ##
==========================================
+ Coverage   89.71%   89.74%   +0.03%     
==========================================
  Files          27       27              
  Lines         914      917       +3     
==========================================
+ Hits          820      823       +3     
  Misses         94       94              
Files with missing lines Coverage Δ
src/numcodecs/checksum32.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Duo-ee

Duo-ee commented Nov 12, 2026

Copy link
Copy Markdown
Author

This duplicates #891, which already fixes #894 (and #890), including bytearray, C/F-ordered matrices and strided destinations. I missed it because #894 does not list #891 under closedByPullRequestsReferences — opening this PR was redundant.

Please treat #891 as the fix; this branch can be closed.

@Duo-ee

Duo-ee commented Nov 13, 2026

Copy link
Copy Markdown
Author

Same note as on the sibling PRs: the red build jobs fail during collection with ValueError: invalid literal for int() with base 10: 'post0' from the zfpy 1.0.1.post0 version string, on every 3.12–3.14 job of every open PR in this repo. The change itself (and #896/#897) compiles and passes wherever zfpy is absent.

@Duo-ee Duo-ee closed this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JenkinsLookup3.decode can repeat a one-byte payload across an oversized output

1 participant