[yuxi-liu-wired]

A decoder made with new_decoder() panics when the first bytes of a potential BOM arrive at the end of one buffer, turn out not to be a BOM, and the next call's output buffer is too short for them. A four-byte buffer is enough to trigger it, and four bytes is the documented minimum for UTF-8:

let mut decoder = IBM866.new_decoder();
let mut dst = [0u8; 16];
decoder.decode_to_utf8(b"\xEF\xBB", &mut dst, false); // InputEmpty, both bytes held
let mut dst = [0u8; 4];
decoder.decode_to_utf8(b"ab", &mut dst, false);
// panicked at src/macros.rs: Output buffer must have been too small.

decode_to_utf_after_one_potential_bom_byte and decode_to_utf_after_two_potential_bom_bytes decode the held bytes into dst and treat OutputFull as unreachable. That's wrong whenever EF and BB decode to more than four UTF-8 bytes together. Going through every encoding, held prefix and next buffer, I found 704 panicking cases, all with EF BB held and a 4- or 5-byte dst:

• x-user-defined (U+F7EF U+F7BB),
• ISO-2022-JP (two U+FFFD),
• IBM866 (я╗),
• windows-874,
• the other single-byte encodings that map EF/BB outside Latin-1.

The same happens through decode_to_str and with the _without_replacement variants.

With this change, both functions return OutputFull and set the life cycle so that the next call replays what wasn't decoded:

• one held byte goes back to its Seen*First state;
• EF BB with nothing decoded goes back to SeenUtf8Second;
• EF decoded but BB not goes to ConvertingWithPendingBB, which the Malformed path already uses for that case.

out_read/first_read stay 0, because none of the held bytes came from the current src.

To check the resumed state, I ran every encoding, held prefix (EF, EF BB, FE, FF), seven next buffers and every output size from the minimum to 8, through decode_to_utf8, decode_to_utf8_without_replacement and decode_to_utf16, retrying after OutputFull. All 19,040 runs give exactly the spec's decode result and final encoding.

The new test test_short_buffer_after_non_bom_bytes covers IBM866, x-user-defined and ISO-2022-JP with every split of the input and buffers of 4–7 bytes, and compares the joined output. It panics without the change. cargo test passes on stable, 1.88, --no-default-features, std, simd-accel (nightly), and the fast-legacy-encode / less-slow-* features. cargo fmt --check passes too.

I found this by fuzzing the streaming decoders against a transcription of the Encoding Standard, with BOM sniffing, random chunking and small output buffers. The test is dedicated to the public domain like the rest of the test module.

[hsivonen]

Thank you!