Promoted extended-depth grayscale (9-15 bits -> u16, 17-31 bits -> u32)
can carry bits above the declared depth after the horizontal predictor,
which reconstructs prev+delta in the promoted storage width rather than
modulo 2^BitsPerSample. This underflowed `max - v` in invert_colors (the
image-rs/image-tiff#396 panic in debug, wrong pixel in release) and left
junk in the high bits of the non-inverted output (#396 review, @197g:
"clamp the channel values in the non-inverted color case").
An n-bit predictor delta is stored in exactly n bits, i.e. modulo 2^n,
so reconstruction is defined modulo 2^n as well. Since 2^n divides the
u16/u32 storage modulus, masking each promoted sample with (1<<n)-1 once
after the predictor yields exactly the mod-2^n value -- including
legitimate wrap-arounds -- and also bounds any out-of-range sample from
malformed input. invert_colors' `max - v` is then always in range,
superseding the targeted saturating_sub from #396.
mask_samples_to_bit_depth runs in every integer decode branch of
expand_chunk; outside the extended-depth unpacking branch it is a
statically provable no-op (promoted layouts, declared < storage, occur
only there -- sub-byte depths keep byte-granular storage and reject the
horizontal predictor, standard depths have declared == storage, and the
float predictor requires IEEEFP samples that are never promoted), so the
branches cannot diverge.
Adds exact-pixel regression tests on hand-built TIFFs: 12/20-bit
wrap-around (plain and WhiteIsZero) and a 9-31-bit encode/decode
identity sweep, alongside the two fuzzer-minimized panic seeds. A
saturating clamp corrupts the wrap-around cases -- 12-bit [4095, 0, 1]
is stored as deltas [4095, 1, 1] and masks back to [4095, 0, 1], but
saturating produced [4095, 4095, 4095] -- so the tests lock the modular
semantics.
`1usize << bits_per_sample` panics when bits_per_sample >= 64.
Use checked_shl to return a LimitsExceeded error instead.
Found by fuzzing decode_comprehensive with a BE TIFF containing
PhotometricInterpretation=RGBPalette and BitsPerSample=83.
Currently, when getting the LZWStatus::NoProgress from the underlying
weezl library, tiff checks that no input bytes were read and no
output bytes were written. Although this is according to what the
weezl library mentions in their doc-comment, it's not what the real
result always looks like.
Our fuzzer found input that triggered exactly this case, where the LZW
reader is able to write some output byte before hitting the NoProgress
status.
This change removes the assert, as the resulting action is anyways to return
an error that can be handled by the caller.
Lilith River