CVE-2023-0796

An attacker supplies a crafted TIFF file and invokes `tiffcrop` with options such as `-e multiple -z ... -R 270` to request cropping and rotation [ref_id=2]. The tool parses the malformed file, which contains invalid strip byte counts and other corrupt tags, and then attempts to extract and rotate sub-regions. During rotation of a sub-region, `rotateImage` incorrectly swaps the image dimensions stored in `image->width` and `image->length`, causing downstream code in `extractContigSamplesShifted24bits` to read beyond the allocated buffer [ref_id=1]. This results in a segmentation fault (denial of service) [ref_id=2].

The out-of-bounds read occurs in `extractContigSamplesShifted24bits` at `tools/tiffcrop.c:3592` [ref_id=2]. The call chain is `main` → `processCropSelections` → `extractSeparateRegion` → `extractContigSamplesShifted24bits` [ref_id=2]. The patch modifies the `rotateImage` function signature and its callers to add a `rot_image_params` parameter, preventing `rotateImage` from unconditionally swapping `image->width` and `image->length` when operating on sub-regions rather than the whole image [ref_id=1].

The fix adds a `rot_image_params` parameter (type `int`) to `rotateImage` and its callers [ref_id=1]. When `rot_image_params` is `TRUE` (whole-image rotation), the function swaps `image->width`/`image->length` and the x/y resolutions as before. When `FALSE` (sub-region rotation), it only updates the local `*img_width`/`*img_length` pointers without modifying the image-level dimensions [ref_id=1]. This prevents the dimension mismatch that caused the out-of-bounds read in `extractContigSamplesShifted24bits`. The patch also removes redundant `buffsize` initialization in `initCropMasks` and cleans up unused variable declarations [ref_id=1].

Build libtiff with AddressSanitizer: `CFLAGS="-g -fsanitize=address -fno-omit-frame-pointer" CXXFLAGS="-g -fsanitize=address -fno-omit-frame-pointer" ./configure --prefix=$PWD/build_asan --disable-shared && make -j && make install`. Then run: `./build_asan/bin/tiffcrop -e multiple -z 1,1,2048,2048:1,2049,2048,4097 -R 270 -i poc /tmp/foo` using the provided poc.zip [ref_id=2].