OpenEXR: Integer overflow in the HTJ2K decoder leads to heap-buffer-overflow

An attacker crafts a malicious HTJ2K-compressed EXR file with a channel width of at least 536870912 for FLOAT data (bytes_per_element=4). When `ht_undo_impl()` computes `width * bytes_per_element` in 32-bit signed arithmetic, the product exceeds INT32_MAX, causing a signed integer overflow [CWE-190]. The corrupted offset is later used for pointer arithmetic, leading to a heap out-of-bounds write [ref_id=1]. No authentication is required; the file is delivered over the network or via any vector that causes the library to decode the crafted image.

The vulnerability resides in `src/lib/OpenEXRCore/internal_ht.cpp` in the `ht_undo_impl()` function. At lines 188–189, `decode->channels[i].width` (int32_t) is multiplied by `decode->channels[i].bytes_per_element` (int8_t) using 32-bit signed arithmetic, and the overflowed result is stored in `raster_line_offset` and later used as a pointer offset at line 313. The same unchecked multiplication pattern appears in two other HTJ2K paths: bytes-per-line accumulation (line 213) and pixel-line pointer advancement (line 278).

The patch casts the operands to `size_t` (or `int64_t`) before multiplication at all three vulnerable locations in `internal_ht.cpp`. By promoting the 32-bit signed values to a wider unsigned or 64-bit type, the multiplication no longer wraps around INT32_MAX, preserving the correct offset. The advisory confirms that the fix eliminates the ASAN finding and passes all 63 core tests [ref_id=1].