Integer overflow and bypass of the check in raw2tiff

Integer overflow and bypass of the check in raw2tiff

Version 4.5.1

File: raw2tiff.c

PoC: raw2tiff -H 10 -l 1 -w -1 -b -1 -i band file.bin output.tif (with any file file.bin larger than 10 bytes).

Lines 128-134: Passing the parameters as in the POC line, Since width and band are converted to unsigned int, width and band are set to MAX_INT.

Line 514: and since the test at guess_size function is bypassed: (thanks to integer overflow), the function will return 1

At line 282: linebytes is set to MAX_INT and at line 291, bufsize set to 1 (integer overflow again). which causes a heap overflow during the memcpy at line 328 (after 1 iteration).

--

CVE-2023-38289

Reported by INTEL ASSERT (Cohen, Yaakov; Butterman, Yocheved;Frolov, Polina; Haenel, Arie)