Fix NULL pointer dereference in TIFFPrintDirectory (bugzilla 2778/CVE-2018-7456)
The TIFFPrintDirectory function relies on the following assumptions, supposed to be guaranteed by the specification:
(a) A Transfer Function field is only present if the TIFF file has photometric type < 3.
(b) If SamplesPerPixel > Color Channels, then the ExtraSamples field has count SamplesPerPixel - (Color Channels) and contains information about supplementary channels.
While respect of (a) and (b) are essential for the well functioning of TIFFPrintDirectory, no checks are realized neither by the callee nor by TIFFPrintDirectory itself. Hence, following scenarios might happen and trigger the NULL pointer dereference:
(1) TIFF File of photometric type 4 or more has illegal Transfer Function field.
(2) TIFF File has photometric type 3 or less and defines a SamplesPerPixel field such that SamplesPerPixel > Color Channels without defining all extra samples in the ExtraSamples fields.
In this patch, we address both issues with respect of the following principles:
(A) In the case of (1), the defined transfer table should be printed safely even if it isn't 'legal'. This allows us to avoid expensive checks in TIFFPrintDirectory. Also, it is quite possible that an alternative photometric type would be developed (not part of the standard) and would allow definition of Transfer Table. We want libtiff to be able to handle this scenario out of the box.
(B) In the case of (2), the transfer table should be printed at its right size, that is if TIFF file has photometric type Palette then the transfer table should have one row and not three, even if two extra samples are declared.
In order to fulfill (A) we simply add a new 'i < 3' end condition to the broken TIFFPrintDirectory loop. This makes sure that in any case where (b) would be respected but not (a), everything stays fine.
(B) is fulfilled by the loop condition 'i < td->td_samplesperpixel - td->td_extrasamples'. This is enough as long as (b) is respected.
Naturally, we also make sure (b) is respected. This is done in the TIFFReadDirectory function by making sure any non-color channel is counted in ExtraSamples.