tif_dir: unset transferfunction field if necessary (CVE-2018-19210)

Hugo Lefeuvre requested to merge hlef/libtiff:master into master

This MR addresses http://bugzilla.maptools.org/show_bug.cgi?id=2820.

Issue summary

This issue is very similar to http://bugzilla.maptools.org/show_bug.cgi?id=2500.

In short: when reading an OJPEG compressed image with missing SamplesPerPixel tag the SamplesPerPixel field is updated after setting the transfer table.

This is because of the OJPEG hack: https://gitlab.com/libtiff/libtiff/blob/master/libtiff/tif_dirread.c#L4028

This leads to the transfer table missing two entries.


I see two solutions.

The easy one: do the same as for issue 2500. If the number of entries is higher than before we invalidate the transfer table field and free previously allocated values. In the other case there's nothing to do, additional tf entries won't harm and properly written code will just ignore them since spp - es < 1.

But there is a problem with this fix, namely that after that the transfer table is missing. I am not sure this is compliant with the specification, but we are already doing it for SMaxSampleValue which is basically the same issue.

The risk is that some code might see that spp - es > 1 and assume that the transfer table contains three fields without checking for the FIELD_TRANSFERFUNCTION flag.

The second solution is similar but instead of removing the transfer table we replace it by the default one from the spec. However this forces us to add a new function to generate a default transfer table to libtiff/tif_dir.c which I find quite dirty.

As for now I have implemented the first fix since this is the most logical regarding what was done in the past.

Merge request reports