webp support

Add support for WebP compression.

added 1 commit

• 1d5e1a45 - add webp support

Compare with previous version

@normanb Sorry, I missed an important point in my review on the GDAL side. I think the logic in TWebPDecode() should be revised to avoid the restriction of https://gitlab.com/libtiff/libtiff/blob/1d5e1a45c82594215ca0e97a0005bc1c6fb95a1c/libtiff/tif_webp.c#L175 The TIFF codec decoding API is in streaming mode. Decode() can potentially be called asking for an arbitrary number of pixels. Typical uses with the libtiff public API are a whole tile, whole streep or a scanline. Your code will handle will not handle the later I had the similar issue with the LERC codec. The solution I came with is to decompress the whole tile/strip in PreDecode() in a internal buffer, and then have Decode() copy the requested chunks of that internal buffer to the user output buffer. See https://github.com/OSGeo/gdal/blob/master/gdal/frmts/gtiff/tif_lerc.c#L496

For the CHUNKY_STRIP_READ_SUPPORT specific issue, you can add an exception similarly to the LERC case: https://gitlab.com/libtiff/libtiff/blob/master/libtiff/tif_read.c#L349 That way PreDecode() will get the whole strip / tile.

Similarly on the writing side, you can accumulate in an internal buffer the uncompressed bytes send to Encode(), and in PostEncode() you compress and flush to disk the buffer.

added 1 commit

• bf3451f8 - progressive decoder

Compare with previous version

@rouault I have added progressive webp decode which I think handles both CHUNKY_STRIP_READ_SUPPORT and reading rows.

Looking at the encode it looks like I need to support encoding by row as well, is that correct?

Looking at the encode it looks like I need to support encoding by row as well, is that correct?

Yes, for the symmetry of operations

added 1 commit

• 35023c78 - scanline read/write

Compare with previous version

added 1 commit

• 6c98c35d - support read and write by strip

Compare with previous version

added 1 commit

• a2869e15 - progressive rgba decode with webp

Compare with previous version

added 1 commit

• d0ca86cd - test decode buffer exists

Compare with previous version

@normanb Can you squash all the commits in a single one ?

added 1 commit

• 9eacd59f - webp in tiff

Compare with previous version

@rouault yes, I have squashed the commits.

mentioned in commit 1a926533

merged

I did a test using GraphicsMagick and it appears that WebP lossless mode is not yet working. It takes a lot more CPU time when enabled, but the result is not lossless.

A likely cause for this is that WebP defaults to "YUVA" but needs to be explicitly set to ARGB mode for lossless.

In GraphicsMagick we have this code:

   /*
     Use ARGB input for lossless (YUVA input is lossy).
   */
   picture.use_argb = 1;
   webp_status = WebPPictureAlloc(&picture)

and it was necessary before WebP was truely lossless.

I tried updating TWebPSetupEncode() to look like

#if WEBP_ENCODER_ABI_VERSION >= 0x0100 sp->sEncoderConfig.lossless = sp->lossless; if (sp->lossless) sp->sPicture.use_argb = 1; #endif

but adding this setting did not solve the problem.

Has anyone else managed to achieve true lossless encode with WebP compression?

Bob

@bobfriesenhahn in https://github.com/OSGeo/gdal/blob/master/autotest/gcore/tiff_write.py#L8068 @rouault added a test for lossless mode with webp. How are you testing lossless output?

@normanb Interestingly my test uses approximate comparison, not approximate comparison. And when using "gdalinfo -checksum" on the result of "gdal_translate ../autotest/gcore/data/md_ge_rgb_0010000.tif out.tif -co compress=webp -co tiled=yes -co WEBP_LOSSLESS=true", I don't get the same checksum as on the source image. Not sure why I didn't raise this issue when I wrote the test.

@bobfriesenhahn in https://github.com/OSGeo/gdal/blob/master/autotest/gcore/tiff_write.py#L8068 @rouault added a test for lossless mode with webp. How are you testing lossless output?

I am using GraphicsMagick's compare utility.

This is the exact same issue we ran into with WebP before in GraphicsMagick's WebP coder. WebP does an implicit colorspace transform as part of its normal operation (regardless of the request for lossless mode) and the colorspace transform is lossy.

To write TIFF with lossless WebP using development GraphicsMagick sources do:

gm convert infile.ppm -compress webp -define tiff:webp-lossless=true outfile.tiff

then compare like:

gm compare -metric PAE infile.ppm outfile.tiff

Bob

I forgot to mention that this weekend I fixed a fall-through switch statement which would also have impacted lossless mode.

Bob

@bobfriesenhahn Thanks for the fix on the fall through switch statement. It seems that the WebPConfigInitInternal function call resets lossless to false. I have put PR against GDAL here https://github.com/OSGeo/gdal/pull/1105/ with a test case. If it is merged then I can update here.