Draft: Save DZI and JPEG through vips

Closing, as the things we've learned from this have been added to !52 (merged)

Not intended to be merged in in current form. As pyvips is specifically for handling large images without holding all in memory, starting to test whether we should be using this for all image saving.

Notes so far - it does seem to produce an image with a greater filesize (when loaded into numpy) than the peak RAM used It does not seem to check if there's enough space on disk to complete the file creation, and we'd need to check manually ourselves. Scanning takes care of this somewhat by checking if we have at least half a GB available before starting a scan... if we're scanning in full resolution, that's not going to be enough for long

For saving a JPEG through VIPs, modifying pyramidal_tiff.py tile = pyvips.Image.new_from_file(filepath) to

tile = pyvips.Image.new_from_file(filepath, access="sequential")

reduces the memory use by a factor of around 4. This errors for saving a TIFF or DZI, as those can't be saved only reading from each time top-to-bottom, likely due to the multiple layers.

access="random" (default behaviour) running a full stitch, producing a JPEG from pyvips

image

access="sequential" running a full stitch, producing a JPEG from pyvips. peak memory is not from the JPEG anymore

image

Creating the JPEG from cv2 rather than vips. Faster, and same peak memory as random

WhatsApp_Image_2025-05-12_at_15.15.34_69fdeb76

Creating the JPEG from cv2, the dzi from deepzoom image creator, and the tiff from pvyips (random)

WhatsApp_Image_2025-05-12_at_15.22.48_45ab334a

Creating the JPEG in pyvips with access sequential, then loading that JPEG to produce the TIFF and DZI

WhatsApp_Image_2025-05-12_at_15.08.25_7f1a1e17

Creating the JPEG in pyvips with access sequential, input tile size (8196, 8196) instead of default (4096, 4096). Faster, but 0.5GB of memory

image

image

Edited by Joe Knapper

Merge request reports

Loading