• Duy Nguyen's avatar
    fsck: use streaming interface for large blobs in pack · ec9d2249
    Duy Nguyen authored
    For blobs, we want to make sure the on-disk data is not corrupted
    (i.e. can be inflated and produce the expected SHA-1). Blob content is
    opaque, there's nothing else inside to check for.
    For really large blobs, we may want to avoid unpacking the entire blob
    in memory, just to check whether it produces the same SHA-1. On 32-bit
    systems, we may not have enough virtual address space for such memory
    allocation. And even on 64-bit where it's not a problem, allocating a
    lot more memory could result in kicking other parts of systems to swap
    file, generating lots of I/O and slowing everything down.
    For this particular operation, not unpacking the blob and letting
    check_sha1_signature, which supports streaming interface, do the job
    is sufficient. check_sha1_signature() is not shown in the diff,
    unfortunately. But if will be called when "data_valid && !data" is
    We will call the callback function "fn" with NULL as "data". The only
    callback of this function is fsck_obj_buffer(), which does not touch
    "data" at all if it's a blob.
    Signed-off-by: Duy Nguyen's avatarNguyễn Thái Ngọc Duy <pclouds@gmail.com>
    Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
pack.h 3.23 KB