1. 14 Mar, 2018 3 commits
  2. 14 Sep, 2017 1 commit
  3. 23 Aug, 2017 1 commit
  4. 26 Sep, 2016 1 commit
  5. 07 Sep, 2016 1 commit
  6. 11 Aug, 2016 1 commit
    • Jeff King's avatar
      provide an initializer for "struct object_info" · 27b5c1a0
      Jeff King authored
      An all-zero initializer is fine for this struct, but because
      the first element is a pointer, call sites need to know to
      use "NULL" instead of "0". Otherwise some static checkers
      like "sparse" will complain; see d099b717 (Fix some sparse
      warnings, 2013-07-18) for example.  So let's provide an
      initializer to make this easier to get right.
      But let's also comment that memset() to zero is explicitly
      OK[1]. One of the callers embeds object_info in another
      struct which is initialized via memset (expand_data in
      builtin/cat-file.c). Since our subset of C doesn't allow
      assignment from a compound literal, handling this in any
      other way is awkward, so we'd like to keep the ability to
      initialize by memset(). By documenting this property, it
      should make anybody who wants to change the initializer
      think twice before doing so.
      There's one other caller of interest. In parse_sha1_header(),
      we did not initialize the struct fully in the first place.
      This turned out not to be a bug because the sub-function it
      calls does not look at any other fields except the ones we
      did initialize. But that assumption might not hold in the
      future, so it's a dangerous construct. This patch switches
      it to initializing the whole struct, which protects us
      against unexpected reads of the other fields.
      [1] Obviously using memset() to initialize a pointer
          violates the C standard, but we long ago decided that it
          was an acceptable tradeoff in the real world.
      Signed-off-by: default avatarJeff King <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
  7. 31 Mar, 2015 1 commit
    • John Keeping's avatar
      streaming.c: fix a memleak · 9ce4ad3e
      John Keeping authored
      When stream_blob_to_fd() opens an input stream with a filter, the
      filter gets discarded upon calling close_istream() before the
      function returns in the normal case.  However, when we fail to open
      the stream, we failed to discard the filter.
      By discarding the filter in the failure case, give a consistent
      life-time rule of the filter to the callers; otherwise the callers
      need to conditionally discard the filter themselves, and this
      function does not give enough hint for the caller to do so
      Signed-off-by: John Keeping's avatarJohn Keeping <[email protected]>
      Signed-off-by: Stefan Beller's avatarStefan Beller <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
  8. 19 Feb, 2014 1 commit
  9. 17 Jan, 2014 1 commit
  10. 12 Dec, 2013 1 commit
  11. 23 Jul, 2013 1 commit
  12. 18 Jul, 2013 1 commit
  13. 12 Jul, 2013 1 commit
    • Jeff King's avatar
      sha1_object_info_extended: make type calculation optional · 5b086407
      Jeff King authored
      Each caller of sha1_object_info_extended sets up an
      object_info struct to tell the function which elements of
      the object it wants to get. Until now, getting the type of
      the object has always been required (and it is returned via
      the return type rather than a pointer in object_info).
      This can involve actually opening a loose object file to
      determine its type, or following delta chains to determine a
      packed file's base type. These effects produce a measurable
      slow-down when doing a "cat-file --batch-check" that does
      not include %(objecttype).
      This patch adds a "typep" query to struct object_info, so
      that it can be optionally queried just like size and
      disk_size. As a result, the return type of the function is
      no longer the object type, but rather 0/-1 for success/error.
      As there are only three callers total, we just fix up each
      caller rather than keep a compatibility wrapper:
        1. The simpler sha1_object_info wrapper continues to
           always ask for and return the type field.
        2. The istream_source function wants to know the type, and
           so always asks for it.
        3. The cat-file batch code asks for the type only when
           %(objecttype) is part of the format string.
      On linux.git, the best-of-five for running:
        $ git rev-list --objects --all >objects
        $ time git cat-file --batch-check='%(objectsize:disk)'
      on a fully packed repository goes from:
        real    0m8.680s
        user    0m8.160s
        sys     0m0.512s
        real    0m7.205s
        user    0m6.580s
        sys     0m0.608s
      Signed-off-by: default avatarJeff King <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
  14. 07 Jul, 2013 1 commit
    • Jeff King's avatar
      zero-initialize object_info structs · 7c07385d
      Jeff King authored
      The sha1_object_info_extended function expects the caller to
      provide a "struct object_info" which contains pointers to
      "query" items that will be filled in. The purpose of
      providing pointers rather than storing the response directly
      in the struct is so that callers can choose not to incur the
      expense in finding particular fields that they do not care
      Right now the only query item is "sizep", and all callers
      set it explicitly to choose whether or not to query it; they
      can then leave the rest of the struct uninitialized.
      However, as we add new query items, each caller will have to
      be updated to explicitly turn off the new ones (by setting
      them to NULL).  Instead, let's teach each caller to
      zero-initialize the struct, so that they do not have to
      learn about each new query item added.
      Signed-off-by: default avatarJeff King <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
  15. 27 Mar, 2013 3 commits
    • Jeff King's avatar
      avoid infinite loop in read_istream_loose · 692f0bc7
      Jeff King authored
      The read_istream_loose function loops on inflating a chunk of data
      from an mmap'd loose object. We end the loop when we run out
      of space in our output buffer, or if we see a zlib error.
      We need to treat Z_BUF_ERROR specially, though, as it is not
      fatal; it is just zlib's way of telling us that we need to
      either feed it more input or give it more output space. It
      is perfectly normal for us to hit this when we are at the
      end of our buffer.
      However, we may also get Z_BUF_ERROR because we have run out
      of input. In a well-formed object, this should not happen,
      because we have fed the whole mmap'd contents to zlib. But
      if the object is truncated or corrupt, we will loop forever,
      never giving zlib any more data, but continuing to ask it to
      We can fix this by considering it an error when zlib returns
      Z_BUF_ERROR but we still have output space left (which means
      it must want more input, which we know is a truncation
      error). It would not be sufficient to just check whether
      zlib had consumed all the input at the start of the loop, as
      it might still want to generate output from what is in its
      internal state.
      Signed-off-by: default avatarJeff King <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
    • Jeff King's avatar
      read_istream_filtered: propagate read error from upstream · 42e7e2a5
      Jeff King authored
      The filter istream pulls data from an "upstream" stream,
      running it through a filter function. However, we did not
      properly notice when the upstream filter yielded an error,
      and just returned what we had read. Instead, we should
      propagate the error.
      Signed-off-by: default avatarJeff King <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
    • Jeff King's avatar
      stream_blob_to_fd: detect errors reading from stream · 45d4bdae
      Jeff King authored
      We call read_istream, but never check its return value for
      errors. This can lead to us looping infinitely, as we just
      keep trying to write "-1" bytes (and we do not notice the
      error, as we simply check that write_in_full reports the
      same number of bytes we fed it, which of course is also -1).
      Signed-off-by: default avatarJeff King <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
  16. 18 May, 2012 1 commit
  17. 03 May, 2012 1 commit
  18. 07 Mar, 2012 1 commit
  19. 22 Jul, 2011 1 commit
    • Jeff King's avatar
      streaming: free git_istream upon closing · 95dea6eb
      Jeff King authored
      Kirill Smelkov noticed that post-1.7.6 "git checkout"
      started leaking tons of memory. The streaming_write_entry
      function properly calls close_istream(), but that function
      did not actually free() the allocated git_istream struct.
      The git_istream struct is totally opaque to calling code,
      and must be heap-allocated by open_istream. Therefore it's
      not appropriate for callers to have to free it.
      This patch makes close_istream() into "close and de-allocate
      all associated resources". We could add a new "free_istream"
      call, but there's not much point in letting callers inspect
      the istream after close. And this patch's semantics make us
      match fopen/fclose, which is well-known and understood.
      Signed-off-by: default avatarJeff King <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
  20. 26 May, 2011 2 commits
    • Junio C Hamano's avatar
      Add streaming filter API · b6691092
      Junio C Hamano authored
      This introduces an API to plug custom filters to an input stream.
      The caller gets get_stream_filter("path") to obtain an appropriate
      filter for the path, and then uses it when opening an input stream
      via open_istream().  After that, the caller can read from the stream
      with read_istream(), and close it with close_istream(), just like an
      unfiltered stream.
      This only adds a "null" filter that is a pass-thru filter, but later
      changes can add LF-to-CRLF and other filters, and the callers of the
      streaming API do not have to change.
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
    • Junio C Hamano's avatar
      stream filter: add "no more input" to the filters · 4ae66704
      Junio C Hamano authored
      Some filters may need to buffer the input and look-ahead inside it
      to decide what to output, and they may consume more than zero bytes
      of input and still not produce any output. After feeding all the
      input, pass NULL as input as keep calling stream_filter() to let
      such filters know there is no more input coming, and it is time for
      them to produce the remaining output based on the buffered input.
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
  21. 21 May, 2011 3 commits