1. 10 Apr, 2007 5 commits
    • Nicolas Pitre's avatar
      compute a CRC32 for each object as stored in a pack · 78d1e84f
      Nicolas Pitre authored
      The most important optimization for performance when repacking is the
      ability to reuse data from a previous pack as is and bypass any delta
      or even SHA1 computation by simply copying the raw data from one pack
      to another directly.
      
      The problem with  this is that any data corruption within a copied object
      would go unnoticed and the new (repacked) pack would be self-consistent
      with its own checksum despite containing a corrupted object.  This is a
      real issue that already happened at least once in the past.
      
      In some attempt to prevent this, we validate the copied data by inflating
      it and making sure no error is signaled by zlib.  But this is still not
      perfect as a significant portion of a pack content is made of object
      headers and references to delta base objects which are not deflated and
      therefore not validated when repacking actually making the pack data reuse
      still not as safe as it could be.
      
      Of course a full SHA1 validation could be performed, but that implies
      full data inflating and delta replaying which is extremely costly, which
      cost the data reuse optimization was designed to avoid in the first place.
      
      So the best solution to this is simply to store a CRC32 of the raw pack
      data for each object in the pack index.  This way any object in a pack can
      be validated before being copied as is in another pack, including header
      and any other non deflated data.
      
      Why CRC32 instead of a faster checksum like Adler32?  Quoting Wikipedia:
      
         Jonathan Stone discovered in 2001 that Adler-32 has a weakness for very
         short messages. He wrote "Briefly, the problem is that, for very short
         packets, Adler32 is guaranteed to give poor coverage of the available
         bits. Don't take my word for it, ask Mark Adler. :-)" The problem is
         that sum A does not wrap for short messages. The maximum value of A for
         a 128-byte message is 32640, which is below the value 65521 used by the
         modulo operation. An extended explanation can be found in RFC 3309,
         which mandates the use of CRC32 instead of Adler-32 for SCTP, the
         Stream Control Transmission Protocol.
      
      In the context of a GIT pack, we have lots of small objects, especially
      deltas, which are likely to be quite small and in a size range for which
      Adler32 is dimed not to be sufficient.  Another advantage of CRC32 is the
      possibility for recovery from certain types of small corruptions like
      single bit errors which are the most probable type of corruptions.
      
      OK what this patch does is to compute the CRC32 of each object written to
      a pack within pack-objects.  It is not written to the index yet and it is
      obviously not validated when reusing pack data yet either.
      Signed-off-by: default avatarNicolas Pitre <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      78d1e84f
    • Nicolas Pitre's avatar
      add overflow tests on pack offset variables · d7dd0223
      Nicolas Pitre authored
      Change a few size and offset variables to more appropriate type, then
      add overflow tests on those offsets.  This prevents any bad data to be
      generated/processed if off_t happens to not be large enough to handle
      some big packs.
      
      Better be safe than sorry.
      Signed-off-by: default avatarNicolas Pitre <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      d7dd0223
    • Nicolas Pitre's avatar
      make overflow test on delta base offset work regardless of variable size · 8723f216
      Nicolas Pitre authored
      This patch introduces the MSB() macro to obtain the desired number of
      most significant bits from a given variable independently of the variable
      type.
      
      It is then used to better implement the overflow test on the OBJ_OFS_DELTA
      base offset variable with the property of always working correctly
      regardless of the type/size of that variable.
      Signed-off-by: default avatarNicolas Pitre <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      8723f216
    • Nicolas Pitre's avatar
      get rid of num_packed_objects() · 57059091
      Nicolas Pitre authored
      The coming index format change doesn't allow for the number of objects
      to be determined from the size of the index file directly.  Instead, Let's
      initialize a field in the packed_git structure with the object count when
      the index is validated since the count is always known at that point.
      
      While at it let's reorder some struct packed_git fields to avoid padding
      due to needed 64-bit alignment for some of them.
      Signed-off-by: default avatarNicolas Pitre <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      57059091
    • René Scharfe's avatar
      git-archive: make tar the default format · 8ff21b1a
      René Scharfe authored
      As noted by Junio, --format=tar should be assumed if no format
      was specified.
      Signed-off-by: default avatarRene Scharfe <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      8ff21b1a
  2. 09 Apr, 2007 8 commits
  3. 08 Apr, 2007 5 commits
  4. 07 Apr, 2007 15 commits
    • Junio C Hamano's avatar
      A new merge stragety 'subtree'. · 68faf689
      Junio C Hamano authored
      This merge strategy largely piggy-backs on git-merge-recursive.
      When merging trees A and B, if B corresponds to a subtree of A,
      B is first adjusted to match the tree structure of A, instead of
      reading the trees at the same level.  This adjustment is also
      done to the common ancestor tree.
      
      If you are pulling updates from git-gui repository into git.git
      repository, the root level of the former corresponds to git-gui/
      subdirectory of the latter.  The tree object of git-gui's toplevel
      is wrapped in a fake tree object, whose sole entry has name 'git-gui'
      and records object name of the true tree, before being used by
      the 3-way merge code.
      
      If you are merging the other way, only the git-gui/ subtree of
      git.git is extracted and merged into git-gui's toplevel.
      
      The detection of corresponding subtree is done by comparing the
      pathnames and types in the toplevel of the tree.
      
      Heuristics galore!  That's the git way ;-).
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      68faf689
    • Junio C Hamano's avatar
      git-push to multiple locations does not stop at the first failure · fd1d1b05
      Junio C Hamano authored
      When pushing into multiple repositories with git push, via
      multiple URL in .git/remotes/$shorthand or multiple url
      variables in [remote "$shorthand"] section, we used to stop upon
      the first failure.  Continue the operation and report the
      failure at the end.
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      fd1d1b05
    • Junio C Hamano's avatar
      git-push reports the URL after failing. · 39878b0c
      Junio C Hamano authored
      This came up on #git when somebody was getting 'unable to create
      ./objects/tmp_oXXXX' but sweared he had write permission to that
      directory.  It turned out that the repository URL was changed
      and he was accessing a repository he does not have a write
      permission anymore.
      
      I am not sure how much this would have helped somebody who
      believed he was accessing location when the permission of that
      location was changed while he was looking the other way, though.
      But giving more information on the error path would be better,
      and the next change would be helped with this as well.
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      39878b0c
    • Junio C Hamano's avatar
      Merge branch 'jc/index-output' · ee9693e2
      Junio C Hamano authored
      * jc/index-output:
        git-read-tree --index-output=<file>
        _GIT_INDEX_OUTPUT: allow plumbing to output to an alternative index file.
      
      Conflicts:
      
      	builtin-apply.c
      ee9693e2
    • Junio C Hamano's avatar
      Merge branch 'fp/make-j' · 39415449
      Junio C Hamano authored
      * fp/make-j:
        Makefile: Add '+' to QUIET_SUBDIR0 to fix parallel make.
      39415449
    • Junio C Hamano's avatar
      Merge branch 'cc/bisect' · 5bba1b35
      Junio C Hamano authored
      * cc/bisect:
        git-bisect: allow bisecting with only one bad commit.
        t6030: add a bit more tests to git-bisect
        git-bisect: modernization
        Documentation: bisect: "start" accepts one bad and many good commits
        Bisect: teach "bisect start" to optionally use one bad and many good revs.
      5bba1b35
    • Junio C Hamano's avatar
      Merge branch 'jc/checkout' (early part) · b7108a16
      Junio C Hamano authored
      * 'jc/checkout' (early part):
        checkout: allow detaching to HEAD even when switching to the tip of a branch
      b7108a16
    • Junio C Hamano's avatar
      Merge branch 'maint' · ced38ea2
      Junio C Hamano authored
      * maint:
        Documentation: tighten dependency for git.{html,txt}
        Makefile: iconv() on Darwin has the old interface
        t5300-pack-object.sh: portability issue using /usr/bin/stat
        t3200-branch.sh: small language nit
        usermanual.txt: some capitalization nits
        Make builtin-branch.c handle the git config file
        rename_ref(): only print a warning when config-file update fails
        Distinguish branches by more than case in tests.
        Avoid composing too long "References" header.
        cvsimport: Improve formating consistency
        cvsimport: Reorder options in documentation for better understanding
        cvsimport: Improve usage error reporting
        cvsimport: Improve documentation of CVSROOT and CVS module determination
        cvsimport: sync usage lines with existing options
      
      Conflicts:
      
      	Documentation/Makefile
      ced38ea2
    • Junio C Hamano's avatar
      Documentation: tighten dependency for git.{html,txt} · d7907392
      Junio C Hamano authored
      Every time _any_ documentation page changed, cmds-*.txt files
      were regenerated, which caused git.{html,txt} to be remade.  Try
      not to update cmds-*.txt files if their new contents match the
      old ones.
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      d7907392
    • Arjen Laarhoven's avatar
      Makefile: iconv() on Darwin has the old interface · 63b4b7a7
      Arjen Laarhoven authored
      The libiconv on Darwin uses the old iconv() interface (2nd argument is a
      const char **, instead of a char **).  Add OLD_ICONV to the Darwin
      variable definitions to handle this.
      Signed-off-by: default avatarArjen Laarhoven <[email protected]>
      Acked-by: default avatarBrian Gernhardt <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      63b4b7a7
    • Arjen Laarhoven's avatar
      t5300-pack-object.sh: portability issue using /usr/bin/stat · d93f7c18
      Arjen Laarhoven authored
      In the test 'compare delta flavors', /usr/bin/stat is used to get file size.
      This isn't portable.  There already is a dependency on Perl, use its '-s'
      operator to get the file size.
      Signed-off-by: default avatarArjen Laarhoven <[email protected]>
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      d93f7c18
    • Junio C Hamano's avatar
      git-bisect: allow bisecting with only one bad commit. · 0a5280a9
      Junio C Hamano authored
      This allows you to say:
      
      	git bisect start
      	git bisect bad $bad
      	git bisect next
      
      to start bisection without knowing a good commit.  This would
      have you try a commit that is half-way since the beginning of
      the history, which is rather wasteful if you already know a good
      commit, but if you don't (or your history is short enough that
      you do not care), there is no reason not to allow this.
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      0a5280a9
    • Junio C Hamano's avatar
      t6030: add a bit more tests to git-bisect · 4f506716
      Junio C Hamano authored
      Verify that git-bisect does not start before getting one bad and
      one good commit.
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      4f506716
    • Arjen Laarhoven's avatar
      t3200-branch.sh: small language nit · 4c84f0dc
      Arjen Laarhoven authored
      Signed-off-by: default avatarJunio C Hamano <[email protected]>
      4c84f0dc
    • Arjen Laarhoven's avatar
      ae25c67a
  5. 06 Apr, 2007 7 commits