1. 20 Nov, 2015 1 commit
  2. 19 Aug, 2015 1 commit
  3. 02 Apr, 2015 2 commits
  4. 14 Mar, 2015 1 commit
  5. 08 Jan, 2015 2 commits
    • Ronnie Sahlberg's avatar
      send-pack.c: add --atomic command line argument · 4ff17f10
      Ronnie Sahlberg authored
      This adds support to send-pack to negotiate and use atomic pushes
      iff the server supports it. Atomic pushes are activated by a new command
      line flag --atomic.
      In order to do this we also need to change the semantics for send_pack()
      slightly. The existing send_pack() function actually doesn't send all the
      refs back to the server when multiple refs are involved, for example
      when using --all. Several of the failure modes for pushes can already be
      detected locally in the send_pack client based on the information from the
      initial server side list of all the refs as generated by receive-pack.
      Any such refs that we thus know would fail to push are thus pruned from
      the list of refs we send to the server to update.
      For atomic pushes, we have to deal thus with both failures that are detected
      locally as well as failures that are reported back from the server. In order
      to do so we treat all local failures as push failures too.
      We introduce a new status code REF_STATUS_ATOMIC_PUSH_FAILED so we can
      flag all refs that we would normally have tried to push to the server
      but we did not due to local failures. This is to improve the error message
      back to the end user to flag that "these refs failed to update since the
      atomic push operation failed."
      Signed-off-by: default avatarRonnie Sahlberg <sahlberg@google.com>
      Signed-off-by: Stefan Beller's avatarStefan Beller <sbeller@google.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Stefan Beller's avatar
      send-pack: rename ref_update_to_be_sent to check_to_send_update · 7582e939
      Stefan Beller authored
      This renames ref_update_to_be_sent to check_to_send_update and inverts
      the meaning of the return value. Having the return value inverted we
      can have different values for the error codes. This is useful in a
      later patch when we want to know if we hit the CHECK_REF_STATUS_REJECTED
      Signed-off-by: Stefan Beller's avatarStefan Beller <sbeller@google.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  6. 29 Dec, 2014 1 commit
    • brian m. carlson's avatar
      pack-objects: use --objects-edge-aggressive for shallow repos · 2dacf26d
      brian m. carlson authored
      When fetching into or pushing from a shallow repository, we want to
      aggressively mark edges as uninteresting, since this decreases the pack
      size.  However, aggressively marking edges can negatively affect
      performance on large non-shallow repositories with lots of refs.
      Teach pack-objects a --shallow option to indicate that we're pushing
      from or fetching into a shallow repository.  Use
      --objects-edge-aggressive only for shallow repositories and otherwise
      use --objects-edge, which performs better in the general case.  Update
      the callers to pass the --shallow option when they are dealing with a
      shallow repository.
      Signed-off-by: brian m. carlson's avatarbrian m. carlson <sandals@crustytoothpaste.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  7. 17 Sep, 2014 1 commit
    • Junio C Hamano's avatar
      signed push: fortify against replay attacks · b89363e4
      Junio C Hamano authored
      In order to prevent a valid push certificate for pushing into an
      repository from getting replayed in a different push operation, send
      a nonce string from the receive-pack process and have the signer
      include it in the push certificate.  The receiving end uses an HMAC
      hash of the path to the repository it serves and the current time
      stamp, hashed with a secret seed (the secret seed does not have to
      be per-repository but can be defined in /etc/gitconfig) to generate
      the nonce, in order to ensure that a random third party cannot forge
      a nonce that looks like it originated from it.
      The original nonce is exported as GIT_PUSH_CERT_NONCE for the hooks
      to examine and match against the value on the "nonce" header in the
      certificate to notice a replay, but returned "nonce" header in the
      push certificate is examined by receive-pack and the result is
      exported as GIT_PUSH_CERT_NONCE_STATUS, whose value would be "OK"
      if the nonce recorded in the certificate matches what we expect, so
      that the hooks can more easily check.
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  8. 15 Sep, 2014 11 commits
    • Junio C Hamano's avatar
      signed push: add "pushee" header to push certificate · 9be89160
      Junio C Hamano authored
      Record the URL of the intended recipient for a push (after
      anonymizing it if it has authentication material) on a new "pushee
      URL" header.  Because the networking configuration (SSH-tunnels,
      proxies, etc.) on the pushing user's side varies, the receiving
      repository may not know the single canonical URL all the pushing
      users would refer it as (besides, many sites allow pushing over
      ssh://host/path and https://host/path protocols to the same
      repository but with different local part of the path).  So this
      value may not be reliably used for replay-attack prevention
      purposes, but this will still serve as a human readable hint to
      identify the repository the certificate refers to.
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Junio C Hamano's avatar
      signed push: remove duplicated protocol info · 4adf569d
      Junio C Hamano authored
      With the interim protocol, we used to send the update commands even
      though we already send a signed copy of the same information when
      push certificate is in use.  Update the send-pack/receive-pack pair
      not to do so.
      The notable thing on the receive-pack side is that it makes sure
      that there is no command sent over the traditional protocol packet
      outside the push certificate.  Otherwise a pusher can claim to be
      pushing one set of ref updates in the signed certificate while
      issuing commands to update unrelated refs, and such an update will
      evade later audits.
      Finally, start documenting the protocol.
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Junio C Hamano's avatar
      send-pack: send feature request on push-cert packet · 20a7558f
      Junio C Hamano authored
      We would want to update the interim protocol so that we do not send
      the usual update commands when the push certificate feature is in
      use, as the same information is in the certificate.  Once that
      happens, the push-cert packet may become the only protocol command,
      but then there is no packet to put the feature request behind, like
      we always did.
      As we have prepared the receiving end that understands the push-cert
      feature to accept the feature request on the first protocol packet
      (other than "shallow ", which was an unfortunate historical mistake
      that has to come before everything else), we can give the feature
      request on the push-cert packet instead of the first update protocol
      packet, in preparation for the next step to actually update to the
      final protocol.
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Junio C Hamano's avatar
      push: the beginning of "git push --signed" · a85b377d
      Junio C Hamano authored
      While signed tags and commits assert that the objects thusly signed
      came from you, who signed these objects, there is not a good way to
      assert that you wanted to have a particular object at the tip of a
      particular branch.  My signing v2.0.1 tag only means I want to call
      the version v2.0.1, and it does not mean I want to push it out to my
      'master' branch---it is likely that I only want it in 'maint', so
      the signature on the object alone is insufficient.
      The only assurance to you that 'maint' points at what I wanted to
      place there comes from your trust on the hosting site and my
      authentication with it, which cannot easily audited later.
      Introduce a mechanism that allows you to sign a "push certificate"
      (for the lack of better name) every time you push, asserting that
      what object you are pushing to update which ref that used to point
      at what other object.  Think of it as a cryptographic protection for
      ref updates, similar to signed tags/commits but working on an
      orthogonal axis.
      The basic flow based on this mechanism goes like this:
       1. You push out your work with "git push --signed".
       2. The sending side learns where the remote refs are as usual,
          together with what protocol extension the receiving end
          supports.  If the receiving end does not advertise the protocol
          extension "push-cert", an attempt to "git push --signed" fails.
          Otherwise, a text file, that looks like the following, is
          prepared in core:
      	certificate version 0.1
      	pusher Junio C Hamano <gitster@pobox.com> 1315427886 -0700
      	7339ca65... 21580ecb... refs/heads/master
      	3793ac56... 12850bec... refs/heads/next
          The file begins with a few header lines, which may grow as we
          gain more experience.  The 'pusher' header records the name of
          the signer (the value of user.signingkey configuration variable,
          falling back to GIT_COMMITTER_{NAME|EMAIL}) and the time of the
          certificate generation.  After the header, a blank line follows,
          followed by a copy of the protocol message lines.
          Each line shows the old and the new object name at the tip of
          the ref this push tries to update, in the way identical to how
          the underlying "git push" protocol exchange tells the ref
          updates to the receiving end (by recording the "old" object
          name, the push certificate also protects against replaying).  It
          is expected that new command packet types other than the
          old-new-refname kind will be included in push certificate in the
          same way as would appear in the plain vanilla command packets in
          unsigned pushes.
          The user then is asked to sign this push certificate using GPG,
          formatted in a way similar to how signed tag objects are signed,
          and the result is sent to the other side (i.e. receive-pack).
          In the protocol exchange, this step comes immediately before the
          sender tells what the result of the push should be, which in
          turn comes before it sends the pack data.
       3. When the receiving end sees a push certificate, the certificate
          is written out as a blob.  The pre-receive hook can learn about
          the certificate by checking GIT_PUSH_CERT environment variable,
          which, if present, tells the object name of this blob, and make
          the decision to allow or reject this push.  Additionally, the
          post-receive hook can also look at the certificate, which may be
          a good place to log all the received certificates for later
      Because a push certificate carry the same information as the usual
      command packets in the protocol exchange, we can omit the latter
      when a push certificate is in use and reduce the protocol overhead.
      This however is not included in this patch to make it easier to
      review (in other words, the series at this step should never be
      released without the remainder of the series, as it implements an
      interim protocol that will be incompatible with the final one).
      As such, the documentation update for the protocol is left out of
      this step.
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Junio C Hamano's avatar
      send-pack: clarify that cmds_sent is a boolean · c67072b9
      Junio C Hamano authored
      We use it to make sure that the feature request is sent only once on
      the very first request packet (ignoring the "shallow " line, which
      was an unfortunate mistake we cannot retroactively fix with existing
      receive-pack already deployed in the field) and we set it to "true"
      with cmds_sent++, not because we care about the actual number of
      updates sent but because it is merely an idiomatic way.
      Set it explicitly to one to clarify that the code that uses this
      variable only cares about its zero-ness.
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Junio C Hamano's avatar
      send-pack: refactor inspecting and resetting status and sending commands · b783aa71
      Junio C Hamano authored
      The main loop over remote_refs list inspects the ref status
      to see if we need to generate pack data (i.e. a delete-only push
      does not need to send any additional data), resets it to "expecting
      the status report" state, and formats the actual update commands
      to be sent.
      Split the former two out of the main loop, as it will become
      conditional in later steps.
      Besides, we should have code that does real thing here, before the
      "Finally, tell the other end!" part ;-)
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Junio C Hamano's avatar
      send-pack: rename "new_refs" to "need_pack_data" · ab2b0c90
      Junio C Hamano authored
      The variable counts how many non-deleting command is being sent, but
      is only checked with 0-ness to decide if we need to send the pack
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Junio C Hamano's avatar
      send-pack: factor out capability string generation · 887f3533
      Junio C Hamano authored
      A run of 'var ? " var" : ""' fed to a long printf string in a deeply
      nested block was hard to read.  Move it outside the loop and format
      it into a strbuf.
      As an added bonus, the trick to add "agent=<agent-name>" by using
      two conditionals is replaced by a more readable version.
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Junio C Hamano's avatar
      send-pack: always send capabilities · 64de20a1
      Junio C Hamano authored
      We tried to avoid sending one extra byte, NUL and nothing behind it
      to signal there is no protocol capabilities being sent, on the first
      command packet on the wire, but it just made the code look ugly.
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Junio C Hamano's avatar
      send-pack: refactor decision to send update per ref · e40671a3
      Junio C Hamano authored
      A new helper function ref_update_to_be_sent() decides for each ref
      if the update is to be sent based on the status previously set by
      set_ref_status_for_push() and also if this is a mirrored push.
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Junio C Hamano's avatar
      send-pack: move REF_STATUS_REJECT_NODELETE logic a bit higher · 621b0599
      Junio C Hamano authored
      20e8b465 (refactor ref status logic for pushing, 2010-01-08)
      restructured the code to set status for each ref to be pushed, but
      did not quite go far enough.  We inspect the status set earlier by
      set_refs_status_for_push() and then perform yet another update to
      the status of a ref with an otherwise OK status to be deleted to
      mark it with REF_STATUS_REJECT_NODELETE when the protocol tells us
      never to delete.
      Split the latter into a separate loop that comes before we enter the
      per-ref loop.  This way we would have one less condition to check in
      the main loop.
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  9. 20 Aug, 2014 1 commit
  10. 06 Jan, 2014 1 commit
  11. 11 Dec, 2013 3 commits
  12. 05 Dec, 2013 1 commit
    • Christian Couder's avatar
      replace {pre,suf}fixcmp() with {starts,ends}_with() · 59556548
      Christian Couder authored
      Leaving only the function definitions and declarations so that any
      new topic in flight can still make use of the old functions, replace
      existing uses of the prefixcmp() and suffixcmp() with new API
      The change can be recreated by mechanically applying this:
          $ git grep -l -e prefixcmp -e suffixcmp -- \*.c |
            grep -v strbuf\\.c |
            xargs perl -pi -e '
      on the result of preparatory changes in this series.
      Signed-off-by: Christian Couder's avatarChristian Couder <chriscool@tuxfamily.org>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  13. 25 Nov, 2013 1 commit
  14. 23 Oct, 2013 1 commit
  15. 23 Jul, 2013 1 commit
  16. 08 Jul, 2013 1 commit
    • Junio C Hamano's avatar
      cache.h: move remote/connect API out of it · 47a59185
      Junio C Hamano authored
      The definition of "struct ref" in "cache.h", a header file so
      central to the system, always confused me.  This structure is not
      about the local ref used by sha1-name API to name local objects.
      It is what refspecs are expanded into, after finding out what refs
      the other side has, to define what refs are updated after object
      transfer succeeds to what values.  It belongs to "remote.h" together
      with "struct refspec".
      While we are at it, also move the types and functions related to the
      Git transport connection to a new header file connect.h
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  17. 20 Feb, 2013 4 commits
    • Jeff King's avatar
      pkt-line: provide a LARGE_PACKET_MAX static buffer · 74543a04
      Jeff King authored
      Most of the callers of packet_read_line just read into a
      static 1000-byte buffer (callers which handle arbitrary
      binary data already use LARGE_PACKET_MAX). This works fine
      in practice, because:
        1. The only variable-sized data in these lines is a ref
           name, and refs tend to be a lot shorter than 1000
        2. When sending ref lines, git-core always limits itself
           to 1000 byte packets.
      However, the only limit given in the protocol specification
      in Documentation/technical/protocol-common.txt is
      LARGE_PACKET_MAX; the 1000 byte limit is mentioned only in
      pack-protocol.txt, and then only describing what we write,
      not as a specific limit for readers.
      This patch lets us bump the 1000-byte limit to
      LARGE_PACKET_MAX. Even though git-core will never write a
      packet where this makes a difference, there are two good
      reasons to do this:
        1. Other git implementations may have followed
           protocol-common.txt and used a larger maximum size. We
           don't bump into it in practice because it would involve
           very long ref names.
        2. We may want to increase the 1000-byte limit one day.
           Since packets are transferred before any capabilities,
           it's difficult to do this in a backwards-compatible
           way. But if we bump the size of buffer the readers can
           handle, eventually older versions of git will be
           obsolete enough that we can justify bumping the
           writers, as well. We don't have plans to do this
           anytime soon, but there is no reason not to start the
           clock ticking now.
      Just bumping all of the reading bufs to LARGE_PACKET_MAX
      would waste memory. Instead, since most readers just read
      into a temporary buffer anyway, let's provide a single
      static buffer that all callers can use. We can further wrap
      this detail away by having the packet_read_line wrapper just
      use the buffer transparently and return a pointer to the
      static storage.  That covers most of the cases, and the
      remaining ones already read into their own LARGE_PACKET_MAX
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Jeff King's avatar
      pkt-line: teach packet_read_line to chomp newlines · 819b929d
      Jeff King authored
      The packets sent during ref negotiation are all terminated
      by newline; even though the code to chomp these newlines is
      short, we end up doing it in a lot of places.
      This patch teaches packet_read_line to auto-chomp the
      trailing newline; this lets us get rid of a lot of inline
      chomping code.
      As a result, some call-sites which are not reading
      line-oriented data (e.g., when reading chunks of packfiles
      alongside sideband) transition away from packet_read_line to
      the generic packet_read interface. This patch converts all
      of the existing callsites.
      Since the function signature of packet_read_line does not
      change (but its behavior does), there is a possibility of
      new callsites being introduced in later commits, silently
      introducing an incompatibility.  However, since a later
      patch in this series will change the signature, such a
      commit would have to be merged directly into this commit,
      not to the tip of the series; we can therefore ignore the
      This is an internal cleanup and should produce no change of
      behavior in the normal case. However, there is one corner
      case to note. Callers of packet_read_line have never been
      able to tell the difference between a flush packet ("0000")
      and an empty packet ("0004"), as both cause packet_read_line
      to return a length of 0. Readers treat them identically,
      even though Documentation/technical/protocol-common.txt says
      we must not; it also says that implementations should not
      send an empty pkt-line.
      By stripping out the newline before the result gets to the
      caller, we will now treat the newline-only packet ("0005\n")
      the same as an empty packet, which in turn gets treated like
      a flush packet. In practice this doesn't matter, as neither
      empty nor newline-only packets are part of git's protocols
      (at least not for the line-oriented bits, and readers who
      are not expecting line-oriented packets will be calling
      packet_read directly, anyway). But even if we do decide to
      care about the distinction later, it is orthogonal to this
      patch.  The right place to tighten would be to stop treating
      empty packets as flush packets, and this change does not
      make doing so any harder.
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Jeff King's avatar
      pkt-line: drop safe_write function · cdf4fb8e
      Jeff King authored
      This is just write_or_die by another name. The one
      distinction is that write_or_die will treat EPIPE specially
      by suppressing error messages. That's fine, as we die by
      SIGPIPE anyway (and in the off chance that it is disabled,
      write_or_die will simulate it).
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Jeff King's avatar
      send-pack: prefer prefixcmp over memcmp in receive_status · 8f9e3e49
      Jeff King authored
      This code predates prefixcmp, so it used memcmp along with
      static sizes. Replacing these memcmps with prefixcmp makes
      the code much more readable, and the lack of static sizes
      will make refactoring it in future patches simpler.
      Note that we used to be unnecessarily liberal in parsing the
      "unpack" status line, and would accept "unpack ok\njunk". No
      version of git has ever produced that, and it violates the
      BNF in Documentation/technical/pack-protocol.txt. Let's take
      this opportunity to tighten the check by converting the
      prefix comparison into a strcmp.
      While we're in the area, let's also fix a vague error
      message that does not follow our usual conventions (it
      writes directly to stderr and does not use the "error:"
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  18. 24 Jan, 2013 1 commit
    • Junio C Hamano's avatar
      push: introduce REJECT_FETCH_FIRST and REJECT_NEEDS_FORCE · 75e5c0dc
      Junio C Hamano authored
      When we push to update an existing ref, if:
       * the object at the tip of the remote is not a commit; or
       * the object we are pushing is not a commit,
      it won't be correct to suggest to fetch, integrate and push again,
      as the old and new objects will not "merge".  We should explain that
      the push must be forced when there is a non-committish object is
      involved in such a case.
      If we do not have the current object at the tip of the remote, we do
      not even know that object, when fetched, is something that can be
      merged.  In such a case, suggesting to pull first just like
      non-fast-forward case may not be technically correct, but in
      practice, most such failures are seen when you try to push your work
      to a branch without knowing that somebody else already pushed to
      update the same branch since you forked, so "pull first" would work
      as a suggestion most of the time.  And if the object at the tip is
      not a commit, "pull first" will fail, without making any permanent
      damage.  As a side effect, it also makes the error message the user
      will get during the next "push" attempt easier to understand, now
      the user is aware that a non-commit object is involved.
      In these cases, the current code already rejects such a push on the
      client end, but we used the same error and advice messages as the
      ones used when rejecting a non-fast-forward push, i.e. pull from
      there and integrate before pushing again.
      Introduce new rejection reasons and reword the messages
      [jc: with help by Peff on message details]
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  19. 02 Dec, 2012 1 commit
  20. 29 Oct, 2012 1 commit
  21. 10 Aug, 2012 2 commits
    • Jeff King's avatar
      do not send client agent unless server does first · d50c3871
      Jeff King authored
      Commit ff5effdf taught both clients and servers of the git protocol
      to send an "agent" capability that just advertises their version for
      statistics and debugging purposes.  The protocol-capabilities.txt
      document however indicates that the client's advertisement is
      actually a response, and should never include capabilities not
      mentioned in the server's advertisement.
      Adding the unconditional advertisement in the server programs was
      OK, then, but the clients broke the protocol.  The server
      implementation of git-core itself does not care, but at least one
      does: the Google Code git server (or any server using Dulwich), will
      hang up with an internal error upon seeing an unknown capability.
      Instead, each client must record whether we saw an agent string from
      the server, and respond with its agent only if the server mentioned
      it first.
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Jeff King's avatar
      send-pack: fix capability-sending logic · ca8e127c
      Jeff King authored
      If we have capabilities to send to the server, we send the
      regular "want" line followed by a NUL, then the
      capabilities; otherwise, we do not even send the NUL.
      However, when checking whether we want to send the "quiet"
      capability, we check args->quiet, which is wrong. That flag
      only tells us whether the client side wanted to be quiet,
      not whether the server supports it (originally, in c207e34f,
      it meant both; however, that was later split into two flags
      by 01fdc21f).
      We still check the right flag when actually printing
      "quiet", so this could only have two effects:
        1. We might send the trailing NUL when we do not otherwise
           need to. In theory, an antique pre-capability
           implementation of git might choke on this (since the
           client is instructed never to respond with capabilities
           that the server has not first advertised).
        2. We might also want to send the quiet flag if the
           args->progress flag is false, but this code path would
           not trigger in that instance.
      In practice, it almost certainly never matters. The
      report-status capability dates back to 2005. Any real-world
      server is going to advertise that, and we will always
      respond with at least that capability.
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  22. 03 Aug, 2012 1 commit
    • Jeff King's avatar
      include agent identifier in capability string · ff5effdf
      Jeff King authored
      Instead of having the client advertise a particular version
      number in the git protocol, we have managed extensions and
      backwards compatibility by having clients and servers
      advertise capabilities that they support. This is far more
      robust than having each side consult a table of
      known versions, and provides sufficient information for the
      protocol interaction to complete.
      However, it does not allow servers to keep statistics on
      which client versions are being used. This information is
      not necessary to complete the network request (the
      capabilities provide enough information for that), but it
      may be helpful to conduct a general survey of client
      versions in use.
      We already send the client version in the user-agent header
      for http requests; adding it here allows us to gather
      similar statistics for non-http requests.
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>