1. 01 Apr, 2019 1 commit
  2. 05 Mar, 2019 2 commits
  3. 03 Mar, 2019 1 commit
    • Jonathan Tan's avatar
      remote-curl: use post_rpc() for protocol v2 also · a97d0079
      Jonathan Tan authored
      When transmitting and receiving POSTs for protocol v0 and v1,
      remote-curl uses post_rpc() (and associated functions), but when doing
      the same for protocol v2, it uses a separate set of functions
      (proxy_rpc() and others). Besides duplication of code, this has caused
      at least one bug: the auth retry mechanism that was implemented in v0/v1
      was not implemented in v2.
      
      To fix this issue and avoid it in the future, make remote-curl also use
      post_rpc() when handling protocol v2. Because line lengths are written
      to the HTTP request in protocol v2 (unlike in protocol v0/v1), this
      necessitates changes in post_rpc() and some of the functions it uses;
      perform these changes too.
      
      A test has been included to ensure that the code for both the unchunked
      and chunked variants of the HTTP request is exercised.
      
      Note: stateless_connect() has been updated to use the lower-level packet
      reading functions instead of struct packet_reader. The low-level control
      is necessary here because we cannot change the destination buffer of
      struct packet_reader while it is being used; struct packet_buffer has a
      peeking mechanism which relies on the destination buffer being present
      in between a peek and a read.
      Signed-off-by: 's avatarJonathan Tan <jonathantanmy@google.com>
      Signed-off-by: 's avatarJunio C Hamano <gitster@pobox.com>
      a97d0079
  4. 22 Feb, 2019 2 commits
    • Jeff Hostetler's avatar
      trace2: create new combined trace facility · ee4512ed
      Jeff Hostetler authored
      Create a new unified tracing facility for git.  The eventual intent is to
      replace the current trace_printf* and trace_performance* routines with a
      unified set of git_trace2* routines.
      
      In addition to the usual printf-style API, trace2 provides higer-level
      event verbs with fixed-fields allowing structured data to be written.
      This makes post-processing and analysis easier for external tools.
      
      Trace2 defines 3 output targets.  These are set using the environment
      variables "GIT_TR2", "GIT_TR2_PERF", and "GIT_TR2_EVENT".  These may be
      set to "1" or to an absolute pathname (just like the current GIT_TRACE).
      
      * GIT_TR2 is intended to be a replacement for GIT_TRACE and logs command
        summary data.
      
      * GIT_TR2_PERF is intended as a replacement for GIT_TRACE_PERFORMANCE.
        It extends the output with columns for the command process, thread,
        repo, absolute and relative elapsed times.  It reports events for
        child process start/stop, thread start/stop, and per-thread function
        nesting.
      
      * GIT_TR2_EVENT is a new structured format. It writes event data as a
        series of JSON records.
      
      Calls to trace2 functions log to any of the 3 output targets enabled
      without the need to call different trace_printf* or trace_performance*
      routines.
      Signed-off-by: 's avatarJeff Hostetler <jeffhost@microsoft.com>
      Signed-off-by: 's avatarJunio C Hamano <gitster@pobox.com>
      ee4512ed
    • Jonathan Tan's avatar
      remote-curl: refactor reading into rpc_state's buf · 78ad9172
      Jonathan Tan authored
      Currently, whenever remote-curl reads pkt-lines from its response file
      descriptor, only the payload is written to its buf, not the 4 characters
      denoting the length. A future patch will require the ability to also
      write those 4 characters, so in preparation for that, refactor this read
      into its own function.
      Signed-off-by: 's avatarJonathan Tan <jonathantanmy@google.com>
      Signed-off-by: 's avatarJunio C Hamano <gitster@pobox.com>
      78ad9172
  5. 14 Feb, 2019 3 commits
  6. 06 Feb, 2019 2 commits
    • Jeff King's avatar
      remote-curl: tighten "version 2" check for smart-http · cbdb8d14
      Jeff King authored
      In a v2 smart-http conversation, the server should reply to our initial
      request with a pkt-line saying "version 2". We check that with
      starts_with(), but really that should be the only thing in that packet.
      A response of "version 20" should not match.
      
      Let's tighten this check to use strcmp(). Note that we don't need to
      worry about a trailing newline here, because the ptk-line code will have
      chomped it for us already.
      Signed-off-by: 's avatarJeff King <peff@peff.net>
      Signed-off-by: 's avatarJunio C Hamano <gitster@pobox.com>
      cbdb8d14
    • Jeff King's avatar
      remote-curl: refactor smart-http discovery · 8ee3e120
      Jeff King authored
      After making initial contact with an http server, we have to decide if
      the server supports smart-http, and if so, which version. Our rules are
      a bit inconsistent:
      
        1. For v0, we require that the content-type indicates a smart-http
           response. We also require the response to look vaguely like a
           pkt-line starting with "#". If one of those does not match, we fall
           back to dumb-http.
      
           But according to our http protocol spec[1]:
      
             Dumb servers MUST NOT return a return type starting with
             `application/x-git-`.
      
           If we see the expected content-type, we should consider it
           smart-http. At that point we can parse the pkt-line for real, and
           complain if it is not syntactically valid.
      
        2. For v2, we do not actually check the content-type. Our v2 protocol
           spec says[2]:
      
             When using the http:// or https:// transport a client makes a
             "smart" info/refs request as described in `http-protocol.txt`[...]
      
           and the http spec is clear that for a smart-http response[3]:
      
             The Content-Type MUST be `application/x-$servicename-advertisement`.
      
           So it is required according to the spec.
      
      These inconsistencies were easy to miss because of the way the original
      code was written as an inline conditional. Let's pull it out into its
      own function for readability, and improve a few things:
      
       - we now predicate the smart/dumb decision entirely on the presence of
         the correct content-type
      
       - we do a real pkt-line parse before deciding how to proceed (and die
         if it isn't valid)
      
       - use skip_prefix() for comparing service strings, instead of
         constructing expected output in a strbuf; this avoids dealing with
         memory cleanup
      
      Note that this _is_ tightening what the client will allow. It's all
      according to the spec, but it's possible that other implementations
      might violate these. However, violating these particular rules seems
      like an odd choice for a server to make.
      
      [1] Documentation/technical/http-protocol.txt, l. 166-167
      [2] Documentation/technical/protocol-v2.txt, l. 63-64
      [3] Documentation/technical/http-protocol.txt, l. 247
      Helped-by: 's avatarJosh Steadmon <steadmon@google.com>
      Signed-off-by: 's avatarJeff King <peff@peff.net>
      Signed-off-by: 's avatarJunio C Hamano <gitster@pobox.com>
      8ee3e120
  7. 10 Jan, 2019 3 commits
  8. 02 Jan, 2019 2 commits
    • Masaya Suzuki's avatar
      pack-protocol.txt: accept error packets in any context · 2d103c31
      Masaya Suzuki authored
      In the Git pack protocol definition, an error packet may appear only in
      a certain context. However, servers can face a runtime error (e.g. I/O
      error) at an arbitrary timing. This patch changes the protocol to allow
      an error packet to be sent instead of any packet.
      
      Without this protocol spec change, when a server cannot process a
      request, there's no way to tell that to a client. Since the server
      cannot produce a valid response, it would be forced to cut a connection
      without telling why. With this protocol spec change, the server can be
      more gentle in this situation. An old client may see these error packets
      as an unexpected packet, but this is not worse than having an unexpected
      EOF.
      
      Following this protocol spec change, the error packet handling code is
      moved to pkt-line.c. Implementation wise, this implementation uses
      pkt-line to communicate with a subprocess. Since this is not a part of
      Git protocol, it's possible that a packet that is not supposed to be an
      error packet is mistakenly parsed as an error packet. This error packet
      handling is enabled only for the Git pack protocol parsing code
      considering this.
      Signed-off-by: 's avatarMasaya Suzuki <masayasuzuki@google.com>
      Signed-off-by: 's avatarJunio C Hamano <gitster@pobox.com>
      2d103c31
    • Masaya Suzuki's avatar
      Use packet_reader instead of packet_read_line · 01f9ec64
      Masaya Suzuki authored
      By using and sharing a packet_reader while handling a Git pack protocol
      request, the same reader option is used throughout the code. This makes
      it easy to set a reader option to the request parsing code.
      Signed-off-by: 's avatarMasaya Suzuki <masayasuzuki@google.com>
      Signed-off-by: 's avatarJunio C Hamano <gitster@pobox.com>
      01f9ec64
  9. 10 Dec, 2018 1 commit
  10. 12 Nov, 2018 1 commit
    • Torsten Bögershausen's avatar
      remote-curl.c: xcurl_off_t is not portable (on 32 bit platfoms) · cb8010bb
      Torsten Bögershausen authored
      When  setting
      DEVELOPER = 1
      DEVOPTS = extra-all
      
      "gcc (Raspbian 6.3.0-18+rpi1+deb9u1) 6.3.0 20170516" errors out with
      "comparison is always false due to limited range of data type"
      "[-Werror=type-limits]"
      
      It turns out that the function xcurl_off_t() has 2 flavours:
      
      - It gives a warning 32 bit systems, like Linux
      - It takes the signed ssize_t as a paramter, but the only caller is using
        a size_t (which is typically unsigned these days)
      
      The original motivation of this function is to make sure that sizes > 2GiB
      are handled correctly. The curl documentation says:
      "For any given platform/compiler curl_off_t must be typedef'ed to a 64-bit
       wide signed integral data type"
      On a 32 bit system "size_t" can be promoted into a 64 bit signed value
      without loss of data, and therefore we may see the
      "comparison is always false" warning.
      On a 64 bit system it may happen, at least in theory, that size_t is > 2^63,
      and then the promotion from an unsigned "size_t" into a signed "curl_off_t"
      may be a problem.
      
      One solution to suppress a possible compiler warning could be to remove
      the function xcurl_off_t().
      
      However, to be on the very safe side, we keep it and improve it:
      
      - The len parameter is changed from ssize_t to size_t
      - A temporally variable "size" is used, promoted int uintmax_t and the compared
        with "maximum_signed_value_of_type(curl_off_t)".
        Thanks to Junio C Hamano for this hint.
      Signed-off-by: 's avatarTorsten Bögershausen <tboegi@web.de>
      Signed-off-by: 's avatarJunio C Hamano <gitster@pobox.com>
      cb8010bb
  11. 05 Sep, 2018 1 commit
  12. 08 Aug, 2018 1 commit
  13. 23 May, 2018 2 commits
  14. 24 Apr, 2018 1 commit
  15. 11 Apr, 2018 1 commit
  16. 15 Mar, 2018 6 commits
  17. 14 Mar, 2018 2 commits
  18. 20 Feb, 2018 1 commit
    • Jeff King's avatar
      remote-curl: unquote incoming push-options · 90dce21e
      Jeff King authored
      The transport-helper protocol c-style quotes the value of
      any options passed to the helper via the "option <key> <value>"
      directive. However, remote-curl doesn't actually unquote the
      push-option values, meaning that we will send the quoted
      version to the other side (whereas git-over-ssh would send
      the raw value).
      
      The pack-protocol.txt documentation defines the push-options
      as a series of VCHARs, which excludes most characters that
      would need quoting. But:
      
        1. You can still see the bug with a valid push-option that
           starts with a double-quote (since that triggers
           quoting).
      
        2. We do currently handle any non-NUL characters correctly
           in git-over-ssh. So even though the spec does not say
           that we need to handle most quoted characters, it's
           nice if our behavior is consistent between protocols.
      
      There are two new tests: the "direct" one shows that this
      already works in the non-http case, and the http one covers
      this bugfix.
      Reported-by: Jon Simons's avatarJon Simons <jon@jonsimons.org>
      Signed-off-by: 's avatarJeff King <peff@peff.net>
      Signed-off-by: 's avatarJunio C Hamano <gitster@pobox.com>
      90dce21e
  19. 08 Feb, 2018 1 commit
  20. 08 Dec, 2017 1 commit
  21. 05 Dec, 2017 1 commit
    • Jonathan Tan's avatar
      introduce fetch-object: fetch one promisor object · 88e2f9ed
      Jonathan Tan authored
      Introduce fetch-object, providing the ability to fetch one object from a
      promisor remote.
      
      This uses fetch-pack. To do this, the transport mechanism has been
      updated with 2 flags, "from-promisor" to indicate that the resulting
      pack comes from a promisor remote (and thus should be annotated as such
      by index-pack), and "no-dependents" to indicate that only the objects
      themselves need to be fetched (but fetching additional objects is
      nevertheless safe).
      
      Whenever "no-dependents" is used, fetch-pack will refrain from using any
      object flags, because it is most likely invoked as part of a dynamic
      object fetch by another Git command (which may itself use object flags).
      An alternative to this is to leave fetch-pack alone, and instead update
      the allocation of flags so that fetch-pack's flags never overlap with
      any others, but this will end up shrinking the number of flags available
      to nearly every other Git command (that is, every Git command that
      accesses objects), so the approach in this commit was used instead.
      
      This will be tested in a subsequent commit.
      Signed-off-by: 's avatarJonathan Tan <jonathantanmy@google.com>
      Signed-off-by: 's avatarJunio C Hamano <gitster@pobox.com>
      88e2f9ed
  22. 15 Jun, 2017 1 commit
  23. 14 Apr, 2017 1 commit
  24. 31 Mar, 2017 1 commit
    • brian m. carlson's avatar
      Rename sha1_array to oid_array · 910650d2
      brian m. carlson authored
      Since this structure handles an array of object IDs, rename it to struct
      oid_array.  Also rename the accessor functions and the initialization
      constant.
      
      This commit was produced mechanically by providing non-Documentation
      files to the following Perl one-liners:
      
          perl -pi -E 's/struct sha1_array/struct oid_array/g'
          perl -pi -E 's/\bsha1_array_/oid_array_/g'
          perl -pi -E 's/SHA1_ARRAY_INIT/OID_ARRAY_INIT/g'
      Signed-off-by: brian m. carlson's avatarbrian m. carlson <sandals@crustytoothpaste.net>
      Signed-off-by: 's avatarJunio C Hamano <gitster@pobox.com>
      910650d2
  25. 28 Mar, 2017 1 commit