1. 07 Oct, 2018 3 commits
    • Jonathan Tan's avatar
      transport: list refs before fetch if necessary · 6ab40557
      Jonathan Tan authored
      The built-in bundle transport and the transport helper interface do not
      work when transport_fetch_refs() is called immediately after transport
      creation. This will be needed in a subsequent patch, so fix this.
      
      Evidence: fetch_refs_from_bundle() relies on data->header being
      initialized in get_refs_from_bundle(), and fetch() in transport-helper.c
      relies on either data->fetch or data->import being set by get_helper(),
      but neither transport_helper_init() nor fetch() calls get_helper().
      
      Up until the introduction of the partial clone feature, this has not
      been a problem, because transport_fetch_refs() is always called after
      transport_get_remote_refs(). With the introduction of the partial clone
      feature, which involves calling transport_fetch_refs() (to fetch objects
      by their OIDs) without transport_get_remote_refs(), this is still not a
      problem, but only coincidentally - we do not support partially cloning a
      bundle, and as for cloning using a transport-helper-using protocol, it
      so happens that before transport_fetch_refs() is called, fetch_refs() in
      fetch-object.c calls transport_set_option(), which means that the
      aforementioned get_helper() is invoked through set_helper_option() in
      transport-helper.c.
      
      This could be fixed by fixing the transports themselves, but it doesn't
      seem like a good idea to me to open up previously untested code paths;
      also, there may be transport helpers in the wild that assume that "list"
      is always called before "fetch". Instead, fix this by having
      transport_fetch_refs() call transport_get_remote_refs() to ensure that
      the latter is always called at least once, unless the transport
      explicitly states that it supports fetching without listing refs.
      Signed-off-by: default avatarJonathan Tan <jonathantanmy@google.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      6ab40557
    • Jonathan Tan's avatar
      transport: do not list refs if possible · 01775651
      Jonathan Tan authored
      When all refs to be fetched are exact OIDs, it is possible to perform a
      fetch without requiring the remote to list refs if protocol v2 is used.
      Teach Git to do this.
      
      This currently has an effect only for lazy fetches done from partial
      clones. The change necessary to likewise optimize "git fetch <remote>
      <sha-1>" will be done in a subsequent patch.
      Signed-off-by: default avatarJonathan Tan <jonathantanmy@google.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      01775651
    • Jonathan Tan's avatar
      transport: allow skipping of ref listing · 99bcb883
      Jonathan Tan authored
      The get_refs_via_connect() function both performs the handshake
      (including determining the protocol version) and obtaining the list of
      remote refs. However, the fetch protocol v2 supports fetching objects
      without the listing of refs, so make it possible for the user to skip
      the listing by creating a new handshake() function. This will be used in
      a subsequent commit.
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      99bcb883
  2. 27 Sep, 2018 17 commits
    • Junio C Hamano's avatar
      Sync with 2.19.1 · f84b9b09
      Junio C Hamano authored
      * maint:
        Git 2.19.1
        Git 2.18.1
        Git 2.17.2
        fsck: detect submodule paths starting with dash
        fsck: detect submodule urls starting with dash
        Git 2.16.5
        Git 2.15.3
        Git 2.14.5
        submodule-config: ban submodule paths that start with a dash
        submodule-config: ban submodule urls that start with dash
        submodule--helper: use "--" to signal end of clone options
      f84b9b09
    • Junio C Hamano's avatar
      Git 2.19.1 · cae598d9
      Junio C Hamano authored
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      cae598d9
    • Junio C Hamano's avatar
      Sync with 2.18.1 · 1958ad50
      Junio C Hamano authored
      * maint-2.18:
        Git 2.18.1
        Git 2.17.2
        fsck: detect submodule paths starting with dash
        fsck: detect submodule urls starting with dash
        Git 2.16.5
        Git 2.15.3
        Git 2.14.5
        submodule-config: ban submodule paths that start with a dash
        submodule-config: ban submodule urls that start with dash
        submodule--helper: use "--" to signal end of clone options
      1958ad50
    • Junio C Hamano's avatar
      Git 2.18.1 · 268fbcd1
      Junio C Hamano authored
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      268fbcd1
    • Junio C Hamano's avatar
      Sync with 2.17.2 · 44f87dac
      Junio C Hamano authored
      * maint-2.17:
        Git 2.17.2
        fsck: detect submodule paths starting with dash
        fsck: detect submodule urls starting with dash
        Git 2.16.5
        Git 2.15.3
        Git 2.14.5
        submodule-config: ban submodule paths that start with a dash
        submodule-config: ban submodule urls that start with dash
        submodule--helper: use "--" to signal end of clone options
      44f87dac
    • Junio C Hamano's avatar
      Git 2.17.2 · 6e9e91e9
      Junio C Hamano authored
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      6e9e91e9
    • Jeff King's avatar
      fsck: detect submodule paths starting with dash · 1a7fd1fb
      Jeff King authored
      As with urls, submodule paths with dashes are ignored by
      git, but may end up confusing older versions. Detecting them
      via fsck lets us prevent modern versions of git from being a
      vector to spread broken .gitmodules to older versions.
      
      Compared to blocking leading-dash urls, though, this
      detection may be less of a good idea:
      
        1. While such paths provide confusing and broken results,
           they don't seem to actually work as option injections
           against anything except "cd". In particular, the
           submodule code seems to canonicalize to an absolute
           path before running "git clone" (so it passes
           /your/clone/-sub).
      
        2. It's more likely that we may one day make such names
           actually work correctly. Even after we revert this fsck
           check, it will continue to be a hassle until hosting
           servers are all updated.
      
      On the other hand, it's not entirely clear that the behavior
      in older versions is safe. And if we do want to eventually
      allow this, we may end up doing so with a special syntax
      anyway (e.g., writing "./-sub" in the .gitmodules file, and
      teaching the submodule code to canonicalize it when
      comparing).
      
      So on balance, this is probably a good protection.
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      1a7fd1fb
    • Jeff King's avatar
      fsck: detect submodule urls starting with dash · a124133e
      Jeff King authored
      Urls with leading dashes can cause mischief on older
      versions of Git. We should detect them so that they can be
      rejected by receive.fsckObjects, preventing modern versions
      of git from being a vector by which attacks can spread.
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      a124133e
    • Junio C Hamano's avatar
      Sync with 2.16.5 · e43aab77
      Junio C Hamano authored
      * maint-2.16:
        Git 2.16.5
        Git 2.15.3
        Git 2.14.5
        submodule-config: ban submodule paths that start with a dash
        submodule-config: ban submodule urls that start with dash
        submodule--helper: use "--" to signal end of clone options
      e43aab77
    • Junio C Hamano's avatar
      Git 2.16.5 · 27d05d1a
      Junio C Hamano authored
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      27d05d1a
    • Junio C Hamano's avatar
      Sync with 2.15.3 · 424aac65
      Junio C Hamano authored
      * maint-2.15:
        Git 2.15.3
        Git 2.14.5
        submodule-config: ban submodule paths that start with a dash
        submodule-config: ban submodule urls that start with dash
        submodule--helper: use "--" to signal end of clone options
      424aac65
    • Junio C Hamano's avatar
      Git 2.15.3 · 924c623e
      Junio C Hamano authored
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      924c623e
    • Junio C Hamano's avatar
      Sync with Git 2.14.4 · 902df9f5
      Junio C Hamano authored
      * maint-2.14:
        Git 2.14.5
        submodule-config: ban submodule paths that start with a dash
        submodule-config: ban submodule urls that start with dash
        submodule--helper: use "--" to signal end of clone options
      902df9f5
    • Junio C Hamano's avatar
      Git 2.14.5 · d0832b28
      Junio C Hamano authored
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      d0832b28
    • Jeff King's avatar
      submodule-config: ban submodule paths that start with a dash · 273c6149
      Jeff King authored
      We recently banned submodule urls that look like
      command-line options. This is the matching change to ban
      leading-dash paths.
      
      As with the urls, this should not break any use cases that
      currently work. Even with our "--" separator passed to
      git-clone, git-submodule.sh gets confused. Without the code
      portion of this patch, the clone of "-sub" added in t7417
      would yield results like:
      
          /path/to/git-submodule: 410: cd: Illegal option -s
          /path/to/git-submodule: 417: cd: Illegal option -s
          /path/to/git-submodule: 410: cd: Illegal option -s
          /path/to/git-submodule: 417: cd: Illegal option -s
          Fetched in submodule path '-sub', but it did not contain b56243f8f4eb91b2f1f8109452e659f14dd3fbe4. Direct fetching of that commit failed.
      
      Moreover, naively adding such a submodule doesn't work:
      
        $ git submodule add $url -sub
        The following path is ignored by one of your .gitignore files:
        -sub
      
      even though there is no such ignore pattern (the test script
      hacks around this with a well-placed "git mv").
      
      Unlike leading-dash urls, though, it's possible that such a
      path _could_ be useful if we eventually made it work. So
      this commit should be seen not as recommending a particular
      policy, but rather temporarily closing off a broken and
      possibly dangerous code-path. We may revisit this decision
      later.
      
      There are two minor differences to the tests in t7416 (that
      covered urls):
      
        1. We don't have a "./-sub" escape hatch to make this
           work, since the submodule code expects to be able to
           match canonical index names to the path field (so you
           are free to add submodule config with that path, but we
           would never actually use it, since an index entry would
           never start with "./").
      
        2. After this patch, cloning actually succeeds. Since we
           ignore the submodule.*.path value, we fail to find a
           config stanza for our submodule at all, and simply
           treat it as inactive. We still check for the "ignoring"
           message.
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      273c6149
    • Jeff King's avatar
      submodule-config: ban submodule urls that start with dash · f6adec4e
      Jeff King authored
      The previous commit taught the submodule code to invoke our
      "git clone $url $path" with a "--" separator so that we
      aren't confused by urls or paths that start with dashes.
      
      However, that's just one code path. It's not clear if there
      are others, and it would be an easy mistake to add one in
      the future. Moreover, even with the fix in the previous
      commit, it's quite hard to actually do anything useful with
      such an entry. Any url starting with a dash must fall into
      one of three categories:
      
       - it's meant as a file url, like "-path". But then any
         clone is not going to have the matching path, since it's
         by definition relative inside the newly created clone. If
         you spell it as "./-path", the submodule code sees the
         "/" and translates this to an absolute path, so it at
         least works (assuming the receiver has the same
         filesystem layout as you). But that trick does not apply
         for a bare "-path".
      
       - it's meant as an ssh url, like "-host:path". But this
         already doesn't work, as we explicitly disallow ssh
         hostnames that begin with a dash (to avoid option
         injection against ssh).
      
       - it's a remote-helper scheme, like "-scheme::data". This
         _could_ work if the receiver bends over backwards and
         creates a funny-named helper like "git-remote--scheme".
         But normally there would not be any helper that matches.
      
      Since such a url does not work today and is not likely to do
      anything useful in the future, let's simply disallow them
      entirely. That protects the existing "git clone" path (in a
      belt-and-suspenders way), along with any others that might
      exist.
      
      Our tests cover two cases:
      
        1. A file url with "./" continues to work, showing that
           there's an escape hatch for people with truly silly
           repo names.
      
        2. A url starting with "-" is rejected.
      
      Note that we expect case (2) to fail, but it would have done
      so even without this commit, for the reasons given above.
      So instead of just expecting failure, let's also check for
      the magic word "ignoring" on stderr. That lets us know that
      we failed for the right reason.
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      f6adec4e
    • Jeff King's avatar
      submodule--helper: use "--" to signal end of clone options · 98afac7a
      Jeff King authored
      When we clone a submodule, we call "git clone $url $path".
      But there's nothing to say that those components can't begin
      with a dash themselves, confusing git-clone into thinking
      they're options. Let's pass "--" to make it clear what we
      expect.
      
      There's no test here, because it's actually quite hard to
      make these names work, even with "git clone" parsing them
      correctly. And we're going to restrict these cases even
      further in future commits. So we'll leave off testing until
      then; this is just the minimal fix to prevent us from doing
      something stupid with a badly formed entry.
      Reported-by: joernchen's avatarjoernchen <joernchen@phenoelit.de>
      Signed-off-by: default avatarJeff King <peff@peff.net>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      98afac7a
  3. 24 Sep, 2018 20 commits