1. 13 Jun, 2018 1 commit
    • Ævar Arnfjörð Bjarmason's avatar
      git-credential-netrc: remove use of "autodie" · 627be153
      Ævar Arnfjörð Bjarmason authored
      The "autodie" module was added in Perl 5.10.1, but our INSTALL
      document says "version 5.8 or later is needed".
      As discussed in <87efhfvxzu.fsf@evledraar.gmail.com> this script is in
      contrib/, so we might not want to apply that policy, however in this
      case "autodie" was recently added as a "gratuitous safeguard" in
      786ef50a ("git-credential-netrc: accept gpg option",
      2018-05-12) (see
      Looking at it more carefully the addition of "autodie" inadvertently
      introduced a logic error, since having it is equivalent to this patch:
          @@ -245,10 +244,10 @@ sub load_netrc {
           	if ($gpgmode) {
           		my @cmd = ($options{'gpg'}, qw(--decrypt), $file);
           		log_verbose("Using GPG to open $file: [@cmd]");
          -		open $io, "-|", @cmd;
          +		open $io, "-|", @cmd or die "@cmd: $!";
           	} else {
           		log_verbose("Opening $file...");
          -		open $io, '<', $file;
          +		open $io, '<', $file or die "$file: $!$!;
           	# nothing to do if the open failed (we log the error later)
      As shown in the context the intent of that code is not do die but to
      log the error later.
      Per my reading of the file this was the only thing autodie was doing
      in this file (there was no other code it altered). So let's remove it,
      both to fix the logic error and to get rid of the dependency.
      1. <87efhfvxzu.fsf@evledraar.gmail.com>
      2. <CAHqJXRE8OKSKcck1APHAHccLZhox+tZi8nNu2RA74RErX8s3Pg@mail.gmail.com>
      Signed-off-by: Ævar Arnfjörð Bjarmason's avatarÆvar Arnfjörð Bjarmason <avarab@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  2. 12 Jun, 2018 1 commit
    • Gábor Szeder's avatar
      completion: correct zsh detection when run from git-completion.zsh · 61d48c66
      Gábor Szeder authored
      v2.18.0-rc0~90^2 (completion: reduce overhead of clearing cached
      --options, 2018-04-18) worked around a bug in bash's "set" builtin on
      MacOS by using compgen instead.  It was careful to avoid breaking zsh
      by guarding this workaround with
      	if [[ -n ${ZSH_VERSION-}} ]]
      Alas, this interacts poorly with git-completion.zsh's bash emulation:
      	ZSH_VERSION='' . "$script"
      Correct it by instead using a new GIT_SOURCING_ZSH_COMPLETION shell
      variable to detect whether git-completion.bash is being sourced from
      git-completion.zsh.  This way, the zsh variant is used both when run
      from zsh directly and when run via git-completion.zsh.
      Reproduction recipe:
       1. cd git/contrib/completion && cp git-completion.zsh _git
       2. Put the following in a new ~/.zshrc file:
       	autoload -U compinit; compinit
      	autoload -U bashcompinit; bashcompinit
      	fpath=(~/src/git/contrib/completion $fpath)
       3. Open zsh and "git <TAB>".
      With this patch:
      Triggers nice git-completion.bash based tab completion
       contrib/completion/git-completion.bash:354: read-only variable: QISUFFIX
       zsh:12: command not found: ___main
       zsh:15: _default: function definition file not found
       _dispatch:70: bad math expression: operand expected at `/usr/bin/g...'
       Segmentation fault
      Reported-by: Rick van Hattem's avatarRick van Hattem <wolph@wol.ph>
      Reported-by: default avatarDave Borowitz <dborowitz@google.com>
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJonathan Nieder <jrnieder@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  3. 01 Jun, 2018 1 commit
  4. 29 May, 2018 4 commits
  5. 23 May, 2018 1 commit
  6. 21 May, 2018 8 commits
    • Duy Nguyen's avatar
      completion: allow to customize the completable command list · 6532f374
      Duy Nguyen authored
      By default we show porcelain, external commands and a couple others
      that are also popular. If you are not happy with this list, you can
      now customize it a new config variable.
      Signed-off-by: Duy Nguyen's avatarNguyễn Thái Ngọc Duy <pclouds@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Duy Nguyen's avatar
      completion: add and use --list-cmds=alias · 3301d36b
      Duy Nguyen authored
      By providing aliases via --list-cmds=, we could simplify command
      collection code in the script. We only issue one git command. Before
      this patch that is "git config", after it's "git --list-cmds=". In
      "git help" completion case we actually reduce one "git" process (for
      getting guides) but that call was added in this series so it does not
      really count.
      A couple of bash functions are removed because they are not needed
      anymore. __git_compute_all_commands() and $__git_all_commands stay
      because they are still needed for completing pager.* config and
      without "alias" group, the result is still cacheable.
      There is a slight (good) change in _git_help() with this patch: before
      "git help <tab>" shows external commands (as in _not_ part of git) as
      well as part of $__git_all_commands. We have finer control over
      command listing now and can exclude that because we can't provide a
      man page for external commands anyway.
      Signed-off-by: Duy Nguyen's avatarNguyễn Thái Ngọc Duy <pclouds@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Duy Nguyen's avatar
    • Duy Nguyen's avatar
      completion: let git provide the completable command list · 84a97131
      Duy Nguyen authored
      Instead of maintaining a separate list of command classification,
      which often could go out of date, let's centralize the information
      back in git.
      While the function in git-completion.bash implies "list porcelain
      commands", that's not exactly what it does. It gets all commands (aka
      --list-cmds=main,others) then exclude certain non-porcelain ones. We
      could almost recreate this list two lists list-mainporcelain and
      others. The non-porcelain-but-included-anyway is added by the third
      category list-complete.
      Note that the current completion script incorrectly classifies
      filter-branch as porcelain and t9902 tests this behavior. We keep it
      this way in t9902 because this test does not really care which
      particular command is porcelain or plumbing, they're just names.
      Signed-off-by: Duy Nguyen's avatarNguyễn Thái Ngọc Duy <pclouds@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Duy Nguyen's avatar
      help: use command-list.txt for the source of guides · 1b81d8cb
      Duy Nguyen authored
      The help command currently hard codes the list of guides and their
      summary in C. Let's move this list to command-list.txt. This lets us
      extract summary lines from Documentation/git*.txt. This also
      potentially lets us list guides in git.txt, but I'll leave that for
      Signed-off-by: Duy Nguyen's avatarNguyễn Thái Ngọc Duy <pclouds@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Duy Nguyen's avatar
      completion: implement and use --list-cmds=main,others · 6bb2dc0b
      Duy Nguyen authored
      This is part of the effort to break down and provide commands by
      category in machine-readable form. This could be helpful later on when
      completion script switches to use --list-cmds for selecting
      completable commands. It would be much easier for the user to choose
      to complete _all_ commands instead of the default selection by passing
      different values to --list-cmds in git-completino.bash.
      While at there, replace "git help -a" in git-completion.bash with
      --list-cmds since it's better suited for this task.
      Signed-off-by: Duy Nguyen's avatarNguyễn Thái Ngọc Duy <pclouds@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Duy Nguyen's avatar
      git.c: convert --list-* to --list-cmds=* · 0089521c
      Duy Nguyen authored
      Even if these are hidden options, let's make them a bit more generic
      since we're introducing more listing types shortly. The code is
      structured to allow combining multiple listing types together because
      we will soon add more types the 'builtins'.
      'parseopt' remains separate because it has separate (SPC) to match
      git-completion.bash needs and will not combine with others.
      Signed-off-by: Duy Nguyen's avatarNguyễn Thái Ngọc Duy <pclouds@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Gábor Szeder's avatar
      completion: don't return with error from __gitcomp_file_direct() · 8b4c2e0b
      Gábor Szeder authored
      In __gitcomp_file_direct() we tell Bash that it should handle our
      possible completion words as filenames with the following piece of
        # use a hack to enable file mode in bash < 4
        compopt -o filenames +o nospace 2>/dev/null ||
        compgen -f /non-existing-dir/ > /dev/null
      Unfortunately, this makes this function always return with error
      when it is not invoked in real completion, but e.g. in tests of
        - First the 'compopt' line errors out
          - either because in Bash v3.x there is no such command,
          - or because in Bash v4.x it complains about "not currently
            executing completion function",
        - then 'compgen' just silently returns with error because of the
          non-existing directory.
      Since __gitcomp_file_direct() is now the last command executed in
      __git_complete_index_file(), that function returns with error as well,
      which prevents it from being invoked in tests directly as is, and
      would require extra steps in test to hide its error code.
      So let's make sure that __gitcomp_file_direct() doesn't return with
      error, because in the tests coming in the following patch we do want
      to exercise __git_complete_index_file() directly,
      __gitcomp_file() contains the same construct, and thus it, too, always
      returns with error.  Update that function accordingly as well.
      While at it, also remove the space from between the redirection
      operator and the filename in both functions.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  7. 13 May, 2018 2 commits
  8. 07 May, 2018 1 commit
    • Florian Gamböck's avatar
      completion: load completion file for external subcommand · 085e2ee0
      Florian Gamböck authored
      Adding external subcommands to Git is as easy as to put an executable
      file git-foo into PATH. Packaging such subcommands for a Linux
      distribution can be achieved by unpacking the executable into /usr/bin
      of the user's system. Adding system-wide completion scripts for new
      subcommands, however, can be a bit tricky.
      Since bash-completion started to use dynamical loading of completion
      scripts since v1.90 (preview of v2.0), it is no longer sufficient to
      drop a completion script of a subcommand into the standard completions
      path, /usr/share/bash-completion/completions, since this script will not
      be loaded if called as a git subcommand.
      For example, look at https://bugs.gentoo.org/544722. To give a short
      summary: The popular git-flow subcommand provides a completion script,
      which gets installed as /usr/share/bash-completion/completions/git-flow.
      If you now type into a Bash shell:
          git flow <TAB>
      You will not get any completions, because bash-completion only loads
      completions for git and git has no idea that git-flow is defined in
      another file. You have to load this script manually or trigger the
      dynamic loader with:
          git-flow <TAB> # Please notice the dash instead of whitespace
      This will not complete anything either, because it only defines a Bash
      function, without generating completions. But now the correct completion
      script has been loaded and the first command can use the completions.
      So, the goal is now to teach the git completion script to consider the
      possibility of external completion scripts for subcommands, but of
      course without breaking current workflows.
      I think the easiest method is to use a function that was defined by
      bash-completion v1.90, namely _completion_loader. It will take care of
      loading the correct script if present. Afterwards, the git completion
      script behaves as usual.
      _completion_loader was introduced in commit 20c05b43 of bash-completion
      (https://github.com/scop/bash-completion.git) back in 2011, so it should
      be available in even older LTS distributions. This function searches for
      external completion scripts not only in the default path
      /usr/share/bash-completion/completions, but also in the user's home
      directory via $XDG_DATA_HOME and in a user specified directory via
      The only "drawback" (if it even can be called as such) is, that if
      _completion_loader does not find a completion script, it automatically
      registers a minimal function for basic path completion. In practice,
      however, this will not matter, because in this case the given command is
      a git command in its dashed form, e.g. 'git-diff-index', and those have
      been deprecated for a long time.
      This way we can leverage bash-completion's dynamic loading for git
      subcommands and make it easier for developers to distribute custom
      completion scripts.
      Signed-off-by: default avatarFlorian Gamböck <mail@floga.de>
      Acked-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  9. 06 May, 2018 1 commit
  10. 02 May, 2018 1 commit
    • Gábor Szeder's avatar
      coccinelle: avoid wrong transformation suggestions from commit.cocci · 279ffad1
      Gábor Szeder authored
      The semantic patch 'contrib/coccinelle/commit.cocci' added in
      2e27bd77 (treewide: replace maybe_tree with accessor methods,
      2018-04-06) is supposed to "ensure that all references to the
      'maybe_tree' member of struct commit are either mutations or accesses
      through get_commit_tree()".  So get_commit_tree() clearly must be able
      to directly access the 'maybe_tree' member, and 'commit.cocci' has a
      bit of a roundabout workaround to ensure that get_commit_tree()'s
      direct access in its return statement is not transformed: after all
      references to 'maybe_tree' have been transformed to a call to
      get_commit_tree(), including the reference in get_commit_tree()
      itself, the last rule transforms back a 'return get_commit_tree()'
      statement, back then found only in get_commit_tree() itself, to a
      direct access.
      Unfortunately, already the very next commit shows that this workaround
      is insufficient: 7b8a21db (commit-graph: lazy-load trees for
      commits, 2018-04-06) extends get_commit_tree() with a condition
      directly accessing the 'maybe_tree' member, and Coccinelle with
      'commit.cocci' promptly detects it and suggests a transformation to
      avoid it.  This transformation is clearly wrong, because calling
      get_commit_tree() to access 'maybe_tree' _in_ get_commit_tree() would
      obviously lead to recursion.  Furthermore, the same commit added
      another, more specialized getter function get_commit_tree_in_graph(),
      whose legitimate direct access to 'maybe_tree' triggers a similar
      wrong transformation suggestion.
      Exclude both of these getter functions from the general rule in
      'commit.cocci' that matches their direct accesses to 'maybe_tree'.
      Also exclude load_tree_for_commit(), which, as static helper funcion
      of get_commit_tree_in_graph(), has legitimate direct access to
      'maybe_tree' as well.
      The last rule transforming back 'return get_commit_tree()' statements
      to direct accesses thus became unnecessary, remove it.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Acked-by: Derrick Stolee's avatarDerrick Stolee <dstolee@microsoft.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  11. 30 Apr, 2018 1 commit
  12. 26 Apr, 2018 2 commits
    • Johannes Schindelin's avatar
      pull: accept --rebase=merges to recreate the branch topology · 1131ec98
      Johannes Schindelin authored
      Similar to the `preserve` mode simply passing the `--preserve-merges`
      option to the `rebase` command, the `merges` mode simply passes the
      `--rebase-merges` option.
      This will allow users to conveniently rebase non-trivial commit
      topologies when pulling new commits, without flattening them.
      Signed-off-by: Johannes Schindelin's avatarJohannes Schindelin <johannes.schindelin@gmx.de>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Johannes Schindelin's avatar
      rebase: introduce the --rebase-merges option · 8f6aed71
      Johannes Schindelin authored
      Once upon a time, this here developer thought: wouldn't it be nice if,
      say, Git for Windows' patches on top of core Git could be represented as
      a thicket of branches, and be rebased on top of core Git in order to
      maintain a cherry-pick'able set of patch series?
      The original attempt to answer this was: git rebase --preserve-merges.
      However, that experiment was never intended as an interactive option,
      and it only piggy-backed on git rebase --interactive because that
      command's implementation looked already very, very familiar: it was
      designed by the same person who designed --preserve-merges: yours truly.
      Some time later, some other developer (I am looking at you, Andreas!
      ;-)) decided that it would be a good idea to allow --preserve-merges to
      be combined with --interactive (with caveats!) and the Git maintainer
      (well, the interim Git maintainer during Junio's absence, that is)
      agreed, and that is when the glamor of the --preserve-merges design
      started to fall apart rather quickly and unglamorously.
      The reason? In --preserve-merges mode, the parents of a merge commit (or
      for that matter, of *any* commit) were not stated explicitly, but were
      *implied* by the commit name passed to the `pick` command.
      This made it impossible, for example, to reorder commits. Not to mention
      to move commits between branches or, deity forbid, to split topic branches
      into two.
      Alas, these shortcomings also prevented that mode (whose original
      purpose was to serve Git for Windows' needs, with the additional hope
      that it may be useful to others, too) from serving Git for Windows'
      Five years later, when it became really untenable to have one unwieldy,
      big hodge-podge patch series of partly related, partly unrelated patches
      in Git for Windows that was rebased onto core Git's tags from time to
      time (earning the undeserved wrath of the developer of the ill-fated
      git-remote-hg series that first obsoleted Git for Windows' competing
      approach, only to be abandoned without maintainer later) was really
      untenable, the "Git garden shears" were born [*1*/*2*]: a script,
      piggy-backing on top of the interactive rebase, that would first
      determine the branch topology of the patches to be rebased, create a
      pseudo todo list for further editing, transform the result into a real
      todo list (making heavy use of the `exec` command to "implement" the
      missing todo list commands) and finally recreate the patch series on
      top of the new base commit.
      That was in 2013. And it took about three weeks to come up with the
      design and implement it as an out-of-tree script. Needless to say, the
      implementation needed quite a few years to stabilize, all the while the
      design itself proved itself sound.
      With this patch, the goodness of the Git garden shears comes to `git
      rebase -i` itself. Passing the `--rebase-merges` option will generate
      a todo list that can be understood readily, and where it is obvious
      how to reorder commits. New branches can be introduced by inserting
      `label` commands and calling `merge <label>`. And once this mode will
      have become stable and universally accepted, we can deprecate the design
      mistake that was `--preserve-merges`.
      Link *1*:
      Link *2*:
      https://github.com/git-for-windows/build-extra/blob/master/shears.shSigned-off-by: Johannes Schindelin's avatarJohannes Schindelin <johannes.schindelin@gmx.de>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  13. 20 Apr, 2018 2 commits
  14. 17 Apr, 2018 10 commits
    • Gábor Szeder's avatar
      completion: reduce overhead of clearing cached --options · 94408dc7
      Gábor Szeder authored
      To get the names of all '$__git_builtin_*' variables caching --options
      of builtin commands in order to unset them, 8b0eaa41 (completion:
      clear cached --options when sourcing the completion script,
      2018-03-22) runs a 'set |sed s///' pipeline.  This works both in Bash
      and in ZSH, but has a higher than necessary overhead with the extra
      In Bash we can do better: run the 'compgen -v __gitcomp_builtin_'
      builtin command, which lists the same variables, but without a
      pipeline and 'sed' it can do so with lower overhead.
      ZSH will still continue to run that pipeline.
      This change also happens to work around an issue in the default Bash
      version shipped in macOS (3.2.57), reported by users of the Powerline
      shell prompt, which was triggered by the same commit 8b0eaa41 as
      well.  Powerline uses several Unicode Private Use Area code points to
      represent some of its pretty text UI elements (arrows and what not),
      and these are stored in the $PS1 variable.  Apparently the 'set'
      builtin of said Bash version on macOS has issues with these code
      points, and produces garbled output where Powerline's special symbols
      should be in the $PS1 variable.  This, in turn, triggers the following
      error message in the downstream 'sed' process:
        sed: RE error: illegal byte sequence
      Other Bash versions, notably 4.4.19 on macOS via homebrew (i.e. a
      newer version on the same platform) and 3.2.25 on CentOS (i.e. a
      slightly earlier version, though on a different platform) are not
      affected.  ZSH in macOS (the versions shipped by default or installed
      via homebrew) or on other platforms isn't affected either.
      With this patch neither the 'set' builtin is invoked to print garbage,
      nor 'sed' to choke on it.
      Issue-on-macOS-reported-by: default avatarStephon Harris <theonestep4@gmail.com>
      Issue-on-macOS-explained-by: Matt Coleman's avatarMatthew Coleman <matt@1eanda.com>
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Gábor Szeder's avatar
      completion: fill COMPREPLY directly when completing paths · 7b003420
      Gábor Szeder authored
      During git-aware path completion, when a lot of path components have
      to be listed, a significant amount of time is spent in
      __gitcomp_file(), or more accurately in the shell loop of
      __gitcompappend(), iterating over all the path components filtering
      path components matching the current word to be completed, adding
      prefix path components, and placing the resulting matching paths into
      the COMPREPLY array.
      Now, a previous patch in this series made 'git ls-files' and 'git
      diff-index' list only paths matching the current word to be completed,
      so an additional filtering in __gitcomp_file() is not necessary
      anymore.  Adding the prefix path components could be done much more
      efficiently in __git_index_files()'s 'awk' script while stripping
      trailing path components and removing duplicates and quoting.  And
      then the resulting paths won't require any more filtering or
      processing before being handed over to Bash, so we could fill the
      COMPREPLY array directly.
      Unfortunately, we can't simply use the __gitcomp_direct() helper
      function to do that, because __gitcomp_file() does one additional
      thing: it tells Bash that we are doing filename completion, so the
      shell will kindly do four important things for us:
        1. Append a trailing space to all filenames.
        2. Append a trailing '/' to all directory names.
        3. Escape any meta, globbing, separator, etc. characters.
        4. List only the current path component when listing possible
           completions (i.e. 'dir/subdir/f<TAB>' will list 'file1', 'file2',
           etc. instead of the whole 'dir/subdir/file1',
      While we could let __git_index_files()'s 'awk' script take care of the
      first two points, the third one gets tricky, and we absolutely need
      the shell's support for the fourth.
      Add the helper function __gitcomp_file_direct(), which, just like
      __gitcomp_direct(), fills the COMPREPLY array with prefiltered and
      preprocessed paths without any additional processing, without a shell
      loop, with just one single compound assignment, and, similar to
      __gitcomp_file(), tells Bash and ZSH that we are doing filename
      completion.  Extend __git_index_files()'s 'awk' script a bit to
      prepend any prefix path components to all listed paths.  Finally,
      modify __git_complete_index_file() to feed __git_index_files()'s
      output to ___gitcomp_file_direct() instead of __gitcomp_file().
      After this patch there is no shell loop left in the path completion
      code path.
      This speeds up path completion when there are a lot of paths matching
      the current word to be completed.  In a pathological repository with
      100k files in a single directory, listing all those files:
        Before this patch, best of five, using GNU awk on Linux:
          $ time cur=dir/ __git_complete_index_file
          real    0m0.983s
          user    0m1.004s
          sys     0m0.033s
          real    0m0.313s
          user    0m0.341s
          sys     0m0.029s
        Difference: -68.2%
        Speedup:      3.1x
        To see the benefits of the whole patch series, the same command with
          real    0m2.736s
          user    0m2.472s
          sys     0m0.610s
        Difference: -88.6%
        Speedup:      8.7x
      Note that this patch changes the output of the __git_index_files()
      helper function by unconditionally prepending the prefix path
      components to every listed path.  This would break users' completion
      scriptlets that directly run:
        __gitcomp_file "$(__git_index_files ...)" "$pfx" "$cur_"
      because that would add the prefix path components once more.
      However, __git_index_files() is kind of a "helper function of a helper
      function", and users' completion scriptlets should have been using
      __git_complete_index_file() for git-aware path completion in the first
      place, so this is likely doesn't worth worrying about.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Gábor Szeder's avatar
      completion: improve handling quoted paths in 'git ls-files's output · 193757f8
      Gábor Szeder authored
      If any pathname contains backslash, double quote, tab, newline, or any
      control characters, 'git ls-files' and 'git diff-index' will enclose
      that pathname in double quotes and escape those special characters
      using C-style one-character escape sequences or \nnn octal values.
      This prevents those files from being listed during git-aware path
      completion, because due to the quoting they will never match the
      current word to be completed.
      Extend __git_index_files()'s 'awk' script to remove all that quoting
      and escaping from unique path components, so even paths containing
      (almost all) such special characters can be completed.
      Paths containing newline characters are still an issue, though.  We
      use newlines as separator character when filling the COMPREPLY array,
      so a path with one or more newline will end up split to two or more
      elements in COMPREPLY, basically breaking completion.  There is
      nothing we can do about it without a significant performance hit, so
      let's just ignore such paths for now.  As far as paths with newlines
      are concerned, this isn't any different from the previous behavior,
      because those paths were always omitted, though in the past they were
      omitted because due to the quoting they didn't match the current word
      to be completed.  Anyway, Bash's own filename completion (Meta-/) can
      complete even those paths, if need be.
        - We don't dequote path components right away as they are coming in,
          because then we would have to dequote each directory name
          repeatedly, as many times as it appears in the input, i.e. as many
          times as the number of listed paths it contains.  Instead, we
          dequote them at the end, as we print unique path components.
        - Even when a directory name itself does not contain any special
          characters, it will still be quoted if any of its trailing path
          components do.  If a directory contains paths both with and
          without special characters, then the name of that directory will
          appear both quoted and unquoted in the output of 'git ls-files'
          and 'git diff-index'.  Consequently, we will add such a directory
          name to the deduplicating associative array twice: once quoted and
          once unquoted.
          This means that we have to be careful after dequoting a directory
          name, and only print it if we haven't seen the same directory name
        - It would be wonderful if we could just pass '-z' to those git
          commands to output \0-separated unquoted paths, and use \0 as
          record separator in the 'awk' script processing their output...
          this patch would be so much simpler, almost trivial even.
          Unfortunately, however, POSIX and most 'awk' implementations don't
          support \0 as record separator (GNU awk does support it).
        - This patch makes the earlier change to list paths with
          'core.quotePath=false' basically redundant, because this could
          decode any \nnn-escaped non-ASCII character just fine, as well.
          However, I suspect that 'git ls-files' can deal with those
          non-ASCII characters faster than this updated 'awk' script; just
          in case someone is burdened with tons of pathnames containing
          non-ASCII characters.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Gábor Szeder's avatar
      completion: remove repeated dirnames with 'awk' during path completion · c1bc0a0e
      Gábor Szeder authored
      During git-aware path completion, after all the trailing path
      components have been removed from the output of 'git ls-files' and
      'git diff-index' (see previous patch), each directory name is repeated
      as many times as the number of listed paths it contains.  This can be
      a lot of repetitions, especially when invoking path completion close
      to the root of a big worktree, which would cause a considerable
      overhead downstream of __git_index_files(), in particular in the shell
      loop that fills the COMPREPLY array.  To reduce this overhead,
      __git_index_files() runs the classic '... |sort |uniq' pattern to
      remove those repetitions from the function's output.
      While removing repeated directory names is effective in reducing the
      number of iterations in that shell loop, it still imposes the overhead
      of fork()+exec()ing two external processes, and two additional stages
      in the pipeline, where potentially relatively large amount of data can
      be passed between two subsequent pipeline stages.
      Extend __git_index_files()'s 'awk' script to remove repeated path
      components by first creating and filling an associative array indexed
      by all encountered path components (after the trailing path components
      have been removed), and then iterating over this array and printing
      the indices, i.e. unique path components.  This way we can remove the
      '|sort |uniq' pipeline stages, and their eliminated overhead results
      in faster path completion.
      Listing all tracked files (12) and directories (23) at the top of the
      worktree in linux.git (over 62k files), i.e. what's doing all the hard
      work behind 'git rm <TAB>':
        Before this patch, best of five, using GNU awk on Linux:
          real    0m0.069s
          user    0m0.089s
          sys     0m0.026s
          real    0m0.052s
          user    0m0.072s
          sys     0m0.014s
        Difference: -24.6%
      Note that this changes order of elements in __git_index_files()'s
      output.  This is not an issue, because this function was only ever
      intended to feed paths into the COMPREPLY array, and Bash will sort
      its elements (according to the users locale) anyway.
      Note also that using 'awk' to remove repeated path components is also
      beneficial for the performance of the next two patches:
        - The first will extend this 'awk' script to dequote quoted paths in
          the output of 'git ls-files' and 'git diff-index'.  With this
          patch it will only have to dequote unique path components, not
        - The second will, among other things, extend this 'awk' script to
          prepend prefix path components from the command line to the
          currently completed path component.  Consequently, each line in
          'awk's output will grow longer.  Without this patch that '|sort
          |uniq' would have to exchange and process that much more data.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Gábor Szeder's avatar
      completion: use 'awk' to strip trailing path components · 105c0eff
      Gábor Szeder authored
      During git-aware path completion we complete one path component at a
      time, i.e. 'git add <TAB>' offers only 'dir/' at first, not
      'dir/subdir/file' right away, just like Bash's own filename
      completion.  However, since both 'git ls-files' and 'git diff-index'
      dive deep into subdirectories, we have to strip all trailing path
      components from the listed paths, keeping only the leading path
      component.  This stripping is currently done in a shell loop in
      __git_index_files(), which can take a significant amount of time when
      it has to iterate through a large number of paths.
      Replace this shell loop with a little 'awk' script using '/' as input
      field separator and printing the first field, which produces the same
      output much faster.
      Listing all tracked files (12) and directories (23) at the top of the
      worktree in linux.git (over 62k files), i.e. what's doing all the hard
      work behind 'git rm <TAB>':
        Before this patch, best of five, using GNU awk on Linux:
          $ time cur= __git_complete_index_file
          real    0m2.149s
          user    0m1.307s
          sys     0m1.086s
          real    0m0.067s
          user    0m0.089s
          sys     0m0.023s
        Difference: -96.9%
        Speedup:     32.1x
      Note that this could be done with 'sed', or even with 'cut', just as
      well, but the upcoming patches require 'awk's scriptability.
      Note also that this change means one more fork()+exec()ed process
      during path completion, adding more overhead especially on Windows,
      but a later patch will more than make up for it by eliminating two
      other processes in the same function.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Gábor Szeder's avatar
      completion: let 'ls-files' and 'diff-index' filter matching paths · a364e984
      Gábor Szeder authored
      During git-aware path completion, e.g. 'git rm dir/fil<TAB>', both
      'git ls-files' and 'git diff-index' list all paths in the given 'dir/'
      matching certain criteria (cached, modified, untracked, etc.)
      appropriate for the given git command, even paths whose names don't
      begin with 'fil'.  This comes with a considerable performance
      penalty when the directory in question contains a lot of paths, but
      the current word can be uniquely completed or when only a handful of
      those paths match the current word.
      Reduce the number of iterations in this codepath from the number of
      paths to the number of matching paths by specifying an appropriate
      globbing pattern to 'git ls-files' and 'git diff-index' to list only
      paths that match the current word to be completed.
      Note that both commands treat backslashes as escape characters in
      their file arguments, e.g. to preserve the literal meaning of globbing
      characters, so we have to double every backslash in the globbing
      pattern.  This is why one of the path completion tests specifically
      checks the completion of a path containing a literal backslash
      character (that test still fails, though, because both commands output
      such paths enclosed in double quotes and the special characters
      escaped; a later patch in this series will deal with those).
      This speeds up path completion considerably when there are a lot of
      non-matching paths to be filtered out.  Uniquely completing a tracked
      filename at the top of the worktree in linux.git (over 62k files),
      i.e. what's doing all the hard work behind 'git rm Mak<TAB>' to
      complete 'Makefile':
        Before this patch, best of five, on Linux:
          $ time cur=Mak __git_complete_index_file
          real    0m2.159s
          user    0m1.299s
          sys     0m1.089s
          real    0m0.033s
          user    0m0.023s
          sys     0m0.015s
        Difference: -98.5%
        Speedup:     65.4x
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Gábor Szeder's avatar
      completion: improve handling quoted paths on the command line · f12785a3
      Gábor Szeder authored
      Our git-aware path completion doesn't work when it has to complete a
      word already containing quoted and/or backslash-escaped characters on
      the command line.  The root cause of the issue is that completion
      functions see all words on the command line verbatim, i.e. including
      all backslash, single and double quote characters that the shell would
      eventually remove when executing the finished command.  These
      quoting/escaping characters cause different issues depending on which
      path component of the word to be completed contains them:
        - The quoting/escaping is in the prefix path component(s).
          Let's suppose we have a directory called 'New Dir', containing two
          untracked files 'file.c' and 'file.o', and we have a gitignore
          rule ignoring object files.  In this case all of these:
            git add New\ Dir/<TAB>
            git add "New Dir/<TAB>
            git add 'New Dir/<TAB>
          should uniquely complete 'file.c' right away, but Bash offers both
          'file.c' and 'file.o' instead.  The reason for this behavior is
          that our completion script uses the prefix directory name like
          'git -C "New\ Dir/" ls-files ...", i.e. with the backslash inside
          double quotes.  Git then tries to enter a directory called
          'New\ Dir', which (most likely) fails because such a directory
          doesn't exists.  As a result our completion script doesn't list
          any files, leaves the COMPREPLY array empty, which in turn causes
          Bash to fall back to its simple filename completion and lists all
          files in that directory, i.e. both 'file.c' and 'file.o'.
        - The quoting/escaping is in the path component to be completed.
          Let's suppose we have two untracked files 'New File.c' and
          'New File.o', and we have a gitignore rule ignoring object files.
          In this case all of these:
            git add New\ Fi<TAB>
            git add "New Fi<TAB>
            git add 'New Fi<TAB>
          should uniquely complete 'New File.c' right away, but Bash offers
          both 'New File.c' and 'New File.o' instead.  The reason for this
          behavior is that our completion script uses this 'New\ Fi' or
          '"New Fi' etc. word to filter matching paths, and of course none
          of the potential filenames will match because of the included
          backslash or double quote.  The end result is the same as above:
          the completion script doesn't list any files, Bash falls back to
          its filename completion, which then lists the matching object file
          as well.
      Add the new helper function __git_dequote() [1], which removes (most
      of[2]) the quoting and escaping from the word it gets as argument.  To
      minimize the overhead of calling this function, store its result in
      the variable $dequoted_word, supposed to be declared local in the
      caller; simply printing the result would require a command
      substitution imposing the overhead of fork()ing a subshell.  Use this
      function in __git_complete_index_file() to dequote the current word,
      i.e. the path, to be completed, to avoid the above described
      quoting-related issues, thereby fixing two of the failing quoted path
      completion tests.
      [1] The bash-completion project already has a dequote() function,
          which I hoped I could borrow to deal with this, but unfortunately
          it doesn't work quite well for this purpose (perhaps that's why
          even the bash-completion project only rarely uses it).  The main
          issue is that their dequote() is implemented as:
            eval printf %s "$1" 2> /dev/null
          where $1 would contain the word to be completed.  While it's a
          short and sweet one-liner, the use of 'eval' requires that $1 is a
          syntactically valid string, which is not the case when quoting the
          path like 'git add "New Dir/<TAB>'.  This causes 'eval' to fail,
          because it can't find the matching closing double quote, and the
          function returns nothing.  The result is totally broken behavior,
          as if the current word were empty, and the completion script would
          then list all files from the current directory.  This is why one
          of the quoted path completion tests specifically checks the
          completion of a path with an opening but without a corresponding
          closing double quote character.  Furthermore, the 'eval' performs
          all kinds of expansions, which may or may not be desired; I think
          it's the latter.  Finally, using this function would require a
          command substitution.
      [2] Bash understands the $'string' quoting as well, which "expands to
          'string', with backslash-escaped characters replaced as specified
          by the ANSI C standard" (quoted from Bash manpage).  Since shell
          metacharacters, field separators, globbing, etc. can all be easily
          entered using standard shell escaping or quoting, this type of
          quoting comes in handly when dealing with control characters that
          are otherwise difficult both to "type" and to see on the command
          line.  Because of this difficulty I would assume that people do
          avoid pathnames with such control characters anyway, so I didn't
          bother implementing it.  This function is already way too long as
          it is.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Gábor Szeder's avatar
      completion: support completing non-ASCII pathnames · 3dfe23ba
      Gábor Szeder authored
      Unless the user has 'core.quotePath=false' somewhere in the
      configuration, both 'git ls-files' and 'git diff-index' will by
      default quote any pathnames that contain bytes with values higher than
      0x80, and escape those bytes as '\nnn' octal values.  This prevents
      completing paths when the current path component to be completed
      contains any non-ASCII, most notably UTF-8, characters, because none
      of the listed quoted paths will match the current word on the command
      Set 'core.quotePath=false' for those 'git ls-files' and 'git
      diff-index' invocations, so they won't consider bytes higher than 0x80
      as "unusual", and won't quote pathnames containing such characters.
      Note that pathnames containing backslash, double quote, or control
      characters will still be quoted; a later patch in this series will
      deal with those.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Gábor Szeder's avatar
      completion: simplify prefix path component handling during path completion · 6bf0ced4
      Gábor Szeder authored
      Once upon a time 'git -C "" cmd' errored out with "Cannot change to
      '': No such file or directory", therefore the completion script took
      extra steps to run 'git -C "." cmd' instead; see fca416a4
      (completion: use "git -C $there" instead of (cd $there && git ...),
      Those extra steps are not needed since 6a536e20 (git: treat "git -C
      '<path>'" as a no-op when <path> is empty, 2015-03-06), so remove
      While at it, also simplify how the trailing '/' is appended to the
      variable holding the prefix path components.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
    • Gábor Szeder's avatar
      completion: move __git_complete_index_file() next to its helpers · 722e31c7
      Gábor Szeder authored
      It's much easier to read, understand and modify the functions related
      to git-aware path completion when they are right next to each other.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  15. 16 Apr, 2018 1 commit
    • Ævar Arnfjörð Bjarmason's avatar
      git{,-blame}.el: remove old bitrotting Emacs code · 6d5ed483
      Ævar Arnfjörð Bjarmason authored
      The git-blame.el mode has been superseded by Emacs's own
      vc-annotate (invoked by C-x v g). Users of the git.el mode are now
      much better off using either Magit or the Git backend for Emacs's own
      VC mode.
      These modes were added over 10 years ago when Emacs's own Git support
      was much less mature, and there weren't other mature modes in the wild
      or shipped with Emacs itself.
      These days these modes have few if any users, and users of git aren't
      well served by us shipping these (some OS's install them alongside git
      by default, which is confusing and leads users astray).
      So let's remove these per Alexandre Julliard's message to the
      ML[1]. If someone still wants these for some reason they're better
      served by hosting these elsewhere (e.g. on ELPA), instead of us
      distributing them with git.
      However, since downstream packagers such as Debian are packaging this
      as git-el it's less disruptive to still carry these files as Elisp
      code that'll error out with a message suggesting alternatives, rather
      than drop the files entirely[2].
      Then rather than receive a cryptic load error when they upgrade
      existing users will get an error directing them to the README file, or
      to just stop requiring these modes. I think it makes sense to link to
      GitHub's hosting of contrib/emacs/README (which'll be updated by the
      time users see this) so they don't have to hunt down the packaged
      README on their local system.
      1. "Re: [PATCH] git.el: handle default excludesfile
         properly" (87muzlwhb0.fsf@winehq.org) --
      2. "Re: [PATCH v3] git{,-blame}.el: remove old bitrotting Emacs
         code" (20180327165751.GA4343@aiede.svl.corp.google.com) --
         https://public-inbox.org/git/20180327165751.GA4343@aiede.svl.corp.google.com/Signed-off-by: Ævar Arnfjörð Bjarmason's avatarÆvar Arnfjörð Bjarmason <avarab@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
  16. 11 Apr, 2018 3 commits