1. 30 Jan, 2018 1 commit
    • Gábor Szeder's avatar
      travis-ci: don't repeat the path of the cache directory · b2cbaa09
      Gábor Szeder authored
      Some of our 'ci/*' scripts repeat the name or full path of the Travis
      CI cache directory, and the following patches will add new places
      using that path.
      
      Use a variable to refer to the path of the cache directory instead, so
      it's hard-coded only in a single place.
      
      Pay extra attention to the 32 bit Linux build: it runs in a Docker
      container, so pass the path of the cache directory from the host to
      the container in an environment variable.  Note that an environment
      variable passed this way is exported inside the container, therefore
      its value is directly available in the 'su' snippet even though that
      snippet is single quoted.  Furthermore, use the variable in the
      container only if it's been assigned a non-empty value, to prevent
      errors when someone is running or debugging the Docker build locally,
      because in that case the variable won't be set as there won't be any
      Travis CI cache.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      b2cbaa09
  2. 03 Jan, 2018 2 commits
    • Gábor Szeder's avatar
      travis-ci: check that all build artifacts are .gitignore-d · b92cb86e
      Gábor Szeder authored
      Every once in a while our explicit .gitignore files get out of sync
      when our build process learns to create new artifacts, like test
      helper executables, but the .gitignore files are not updated
      accordingly.
      
      Use Travis CI to help catch such issues earlier: check that there are
      no untracked files at the end of any build jobs building Git (i.e. the
      64 bit Clang and GCC Linux and OSX build jobs, plus the GETTEXT_POISON
      and 32 bit Linux build jobs) or its documentation, and fail the build
      job if there are any present.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      b92cb86e
    • Gábor Szeder's avatar
      travis-ci: don't store P4 and Git LFS in the working tree · 88e00b70
      Gábor Szeder authored
      The Clang and GCC 64 bit Linux build jobs download and store the P4
      and Git LFS executables under the current directory, which is the
      working tree that we are about to build and test.  This means that Git
      commands like 'status' or 'ls-files' would list these files as
      untracked.  The next commit is about to make sure that there are no
      untracked files present after the build, and the downloaded
      executables in the working tree are interfering with those upcoming
      checks.
      
      Therefore, let's download P4 and Git LFS in the home directory,
      outside of the working tree.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      88e00b70
  3. 02 Jan, 2018 3 commits
    • Gábor Szeder's avatar
      travis-ci: record and skip successfully built trees · 9cc2c76f
      Gábor Szeder authored
      Travis CI dutifully builds and tests each new branch tip, even if its
      tree has previously been successfully built and tested.  This happens
      often enough in contributors' workflows, when a work-in-progress
      branch is rebased changing e.g. only commit messages or the order or
      number of commits while leaving the resulting code intact, and is then
      pushed to a Travis CI-enabled GitHub fork.
      
      This is wasting Travis CI's resources and is sometimes scary-annoying
      when the new tip commit with a tree identical to the previous,
      successfully tested one is suddenly reported in red, because one of
      the OSX build jobs happened to exceed the time limit yet again.
      
      So extend our Travis CI build scripts to skip building commits whose
      trees have previously been successfully built and tested.  Use the
      Travis CI cache feature to keep a record of the object names of trees
      that tested successfully, in a plain and simple flat text file, one
      line per tree object name.  Append the current tree's object name at
      the end of every successful build job to this file, along with a bit
      of additional info about the build job (commit object name, Travis CI
      job number and id).  Limit the size of this file to 1000 records, to
      prevent it from growing too large for git/git's forever living
      integration branches.  Check, using a simple grep invocation, in each
      build job whether the current commit's tree is already in there, and
      skip the build if it is.  Include a message in the skipped build job's
      trace log, containing the URL to the build job successfully testing
      that tree for the first time and instructions on how to force a
      re-build.  Catch the case when a build job, which successfully built
      and tested a particular tree for the first time, is restarted and omit
      the URL of the previous build job's trace log, as in this case it's
      the same build job and the trace log has just been overwritten.
      
      Note: this won't kick in if two identical trees are on two different
      branches, because Travis CI caches are not shared between build jobs
      of different branches.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Reviewed-by: default avatarLars Schneider <larsxschneider@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      9cc2c76f
    • Gábor Szeder's avatar
      travis-ci: create the cache directory early in the build process · b4a2fdc9
      Gábor Szeder authored
      It seems that Travis CI creates the cache directory for us anyway,
      even when a previous cache doesn't exist for the current build job.
      Alas, this behavior is not explicitly documented, therefore we don't
      rely on it and create the cache directory ourselves in those build
      jobs that read/write cached data (currently only the prove state).
      
      In the following commit we'll start to cache additional data in every
      build job, and will access the cache much earlier in the build
      process.
      
      Therefore move creating the cache directory to 'ci/lib-travisci.sh' to
      make sure that it exists at the very beginning of every build job.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Reviewed-by: default avatarLars Schneider <larsxschneider@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      b4a2fdc9
    • Gábor Szeder's avatar
      travis-ci: print the "tip of branch is exactly at tag" message in color · 495ea6cd
      Gábor Szeder authored
      To make this info message stand out from the regular build job trace
      output.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Reviewed-by: default avatarLars Schneider <larsxschneider@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      495ea6cd
  4. 27 Dec, 2017 1 commit
    • Gábor Szeder's avatar
      travis-ci: fine tune the use of 'set -x' in 'ci/*' scripts · a8b8b6b8
      Gábor Szeder authored
      The change in commit 4f263666 (travis-ci: use 'set -x' in 'ci/*'
      scripts for extra tracing output, 2017-12-12) left a couple of rough
      edges:
      
        - 'ci/run-linux32-build.sh' is executed in a Docker container and
          therefore doesn't source 'ci/lib-travisci.sh', which would enable
          tracing executed commands.  Enable 'set -x' in this script, too.
      
        - 'ci/print-test-failures.sh' iterates over all the files containing
          the exit codes of all the executed test scripts.  Since there are
          over 800 such files, the loop produces way too much noise with
          tracing executed commands enabled, so disable 'set -x' for this
          script.
      
        - 'ci/run-windows-build.sh' busily waits in a loop for the result of
          the Windows build, producing too much noise with tracing executed
          commands enabled as well.  Disable 'set -x' for the duration of
          that loop.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      a8b8b6b8
  5. 12 Dec, 2017 4 commits
    • Gábor Szeder's avatar
      travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output · 4f263666
      Gábor Szeder authored
      While the build logic was embedded in our '.travis.yml', Travis CI
      used to produce a nice trace log including all commands executed in
      those embedded scriptlets.  Since 657343a6 (travis-ci: move Travis CI
      code into dedicated scripts, 2017-09-10), however, we only see the
      name of the dedicated scripts, but not what those scripts are actually
      doing, resulting in a less useful trace log.  A patch later in this
      series will move setting environment variables from '.travis.yml' to
      the 'ci/*' scripts, so not even those will be included in the trace
      log.
      
      Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other
      'ci/*' scripts, so we get trace log about the commands executed in all
      of those scripts.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      4f263666
    • Gábor Szeder's avatar
      travis-ci: set GIT_TEST_HTTPD in 'ci/lib-travisci.sh' · a1157b76
      Gábor Szeder authored
      Commit 657343a6 (travis-ci: move Travis CI code into dedicated
      scripts, 2017-09-10) converted '.travis.yml's default 'before_install'
      scriptlet to the 'ci/install-dependencies.sh' script, and while doing
      so moved setting GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang
      Linux build jobs to that script.  This is wrong for two reasons:
      
       - The purpose of that script is, as its name suggests, to install
         dependencies, not to set any environment variables influencing
         which tests should be run (though, arguably, this was already an
         issue with the original 'before_install' scriptlet).
      
       - Setting the variable has no effect anymore, because that script is
         run in a separate shell process, and the variable won't be visible
         in any of the other scripts, notably in 'ci/run-tests.sh'
         responsible for, well, running the tests.
      
      Luckily, this didn't have a negative effect on our Travis CI build
      jobs, because GIT_TEST_HTTPD is a tri-state variable defaulting to
      "auto" and a functioning web server was installed in those Linux build
      jobs, so the httpd tests were run anyway.
      
      Apparently the httpd tests run just fine without GIT_TEST_HTTPD being
      set, therefore we could simply remove this environment variable.
      However, if a bug were to creep in to change the Travis CI build
      environment to run the tests as root or to not install Apache, then
      the httpd tests would be skipped and the build job would still
      succeed.  We would only notice if someone actually were to look
      through the build job's trace log; but who would look at the trace log
      of a successful build job?!
      
      Since httpd tests are important, we do want to run them and we want to
      be loudly reminded if they can't be run.  Therefore, move setting
      GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang Linux build jobs
      to 'ci/lib-travisci.sh' to ensure that the build job fails when the
      httpd tests can't be run.  (We could set it in 'ci/run-tests.sh' just
      as well, but it's better to keep all environment variables in one
      place in 'ci/lib-travisci.sh'.)
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      a1157b76
    • Gábor Szeder's avatar
      travis-ci: move setting environment variables to 'ci/lib-travisci.sh' · e3371e92
      Gábor Szeder authored
      Our '.travis.yml's 'env.global' section sets a bunch of environment
      variables for all build jobs, though none of them actually affects all
      build jobs.  It's convenient for us, and in most cases it works just
      fine, because irrelevant environment variables are simply ignored.
      
      However, $GIT_SKIP_TESTS is an exception: it tells the test harness to
      skip the two test scripts that are prone to occasional failures on
      OSX, but as it's set for all build jobs those tests are not run in any
      of the build jobs that are capable to run them reliably, either.
      
      Therefore $GIT_SKIP_TESTS should only be set in the OSX build jobs,
      but those build jobs are included in the build matrix implicitly (i.e.
      by combining the matrix keys 'os' and 'compiler'), and there is no way
      to set an environment variable only for a subset of those implicit
      build jobs.  (Unless we were to add new scriptlets to '.travis.yml',
      which is exactly the opposite direction that we took with commit
      657343a6 (travis-ci: move Travis CI code into dedicated scripts,
      2017-09-10)).
      
      So move setting $GIT_SKIP_TESTS to 'ci/lib-travisci.sh', where it can
      trivially be set only for the OSX build jobs.
      
      Furthermore, move setting all other environment variables from
      '.travis.yml' to 'ci/lib-travisci.sh', too, because a couple of
      environment variables are already set there, and this way all
      environment variables will be set in the same place.  All the logic
      controlling our builds is already in the 'ci/*' scripts anyway, so
      there is really no good reason to keep the environment variables
      separately.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      e3371e92
    • Gábor Szeder's avatar
      travis-ci: introduce a $jobname variable for 'ci/*' scripts · bf427a94
      Gábor Szeder authored
      A couple of 'ci/*' scripts are shared between different build jobs:
      'ci/lib-travisci.sh', being a common library, is sourced from almost
      every script, while 'ci/install-dependencies.sh', 'ci/run-build.sh'
      and 'ci/run-tests.sh' are shared between the "regular" GCC and Clang
      Linux and OSX build jobs, and the latter two scripts are used in the
      GETTEXT_POISON Linux build job as well.
      
      Our builds could benefit from these shared scripts being able to
      easily tell which build job they are taking part in.  Now, it's
      already quite easy to tell apart Linux vs OSX and GCC vs Clang build
      jobs, but it gets trickier with all the additional Linux-based build
      jobs included explicitly in the build matrix.
      
      Unfortunately, Travis CI doesn't provide much help in this regard.
      The closest we've got is the $TRAVIS_JOB_NUMBER variable, the value of
      which is two dot-separated integers, where the second integer
      indicates a particular build job.  While it would be possible to use
      that second number to identify the build job in our shared scripts, it
      doesn't seem like a good idea to rely on that:
      
        - Though the build job numbering sequence seems to be stable so far,
          Travis CI's documentation doesn't explicitly states that it is
          indeed stable and will remain so in the future.  And even if it
          were stable,
      
        - if we were to remove or insert a build job in the middle, then the
          job numbers of all subsequent build jobs would change accordingly.
      
      So roll our own means of simple build job identification and introduce
      the $jobname environment variable in our builds, setting it in the
      environments of the explicitly included jobs in '.travis.yml', while
      constructing one in 'ci/lib-travisci.sh' as the combination of the OS
      and compiler name for the GCC and Clang Linux and OSX build jobs.  Use
      $jobname instead of $TRAVIS_OS_NAME in scripts taking different
      actions based on the OS and build job (when installing P4 and Git LFS
      dependencies and including them in $PATH).  The following two patches
      will also rely on $jobname.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      bf427a94
  6. 02 Nov, 2017 1 commit
    • Gábor Szeder's avatar
      travis-ci: fix running P4 and Git LFS tests in Linux build jobs · 83d1efe5
      Gábor Szeder authored
      Linux build jobs on Travis CI skip the P4 and Git LFS tests since
      commit 657343a6 (travis-ci: move Travis CI code into dedicated
      scripts, 2017-09-10), claiming there are no P4 or Git LFS installed.
      
      The reason is that P4 and Git LFS binaries are not installed to a
      directory in the default $PATH, but their directories are prepended to
      $PATH.  This worked just fine before said commit, because $PATH was
      set in a scriptlet embedded in our '.travis.yml', thus its new value
      was visible during the rest of the build job.  However, after these
      embedded scriptlets were moved into dedicated scripts executed in
      separate shell processes, any variable set in one of those scripts is
      only visible in that single script but not in any of the others.  In
      this case, 'ci/install-dependencies.sh' downloads P4 and Git LFS and
      modifies $PATH, but to no effect, because 'ci/run-tests.sh' only sees
      Travis CI's default $PATH.
      
      Move adjusting $PATH to 'ci/lib-travisci.sh', which is sourced in all
      other 'ci/' scripts, so all those scripts will see the updated $PATH
      value.
      Signed-off-by: Gábor Szeder's avatarSZEDER Gábor <szeder.dev@gmail.com>
      Signed-off-by: default avatarJunio C Hamano <gitster@pobox.com>
      83d1efe5
  7. 22 Sep, 2017 1 commit
  8. 11 Sep, 2017 2 commits