How to migrate your patches from Gerrit
Gitlab workflow basics
We follow the suggestions by the Gitlab workflow reference.
- Merge master into your branch
- Create merge commits to resolve conflicts when merging in master
- Don't rebase
- Always squash branches before merging, to make sure that we can bisect efficiently.
Fast-forward merge not possible
When gitlab states
Fast-forward merge is not possible. Rebase the source branch onto the target branch. use the following
- git fetch
- check out your local version of the development branch
- git merge origin/master into your branch locally
- push up the branch again to gitlab
- don't rebase - it is more work and you need to force push to upload
Naming of branches
If closing an issue use the issuenumber-title as suggested, otherwise use mr-title
Manual squash of Merge request changes to rename initial commit message
Sometimes the meaning of a MR changes over the course of feedback and the initial commit message (usually used to generate the squash commit message) becomes outdated.
To update the commit message and squash the changes on the branch do the following
- git fetch
- git checkout your-development-branch
- git reset $(git merge-base origin/target-branch your-development-branch)
- git add
- git commit # add new and improved commit message here
- git push -f origin your-development-branch
Larger features that need interdependent merge requests
Always keep individual merge requests very small. Where features require lots of changes consider
- Starting with merge requests that explain the overall structure
- Merge requests can be made depend on another
- Seeing incremental changes between dependent merge requests is not straightforward in gitlab, but possible - to compare dependent branches In GitLab menu, go to Repository -> Compare. Select the branch that corresponds to the child change as a Source in the drop-down menu, choose parent change as the Target. Click Compare button and copy the link from the browser address bar. Add "Compare to the parent: https://gitlab.com/gromacs/gromacs/-/compare/PARENT_BRANCH...CHILD_BRANCH" to the description of the merge request. You will have to keep this dependency up to date for the link to work properly. For example, if you update the parent, you will need to merge its branch to the child branch right away. Otherwise your recent updates will show up in comparison.
- Have a fork where the complete feature is visible
- Pick individual commits from the fork and turn them into merge requests
Updating regression tests
Changes in |GROMACS| that require changes in regression tests are notoriously hard, because a merge request that tests against the non-updated version of the regression tests will necessarily fail, while updating regression tests while the current change is not integrated into master, might cause other merge request pipelines to fail.
The solution is a new regression test branch, uploaded to gitlab. Then set that regression test branch as a variable when running the specific pipeline that requires the regressiontest-update.
In order to run the pipeline for a custom combination of
gromacs-regressiontest branch and
gromacs branch, do it from the
gromacs-regressiontest repo, and set the
GROMACSSOURCEBRANCH variable to the desired branch in
Variables for individual piplelines are set in the gitlab interface under
Pipelines. Then chose in the top right corner
Run for, the desired branch may be selected, and variables may be set
in the fields below.
Large tasks for 2021
- GPU infrastructure development: #3311 (Artem Zhmurov)
- Improve GPU update-constraints module: #3114 (Artem Zhmurov)
- Further improvements to GPU Buffer Ops and Comms: #3370 (Alan Gray)
- Simulation workload: ??? (Szilard P)
New simulation features
- QM/MM Interface with CP2K: #3172 (Christian Blau, Dmitry Morozov)
- Constant pH: (Paul Bauer)
- Modular simulator: #3418 (Pascal Merz)
- External API #2045 (Eric Irrgang)
- NB-LIB Integration (Joe Jordan, Prashanth Kanduri, Sebastian Keller, Victor Holanda Rusu)
- MDModularisation of Pull code, essential dynamics, interactive md, awh, ion swapping and restraint module - #3201, #2590, #3040 (Christian Blau, Pascal Merz)
- Public API
- GitLab migration: done (Paul Bauer)
Large tasks for 2022
- Notes from M. Eric Irrgang, U. Virginia
- Library cleanup to suit large workflows from Mark Abraham, ENCCS
GitLab Runner admin
High level description:
GROMACS CI jobs are described by .yml files in
admin/gitlab-ci. These are interpreted
by GitLab Runner, who runs some jobs on general resources available from GitLab, and partly on the tcblab k8s infrastructure. Several such nodes have e.g. GPUs and are available to run CI pods based on resource requirements in the .yml files.
Sometimes the GPU nodes have issues like incompatible drivers. These are often seen when jobs fail at the end of configuration stages, when the
build containers cannot be scheduled:
ERROR: Job failed (system failure): unable to upgrade connection: container not found ("build")
In such cases, someone with admin rights needs to update the tcblab salt configuration to update the driver packages, and then use MAAS to reboot the nodes. More docs are available in the repo in
/srv/salt on the salt machine.
See /mnt/cephfs/resources/gromacs/infrastructure/gitlab_runner/README.txt on tcblab filesystem.
Few docs are kept here in world-readable form!