Automated Dependency Management should be part of the one Devops platform
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
Closely related to #260291.
Introduction
Over time, codebases tend to gather dependencies. Unless these dependencies are actively maintained, they'll quickly go stale, opening a project to the potential vulnerabilities in upstream dependencies.
Dependency scanners help to reduce this risk, but only require action to be taken when a vulnerability has been detected. If the dependency is sufficiently out-of-date, by several major versions for example, upgrading to a more recent patched version can be a major task on it's own, delaying the vulnerability mitigation process.
Keeping dependencies up-to-date and current also reduces risk as each individual upgrade is smaller and easier to rollback in the event of an issue, rather than forcing occasional, but potentially disruptive major changes.
This can be automated with tooling. GitHub offers dependabot, and another widely used, and much more flexible tool is Renovate. Creating these updates manually is time-consuming, and without automation, unlikely to happen.
Proposal
Add automated dependency management tooling to GitLab. This could be done via a UI checkbox or a GitLab CI Template.
This tooling should monitor project dependencies and automatically create merge-requests containing updates, either on a schedule, or immediately when the update is to avoid a vulnerability.
Case-Study
Over the past few months, GitLab Infrastructure teams have adopted Renovate to maintain and automatically open dependency upgrade merge-requests across our codebases.
To date, ~700 merge requests have been opened (note that many of these MRs are within the Dedicated namespace, so not publicly accessible at present).
Over each weekend, Renovate will check most of project dependencies -- including development-time-only tools (eg, code generators) -- and automatically opens MRs to update projects. In many cases, these changes are patch-version upgrades and these can easily be merged. Renovate helpfully includes changelogs in the MRs, allowing for maintainers to easily understand the differences.
When a vulnerability is detected in a dependency, Renovate is configured to offer a patch MR immediately, skipping the weekend schedule and avoiding delay.
Projects can use a simple include in their .gitlab-ci.yml file to obtain this functionality. The documentation on the Infrastructure Renovate Bot configuration is available here: https://gitlab.com/gitlab-com/gl-infra/common-ci-tasks/-/blob/main/renovate-bot.md.
The Infrastructure Renovate configuration even updates include:file:ref dependencies in .gitlab-ci.yml files. This ensures that we can safely upgrade GitLab-CI includes while still ensuring that our builds are recreatible (without a ref, template upgrades can be more difficult to perform safely).
Benefits
This practice has many benefits. The main one being that its generally easier to stay ahead of vulnerabilities in dependencies when using automated dependency management. There are other advantages too:
- Pinned versions: Since dependency management is now automated, its easier to switch to pinned-versions instead of specifying version ranges. Pinned versions are less likely to be exploited via supply-chain attacks, where an attacker publishes a new version of a previously safe dependency, and that version is included automatically in a build.
- Encouraging better testing: Since the dependency updates are automated, and there will be generally more updates, better testing practices become important. For example, there may be parts of the codebase that contributors implicitly rely on manual testing by contributors before submitting the change. With automated changes, this will not be done, which encourages maintainers to ensure that those parts of the code are covered by automated tests.
- Encourages engineering teams to consider whether a dependency is really necessary: adding a dependency to a codebase implicitly adds responsibility to the team to maintain the dependency later on, but knowing that the automated dependency tooling will be offering updates to new versions can bring this into focus and encourage discussion on whether or not the dependency is worth that cost.
cc @bmarnane