Skip to content

Creating signatures and/or hashes for all Omnibus packages

I'm going to add some more detail about why this should be done. I don't think there was enough information in the previous description.

What we're trying to protect against:

  • Malicious modifications of packages on the package server.
  • Hijacked upgrades for existing installations by modifying DNS to point to a rogue package server (SSL should hopefully prevent this).
  • Other forms of man-in-the-middle attacks.

Our options for protecting users:

  • Signed packages.
  • Package hashes.

There are also infrastructure protections that help but those are handled in other trackers and many are already in-place.

Known knowns:

  • Packagecloud does not provide support for automatically signing packages as they arrive. It provides signatures for metadata only, using an automatically generated key stored on the server.
  • Packagecloud does automatically create hashes for each file and distributes them on the package download page.
  • Omnibus also creates hashes and provides them in the form of a <package>.metadata.json file along with the package when it's finished.
  • Both RPM and DEB packages support GnuPG signatures inside the packages themselves.
  • Hashes do not provide any form of automated protection, but they are trivial to generate. Users must manually verify hashes against the known-good list. This list must be kept in a safe place (i.e. not the package server) as it is trivial to modify.
  • It is possible to sign a hash using GnuPG so that it cannot be modified. The easiest thing to do is sign the file that contains all of the hashes.

Publishing hashes for all releases was seen as a good intermediate step towards package signatures, as they are simple to create and distribute once you have chosen a safe location to publish them.

To properly implement package hashing we need:

  • Changes to the build tools to automatically generate hashes, preferably in one file per release so users don't have to search through a large list of files to find the right hash.
  • A published location where these hashes will be uploaded for every release.
  • Updates to the installation instructions to notify users how to verify hashes and where they are located.

Should we decide to sign DEB and RPM packages (which I think is a good idea) we will need the following steps:

  • A signing key, safely stored in a location available to release managers as needed.
  • A public key and fingerprint published in a location available to all users and to automated package verification tools.
  • Modifications to the release tools to support package signing.

I hope that clears up some of the intent behind this and how I hoped we'd get there. This is still a discussion issue so I don't want to give the impression I'm forcing a set of actions. Please let me know if you agree/disagree.

Pending work

Appended 27 July 2017 (by @WarheadsSE), Updated 8 August 2017 (by @WarheadsSE)

  • Implement new keypair #2632
    • Generate new keypair
    • Upload to appropriate location
    • Change/add values for GPG_PASSPHRASE and SECRET_AWS_*
    • Upload pubkeys to PackageCloud repositories
      • Upload to unstable (@WarheadsSE completed 2017-08-07)
      • Upload to public gitlab-ce/gitlab-ee on 2017-08-21
    • Write documentation on generation / maintenance of keys. gitlab-com/runbooks#10 (moved)
    • Write documentation for users to enable the checking of package signatures !1823 (merged)
  • Backport into supported stable trees #2621 (closed)
  • Activate in production branches !1771 (merged)
  • Upstream changes to chef/Ominbus
Edited by Joshua Lambert