[META] Install hashicorp vault and secure it

Progress:

• Vault terraform prerequisites: module, security groups, networks, etc: https://gitlab.com/gitlab-com/gitlab-com-infrastructure/merge_requests/86
• Vault VM: https://gitlab.com/gitlab-com/gitlab-com-infrastructure/merge_requests/92
• [prerequisite] internal apt mirror (https://dev.gitlab.org/cookbooks/gitlab_aptly)
• Package Vault in .deb (https://gitlab.com/gitlab-pkg/gitlab-vault)
  • With config
  • With systemd unit file
  • With integration tests
  • With in-memory backend for bootstraping
  • With PGP encrypted root token and unseal keys by default
• Collect @gl-infra PGP keys (https://gitlab.com/gitlab-com/infrastructure/issues/2851)
• Setup CI vaults with reusable backend on S3
• Implement CA
• Setup staging instance of the vault and let people play with it (at this point #2604 (closed) should be unblocked)
• Bootstrap CA on staging vault
• With CA, set up monitoring and rsyslog for staging vault.
• Switch staging vault to use consul backend, TLS being protected by the above CA
• Implement other secret backends (TBD)
• Implement HA setup in staging (second vault, cluster address, update monitoring).
• Disaster recovery testing in staging.
• Setup production HA setup.
• Write runbooks on how to use/setup/destroy/maintain the vault.
• Write runbook and test changing the set of unseal GPG keys due to loss/compromise/etc.
• Disable SSH by default.

mentioned in issue #1504 (closed)

assigned to @ilyaf

added cloud native label

changed milestone to %WoW ending 2017-07-18

mentioned in issue #1816 (closed)

added goal label

@gl-infra Please dump all your wishes/ideas on what do you need from Vault here, and I'll make it happen :)

• Must Use Two-Man rule leveraging Shamir's Secret Sharing technique for unsealing the vault.

• Leverage Vault as a PKI & CA per #1574 (closed)

as an FYI: i gleaned a pretty good overview of how to integrate it when playing with it from here: https://www.hashicorp.com/blog/using-hashicorps-vault-with-chef/

changed milestone to %WoW ending 2017-07-25

changed milestone to %WoW ending 2017-08-01

changed the description

changed milestone to %WoW ending 2017-08-08

added moved 1 label

changed the description

marked the checklist item Vault VM: https://gitlab.com/gitlab-com/gitlab-com-infrastructure/merge_requests/92 as completed

changed the description

changed milestone to %WoW ending 2017-08-15

removed goal label

added goal label

@ilyaf with all the work for Geo, how realistic is that this will happen?

@pcarranza I don't think I'll deliver vault this WoW, unless we really-really rush it. But since we decided not to do that in prod call, I'll refactor this into some sort of meta issue, and focus on building blocks for it, like aptly and deb package pipeline.

Let me think about it for some time. We can drop the goal label though, and I will add it to the bits that I'm planning to deliver.

Does this make sense?

removed goal label

It makes total sense to me @ilyaf

changed milestone to %WoW ending 2017-08-22

changed milestone to %WoW ending 2017-08-29

removed assignee

changed milestone to %WoW ending 2017-09-19

mentioned in issue #2597 (closed)

mentioned in issue #1212 (closed)

mentioned in issue #2604 (closed)

changed milestone to %WoW ending 2017-10-10

assigned to @ilyaf

Assigning back to myself, as I made some progress on the prerequisites, and now its almost only vault (next steps will require some consul/monitoring dependencies)

changed the description

changed the description

changed milestone to %WoW ending 2017-10-03

mentioned in issue #1865 (closed)

changed milestone to %WoW ending 2017-10-10

added moved 2 and removed moved 1 labels

@ilyaf can you add a point for "Unblock https://gitlab.com/gitlab-com/infrastructure/issues/2604" to list where you deem appropriate?

changed the description

marked the checklist item Setup CI vaults with reusable backend on S3 as completed

changed milestone to %WoW ending 2017-10-17

added moved 1 moved 3 labels

added moved 4 label

changed milestone to %WoW ending 2017-10-31

removed milestone

added GCP Migration label

removed milestone

marked the checklist item Collect @gl-infra PGP keys (https://gitlab.com/gitlab-com/infrastructure/issues/2851) as completed

added meta label

changed title from Install hashicorp vault and secure it to [META] Install hashicorp vault and secure it

unassigned @ilyaf

removed GCP Migration label

assigned to @ctbarrett

added Planning label

Resurrecting this to re-evaluate progress and look at planning for (re-)deploying within GCP/GKE. I'll also reach out to Hashicorp to get info on support plans and pricing for Vault Enterprise

I'll also reach out to Hashicorp to get info on support plans and pricing for Vault Enterprise

I sent a request on September 7th for more information on Terraform Enterprise as part of #4872 (closed), with a comment expressing possible interest in the enterprise-level pricing and features for other products (Consul, Vault, ...?), as well.

A customer is interested in this: https://gitlab.my.salesforce.com/00161000003RIGC

Hi @ctbarrett,

This issue does not appear to have an issue weight set.

As a general guidelines use a weight of 1 for an access request issue or a simple configuration update. Use this as a multiplier for setting the weight. If you are unsure about what weight to set it is better to add a generous estimate and change it later. If the weight on this issue is 8 or larger then it might be a good idea to consider splitting this issue up into smaller pieces.

Thanks for your help!

---

You are welcome to help improve this comment.

unassigned @ctbarrett

@ctbarrett was there something driving you towards Vault Enterprise? The base Vault Open Source package will do everything that we need, including use GKMS as a backend.

@northrup not specifically, no - it was more a question of learning about the features to see if there was anything useful for us. Given that I never received any response to my request for more information, I'm disinclined to investigate any of their enterprise options at this point.

added 1 deleted label

added to epic &25 (closed)

changed epic to &55 (closed)

added severity4 label

added priority4 label

This work has been completed in other issues. Closing this as stale.

closed