index.html.md 8.71 KB
Newer Older
Seth Itschner's avatar
Seth Itschner committed
1
---
2
layout: handbook-page-toc
Sid Sijbrandij's avatar
Sid Sijbrandij committed
3
title: "Infrastructure Environments"
Seth Itschner's avatar
Seth Itschner committed
4
---
5

6
## On this page
7
{:.no_toc .hidden-md .hidden-lg}
8 9

- TOC
10
{:toc .hidden-md .hidden-lg}
11

Seth Itschner's avatar
Seth Itschner committed
12
## Environments
13

Seth Itschner's avatar
Seth Itschner committed
14 15
### Development

16 17
| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
| --- | --- | --- | --- | --- | --- |
Seth Itschner's avatar
Seth Itschner committed
18
|Development | various | Development| on save | Fixture | individual dev|
Seth Itschner's avatar
Seth Itschner committed
19

20
Development happens on a local machine. Therefore there is no way to provide any SLA. Access is to the individual dev. This could be either EE/CE depending on what the developer is working on.
Seth Itschner's avatar
Seth Itschner committed
21

Seth Itschner's avatar
Seth Itschner committed
22
### Demo
23

24
| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
25
| --- | --- | --- | --- | --- | --- |
Brittany Rohde's avatar
Brittany Rohde committed
26
|Demo | "GitLab Sales Demo Domains - Internal only" (found on the google drive) | Sales| Release | Fixture | Production Team|
27

28
This should be a fully featured version of the current EE release. The high SLA and tightened access is to ensure it is always available for sales. There are no features (feature flags/canary/etc) that we do not ship.
Seth Itschner's avatar
Seth Itschner committed
29

30 31 32 33 34 35 36 37 38 39 40 41 42 43
### Env-Zero

| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
| --- | --- | --- | --- | --- | --- |
| Env-Zero | N/A | Bootstrap GCP | N/A | N/A | N/A|

This environment is used as a genesis project from which all other GCP projects used to
support/manage/host gitlab.com are provisioned. No compute resources are present in the project, and
it is used solely to provide a mechanism for centrally managing GCP projects, provisioning IAM
roles/service accounts for infrastructure deployments within those projects, and controlling which
APIs are enabled for each GCP project via Infrastructure as Code (terraform).

`TODO: Add reference link to README sections in gitlab-com-infrastructure repository and group-projects`

44
### .org
Seth Itschner's avatar
Seth Itschner committed
45

46 47
| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
| --- | --- | --- | --- | --- | --- |
48
| .org | [dev.gitlab.org](https://dev.gitlab.org) | Tools for Gitlab.com | Nightly | Real Work | Production and build team |
Seth Itschner's avatar
Seth Itschner committed
49

50 51 52 53 54 55
Currently there are two main uses for the .org environment:

- Builds.
- Repos that are needed in case GitLab.com is offline.

This is a critical piece of infrastructure that is always growing in size due to build artifacts. There are discussions to make a new build server where nightly CE/EE builds can be deployed or to move the infra repos to a new host that would be an separate (not gitlab.com) EE instance. Although the environment has dev in its domain name don't refer to it as dev since that could be confused with a local development environment.
Seth Itschner's avatar
Seth Itschner committed
56

57 58
### Review Apps

59
| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
60
| --- | --- | --- | --- | --- | --- |
Seth Itschner's avatar
Seth Itschner committed
61
|Review apps | various | Test proposal| on commit | Fixture | Review app owner |
62

63
Ephemeral app environments that are created dynamically every time you push a new branch up to GitLab, and they're automatically deleted when the branch is deleted. Single container with limited access.
Seth Itschner's avatar
Seth Itschner committed
64

65 66
### One-off

67
| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
68
| --- | --- | --- | --- | --- | --- |
Seth Itschner's avatar
Seth Itschner committed
69
| one-off  | various | Testing specific features in a large environment | Release + patches | User specific  | Team developing feature |
70

71
This is less of a staging environment more like a large scale development environments. This could be because of the number of repos required, or a full sized db is required. A version of CE/EE is installed and then patches are applied as work progresses. These should be very limited in number.
Seth Itschner's avatar
Seth Itschner committed
72 73 74 75 76

**Version 1.0**

K8s & helm charts (cloud native)

77
The final version of staging is multiple container deploys managed by K8s via helm charts. This could be mapped to Master and re-deployed every time there is a successful merge to master.  There is already work started to move us to containers.   [https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2420)
Seth Itschner's avatar
Seth Itschner committed
78

79 80 81 82
### Ops

| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
| --- | --- | --- | --- | --- | --- |
83
| ops | [ops.gitlab.net](https://ops.gitlab.net/) | GitLab.com Operations | official ee releases | Fixture | SREs |
84 85 86 87 88 89 90 91 92 93 94

The ops environment hold all infrastructure that is critical for managing GitLab.com infrastructure.

At this time it includes:

* Proxy for ElasticCloud.
* Internal monitoring infrastructure that serves dashboards.gitlab.net
* An isolated GitLab deployment that serves as a backup for all operations related GitLab repositories.
* CICD jobs for critical operations tasks such as backups and maintenance.
* Runners that need to connect to production infrastructure, such as GitLab chatops.

95
### Production
96 97 98

| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
| --- | --- | --- | --- | --- | --- |
99
|Production  | [gitlab.com](https://gitlab.com/) | Production| Release Candidate | Production | Production team |
John Jarvis's avatar
John Jarvis committed
100

John Jarvis's avatar
John Jarvis committed
101
Production will be full scale and size with the ability to have a canary deploy. Production has limited access.
John Jarvis's avatar
John Jarvis committed
102
It consists of two stages:
John Jarvis's avatar
John Jarvis committed
103

104
* The canary stage is a subset of infrastructure that reaches a limited number of members of the community. We deploy to this stage first. For more information see [canary testing](/handbook/engineering/#canary-testing).
John Jarvis's avatar
John Jarvis committed
105
* The main stage serves the remaining traffic for the wider GitLab community.
106

John Jarvis's avatar
John Jarvis committed
107
### Staging
108

109
| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
110
| --- | --- | --- | --- | --- | --- |
111
|Staging  | [staging.gitlab.com](https://staging.gitlab.com/users/sign_in) | Pre-production testing | Frequently | [Pseudonymization of prod](https://en.wikipedia.org/wiki/Pseudonymization) | all engineers |
112

113
Staging has the same topology as Production and includes the same components, since they share the same [terraform configuration](https://gitlab.com/gitlab-com/gitlab-com-infrastructure/tree/master/shared/gstg-gprd).
Seth Itschner's avatar
Seth Itschner committed
114

115
Staging deployments precede production deployments as described in [releases](/handbook/engineering/releases), but Staging is deployed a lot more frequently (at least every few hours, given the build is green). This would be a static environment with an pseudonymized production database. The DB is a snapshot of the production DB (updated only often enough to keep migration times to a minimum).
116

117
If you need an account to test QA issues assigned to you on Staging, you may already have an account as Production accounts are brought across to Staging. Otherwise, if you require an account to be created, create an issue in [the access-request project](https://gitlab.com/gitlab-com/access-requests#pick-a-template) and assign to your manager for review. Requests for access to database and server environments require the approval of your manager as well as that of one of the Infrastructure managers. The same [access-request tracker](https://gitlab.com/gitlab-com/access-requests#pick-a-template) should be used to request this type of access.
John Jarvis's avatar
John Jarvis committed
118 119 120 121 122

### Pre

| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
| --- | --- | --- | --- | --- | --- |
John Jarvis's avatar
John Jarvis committed
123
| pre | pre.gitlab.com | GitLab.com PreProd | Security releases and production patches | Separate and local | SREs |
John Jarvis's avatar
John Jarvis committed
124 125 126 127

The pre environment is an environment used for validating security releases and
production patches. It does not have a full production HA topology or a
copy of the production database.
128

129 130 131 132 133 134
### GitLap

| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
| --- | --- | --- | --- | --- | --- |
| gitlap | gitlap.com | GitLab support testing | ?? | ?? | SREs |
| dev.gitlap | *.dev.gitlap.com | GitLab support testing | N/A | N/A | SRE and support owner |
135
| do.gitlap | *.do.gitlap.com | GitLab support testing | N/A | N/A | SRE and support owner |
136 137

The GitLap environment is an older domain primarily used for support testing.
138 139
All DNS records under `*.dev.gitlap.com` and `*.do.gitlap.com` are controlled
via terraform in the [dev-resources repository](https://gitlab.com/gitlab-com/dev-resources/).
140 141 142 143

The only important system is `gitlab-runner-builder.gitlap.com` which is used
as a CI runner by the [gitlab-runner project](https://gitlab.com/gitlab-org/gitlab-runner).

144
## Self-Managed
145 146 147

| **Name** | **URL** | **Purpose** | **Deploy** | **Database** | **Terminal access** |
| --- | --- | --- | --- | --- | --- |
148
|Self-Managed  | various | Self hosted versions of CE & EE | User specific  | User specific  | User specific |
Seth Itschner's avatar
Seth Itschner committed
149

150
These are environments that are run on-premises by the end-user. We have no influence, access or control of these environments.
Sid Sijbrandij's avatar
Sid Sijbrandij committed
151 152 153

## Nodes

154
If you work at GitLab, also see the [list of nodes managed by Chef](https://ops.gitlab.net/gitlab-cookbooks/chef-repo/tree/master/nodes) to get an idea.