Skip to content
Snippets Groups Projects

Add Go guidelines to developer docs

Merged Philippe Lafoucrière requested to merge docs/go-standards into master
@@ -6,7 +6,7 @@ projects using the [Go language][go].
@@ -6,7 +6,7 @@ projects using the [Go language][go].
## Overview
## Overview
GitLab is built on top of [Ruby on Rails][rails], but we're also using [Go][go]
GitLab is built on top of [Ruby on Rails][rails], but we're also using [Go][go]
for projects where it makes sense. [Go][go] is a very powerful languages, with
for projects where it makes sense. [Go][go] is a very powerful language, with
many advantages, and is best suited for projects with a lot of IO (disk/network
many advantages, and is best suited for projects with a lot of IO (disk/network
access), http requests, parallel processing, etc. Since we have both [Ruby on
access), http requests, parallel processing, etc. Since we have both [Ruby on
Rails][rails] and [Go][go] at GitLab, we should evaluate carefully which of the
Rails][rails] and [Go][go] at GitLab, we should evaluate carefully which of the
@@ -24,20 +24,18 @@ https://github.com/golang/go/wiki/CodeReviewComments
@@ -24,20 +24,18 @@ https://github.com/golang/go/wiki/CodeReviewComments
Reviewers and maintainers should pay attention to:
Reviewers and maintainers should pay attention to:
- defer functions (add examples here)
- `defer` functions: Ensure presence when needed, and after `err` check.
- inject dependencies as parameters
- inject dependencies as parameters
- void structs when marshalling to JSON (generates `null` instead of `[]`)
- void structs when marshalling to JSON (generates `null` instead of `[]`)
TODO: Give examples
### Security
### Security
Security is our top-priority at GitLab. During code reviews, we must take care
Security is our top-priority at GitLab. During code reviews, we must take care
of possible security breaches in our code:
of possible security breaches in our code:
- XSS using text/template
- XSS when using text/template
- CSRF Protection using Gorilla
- CSRF Protection using Gorilla
- Use a Go version without know vulnerabilities
- Use a Go version without known vulnerabilities
- Don't leak secret tokens
- Don't leak secret tokens
- SQL injections
- SQL injections
@@ -46,54 +44,53 @@ Remember to run
@@ -46,54 +44,53 @@ Remember to run
your project, or at least the [gosec
your project, or at least the [gosec
analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/gosec).
analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/gosec).
Web servers can take advantages of middlewares like https://github.com/unrolled/secure
Web servers can take advantages of middlewares like https://github.com/unrolled/secure
### Finding a reviewer
### Finding a reviewer
Many of our projects are too small to have full-time maintainers. That's why we
Many of our projects are too small to have full-time maintainers. That's why we
have a shared pool of Go reviewers at GitLab. To find a reviewer, use the
have a shared pool of Go reviewers at GitLab. To find a reviewer, use the
[Engineering Projects](https://about.gitlab.com/handbook/engineering/projects/)
[Engineering Projects](https://about.gitlab.com/handbook/engineering/projects/)
page in the handbook. "GitLab Community Edition (CE)" and "GitLab Community
page in the handbook. "GitLab Community Edition (CE)" and "GitLab Community
Edition (EE)" both have a "Go" section with
Edition (EE)" both have a "Go" section with its list of reviewers.
To add yourself as a GitLab Go reviewer, add the gitlab-ce and/or ee `go`
To add yourself to this list, add the following to your profile in the
subproject to the
[team.yml](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/team.yml)
[team.yml](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/team.yml)
file.
file.
```yaml
```yaml
projects:
projects:
gitlab-ee: reviewer go
gitlab-ee: reviewer go
gitlab-ce: reviewer go
gitlab-ce: reviewer go
```
```
## Code style and format
## Code style and format
- Avoid global variables, even in packages. Doing so will introduce side
TODO
effects if the package is included multiple times.
- Avoid global variables, even in packages
- Use `go fmt` before committing
- use `go fmt`
### Automatic linting
### Automatic linting
All go projects should include these GitLab-CI jobs:
All go projects should include these GitLab-CI jobs:
```yaml
```yaml
go lint:
go lint:
image: golang:1.11
image: golang:1.11
script:
script:
- go get -u golang.org/x/lint/golint
- go get -u golang.org/x/lint/golint
- golint -set_exit_status
- golint -set_exit_status
```
```
(TODO: Share templates once nested includes are supported, like:
Once [recursive includes](https://gitlab.com/gitlab-org/gitlab-ce/issues/56836)
https://gitlab.com/gitlab-org/security-products/ci-templates/raw/master/includes-dev/analyzer.yml)
will be available, we will be able to share job templates like:
 
https://gitlab.com/gitlab-org/security-products/ci-templates/raw/master/includes-dev/analyzer.yml
## Dependencies
## Dependencies
Dependencies should be kept to the minimum. The introduction of a new
Dependencies should be kept to the minimum. The introduction of a new
dependency should be argued in the Merge Request. Both License Management
dependency should be argued in the Merge Request. Both License Management and
and Dependency Scanning should be activated on all projects to ensure new
Dependency Scanning should be activated on all projects to ensure new
dependencies security status and licence compatibility.
dependencies security status and licence compatibility.
### Modules
### Modules
@@ -103,23 +100,33 @@ Modules](https://github.com/golang/go/wiki/Modules)". It provides a way to
@@ -103,23 +100,33 @@ Modules](https://github.com/golang/go/wiki/Modules)". It provides a way to
define and lock dependencies for reproducible builds. It should be used
define and lock dependencies for reproducible builds. It should be used
whenever possible.
whenever possible.
 
There was a [bug](https://github.com/golang/go/issues/29278) on Modules
 
checksums in go <1.11.4, so make sure to use at least this version to avoid
 
`checksum mismatch` errors.
 
### ORM
### ORM
We don't use ORMs at GitLab (except ActiveRecord, in Ruby on Rails of course).
We don't use ORMs at GitLab (except ActiveRecord, in Ruby on Rails of course).
Projects can be structured with services to avoid them.
Projects can be structured with services to avoid them.
[PQ](https://github.com/lib/pq) should be enough to interact with PostgreSQL databases.
[PQ](https://github.com/lib/pq) should be enough to interact with PostgreSQL
 
databases.
### Migrations
### Migrations
TODO: https://github.com/db-journey/journey?
In the rare event of managing a hosted database, it's necessary to use a
 
migration system like ActiveRecord is providing. One simple library can be
 
used: https://github.com/db-journey/journey It was designed to be used in
 
`postgres` containers, that can be deployed as long-running pods. New versions
 
will deploy a new pod, migrating the data automatically.
## Testing
## Testing
We don't use any specific framework for testing, as the standard library
We should not use any specific library or framework for testing, as the standard library
provides already everything to get started. Some external libraries might be
provides already everything to get started. For example, some external dependencies might be
used if needed:
worth considering in case we decide to use a specific library or framework:
 
- https://github.com/stretchr/testify
- https://github.com/stretchr/testify
- ?
- https://github.com/gavv/httpexpect
Use [subtests](https://blog.golang.org/subtests) whenever possible to improve
Use [subtests](https://blog.golang.org/subtests) whenever possible to improve
code readability and test output.
code readability and test output.
@@ -133,7 +140,7 @@ Benchmarks, to ensure performance consistency over time.
@@ -133,7 +140,7 @@ Benchmarks, to ensure performance consistency over time.
Every Go program is launched from the command line.
Every Go program is launched from the command line.
[cli](https://github.com/urfave/cli) is a convenient package to create command
[cli](https://github.com/urfave/cli) is a convenient package to create command
line apps. It should be used whether the project is a daemon or a simple cli
line apps. It should be used whether the project is a daemon or a simple cli
tool. Flags can be mapped to [environment
tool. Flags can be mapped to [environment
variables](https://github.com/urfave/cli#values-from-the-environment) directly,
variables](https://github.com/urfave/cli#values-from-the-environment) directly,
which documents and centralizes at the same time all the possible command line
which documents and centralizes at the same time all the possible command line
@@ -146,11 +153,11 @@ in the code.
@@ -146,11 +153,11 @@ in the code.
The usage of a logging library is strongly recommended for daemons. Even though
The usage of a logging library is strongly recommended for daemons. Even though
there is a `log` package in the standard library, we generally use
there is a `log` package in the standard library, we generally use
[logrus](https://github.com/sirupsen/logrus). Its plugin ("hooks") system
[logrus](https://github.com/sirupsen/logrus). Its plugin ("hooks") system
makes it a powerful logging library, with the ability to add notifiers and
makes it a powerful logging library, with the ability to add notifiers and
formatters at the logger level directly.
formatters at the logger level directly.
### Tracing
### Tracing and Correlation
[LabKit](https://gitlab.com/gitlab-org/labkit) is a place to keep common
[LabKit](https://gitlab.com/gitlab-org/labkit) is a place to keep common
libraries for Go services. Currently it's vendored into two projects:
libraries for Go services. Currently it's vendored into two projects:
@@ -164,7 +171,7 @@ functionality:
@@ -164,7 +171,7 @@ functionality:
for distributed tracing.
for distributed tracing.
This gives us a thin abstraction over underlying implementations that is
This gives us a thin abstraction over underlying implementations that is
consistent across Workhorse, Gitaly, and, in future, other Go servers. For
consistent across Workhorse, Gitaly, and, in future, other Go servers. For
example, in the case of `gitlab.com/gitlab-org/labkit/tracing` we can switch
example, in the case of `gitlab.com/gitlab-org/labkit/tracing` we can switch
from using Opentracing directly to using Zipkin or Gokit's own tracing wrapper
from using Opentracing directly to using Zipkin or Gokit's own tracing wrapper
without changes to the application code, while still keeping the same
without changes to the application code, while still keeping the same
@@ -175,7 +182,7 @@ variable).
@@ -175,7 +182,7 @@ variable).
Since daemons are long-running applications, they should have mechanisms to
Since daemons are long-running applications, they should have mechanisms to
manage cancellations, and avoid unnecessary resources consumption (which could
manage cancellations, and avoid unnecessary resources consumption (which could
lead to DDOS vulnerabilities). Go Context should be used in functions that can
lead to DDOS vulnerabilities). Go Context should be used in functions that can
block, and passed as the first parameter:
block, and passed as the first parameter:
https://github.com/golang/go/wiki/CodeReviewComments#contexts
https://github.com/golang/go/wiki/CodeReviewComments#contexts
@@ -183,7 +190,7 @@ https://github.com/golang/go/wiki/CodeReviewComments#contexts
@@ -183,7 +190,7 @@ https://github.com/golang/go/wiki/CodeReviewComments#contexts
## Dockerfiles
## Dockerfiles
Every project should have a `Dockerfile` at the root of their repository, to
Every project should have a `Dockerfile` at the root of their repository, to
build and run the project. Since Go program are static binaries, they should
build and run the project. Since Go program are static binaries, they should
not require any external dependency, and shells in the final image are useless.
not require any external dependency, and shells in the final image are useless.
We encourage [Multistage
We encourage [Multistage
builds](https://docs.docker.com/develop/develop-images/multistage-build/):
builds](https://docs.docker.com/develop/develop-images/multistage-build/):
@@ -199,4 +206,5 @@ it will display its help message (if `cli` has been used).
@@ -199,4 +206,5 @@ it will display its help message (if `cli` has been used).
[Return to Development documentation](../README.md)
[Return to Development documentation](../README.md)
[rails]: http://rubyonrails.org/ [go]: https://golang.org
[rails]: http://rubyonrails.org/
 
[go]: https://golang.org
Loading