Skip to content
Snippets Groups Projects

Add Go guidelines to developer docs

Merged Philippe Lafoucrière requested to merge docs/go-standards into master
All threads resolved!
# Go standards and style guidelines
This document describes various guidelines and best practices for GitLab
projects using the [Go language][go].
projects using the [Go language][https://golang.org].
## Overview
GitLab is built on top of [Ruby on Rails][rails], but we're also using [Go][go]
for projects where it makes sense. [Go][go] is a very powerful languages, with
many advantages, and is best suited for projects with a lot of IO (disk/network
access), http requests, parallel processing, etc. Since we have both [Ruby on
Rails][rails] and [Go][go] at GitLab, we should evaluate carefully which of the
two is best for the job.
GitLab is built on top of [Ruby on Rails][https://rubyonrails.org/], but we're
also using Go for projects where it makes sense. [Go][go] is a very powerful
language, with many advantages, and is best suited for projects with a lot of
IO (disk/network access), HTTP requests, parallel processing, etc. Since we
have both Ruby on Rails and Go at GitLab, we should evaluate carefully which of
the two is best for the job.
This page aims to define and organize our Go guidelines, based on our various
experiences. Several projects were started with different standards and they
experiences. Several projects were started with different standards and they
can still have specifics. They will be described in their respective
`README.md` or `PROCESS.md` files.
## Code Review
We follow the common principles of
https://github.com/golang/go/wiki/CodeReviewComments
[Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments).
Reviewers and maintainers should pay attention to:
- defer functions (add examples here)
- inject dependencies as parameters
- void structs when marshalling to JSON (generates `null` instead of `[]`)
TODO: Give examples
- `defer` functions: ensure the presence when needed, and after `err` check.
- Inject dependencies as parameters
- Void structs when marshalling to JSON (generates `null` instead of `[]`)
### Security
Security is our top-priority at GitLab. During code reviews, we must take care
Security is our top priority at GitLab. During code reviews, we must take care
of possible security breaches in our code:
- XSS using text/template
- XSS when using text/template
- CSRF Protection using Gorilla
- Use a Go version without know vulnerabilities
- Use a Go version without known vulnerabilities
- Don't leak secret tokens
- SQL injections
Remember to run
[SAST](https://docs.gitlab.com/ee/user/project/merge_requests/sast.html) on
your project, or at least the [gosec
analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/gosec).
[SAST](https://docs.gitlab.com/ee/user/project/merge_requests/sast.html)
**[ULTIMATE]** on your project (or at least the [gosec
analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/gosec)),
and to follow our [Security
requirements](https://docs.gitlab.com/ee/development/code_review.html#security-requirements).
Web servers can take advantages of middlewares like https://github.com/unrolled/secure
Web servers can take advantages of middlewares like [Secure](https://github.com/unrolled/secure).
### Finding a reviewer
Many of our projects are too small to have full-time maintainers. That's why we
have a shared pool of Go reviewers at GitLab. To find a reviewer, use the
have a shared pool of Go reviewers at GitLab. To find a reviewer, use the
[Engineering Projects](https://about.gitlab.com/handbook/engineering/projects/)
page in the handbook. "GitLab Community Edition (CE)" and "GitLab Community
Edition (EE)" both have a "Go" section with
page in the handbook. "GitLab Community Edition (CE)" and "GitLab Community
Edition (EE)" both have a "Go" section with its list of reviewers.
To add yourself as a GitLab Go reviewer, add the gitlab-ce and/or ee `go`
subproject to the
To add yourself to this list, add the following to your profile in the
[team.yml](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/team.yml)
file.
file and ask your manager to review and merge.
```yaml
projects:
gitlab-ee: reviewer go
```yaml
projects:
gitlab-ee: reviewer go
gitlab-ce: reviewer go
```
## Code style and format
TODO
- Avoid global variables, even in packages
- use `go fmt`
- Avoid global variables, even in packages. By doing so you will introduce side
effects if the package is included multiple times.
- Use `go fmt` before committing ([Gofmt](https://golang.org/cmd/gofmt/) is a
tool that automatically formats Go source code).
### Automatic linting
All go projects should include these GitLab-CI jobs:
All Go projects should include these GitLab CI/CD jobs:
```yaml
go lint: extends: .go script:
go lint:
image: golang:1.11
script:
- go get -u golang.org/x/lint/golint
- golint -set_exit_status
```
(TODO: Share templates once nested includes are supported, like:
https://gitlab.com/gitlab-org/security-products/ci-templates/raw/master/includes-dev/analyzer.yml)
Once [recursive includes](https://gitlab.com/gitlab-org/gitlab-ce/issues/56836)
will be available, we will be able to share job templates like this
[analyzer](https://gitlab.com/gitlab-org/security-products/ci-templates/raw/master/includes-dev/analyzer.yml).
## Dependencies
Dependencies should be kept to the minimum. The introduction of a new
dependency should be argued in the Merge Request. Both License Management
and Dependency Scanning should be activated on all projects to ensure new
dependencies security status and licence compatibility.
Dependencies should be kept to the minimum. The introduction of a new
dependency should be argued in the Merge Request, as per our [Approval
Guidelines](https://docs.gitlab.com/ee/development/code_review.html#approval-guidelines).
Both License Management and Dependency Scanning should be activated on all
projects to ensure new dependencies security status and licence compatibility.
### Modules
Since Go 1.11, a standard Dependency system is available behind the name "[Go
Modules](https://github.com/golang/go/wiki/Modules)". It provides a way to
Since Go 1.11, a standard Dependency system is available behind the name [Go
Modules](https://github.com/golang/go/wiki/Modules). It provides a way to
define and lock dependencies for reproducible builds. It should be used
whenever possible.
There was a [bug on Modules
checksums](https://github.com/golang/go/issues/29278) in Go < v1.11.4, so make
sure to use at least this version to avoid `checksum mismatch` errors.
### ORM
We don't use ORMs at GitLab (except ActiveRecord, in Ruby on Rails of course).
Projects can be structured with services to avoid them.
[PQ](https://github.com/lib/pq) should be enough to interact with PostgreSQL databases.
We don't use Object-relational mapping libraries (ORMs) at GitLab (except
[ActiveRecord](https://guides.rubyonrails.org/active_record_basics.html) in
Ruby on Rails). Projects can be structured with services to avoid them.
[PQ](https://github.com/lib/pq) should be enough to interact with PostgreSQL
databases.
### Migrations
TODO: https://github.com/db-journey/journey?
In the rare event of managing a hosted database, it's necessary to use a
migration system like ActiveRecord is providing. A simple library like
[Journey](https://github.com/db-journey/journey), designed to be used in
`postgres` containers, can be deployed as long-running pods. New versions will
deploy a new pod, migrating the data automatically.
## Testing
We don't use any specific framework for testing, as the standard library
provides already everything to get started. Some external libraries might be
used if needed:
- https://github.com/stretchr/testify
- ?
We should not use any specific library or framework for testing, as the
[standard library](https://golang.org/pkg/) provides already everything to get
started. For example, some external dependencies might be worth considering in
case we decide to use a specific library or framework:
- [Testify](https://github.com/stretchr/testify)
- [httpexpect](https://github.com/gavv/httpexpect)
Use [subtests](https://blog.golang.org/subtests) whenever possible to improve
code readability and test output.
@@ -125,13 +140,14 @@ code readability and test output.
### Benchmarks
Programs handling a lot of IO or complex operations should always include
Benchmarks, to ensure performance consistency over time.
[benchmarks](https://golang.org/pkg/testing/#hdr-Benchmarks), to ensure
performance consistency over time.
## CLIs
Every Go program is launched from the command line.
[cli](https://github.com/urfave/cli) is a convenient package to create command
line apps. It should be used whether the project is a daemon or a simple cli
line apps. It should be used whether the project is a daemon or a simple cli
tool. Flags can be mapped to [environment
variables](https://github.com/urfave/cli#values-from-the-environment) directly,
which documents and centralizes at the same time all the possible command line
@@ -144,25 +160,24 @@ in the code.
The usage of a logging library is strongly recommended for daemons. Even though
there is a `log` package in the standard library, we generally use
[logrus](https://github.com/sirupsen/logrus). Its plugin ("hooks") system
[logrus](https://github.com/sirupsen/logrus). Its plugin ("hooks") system
makes it a powerful logging library, with the ability to add notifiers and
formatters at the logger level directly.
### Tracing
### Tracing and Correlation
[LabKit](https://gitlab.com/gitlab-org/labkit) is a place to keep common
libraries for Go services. Currently it's vendored into two projects:
Workhorse and Gitaly, and it exports two main (but related) pieces of
functionality:
* `gitlab.com/gitlab-org/labkit/correlation` - for propagating and extracting
correlation ids between services.
* `gitlab.com/gitlab-org/labkit/tracing` - for instrumenting Go libraries
for distributed tracing.
* [`gitlab.com/gitlab-org/labkit/correlation`](https://gitlab.com/gitlab-org/labkit/tree/master/correlation)
- for propagating and extracting correlation ids between services.
* [`gitlab.com/gitlab-org/labkit/tracing`](https://gitlab.com/gitlab-org/labkit/tree/master/tracing)
- for instrumenting Go libraries for distributed tracing.
This gives us a thin abstraction over underlying implementations that is
consistent across Workhorse, Gitaly, and, in future, other Go servers. For
consistent across Workhorse, Gitaly, and, in future, other Go servers. For
example, in the case of `gitlab.com/gitlab-org/labkit/tracing` we can switch
from using Opentracing directly to using Zipkin or Gokit's own tracing wrapper
without changes to the application code, while still keeping the same
@@ -173,20 +188,20 @@ variable).
Since daemons are long-running applications, they should have mechanisms to
manage cancellations, and avoid unnecessary resources consumption (which could
lead to DDOS vulnerabilities). Go Context should be used in functions that can
block, and passed as the first parameter:
https://github.com/golang/go/wiki/CodeReviewComments#contexts
lead to DDOS vulnerabilities). [Go
Context](https://github.com/golang/go/wiki/CodeReviewComments#contexts) should
be used in functions that can block and passed as the first parameter.
## Dockerfiles
Every project should have a `Dockerfile` at the root of their repository, to
build and run the project. Since Go program are static binaries, they should
build and run the project. Since Go program are static binaries, they should
not require any external dependency, and shells in the final image are useless.
We encourage [Multistage
builds](https://docs.docker.com/develop/develop-images/multistage-build/):
- They let the user build the project with the right Go version and
dependencies
dependencies.
- They generate a small, self-contained image, derived from `Scratch`.
Generated docker images should have the program at their `Entrypoint` to create
@@ -195,6 +210,4 @@ it will display its help message (if `cli` has been used).
---
[Return to Development documentation](../README.md)
[rails]: http://rubyonrails.org/ [go]: https://golang.org
[Return to Development documentation](../README.md).
Loading