Upgrade to HAProxy 2.6 LTS & adapt config for this change

The updated documentation for configuring the new features is in the README.md as a high level overview, and at the top of the default attributes file.

The old code-path for Ubuntu 18.04 and HAProxy 1.8 is fully regression tested via InSpec tests in test/integration/*/regression_test.rb (included in CI pipeline)
The new code is tested in-line with InSpec tests on 20.04 CI runs.

Any code scoped to just run on 18.04/bionic are being removed later. They are still kept for compatibility during the migration phase.

Changes (unless mentioned, only affects HAP 2.6):

• HAProxy 1.8 code was changed minimally invasive to guarantee to continue to work on existing nodes (incl. systemd overrides and logs), while allowing 20.04 based machines to use HAProxy 2.6+. Regression tests here
  • Node attributes should be moved to the new namespace in the chef roles. Currently the cookbook provides a migration path.
• To avoid confusion with HAProxy nomenclature, frontend as a cluster type is replaced with primary. e.g. the main HAProxy for GitLab.com is now the primary flavour, instead of being called frontend.
• No systemd unit override is required for HAP 2.6+
• The new process management model is used, so HAProxy spawns worker processes that handle the actual traffic, and allow hitless restarts without hackery.
• Automatic use of all available cores (no more setting nbthreads in attributes)
• Simplified traffic routing (api_rate_limit backend and frontend are gone. There is no rate limit on GitLab.com HTTP(s) traffic imposed by HAProxy anymore.
• Updated peer list to new format, and using a new port. (to prevent accidental cross-replication between versions during the migration)
• Updates backend server list, to deduplicate redundant check and ssl options.
• Updated health checks to the new format.
• Backends and Frontends are now standalone templates which can be enabled and disabled by flavour.
• Logs will be fed through the systemd journal.
  • They will still be picked up by rsyslog and written to the known log file location, and being rotated by logrotate.
  • This is in preparation for maybe getting rid of rsyslog, logrotate and file-based fluentd logs, and pipe them directly from journald into fluentd. (Not implemented yet).
• Tuned timeouts to reasonable values (this takes special cases such as https-git traffic, etc. into account)
• Native prometheus exporter is being used instead of the haproxy_exporter
• Test-Kitchen and friends were updated to the most recent versions still compatible with chef 14.
• The test-kitchen setup represents a full gprd installation now.
• Major version of the cookbook is bumped to 5.0

This commit can be squashed.