Geo: A few projects' repository files view never loads
Problem
Postgres streaming replication requires both the primary and the read-replica to run on OSes with compatible C library locale data. Otherwise, the read-replica may return erroneous query results and other incorrect behavior.
See https://wiki.postgresql.org/wiki/Locale_data_changes for details.
Steps to reproduce
- Set up a GitLab site with the Postgres database running on a VM on Ubuntu 18.04
- Set up a Geo secondary site with the Postgres read-replica DB running on Ubuntu 18.10
- Have many projects with Git repositories
- Visit each project, and visit Repository > Files
- Some projects' files view will spin forever, i.e. never load.
- Because a GraphQL query response contains
project: null
even though the project clearly exists. - Because a SQL query doesn't return results
- Because a PG index should be sorted differently on Ubuntu 18.10 vs 18.04
- Because a GraphQL query response contains
Proposal
-
In https://docs.gitlab.com/ee/administration/geo/replication/troubleshooting.html: Document the problem, how to confirm it, and how to resolve it. -
In https://docs.gitlab.com/ee/administration/geo/ under Requirements: Specify that PG replicas must run on the same OS as the primary, due to this PG limitation. This is highly relevant because customers often use Geo to migrate to new infrastructure. -
In https://docs.gitlab.com/omnibus/settings/database.htmldoc/install/requirements.md
: Add a small section to warn about this data corruption even without Geo.
Edited by Michael Kozono