Skip to content

Praefect ping-node must verify storage locations are served

Pavlo Strokov requested to merge ps-verify-storage-locations into master

Before we start praefect we must be sure in it configured properly and dependencies are ready to use.
The gitaly nodes could be accessible to the praefect, but we also need to check if storages are properly configured to be used without issue.

Closes: #2350 (closed)

Examples of check: Invalid configuration (1 virtual storage with 2 nodes, but praefect also has one more storage configured):

2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: dialing...
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: dialing...
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: dialed successfully!
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: checking health...
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: dialed successfully!
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: SUCCESS: node is healthy!
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: checking health...
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: checking consistency...
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: SUCCESS: node is healthy!
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: checking consistency...
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: SUCCESS: "praefect-internal-1" is served by Gitaly
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: SUCCESS: "praefect-internal-0" is served by Gitaly
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: ERROR: configured storage was not reported by remote: "praefect-internal-2"
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: SUCCESS: node is consistent!
2020/03/16 23:05:37 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: ERROR: consistency check failed

Valid configuration (1 virtual storage with 2 nodes):

2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: dialing...
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: dialing...
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: dialed successfully!
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: checking health...
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: dialed successfully!
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: checking health...
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: SUCCESS: node is healthy!
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: checking consistency...
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: SUCCESS: node is healthy!
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: checking consistency...
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: SUCCESS: "praefect-internal-1" is served by Gitaly
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: SUCCESS: "praefect-internal-0" is served by Gitaly
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-1.socket]: SUCCESS: node is consistent!
2020/03/16 22:55:35 [unix:/Users/pstrokov/Workspace/gitlab-development-kit/gitaly-praefect-0.socket]: SUCCESS: node is consistent!
Edited by GitLab Release Tools Bot

Merge request reports