Make staging work again

Pages broken due to missing filesystem

They're missing because the database thinks that they exist. Here's an example page:

https://staging.gitlab.com/godfat/rest-firebase

It's giving me this error: https://sentry.gitlap.com/gitlab/staginggitlabcom/issues/102971/

GRPC::InvalidArgumentlib/gitlab/gitaly_client.rb in call at line 78
error3:Storage can not be found by name 'nfs-file10'

We can't just ignore this because this would prevent me from reading my dashboard: https://staging.gitlab.com/dashboard

Basically the same error. Here's what we could possibly do to resolve them:

  • Remove projects with repository located on missing filesystem
  • Replicate the filesystem somehow

Or we could resolve it from the perspective of GitLab the software:

  1. Make that only the project page would be broken, maybe. Not sure what other pages could break
  2. Make GitLab more robust so that even missing filesystem would not cause 500 error

The above actions are not exclusive to each other.

Other issues & needed improvements (link to issue as these are worked on)

  • Can’t log in with BitBucket — https://gitlab.com/gitlab-com/infrastructure/issues/3081
  • Can’t go through sign up flow — this is currently intentional to prevent random people from signing up on staging
  • Object storage not enabled
  • Having a fleet of runners available on staging to allow testing of CI/CD features - https://gitlab.com/gitlab-com/infrastructure/issues/3518
  • Staging database is not synced frequently enough with production
    • When syncing the database, pending CI jobs should be removed
  • GEO is not set up on Staging
Edited Sep 22, 2020 by Rémy Coutable
Assignee Loading
Time tracking Loading