omnibus-gitlab issueshttps://gitlab.com/gitlab-org/omnibus-gitlab/-/issues2024-03-19T06:44:45Zhttps://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8462External URL in home network running in K3s2024-03-19T06:44:45ZAndréExternal URL in home network running in K3sI run gitlab in my k3s cluster in my home network with self signed certificates.
I followed the instruction [configure-https-manually](https://docs.gitlab.com/omnibus/settings/ssl/index.html#configure-https-manually)
config:
```
externa...I run gitlab in my k3s cluster in my home network with self signed certificates.
I followed the instruction [configure-https-manually](https://docs.gitlab.com/omnibus/settings/ssl/index.html#configure-https-manually)
config:
```
external_url 'https://gitlab-test.k3s.home'
letsencrypt['enable'] = false
nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab-test.k3s.home.crt "
nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/gitlab-test.k3s.home.key"
```
But it returns "bad gateway" all the time but there is no error in the logs.
As soon as I comment external_url the instance starts as expected.
any idea what's wrong?
Version: gitlab/gitlab-ce:16.9.2-ce.0https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8387Migration Gitlab-EE 16.8.0 PQsocket() can't get socket descriptor2024-02-27T16:57:32ZPierre JoMigration Gitlab-EE 16.8.0 PQsocket() can't get socket descriptor### Summary
I would like to change my Gitlab server (Centos to Debian 12) and I have carried out the migration procedure but I seem to be experiencing connection problems with PostgreSQL.
I've already tried restarting my server, reconf...### Summary
I would like to change my Gitlab server (Centos to Debian 12) and I have carried out the migration procedure but I seem to be experiencing connection problems with PostgreSQL.
I've already tried restarting my server, reconfiguring it (`gitlab-ctl reconfigure`), and redoing the process, but nothing works.
### Steps to reproduce
This steps:
- https://docs.gitlab.com/ee/administration/backup_restore/migrate_to_new_server.html
- https://docs.gitlab.com/ee/administration/backup_restore/restore_gitlab.html
### What is the current *bug* behavior?
Gitlab works but it doesn't have my data and I can't connect it.
### What is the expected *correct* behavior?
Gitlab works with my data and I can connect it.
### Relevant logs
<details>
<summary> Relevant logs </summary>
<pre>
```bash
==> /var/log/gitlab/gitlab-exporter/current <==
2024-01-22_14:31:18.98322 - -> /ruby
2024-01-22_14:31:28.41162 ::1 - - [22/Jan/2024:14:31:28 UTC] "GET /sidekiq HTTP/1.1" 200 154722
2024-01-22_14:31:28.41164 - -> /sidekiq
2024-01-22_14:31:32.28404 E, [2024-01-22T14:31:32.283942 #17802] ERROR -- : Error connecting to the database: PQsocket() can't get socket descriptor
2024-01-22_14:31:32.28421 E, [2024-01-22T14:31:32.284149 #17802] ERROR -- : Error connecting to the database: PQsocket() can't get socket descriptor
2024-01-22_14:31:32.28436 E, [2024-01-22T14:31:32.284298 #17802] ERROR -- : Error connecting to the database: PQsocket() can't get socket descriptor
2024-01-22_14:31:32.30738 ::1 - - [22/Jan/2024:14:31:32 UTC] "GET /database HTTP/1.1" 200 0
2024-01-22_14:31:32.30739 - -> /database
2024-01-22_14:31:33.99938 ::1 - - [22/Jan/2024:14:31:33 UTC] "GET /ruby HTTP/1.1" 200 1087
2024-01-22_14:31:33.99940 - -> /ruby
```
</pre>
</details>
### Details of package version
<details>
<summary>Provide the package version installation details</summary>
<pre>
||/ Name Version Architecture Description
+++-==============-============-============-============================================================
un gitlab-ce <none> <none> (no description available)
ii gitlab-ee 16.8.0-ee.0 amd64 GitLab Enterprise Edition (including NGINX, Postgres, Redis)
un gitlab-fips <none> <none> (no description available)
</pre>
</details>
### Environment details
* Operating System: `Debian 12`
* Installation Target, remove incorrect values:
* Bare Metal Machine
* Installation Type, remove incorrect values:
* New Installation
* Is there any other software running on the machine: `No`
* Is this a single or multiple node installation?
* Resources
* CPU: `16`
* Memory total: `32Go`https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8342Upgrade to 16.6.1 can't running gitlab-ctl reconfigure2024-02-24T06:49:12ZWu ChenzhiUpgrade to 16.6.1 can't running gitlab-ctl reconfigure<!---
Please read this!
Before you create a new issue, please make sure that you already ran
`sudo gitlab-ctl reconfigure`
Also make sure to search in https://gitlab.com/gitlab-org/omnibus-gitlab/issues,
to verify the issue you're abo...<!---
Please read this!
Before you create a new issue, please make sure that you already ran
`sudo gitlab-ctl reconfigure`
Also make sure to search in https://gitlab.com/gitlab-org/omnibus-gitlab/issues,
to verify the issue you're about to submit isn't a duplicate.
--->
### Summary
<!-- Describe in short the bug you've encountered -->
Upgrade to 16.6.1 can't running gitlab-ctl reconfigure
### Steps to reproduce
<!-- How one can reproduce the issue - this is very important -->
```bash
docker exec -t gitlab rm -rf /var/lib/gitlab/data/backups/db
docker exec -t gitlab rm -rf /var/lib/gitlab/data/backups/repositories
docker exec -t gitlab rm /opt/gitlab/embedded/service/gitlab-rails/tmp/backup_restore.pid
docker exec -t gitlab gitlab-backup create # 在更新前先進行備份
# sudo rsync -e 'ssh -p 2023' -arvhz --progress --delete /var/lib/gitlab/data/backups/ timmy@nas.atcatw.org:/mnt/NAS/backup/gitlab/ # 異地備援
# sudo rsync -e 'ssh -p 2023' -arvhi --progress --delete --partial --append /var/lib/gitlab/ atca@docker.system.org:/var/lib/gitlab/
helm repo add gitlab https://charts.gitlab.io
helm repo add traefik https://traefik.github.io/charts
helm repo add filestash https://sebagarayco.github.io/helm-filestash
helm search repo filestash
docker stop gitlab
docker rm gitlab
helm delete gitlab-runner -n gitlab
minikube stop
minikube delete
sudo apt-get update
sudo apt-get upgrade -y
# flatpak update -y
# sudo ddclient -daemon=0 -debug -verbose -noquiet
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb
sudo dpkg -i minikube_latest_amd64.deb
docker pull gitlab/gitlab-ce:latest
docker run --cpus=6 --cpuset-cpus 0-5 -d \
--publish 80:80 \
--publish 443:443 \
--publish 22:22 \
--publish 25:25 \
--publish 465:465 \
--publish 587:587 \
--name gitlab \
--restart always \
--volume /var/lib/gitlab/config/:/etc/gitlab \
--volume /var/lib/gitlab/logs/:/var/log/gitlab \
--volume /var/lib/gitlab/data/:/var/opt/gitlab \
gitlab/gitlab-ce:latest
# export HTTP_PROXY=http://10.0.0.3:3128
# export HTTPS_PROXY=https://<proxy hostname:port>
# export NO_PROXY=localhost,127.0.0.1,10.96.0.0/12,192.168.59.0/24,192.168.49.0/24,192.168.39.0/24,10.0.0.0/24,10.1.1.0/24,10.2.2.0/24,50.0.0.0/24
minikube start --cpus 6 --driver=kvm2
helm repo update
kubectl create namespace gitlab
# helm install --namespace gitlab gitlab-runner -f gitlab-values.yaml gitlab/gitlab-runner
helm install --namespace gitlab atca-gitlab-runner -f atca-values.yaml gitlab/gitlab-runner
# helm upgrade --namespace gitlab gitlab-runner -f gitlab-values.yaml gitlab/gitlab-runner
helm upgrade --namespace gitlab atca-gitlab-runner -f atca-values.yaml gitlab/gitlab-runner
#helm search repo filestash
# install
#helm install filestash filestash/filestash \
# --namespace='filestash' --create-namespace \
# kubectl create ns traefik-v2
# Install in the namespace "traefik-v2"
# helm install --namespace=traefik-v2 \
# traefik traefik/traefik
# docker compose -f ispyagentdvr-docker-compose.yaml up -d --remove-orphans
```
### What is the current *bug* behavior?
<!-- What actually happens -->
```text
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
```
### What is the expected *correct* behavior?
<!-- What you should see instead -->
### Relevant logs
<details>
<summary> Relevant logs </summary>
<!--
Paste any relevant logs.
Live log output can be found with `sudo gitlab-ctl tail`.
Log files live in `/var/log/gitlab` by default.
-->
```text
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:10:56+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:10:57+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:10:57+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:10:57+00:00] INFO: Cinc-client pid: 29
[2023-12-08T14:10:59+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:10:59+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:10:59+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:10:59+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:10:59+00:00] INFO: Running start handlers
[2023-12-08T14:10:59+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:10:59+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/bootstrap_disable.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/metadata.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/add_trusted_certs.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/config.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/attributes/default.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/database_reindexing_disable.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/gitlab-shell.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/database_reindexing_enable.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/README.md in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/bootstrap.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/gitlab-workhorse_disable.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/default.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/mailroom.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/gitlab-healthcheck.rb in the cache.
[2023-12-08T14:10:59+00:00] INFO: Storing updated cookbooks/gitlab/recipes/puma.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/database_migrations.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/rails_pages_shared_path.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/nginx_disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/nginx.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/puma_disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/gitlab-workhorse.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/remote-syslog_disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/gitlab-rails.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/mailroom_disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/show_config.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/letsencrypt_renew.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/remote-syslog.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/selinux.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/sidekiq_disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/remove_accounts.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/users.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/web-server.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/storage-check_disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/account_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/bash_hide_env.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/gitlab_exporter.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/sidekiq.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/gitlab_sshd_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/gitlab_mattermost.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/gitlab_workhorse.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/authorizer_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/recipes/storage-check.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/base_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/gitlab_rails.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/gitlab_rails_env_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/gitlab_shell.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/gitlab_rails.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/pg_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/base_pg_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/web_server_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/incoming_email.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/logfiles_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/gitlab_workhorse_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/metrics_exporter_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/logging.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/mailroom_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/geo_pg_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/patroni.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/puma.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/grafana_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/rails_migration_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/helpers/pg_status_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/postgresql.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/redis.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/redis_uri.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/nginx.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/redis_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/registry.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/gitconfig-system.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/sidekiq.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/database.yml.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/pg_version.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/smtp_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/gitlab-healthcheck-rc.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/gitlab-rails-error.html.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/libraries/storage_check.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/gitlab-rails-rc.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/gitlab-shell-config.yml.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/gitlab.yml.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/make_metrics_rundir.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/mount_point_check.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/cable.yml.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/gitconfig.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/nginx-gitlab-health.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/nginx-gitlab-http.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/nginx-gitlab-kas-http.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/nginx-gitlab-mattermost-http.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/nginx-gitlab-registry-http.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/nginx-status.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/nginx.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/nginx-gitlab-pages-http.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/secrets.yml.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/puma.rb.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/resque.yml.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/secret_token.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/smtp_settings.rb.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/redis.yml.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-gitlab-sshd-log-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-gitlab-sshd-log-config.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-gitlab-sshd-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-gitlab-workhorse-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-gitlab-workhorse-log-config.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-gitlab-workhorse-log-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/remote_syslog.yml.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-puma-h.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-mailroom-log-config.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-mailroom-log-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-puma-log-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-puma-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-puma-log-config.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-mailroom-finish.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-remote-syslog-log-config.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-remote-syslog-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-remote-syslog-log-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-mailroom-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-puma-t.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-storage-check-log-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-sidekiq-log-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-storage-check-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/resources/database_objects.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/resources/puma_config.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/resources/rails_migration.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-storage-check-log-config.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-sidekiq-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/sv-sidekiq-log-config.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/templates/default/workhorse-config.toml.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/attributes/default.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/metadata.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/recipes/runit.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/files/default/gitlab-runsvdir.conf in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/recipes/runit_upstart.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/recipes/sysctl.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/gitlab/resources/sidekiq_service.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/gitlab.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/default.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/geo_primary.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/geo_secondary.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/certificate_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/pages.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/patroni.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/pgbouncer.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/recipes/runit_systemd.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/application.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/redis_master.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/redis_sentinel.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/sidekiq.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/spamcheck.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/services.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config_mash.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/recipes/runit_sysvinit.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/consul.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/monitoring.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/formatters/gitlab.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/config/roles/postgres.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helpers/logging_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helpers/node_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/deprecations.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/gitlab_cluster.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/handlers/gitlab.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helpers/selinux_distro_helper.rb in the cache.
- gitlab (0.0.1)
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helpers/output_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helpers/selinux_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helpers/secrets_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helpers/services_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helpers/roles_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/package.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helpers/systemd_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/omnibus_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helpers/version_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/object_proxy.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/helpers/shell_out_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/settings_dsl.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/templates/default/gitlab-runsvdir.service.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/libraries/storage_directory_helper.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/resources/gitlab_sysctl.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/resources/env_dir.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/resources/account.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/resources/templatesymlink.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/resources/storage_directory.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/package/resources/version_file.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/logrotate/recipes/disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/logrotate/attributes/default.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/logrotate/recipes/enable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/logrotate/templates/default/logrotate-service.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/logrotate/templates/default/sv-logrotate-log-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/logrotate/recipes/folders_and_configs.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/logrotate/templates/default/sv-logrotate-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/README.md in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/attributes/default.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/metadata.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/recipes/bin.rb in the cache.
- package (0.1.0)
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/logrotate/templates/default/sv-logrotate-log-config.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/logrotate/templates/default/logrotate.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/recipes/directory_locations.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/recipes/disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/logrotate/templates/default/sv-logrotate-t.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/recipes/user.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/templates/default/pg_hba.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/recipes/standalone.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/templates/default/postgresql-runtime.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/recipes/enable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/recipes/sysctl.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/templates/default/gitlab-psql-rc.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/templates/default/sv-postgresql-log-config.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/templates/default/sv-postgresql-log-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/templates/default/pg_ident.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/templates/default/postgresql.conf.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/resources/database.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/resources/query.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/resources/schema.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/templates/default/sv-postgresql-t.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/templates/default/sv-postgresql-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/resources/user.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/redis/recipes/disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/logrotate/metadata.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/resources/config.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/postgresql/resources/extension.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/redis/metadata.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/redis/attributes/attributes.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/redis/recipes/enable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/redis/templates/default/sv-redis-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/redis/templates/default/gitlab-redis-cli-rc.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/redis/templates/default/sv-redis-log-run.erb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/redis/templates/default/sv-redis-log-config.erb in the cache.
- logrotate (0.1.0)
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/redis/templates/default/redis.conf.erb in the cache.
- postgresql (0.1.0)
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/redis/resources/service.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/default.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/attributes/default.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/alertmanager.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/gitlab-exporter.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/metadata.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/gitlab-exporter_disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/alertmanager_disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/grafana.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/grafana_disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/node-exporter.rb in the cache.
- redis (0.1.0)
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/node-exporter_disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/pgbouncer-exporter.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/prometheus.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/pgbouncer-exporter_disable.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/postgres-exporter.rb in the cache.
[2023-12-08T14:11:00+00:00] INFO: Storing updated cookbooks/monitoring/recipes/user.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/gitlab-exporter.yml.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/grafana_ini.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/recipes/redis-exporter_disable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/mount_point_check.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/postgres-queries.yaml.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/recipes/prometheus_disable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/recipes/redis-exporter.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/libraries/grafana.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/rules/node.rules in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/recipes/postgres-exporter_disable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/libraries/prometheus_helper.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/rules/gitlab.rules in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/libraries/prometheus.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-gitlab-exporter-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-gitlab-exporter-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-alertmanager-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-grafana-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-grafana-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-alertmanager-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-alertmanager-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-node-exporter-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-node-exporter-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-node-exporter-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-pgbouncer-exporter-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-pgbouncer-exporter-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-gitlab-exporter-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-postgres-exporter-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-postgres-exporter-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-grafana-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-pgbouncer-exporter-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-prometheus-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-prometheus-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-prometheus-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-redis-exporter-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-postgres-exporter-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/registry/templates/default/registry-config.yml.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/registry/templates/default/sv-registry-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/registry/templates/default/sv-registry-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/registry/templates/default/sv-registry-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/mattermost/attributes/default.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/mattermost/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/mattermost/recipes/disable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/mattermost/recipes/enable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/mattermost/libraries/mattermost_helper.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/mattermost/templates/default/sv-mattermost-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/registry/attributes/default.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-redis-exporter-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/monitoring/templates/sv-redis-exporter-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/registry/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/registry/recipes/disable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/attributes/watchers.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/mattermost/templates/default/sv-mattermost-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/attributes/services.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/recipes/configure_services.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/recipes/disable_daemon.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/registry/recipes/enable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/mattermost/templates/default/sv-mattermost-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/attributes/default.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/recipes/disable.rb in the cache.
- monitoring (0.1.0)
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/recipes/enable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/recipes/disable_service_postgresql.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/libraries/consul.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/recipes/watchers.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/recipes/enable_daemon.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/recipes/enable_service_postgresql.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/libraries/consul_helper.rb in the cache.
- registry [0.1.0](2023-12-08T14:11:01+00:00) INFO: Storing updated cookbooks/consul/libraries/failover_helper.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/libraries/watch_helper.rb in the cache.
- mattermost (0.1.0)
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/templates/default/sv-consul-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/templates/default/sv-consul-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/templates/default/watcher_scripts/failover_pgbouncer.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/resources/consul_service.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitaly/attributes/default.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitaly/recipes/disable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitaly/libraries/gitaly.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/consul/templates/default/sv-consul-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitaly/templates/default/gitaly-config.toml.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitaly/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitaly/templates/default/sv-gitaly-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitaly/recipes/enable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/praefect/recipes/database_migrations.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/praefect/recipes/disable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitaly/templates/default/sv-gitaly-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitaly/templates/default/sv-gitaly-log-run.erb in the cache.
- consul (0.1.0)
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/praefect/attributes/default.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/praefect/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/praefect/templates/default/sv-praefect-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-kas/attributes/default.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/praefect/recipes/enable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/praefect/templates/default/sv-praefect-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/praefect/templates/default/sv-praefect-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-kas/recipes/disable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/praefect/templates/default/praefect-config.toml.erb in the cache.
- gitaly (0.1.0)
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-kas/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-kas/recipes/enable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-kas/libraries/gitlab_kas.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-kas/templates/default/sv-gitlab-kas-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-kas/templates/default/gitlab-kas-config.yml.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-kas/templates/default/sv-gitlab-kas-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-kas/templates/default/sv-gitlab-kas-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-pages/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-pages/attributes/default.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-pages/recipes/disable.rb in the cache.
- praefect (0.1.0)
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-pages/recipes/enable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-pages/templates/default/sv-gitlab-pages-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-pages/templates/default/secret_token.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/letsencrypt/attributes/default.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-pages/templates/default/sv-gitlab-pages-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-pages/templates/default/mount_point_check.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-pages/libraries/gitlab_pages.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-pages/templates/default/gitlab-pages-config.erb in the cache.
- gitlab-kas (0.1.0)
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/letsencrypt/recipes/disable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/letsencrypt/libraries/lets_encrypt.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/letsencrypt/recipes/enable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/letsencrypt/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/gitlab-pages/templates/default/sv-gitlab-pages-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/letsencrypt/README.md in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/nginx/templates/default/sv-nginx-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/letsencrypt/resources/certificate.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/nginx/templates/default/sv-nginx-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/letsencrypt/libraries/helper.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/nginx/templates/default/sv-nginx-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/runit/LICENSE in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/nginx/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/runit/README.md in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/runit/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/nginx/recipes/enable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/runit/libraries/helpers.rb in the cache.
- nginx (0.1.0)
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/letsencrypt/recipes/http_authorization.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/runit/libraries/resource_runit_service.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/LICENSE in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/runit/libraries/provider_runit_service.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/attributes/default.rb in the cache.
- letsencrypt (0.1.0)
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/recipes/default.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/kitchen.digitalocean.yml in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/README.md in the cache.
- gitlab-pages (0.1.0)
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/chefignore in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/kitchen.dokken.yml in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/kitchen.yml in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/Berksfile in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/libraries/acme.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/.rubocop.yml in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/resources/selfsigned.rb in the cache.
- runit (5.1.7)
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/templates/acme-challange.nginx.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/crond/metadata.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/crond/recipes/disable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/resources/ssl_key.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/crond/recipes/enable.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/acme/resources/certificate.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/crond/attributes/default.rb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/crond/templates/default/sv-crond-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/crond/templates/default/sv-crond-log-config.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/crond/templates/default/sv-crond-log-run.erb in the cache.
[2023-12-08T14:11:01+00:00] INFO: Storing updated cookbooks/crond/resources/job.rb in the cache.
- acme (4.1.6)
- crond (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:11:02+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:11:02+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 06 seconds
[2023-12-08T14:11:02+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:11:02+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:11:02+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:11:02+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:11:02+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:11:24+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:11:24+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:11:24+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:11:24+00:00] INFO: Cinc-client pid: 29
[2023-12-08T14:11:25+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:11:25+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:11:25+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:11:25+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:11:25+00:00] INFO: Running start handlers
[2023-12-08T14:11:25+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:11:26+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- gitlab (0.0.1)
- package (0.1.0)
- logrotate (0.1.0)
- postgresql (0.1.0)
- redis (0.1.0)
- monitoring (0.1.0)
- registry (0.1.0)
- mattermost (0.1.0)
- consul (0.1.0)
- gitaly (0.1.0)
- praefect (0.1.0)
- gitlab-kas (0.1.0)
- letsencrypt (0.1.0)
- gitlab-pages (0.1.0)
- nginx (0.1.0)
- runit (5.1.7)
- acme (4.1.6)
- crond (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:11:27+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:11:27+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 03 seconds
[2023-12-08T14:11:27+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:11:27+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:11:27+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:11:27+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:11:27+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:11:49+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:11:50+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:11:50+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:11:50+00:00] INFO: Cinc-client pid: 29
[2023-12-08T14:11:51+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:11:51+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:11:51+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:11:51+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:11:51+00:00] INFO: Running start handlers
[2023-12-08T14:11:51+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:11:52+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- logrotate (0.1.0)
- gitlab (0.0.1)
- package (0.1.0)
- postgresql (0.1.0)
- redis (0.1.0)
- registry (0.1.0)
- mattermost (0.1.0)
- monitoring (0.1.0)
- consul (0.1.0)
- praefect (0.1.0)
- gitaly (0.1.0)
- gitlab-kas (0.1.0)
- gitlab-pages (0.1.0)
- nginx (0.1.0)
- letsencrypt (0.1.0)
- runit (5.1.7)
- acme (4.1.6)
- crond (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:11:53+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:11:53+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 04 seconds
[2023-12-08T14:11:53+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:11:53+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:11:53+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:11:53+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:11:53+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:12:13+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:12:14+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:12:14+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:12:14+00:00] INFO: Cinc-client pid: 29
[2023-12-08T14:12:15+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:12:15+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:12:15+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:12:15+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:12:15+00:00] INFO: Running start handlers
[2023-12-08T14:12:15+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:12:16+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- postgresql (0.1.0)
- gitlab (0.0.1)
- package (0.1.0)
- redis (0.1.0)
- logrotate (0.1.0)
- monitoring (0.1.0)
- registry (0.1.0)
- gitaly (0.1.0)
- gitlab-kas (0.1.0)
- mattermost (0.1.0)
- letsencrypt (0.1.0)
- nginx (0.1.0)
- consul (0.1.0)
- gitlab-pages (0.1.0)
- praefect (0.1.0)
- runit (5.1.7)
- crond (0.1.0)
- acme (4.1.6)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:12:16+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:12:16+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 03 seconds
[2023-12-08T14:12:16+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:12:16+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:12:16+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:12:16+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:12:16+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:12:38+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:12:39+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:12:39+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:12:39+00:00] INFO: Cinc-client pid: 29
[2023-12-08T14:12:41+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:12:41+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:12:41+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:12:41+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:12:41+00:00] INFO: Running start handlers
[2023-12-08T14:12:41+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:12:41+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- gitlab (0.0.1)
- package (0.1.0)
- logrotate (0.1.0)
- postgresql (0.1.0)
- redis (0.1.0)
- registry (0.1.0)
- mattermost (0.1.0)
- monitoring (0.1.0)
- consul (0.1.0)
- gitaly (0.1.0)
- praefect (0.1.0)
- gitlab-kas (0.1.0)
- gitlab-pages (0.1.0)
- letsencrypt (0.1.0)
- nginx (0.1.0)
- runit (5.1.7)
- acme (4.1.6)
- crond (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:12:42+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:12:42+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 04 seconds
[2023-12-08T14:12:42+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:12:42+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:12:42+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:12:42+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:12:42+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:13:02+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:13:03+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:13:03+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:13:03+00:00] INFO: Cinc-client pid: 30
[2023-12-08T14:13:04+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:13:04+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:13:04+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:13:04+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:13:04+00:00] INFO: Running start handlers
[2023-12-08T14:13:04+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:13:05+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- logrotate (0.1.0)
- gitlab (0.0.1)
- package (0.1.0)
- redis (0.1.0)
- postgresql (0.1.0)
- monitoring (0.1.0)
- registry (0.1.0)
- mattermost (0.1.0)
- gitaly (0.1.0)
- praefect (0.1.0)
- gitlab-kas (0.1.0)
- gitlab-pages (0.1.0)
- letsencrypt (0.1.0)
- acme (4.1.6)
- crond (0.1.0)
- nginx (0.1.0)
- runit (5.1.7)
- consul (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:13:06+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:13:06+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 03 seconds
[2023-12-08T14:13:06+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:13:06+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:13:06+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:13:06+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:13:06+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:13:24+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:13:24+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:13:24+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:13:24+00:00] INFO: Cinc-client pid: 29
[2023-12-08T14:13:26+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:13:26+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:13:26+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:13:26+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:13:26+00:00] INFO: Running start handlers
[2023-12-08T14:13:26+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:13:27+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- gitlab (0.0.1)
- package (0.1.0)
- logrotate (0.1.0)
- redis (0.1.0)
- postgresql (0.1.0)
- registry (0.1.0)
- monitoring (0.1.0)
- mattermost (0.1.0)
- gitlab-pages (0.1.0)
- nginx (0.1.0)
- letsencrypt (0.1.0)
- runit (5.1.7)
- gitaly (0.1.0)
- crond (0.1.0)
- praefect (0.1.0)
- gitlab-kas (0.1.0)
- acme (4.1.6)
- consul (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:13:28+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:13:28+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 03 seconds
[2023-12-08T14:13:28+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:13:28+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:13:28+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:13:28+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:13:28+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:13:48+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:13:49+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:13:49+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:13:49+00:00] INFO: Cinc-client pid: 32
[2023-12-08T14:13:50+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:13:50+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:13:50+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:13:50+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:13:50+00:00] INFO: Running start handlers
[2023-12-08T14:13:50+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:13:52+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- gitlab (0.0.1)
- package (0.1.0)
- logrotate (0.1.0)
- postgresql (0.1.0)
- redis (0.1.0)
- consul (0.1.0)
- monitoring (0.1.0)
- gitaly (0.1.0)
- registry (0.1.0)
- letsencrypt (0.1.0)
- nginx (0.1.0)
- runit (5.1.7)
- praefect (0.1.0)
- mattermost (0.1.0)
- gitlab-kas (0.1.0)
- gitlab-pages (0.1.0)
- acme (4.1.6)
- crond (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:13:53+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:13:53+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 04 seconds
[2023-12-08T14:13:53+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:13:53+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:13:53+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:13:53+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:13:53+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:14:17+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:14:19+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:14:19+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:14:19+00:00] INFO: Cinc-client pid: 29
[2023-12-08T14:14:20+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:14:20+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:14:20+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:14:20+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:14:20+00:00] INFO: Running start handlers
[2023-12-08T14:14:20+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:14:20+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- gitlab (0.0.1)
- package (0.1.0)
- redis (0.1.0)
- logrotate (0.1.0)
- postgresql (0.1.0)
- registry (0.1.0)
- mattermost (0.1.0)
- consul (0.1.0)
- gitaly (0.1.0)
- praefect (0.1.0)
- gitlab-kas (0.1.0)
- gitlab-pages (0.1.0)
- letsencrypt (0.1.0)
- nginx (0.1.0)
- crond (0.1.0)
- runit (5.1.7)
- monitoring (0.1.0)
- acme (4.1.6)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:14:21+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:14:21+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 04 seconds
[2023-12-08T14:14:21+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:14:21+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:14:21+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:14:21+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:14:21+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:14:42+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:14:42+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:14:42+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:14:42+00:00] INFO: Cinc-client pid: 28
[2023-12-08T14:14:44+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:14:44+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:14:44+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:14:44+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:14:44+00:00] INFO: Running start handlers
[2023-12-08T14:14:44+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:14:44+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- package (0.1.0)
- redis (0.1.0)
- logrotate (0.1.0)
- mattermost (0.1.0)
- gitlab (0.0.1)
- postgresql (0.1.0)
- registry (0.1.0)
- consul (0.1.0)
- gitaly (0.1.0)
- monitoring (0.1.0)
- praefect (0.1.0)
- nginx (0.1.0)
- gitlab-kas (0.1.0)
- crond (0.1.0)
- runit (5.1.7)
- letsencrypt (0.1.0)
- gitlab-pages (0.1.0)
- acme (4.1.6)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:14:45+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:14:45+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 03 seconds
[2023-12-08T14:14:45+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:14:45+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:14:45+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:14:45+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:14:45+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:15:05+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:15:07+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:15:07+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:15:07+00:00] INFO: Cinc-client pid: 29
[2023-12-08T14:15:08+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:15:08+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:15:08+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:15:08+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:15:08+00:00] INFO: Running start handlers
[2023-12-08T14:15:08+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:15:09+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- gitlab (0.0.1)
- package (0.1.0)
- logrotate (0.1.0)
- postgresql (0.1.0)
- redis (0.1.0)
- monitoring (0.1.0)
- registry (0.1.0)
- mattermost (0.1.0)
- consul (0.1.0)
- gitaly (0.1.0)
- praefect (0.1.0)
- gitlab-kas (0.1.0)
- gitlab-pages (0.1.0)
- letsencrypt (0.1.0)
- nginx (0.1.0)
- runit (5.1.7)
- acme (4.1.6)
- crond (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:15:10+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:15:10+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 04 seconds
[2023-12-08T14:15:10+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:15:10+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:15:10+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:15:10+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:15:10+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:15:32+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:15:33+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:15:33+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:15:33+00:00] INFO: Cinc-client pid: 29
[2023-12-08T14:15:34+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:15:34+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:15:34+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:15:34+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:15:34+00:00] INFO: Running start handlers
[2023-12-08T14:15:34+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:15:35+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- logrotate (0.1.0)
- postgresql (0.1.0)
- redis (0.1.0)
- monitoring (0.1.0)
- registry (0.1.0)
- mattermost (0.1.0)
- consul (0.1.0)
- gitaly (0.1.0)
- gitlab (0.0.1)
- package (0.1.0)
- runit (5.1.7)
- gitlab-kas (0.1.0)
- gitlab-pages (0.1.0)
- nginx (0.1.0)
- letsencrypt (0.1.0)
- praefect (0.1.0)
- acme (4.1.6)
- crond (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:15:36+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:15:36+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 04 seconds
[2023-12-08T14:15:36+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:15:36+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:15:36+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:15:36+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:15:36+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:15:55+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:15:56+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:15:56+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:15:56+00:00] INFO: Cinc-client pid: 30
[2023-12-08T14:15:57+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:15:57+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:15:57+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:15:57+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:15:57+00:00] INFO: Running start handlers
[2023-12-08T14:15:57+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:15:58+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- logrotate (0.1.0)
- postgresql (0.1.0)
- redis (0.1.0)
- package (0.1.0)
- registry (0.1.0)
- mattermost (0.1.0)
- gitaly (0.1.0)
- gitlab (0.0.1)
- monitoring (0.1.0)
- gitlab-pages (0.1.0)
- letsencrypt (0.1.0)
- nginx (0.1.0)
- runit (5.1.7)
- crond (0.1.0)
- consul (0.1.0)
- praefect (0.1.0)
- acme (4.1.6)
- gitlab-kas (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:15:59+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:15:59+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 03 seconds
[2023-12-08T14:15:59+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:15:59+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:15:59+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:15:59+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:15:59+00:00] FATAL: NoMatchingPatternError: server
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=16.0.4-ce.0
Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:
docker exec -it gitlab editor /etc/gitlab/gitlab.rb
docker restart gitlab
For a comprehensive list of configuration options please see the Omnibus GitLab readme
<https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md>
If this container fails to start due to permission problems try to fix it by executing:
docker exec -it gitlab update-permissions
docker restart gitlab
Cleaning stale PIDs & sockets
Preparing services...
Starting services...
Configuring GitLab...
[2023-12-08T14:16:23+00:00] INFO: Started Cinc Zero at chefzero://localhost:1 with repository at /opt/gitlab/embedded (One version per cookbook)
Cinc Client, version 17.10.0
Patents: <https://www.chef.io/patents>
Infra Phase starting
[2023-12-08T14:16:24+00:00] INFO: ***Cinc Client 17.10.0***
[2023-12-08T14:16:24+00:00] INFO: Platform: x86_64-linux
[2023-12-08T14:16:24+00:00] INFO: Cinc-client pid: 29
[2023-12-08T14:16:25+00:00] INFO: Setting the run_list to ["recipe[gitlab]"] from CLI options
[2023-12-08T14:16:25+00:00] INFO: Run List is [recipe[gitlab]]
[2023-12-08T14:16:25+00:00] INFO: Run List expands to [gitlab]
[2023-12-08T14:16:25+00:00] INFO: Starting Cinc Client Run for 814febbbd47d
[2023-12-08T14:16:25+00:00] INFO: Running start handlers
[2023-12-08T14:16:25+00:00] INFO: Start handlers complete.
Resolving cookbooks for run list: ["gitlab"]
[2023-12-08T14:16:26+00:00] INFO: Loading cookbooks [gitlab@0.0.1, package@0.1.0, logrotate@0.1.0, postgresql@0.1.0, redis@0.1.0, monitoring@0.1.0, registry@0.1.0, mattermost@0.1.0, consul@0.1.0, gitaly@0.1.0, praefect@0.1.0, gitlab-kas@0.1.0, gitlab-pages@0.1.0, letsencrypt@0.1.0, nginx@0.1.0, runit@5.1.7, acme@4.1.6, crond@0.1.0]
Synchronizing cookbooks:
- postgresql (0.1.0)
- redis (0.1.0)
- package (0.1.0)
- gitlab (0.0.1)
- registry (0.1.0)
- logrotate (0.1.0)
- praefect (0.1.0)
- gitlab-pages (0.1.0)
- monitoring (0.1.0)
- consul (0.1.0)
- nginx (0.1.0)
- runit (5.1.7)
- acme (4.1.6)
- crond (0.1.0)
- mattermost (0.1.0)
- gitaly (0.1.0)
- letsencrypt (0.1.0)
- gitlab-kas (0.1.0)
Installing cookbook gem dependencies:
Compiling cookbooks...
Loading Cinc Auditor profile files:
Loading Cinc Auditor input files:
Loading Cinc Auditor waiver files:
================================================================================
Recipe Compile Error in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
================================================================================
NoMatchingPatternError
----------------------
server
Cookbook Trace: (most recent call first)
----------------------------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in`auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`block in from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/config_mash.rb:35:in `auto_vivify'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:142:in`from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/config.rb:22:in `from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:27:in`from_file'
Relevant File Content
----------------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/settings_dsl.rb:
135:
136: def from_file(_file_path)
137: # Throw errors for unrecognized top level calls (usually spelling mistakes)
138: config_strict_mode true
139: # Turn on node deprecation messages
140: Gitlab::Deprecations::NodeAttribute.log_deprecations = true
141: # Allow auto mash creation during from_file call
142>> Gitlab::ConfigMash.auto_vivify { super }
143: ensure
144: config_strict_mode false
145: Gitlab::Deprecations::NodeAttribute.log_deprecations = false
146: end
147:
148: # Enhance set so strict mode errors aren't thrown as long as the setting is witin our defined config
149: def internal_set(symbol, value)
150: if configuration.key?(symbol)
151: configuration[symbol] = value
System Info
------------
chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Running handlers:
[2023-12-08T14:16:27+00:00] ERROR: Running exception handlers
Running handlers complete
[2023-12-08T14:16:27+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 03 seconds
[2023-12-08T14:16:27+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-12-08T14:16:27+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:16:27+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-12-08T14:16:27+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-12-08T14:16:27+00:00] FATAL: NoMatchingPatternError: server
```
</details>
### Details of package version
<details>
<summary>Provide the package version installation details</summary>
<pre>
<!--
For Debian, Ubuntu based systems, paste the output of:
`dpkg-query -l "gitlab-*"`
For RHEL based systems (CentOS, RHEL, OL, Scientific, OpenSUSE, SLES), paste the output of:
`rpm -qa | grep 'gitlab'`
If you can't provide these details, see https://about.gitlab.com/getting-help/ page on where to ask your question.
-->
Current version: gitlab-ce=16.0.4-ce.0
</pre>
</details>
### Environment details
- Operating System: `Ubuntu`
- Installation Target, remove incorrect values:
- Bare Metal Machine
- VM: Digital Ocean, AWS, GCP, Azure, Other `conti`
- Other: `REPLACE-WITH-DETAILS`
- Installation Type, remove incorrect values:
- New Installation
- Upgrade from version `16.5.1-ce.0`
- Other: `REPLACE-WITH-DETAILS`
- Is there any other software running on the machine: `Docker`
- Is this a single or multiple node installation?
- Resources
- CPU: `Intel Core i7-8700K`
- Memory total: `64GB`
- chef_version=17.10.0
- platform=ubuntu
- platform_version=22.04
- ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
- program_name=/opt/gitlab/embedded/bin/cinc-client
- executable=/opt/gitlab/embedded/bin/cinc-client
### Configuration details
<details>
<summary> Provide the relevant sections of `/etc/gitlab/gitlab.rb` </summary>
<pre>
<!--
Paste the details but omit lines starting with `#`. Relevant lines can be retrieved by
running `grep -v -e '^#' -e '^$' /etc/gitlab/gitlab.rb`. Be certain to sanitize sensitive
configuration such as passwords and secrets.
-->
external_url '<https://gitlab.example.com>'
gitlab_rails['gitlab_email_enabled'] = true
gitlab_rails['gitlab_email_from'] = '<gitlab@example.com>'
gitlab_rails['gitlab_email_display_name'] = 'GitLab'
gitlab_rails['gitlab_email_reply_to'] = '<noreply@example.com>'
gitlab_rails['gitlab_email_subject_suffix'] = ''
gitlab_rails['gitlab_email_smime_enabled'] = false
gitlab_rails['gitlab_email_smime_key_file'] = '/etc/gitlab/ssl/gitlab_smime.key'
gitlab_rails['gitlab_email_smime_cert_file'] = '/etc/gitlab/ssl/gitlab_smime.crt'
gitlab_rails['incoming_email_enabled'] = true
gitlab_rails['incoming_email_address'] = "gitlab+gitlab-incoming+%{key}@gmail.com"
gitlab_rails['incoming_email_email'] = "<gitlab@gmail.com>"
gitlab_rails['incoming_email_password'] = "xxxxxxxx"
gitlab_rails['incoming_email_host'] = "imap.gmail.com"
gitlab_rails['incoming_email_port'] = 993
gitlab_rails['incoming_email_ssl'] = true
gitlab_rails['incoming_email_start_tls'] = false
gitlab_rails['incoming_email_mailbox_name'] = "inbox"
gitlab_rails['incoming_email_idle_timeout'] = 60
gitlab_rails['incoming_email_delete_after_delivery'] = true
gitlab_rails['incoming_email_expunge_deleted'] = true
gitlab_rails['initial_root_password'] = "xxxxxxxx"
gitlab_rails['initial_shared_runners_registration_token'] = "xxxxxxxx"
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.gmail.com"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "<gitlab@gmail.com>"
gitlab_rails['smtp_password'] = "xxxxxxxx"
gitlab_rails['smtp_domain'] = "smtp.gmail.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['gitlab_email_from'] = '<gitlab@example.com>'
gitlab_rails['gitlab_email_reply_to'] = '<noreply@example.com>'
gitlab_rails['smtp_openssl_verify_mode'] = 'peer'
nginx['redirect_http_to_https'] = true
nginx['redirect_http_to_https_port'] = 80
registry_nginx['redirect_http_to_https'] = true
mattermost_nginx['redirect_http_to_https'] = true
alertmanager['admin_email'] = '<admin@example.com>'
letsencrypt['enable'] = true
letsencrypt['contact_emails'] = ['gitlab@gmail.com'] # This should be an array of email addresses to add as contacts
letsencrypt['group'] = 'root'
letsencrypt['key_size'] = 2048
letsencrypt['owner'] = 'root'
letsencrypt['wwwroot'] = '/var/opt/gitlab/nginx/www'
letsencrypt['auto_renew'] = true
letsencrypt['auto_renew_hour'] = "12"
letsencrypt['auto_renew_minute'] = "30" # Should be a number or cron expression, if specified.
letsencrypt['auto_renew_day_of_month'] = "*/7"
package['detect_init'] = true
package['systemd_tasks_max'] = 4915
package['systemd_after'] = 'multi-user.target'
package['systemd_wanted_by'] = 'multi-user.target'
</pre>
</details>https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8296�Setting user for Redis 6 >=2023-11-06T03:38:34ZChiwoo�Setting user for Redis 6 >=<!-- This template is a great use for issues that are feature::additions or technical tasks for larger issues.-->
### Proposal
I want to use gitlab using an omnibus installation, but an internal security policy prevents me from using t...<!-- This template is a great use for issues that are feature::additions or technical tasks for larger issues.-->
### Proposal
I want to use gitlab using an omnibus installation, but an internal security policy prevents me from using the "default" account.
Therefore, �I propose to modify the gitlab.rb file to allow you to use a different account.https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8196GitLab with Docker-compose on Mac OS fails to work2023-09-08T19:55:09ZArsenii GorkinGitLab with Docker-compose on Mac OS fails to workHi!
I am trying 4th day in order to get GitLab working in the docker under Mac OS X Ventura.
Here is my docker-compose file:
```
version: '3.7'
services:
db:
image: 'postgres:latest'
restart: always
container_name: postg...Hi!
I am trying 4th day in order to get GitLab working in the docker under Mac OS X Ventura.
Here is my docker-compose file:
```
version: '3.7'
services:
db:
image: 'postgres:latest'
restart: always
container_name: postgres
environment:
POSTGRES_USER: gitlab
POSTGRES_PASSWORD: gitlab
POSTGRES_DB: gitlabhq_production
networks:
- gitlab-net
# ports:
# - '5432:5432'
volumes:
- '$GITLAB_HOME/db_data:/var/lib/postgresql/data'
web:
env_file:
- .env
image: 'gitlab/gitlab-ce:latest'
restart: always
hostname: localhost
container_name: gitlab
environment:
GITLAB_OMNIBUS_CONFIG: |
postgresql['enable'] = false;
gitlab_rails['db_adapter'] = 'postgresql';
gitlab_rails['db_encoding'] = 'utf8';
gitlab_rails['db_username'] = 'gitlab';
gitlab_rails['db_password'] = 'gitlab';
gitlab_rails['db_host'] = 'db';
gitlab_rails['db_port'] = '5432';
gitlab_rails['db_database'] = 'gitlabhq_production';
gitlab_rails['redis_host'] = 'redis';
gitlab_rails['redis_port'] = 6379;
ports:
- '18080:80'
- '18443:443'
- '18022:22'
- '18021:21'
networks:
- gitlab-net
depends_on:
- db
- redis
volumes:
- '$GITLAB_HOME/config:/etc/gitlab'
- '$GITLAB_HOME/logs:/var/log/gitlab'
- '$GITLAB_HOME/data:/var/opt/gitlab'
redis:
image: redis:latest
container_name: redis
networks:
- gitlab-net
# ports:
# - '6379:6379'
networks:
gitlab-net:
driver: bridge
```
If I try without redis and postgres containers I cannot get it work at all.
With this configuration I was able to make it work UNTIL I have turned off the containers for a night and re-started next day... Then.. 502 error chases me.
I have errors dedicated to impossibility of listening of sockets for postgesql, redis, gitlab.
If I completely clean the binding folders in the host machine and try to reconfigure - everything will work until the next restart.
**Very bad workaround :smile: :**
I could make it work ONLY if EVERY TIME before start the container I remove all the dirs in the /data bound directory, excepting next:
/data/.ssh:/var/opt/gitlab/.ssh (to have ssh access for users)
/data/git-data:/var/opt/gitlab/git-data (for repos)
/data/gitlab-rails/uploads (for avatars and some other uploads)
Any help is highly appreciated!https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8066Unresponsive website and high CPU usage of Ruby - Migrated to 16.1.22024-03-12T00:20:42ZJonas SchoenwaldUnresponsive website and high CPU usage of Ruby - Migrated to 16.1.2### Summary
After a successful migration from 14.9.0-ee to 16.1.2-ee gitlab becomes unresponsive after a while (seems to take between 1 and 3 days) only returning a 502 or nothing at all (timeout).
The issue seems to be related with se...### Summary
After a successful migration from 14.9.0-ee to 16.1.2-ee gitlab becomes unresponsive after a while (seems to take between 1 and 3 days) only returning a 502 or nothing at all (timeout).
The issue seems to be related with several sockets throwing gateway exceptions. These exceptions cause the ruby processes to spike at 100% cpu utilization.
There is no clear indication on why exactly that happens.
Restarting the container only helps temporarily, until it deadlocks itself again.
(this is my first issue here, so sorry if I'm missing some information)
### Steps to reproduce
Don't really know what steps could be taken to reproduce my issue, but I'll gladly help to be able to reproduce the issue.
<!-- How one can reproduce the issue - this is very important -->
### What is the current *bug* behavior?
Gitlab starts up just fine. After a while it becomes no longer accessible and the container uses all available CPU ressources
### What is the expected *correct* behavior?
Not that.
### Relevant logs
<details>
<summary> Relevant logs </summary>
<pre>
==> /var/log/gitlab/gitlab-rails/production.log <==
lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'
lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
lib/gitlab/middleware/request_context.rb:15:in `call'
lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call'
config/initializers/fix_local_cache_middleware.rb:11:in `call'
lib/gitlab/middleware/compressed_json.rb:44:in `call'
lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'
lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:79:in `call'
lib/gitlab/middleware/release_env.rb:13:in `call'
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-07-11T21:58:47.015Z","params":[],"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047628,"mem_objects":2130,"mem_bytes":3319952,"mem_mallocs":11380,"mem_total_bytes":3405152,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"7568d3ee-4247-4e05-af8a-69a044633b7f","db_duration_s":0.0,"view_duration_s":0.00046,"duration_s":0.04406}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-07-11T21:59:02.020Z","params":[],"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052304,"mem_objects":2130,"mem_bytes":3319952,"mem_mallocs":11380,"mem_total_bytes":3405152,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"4595f4ad-44bc-4b7c-9954-aac7375607aa","db_duration_s":0.0,"view_duration_s":0.00047,"duration_s":0.04848}
{"method":"GET","path":"/help","format":"*/*","controller":"HelpController","action":"index","status":200,"time":"2023-07-11T21:59:11.656Z","params":[],"correlation_id":"01H53FEBCYRTY6R6NP16YZCF0V","meta.caller_id":"HelpController#index","meta.remote_ip":"127.0.0.1","meta.feature_category":"not_owned","meta.client_id":"ip/127.0.0.1","remote_ip":"127.0.0.1","ua":"curl/8.0.1-DEV","queue_duration_s":0.022315,"response_bytes":71557,"request_urgency":"default","target_duration_s":1,"redis_calls":18,"redis_duration_s":0.0039759999999999995,"redis_read_bytes":2445,"redis_write_bytes":2815,"redis_cache_calls":8,"redis_cache_duration_s":0.001651,"redis_cache_read_bytes":498,"redis_cache_write_bytes":2180,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002325,"redis_feature_flag_read_bytes":1947,"redis_feature_flag_write_bytes":635,"db_count":10,"db_write_count":0,"db_cached_count":5,"db_replica_count":0,"db_primary_count":10,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":5,"db_main_cached_count":5,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.113579,"mem_objects":105084,"mem_bytes":18238050,"mem_mallocs":78419,"mem_total_bytes":22441410,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"db_duration_s":0.00265,"view_duration_s":0.10528,"duration_s":0.11055}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-07-11T21:59:17.022Z","params":[],"redis_calls":1,"redis_duration_s":0.000235,"redis_read_bytes":203,"redis_write_bytes":52,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000235,"redis_feature_flag_read_bytes":203,"redis_feature_flag_write_bytes":52,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054755,"mem_objects":2355,"mem_bytes":3370608,"mem_mallocs":11460,"mem_total_bytes":3464808,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"27c2775d-1e81-44c7-b77e-16d299254466","db_duration_s":0.0,"view_duration_s":0.00041,"duration_s":0.05113}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-07-11T21:59:32.016Z","params":[],"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049082,"mem_objects":2130,"mem_bytes":3319952,"mem_mallocs":11380,"mem_total_bytes":3405152,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"e7183ac6-3418-4a8a-8558-5b9d4f4618b9","db_duration_s":0.0,"view_duration_s":0.00046,"duration_s":0.04524}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-07-11T21:59:47.011Z","params":[],"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045468,"mem_objects":2130,"mem_bytes":3319928,"mem_mallocs":11380,"mem_total_bytes":3405128,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"9ca46d13-e66a-4836-9032-6720d994833b","db_duration_s":0.0,"view_duration_s":0.00047,"duration_s":0.04216}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-07-11T22:00:02.015Z","params":[],"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048046,"mem_objects":2130,"mem_bytes":3319928,"mem_mallocs":11380,"mem_total_bytes":3405128,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"412bc972-687b-4468-b4a6-5e562f68d8da","db_duration_s":0.0,"view_duration_s":0.00043,"duration_s":0.04408}
{"method":"GET","path":"/help","format":"*/*","controller":"HelpController","action":"index","status":200,"time":"2023-07-11T22:00:12.086Z","params":[],"correlation_id":"01H53FG6D1JGG5SR64ZDKDFBXJ","meta.caller_id":"HelpController#index","meta.remote_ip":"127.0.0.1","meta.feature_category":"not_owned","meta.client_id":"ip/127.0.0.1","remote_ip":"127.0.0.1","ua":"curl/8.0.1-DEV","queue_duration_s":0.021132,"response_bytes":71557,"request_urgency":"default","target_duration_s":1,"redis_calls":19,"redis_duration_s":0.0042450000000000005,"redis_read_bytes":2329,"redis_write_bytes":2991,"redis_cache_calls":9,"redis_cache_duration_s":0.001778,"redis_cache_read_bytes":382,"redis_cache_write_bytes":2356,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002467,"redis_feature_flag_read_bytes":1947,"redis_feature_flag_write_bytes":635,"db_count":11,"db_write_count":0,"db_cached_count":5,"db_replica_count":0,"db_primary_count":11,"db_main_count":11,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":5,"db_main_cached_count":5,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116962,"mem_objects":105585,"mem_bytes":18304090,"mem_mallocs":78456,"mem_total_bytes":22527490,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"db_duration_s":0.00474,"view_duration_s":0.11407,"duration_s":0.12166}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-07-11T22:00:17.013Z","params":[],"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047021,"mem_objects":2131,"mem_bytes":3319928,"mem_mallocs":11380,"mem_total_bytes":3405168,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"8194d6b8-3c54-4853-bb6d-fa41084a4b98","db_duration_s":0.0,"view_duration_s":0.0004,"duration_s":0.04336}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-07-11T22:00:32.013Z","params":[],"redis_calls":1,"redis_duration_s":0.000208,"redis_read_bytes":203,"redis_write_bytes":52,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000208,"redis_feature_flag_read_bytes":203,"redis_feature_flag_write_bytes":52,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046897,"mem_objects":2355,"mem_bytes":3370584,"mem_mallocs":11460,"mem_total_bytes":3464784,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"63a6b20b-aa8c-4452-8756-385417ce2a67","db_duration_s":0.0,"view_duration_s":0.00039,"duration_s":0.04353}
==> /var/log/gitlab/gitlab-rails/application_json.log <==
{"severity":"INFO","time":"2023-07-11T19:00:05.807Z","meta.caller_id":"StuckCiJobsWorker","correlation_id":"61f3a94b177bd29c22193db03e8b7843","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","message":"Ci::StuckBuilds::DropPendingService: Cleaning pending timed-out builds"}
{"severity":"INFO","time":"2023-07-11T19:20:06.996Z","meta.caller_id":"Ci::StuckBuilds::DropRunningWorker","correlation_id":"d6fa38fbb13293eb5393e3b672f5088c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","message":"Ci::StuckBuilds::DropRunningService: Cleaning running, timed-out builds"}
{"severity":"INFO","time":"2023-07-11T19:40:06.431Z","meta.caller_id":"Ci::StuckBuilds::DropScheduledWorker","correlation_id":"2fe95c5cbff248668ea8afce94f8ea1c","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","message":"Ci::StuckBuilds::DropScheduledService: Cleaning scheduled, timed-out builds"}
{"severity":"INFO","time":"2023-07-11T20:00:04.946Z","meta.caller_id":"StuckCiJobsWorker","correlation_id":"02dfe3adede5a2f50a10f5a9ad75ee7b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","message":"Ci::StuckBuilds::DropPendingService: Cleaning pending timed-out builds"}
{"severity":"INFO","time":"2023-07-11T20:20:10.716Z","meta.caller_id":"Ci::StuckBuilds::DropRunningWorker","correlation_id":"7615fa87bfee4146175d8c4cd6a66be5","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","message":"Ci::StuckBuilds::DropRunningService: Cleaning running, timed-out builds"}
{"severity":"INFO","time":"2023-07-11T20:40:05.371Z","meta.caller_id":"Ci::StuckBuilds::DropScheduledWorker","correlation_id":"ad57a5f873255c8c2d05a34f9f605852","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","message":"Ci::StuckBuilds::DropScheduledService: Cleaning scheduled, timed-out builds"}
{"severity":"INFO","time":"2023-07-11T21:00:05.687Z","meta.caller_id":"StuckCiJobsWorker","correlation_id":"eca35e391a60d73f03a2ac3cc408674b","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","message":"Ci::StuckBuilds::DropPendingService: Cleaning pending timed-out builds"}
{"severity":"INFO","time":"2023-07-11T21:20:10.056Z","meta.caller_id":"Ci::StuckBuilds::DropRunningWorker","correlation_id":"f45fcf518bf06f9a6c4be6fd072d77ba","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","message":"Ci::StuckBuilds::DropRunningService: Cleaning running, timed-out builds"}
{"severity":"INFO","time":"2023-07-11T21:40:08.180Z","meta.caller_id":"Ci::StuckBuilds::DropScheduledWorker","correlation_id":"77699ba0c7108bcd0049b85f645f3607","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","message":"Ci::StuckBuilds::DropScheduledService: Cleaning scheduled, timed-out builds"}
{"severity":"INFO","time":"2023-07-11T22:00:03.754Z","meta.caller_id":"StuckCiJobsWorker","correlation_id":"09aa2dd9805a86065d61f2bc9be3cc50","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","message":"Ci::StuckBuilds::DropPendingService: Cleaning pending timed-out builds"}
==> /var/log/gitlab/gitlab-rails/web_exporter.log <==
==> /var/log/gitlab/gitlab-rails/llm.log <==
# Logfile created on 2023-06-29 14:53:20 +0000 by logger.rb/v1.4.3
==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2023-06-29-11-08-45.log <==
main: == 20230418164957 QueueUpdateCodeSuggestionsForNamespaceSettings: migrating ===
main: == 20230418164957 QueueUpdateCodeSuggestionsForNamespaceSettings: migrated (0.0256s)
main: == 20230419164438 ChangeCodeSuggestionsDefaultFalseInNamespaceSettings: migrating
main: -- change_column_default(:namespace_settings, :code_suggestions, {:from=>true, :to=>false})
main: -> 0.0071s
main: == 20230419164438 ChangeCodeSuggestionsDefaultFalseInNamespaceSettings: migrated (0.0322s)
main: == [advisory_lock_connection] object_id: 227240, pg_backend_pid: 483
==> /var/log/gitlab/gitlab-rails/migrations.log <==
{"severity":"INFO","time":"2023-06-29T12:28:58.164Z","correlation_id":"f637db546db2e35c750ff24beb7a7c35","method":"with_lock_retries","class":"CleanupBigintConversionForVulnerabilityUserMentionsForGitlabCom","message":"Lock timeout is set","current_iteration":1,"lock_timeout_in_ms":100}
{"severity":"INFO","time":"2023-06-29T12:28:58.183Z","correlation_id":"f637db546db2e35c750ff24beb7a7c35","method":"with_lock_retries","class":"CleanupBigintConversionForVulnerabilityUserMentionsForGitlabCom","message":"Migration finished","current_iteration":1,"lock_timeout_in_ms":100}
{"severity":"INFO","time":"2023-06-29T12:28:58.215Z","correlation_id":"f637db546db2e35c750ff24beb7a7c35","method":"with_lock_retries","class":"CleanupBigintConversionForCommitUserMentionsForGitlabCom","message":"Lock timeout is set","current_iteration":1,"lock_timeout_in_ms":100}
{"severity":"INFO","time":"2023-06-29T12:28:58.233Z","correlation_id":"f637db546db2e35c750ff24beb7a7c35","method":"with_lock_retries","class":"CleanupBigintConversionForCommitUserMentionsForGitlabCom","message":"Migration finished","current_iteration":1,"lock_timeout_in_ms":100}
{"severity":"INFO","time":"2023-06-29T12:28:58.265Z","correlation_id":"f637db546db2e35c750ff24beb7a7c35","method":"with_lock_retries","class":"CleanupBigintConversionForMergeRequestUserMentionsForGitlabCom","message":"Lock timeout is set","current_iteration":1,"lock_timeout_in_ms":100}
{"severity":"INFO","time":"2023-06-29T12:28:58.284Z","correlation_id":"f637db546db2e35c750ff24beb7a7c35","method":"with_lock_retries","class":"CleanupBigintConversionForMergeRequestUserMentionsForGitlabCom","message":"Migration finished","current_iteration":1,"lock_timeout_in_ms":100}
{"severity":"INFO","time":"2023-06-29T12:28:58.315Z","correlation_id":"f637db546db2e35c750ff24beb7a7c35","method":"with_lock_retries","class":"CleanupBigintConversionForTimelogsForGitlabCom","message":"Lock timeout is set","current_iteration":1,"lock_timeout_in_ms":100}
{"severity":"INFO","time":"2023-06-29T12:28:58.331Z","correlation_id":"f637db546db2e35c750ff24beb7a7c35","method":"with_lock_retries","class":"CleanupBigintConversionForTimelogsForGitlabCom","message":"Migration finished","current_iteration":1,"lock_timeout_in_ms":100}
{"severity":"INFO","time":"2023-06-29T12:28:58.355Z","correlation_id":"f637db546db2e35c750ff24beb7a7c35","method":"with_lock_retries","class":"CleanupBigintConversionForAwardEmojiForGitlabCom","message":"Lock timeout is set","current_iteration":1,"lock_timeout_in_ms":100}
{"severity":"INFO","time":"2023-06-29T12:28:58.372Z","correlation_id":"f637db546db2e35c750ff24beb7a7c35","method":"with_lock_retries","class":"CleanupBigintConversionForAwardEmojiForGitlabCom","message":"Migration finished","current_iteration":1,"lock_timeout_in_ms":100}
==> /var/log/gitlab/gitlab-rails/grpc.log <==
# Logfile created on 2022-03-22 22:06:30 +0000 by logger.rb/v1.4.2
W, [2022-07-16T08:33:41.029643 #11052] WARN -- GRPC: In each_remote_read_then_finish: call#run_batch failed somehow
W, [2022-07-16T08:33:53.054659 #10993] WARN -- GRPC: In each_remote_read_then_finish: call#run_batch failed somehow
==> /var/log/gitlab/gitlab-rails/backup_json.log <==
{"severity":"INFO","time":"2023-06-29T08:53:38.672Z","correlation_id":null,"message":"Dumping pages ... "}
{"severity":"INFO","time":"2023-06-29T08:53:38.708Z","correlation_id":null,"message":"done"}
{"severity":"INFO","time":"2023-06-29T08:53:38.708Z","correlation_id":null,"message":"Dumping lfs objects ... "}
{"severity":"INFO","time":"2023-06-29T08:53:57.897Z","correlation_id":null,"message":"done"}
{"severity":"INFO","time":"2023-06-29T08:53:57.897Z","correlation_id":null,"message":"Dumping terraform states ... "}
{"severity":"INFO","time":"2023-06-29T08:53:57.906Z","correlation_id":null,"message":"done"}
{"severity":"INFO","time":"2023-06-29T08:53:57.907Z","correlation_id":null,"message":"Dumping container registry images ... "}
{"severity":"INFO","time":"2023-06-29T08:53:57.907Z","correlation_id":null,"message":"[DISABLED]"}
{"severity":"INFO","time":"2023-06-29T08:53:57.907Z","correlation_id":null,"message":"Dumping packages ... "}
{"severity":"INFO","time":"2023-06-29T08:53:57.916Z","correlation_id":null,"message":"done"}
==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2023-06-29-12-26-50.log <==
main: == 20230615202125 PrepareIndexIssuesOnProjectHealthStatusAscWorkItemType: migrated (0.0790s)
main: == 20230615202511 PrepareIndexIssuesOnProjectHealthStatusDescWorkItemType: migrating
main: -- index_exists?(:issues, [:project_id, :health_status, :id, :state_id, :work_item_type_id], {:order=>{:health_status=>"DESC NULLS LAST", :id=>:desc}, :name=>"index_issues_on_project_health_status_desc_work_item_type", :algorithm=>:concurrently})
main: -> 0.0197s
main: -- add_index_options(:issues, [:project_id, :health_status, :id, :state_id, :work_item_type_id], {:order=>{:health_status=>"DESC NULLS LAST", :id=>:desc}, :name=>"index_issues_on_project_health_status_desc_work_item_type", :algorithm=>:concurrently})
main: -> 0.0000s
main: == 20230615202511 PrepareIndexIssuesOnProjectHealthStatusDescWorkItemType: migrated (0.0517s)
main: == [advisory_lock_connection] object_id: 173640, pg_backend_pid: 498
==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2023-07-05-21-57-39.log <==
Running db:migrate rake task
main: == [advisory_lock_connection] object_id: 173640, pg_backend_pid: 504
main: == [advisory_lock_connection] object_id: 173640, pg_backend_pid: 504
==> /var/log/gitlab/gitlab-rails/application.log <==
2023-06-29T10:29:48.507Z: ActiveRecord connection established
2023-06-29T10:35:08.516Z: ActiveRecord connection established
2023-06-29T10:39:08.527Z: ActiveRecord connection established
2023-06-29T10:40:20.848Z: Ci::StuckBuilds::DropScheduledService: Cleaning scheduled, timed-out builds
2023-06-29T10:43:08.544Z: ActiveRecord connection established
2023-06-29T10:48:48.557Z: ActiveRecord connection established
2023-06-29T10:55:08.563Z: ActiveRecord connection established
2023-06-29T10:59:08.567Z: ActiveRecord connection established
2023-06-29T11:00:13.416Z: Ci::StuckBuilds::DropPendingService: Cleaning pending timed-out builds
2023-06-29T11:04:08.632Z: ActiveRecord connection established
==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2023-06-29-12-18-27.log <==
main: -> 0.0055s
main: -- execute("SET statement_timeout TO 0")
main: -> 0.0005s
main: -- execute("ALTER TABLE organizations VALIDATE CONSTRAINT check_d130d769e0;")
main: -> 0.0357s
main: -- execute("RESET statement_timeout")
main: -> 0.0013s
main: == 20230515111314 AddTextLimitOnOrganizationName: migrated (0.1576s) ==========
main: == [advisory_lock_connection] object_id: 221740, pg_backend_pid: 504
==> /var/log/gitlab/gitlab-rails/service_measurement.log <==
# Logfile created on 2022-03-22 22:06:30 +0000 by logger.rb/v1.4.2
==> /var/log/gitlab/gitlab-rails/api_json.log <==
{"time":"2023-07-11T19:34:19.995Z","severity":"INFO","duration_s":0.01618,"db_duration_s":0.00405,"view_duration_s":0.01213,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"14.0.1","revision":"c1edb478","platform":"windows","architecture":"amd64","executor":"shell","shell":"pwsh","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"}],"host":"localhost","remote_ip":"172.18.0.1, 127.0.0.1","ua":"gitlab-runner 14.0.1 (refs/pipelines/326100216; go1.13.8; windows/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.008621,"redis_calls":5,"redis_duration_s":0.001706,"redis_read_bytes":608,"redis_write_bytes":378,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000952,"redis_feature_flag_read_bytes":608,"redis_feature_flag_write_bytes":234,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000754,"redis_shared_state_write_bytes":144,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":3,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02171,"mem_objects":6786,"mem_bytes":597616,"mem_mallocs":1804,"mem_total_bytes":869056,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01H53753HYCGP5W0RY3RK761AJ","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"172.18.0.1","meta.feature_category":"continuous_integration","meta.client_id":"ip/172.18.0.1","content_length":"641","request_urgency":"low","target_duration_s":5,"response_bytes":27}
{"time":"2023-07-11T19:35:20.018Z","severity":"INFO","duration_s":0.00757,"db_duration_s":0.00163,"view_duration_s":0.00594,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"14.0.1","revision":"c1edb478","platform":"windows","architecture":"amd64","executor":"shell","shell":"pwsh","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"}],"host":"localhost","remote_ip":"172.18.0.1, 127.0.0.1","ua":"gitlab-runner 14.0.1 (refs/pipelines/326100216; go1.13.8; windows/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.005294,"redis_calls":5,"redis_duration_s":0.0006479999999999999,"redis_read_bytes":608,"redis_write_bytes":378,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000388,"redis_feature_flag_read_bytes":608,"redis_feature_flag_write_bytes":234,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_write_bytes":144,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":3,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01233,"mem_objects":6966,"mem_bytes":577392,"mem_mallocs":1809,"mem_total_bytes":856032,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01H5376Y63K69123155723RGXV","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"172.18.0.1","meta.feature_category":"continuous_integration","meta.client_id":"ip/172.18.0.1","content_length":"641","request_urgency":"low","target_duration_s":5,"response_bytes":27}
{"time":"2023-07-11T20:32:20.955Z","severity":"INFO","duration_s":0.10506,"db_duration_s":0.07265,"view_duration_s":0.03241,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"14.0.1","revision":"c1edb478","platform":"windows","architecture":"amd64","executor":"shell","shell":"pwsh","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"}],"host":"localhost","remote_ip":"172.18.0.1, 127.0.0.1","ua":"gitlab-runner 14.0.1 (refs/pipelines/326100216; go1.13.8; windows/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.005004,"redis_calls":8,"redis_duration_s":0.0015440000000000002,"redis_read_bytes":6,"redis_write_bytes":1320,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.000989,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1104,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000555,"redis_shared_state_write_bytes":216,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":6,"db_main_count":3,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.056,"db_main_duration_s":0.001,"db_ci_duration_s":0.054,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01887,"mem_objects":10097,"mem_bytes":848632,"mem_mallocs":2440,"mem_total_bytes":1252512,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01H53AFAVBMTJSJTY8TDWEHRJR","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"172.18.0.1","meta.feature_category":"continuous_integration","meta.client_id":"ip/172.18.0.1","content_length":"641","request_urgency":"low","target_duration_s":5,"response_bytes":27}
{"time":"2023-07-11T20:33:20.985Z","severity":"INFO","duration_s":0.00878,"db_duration_s":0.00272,"view_duration_s":0.00606,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"14.0.1","revision":"c1edb478","platform":"windows","architecture":"amd64","executor":"shell","shell":"pwsh","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"}],"host":"localhost","remote_ip":"172.18.0.1, 127.0.0.1","ua":"gitlab-runner 14.0.1 (refs/pipelines/326100216; go1.13.8; windows/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.00535,"redis_calls":5,"redis_duration_s":0.000807,"redis_read_bytes":609,"redis_write_bytes":378,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000495,"redis_feature_flag_read_bytes":609,"redis_feature_flag_write_bytes":234,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000312,"redis_shared_state_write_bytes":144,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":3,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012401,"mem_objects":6848,"mem_bytes":598016,"mem_mallocs":1807,"mem_total_bytes":871936,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01H53AH5J8VG9KK21P0PH7WM1T","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"172.18.0.1","meta.feature_category":"continuous_integration","meta.client_id":"ip/172.18.0.1","content_length":"641","request_urgency":"low","target_duration_s":5,"response_bytes":27}
{"time":"2023-07-11T20:34:21.018Z","severity":"INFO","duration_s":0.00813,"db_duration_s":0.00223,"view_duration_s":0.0059,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"14.0.1","revision":"c1edb478","platform":"windows","architecture":"amd64","executor":"shell","shell":"pwsh","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"}],"host":"localhost","remote_ip":"172.18.0.1, 127.0.0.1","ua":"gitlab-runner 14.0.1 (refs/pipelines/326100216; go1.13.8; windows/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.004895,"redis_calls":5,"redis_duration_s":0.000763,"redis_read_bytes":609,"redis_write_bytes":378,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000495,"redis_feature_flag_read_bytes":609,"redis_feature_flag_write_bytes":234,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000268,"redis_shared_state_write_bytes":144,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":3,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011724,"mem_objects":6786,"mem_bytes":597616,"mem_mallocs":1804,"mem_total_bytes":869056,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01H53AK06BJRKMHYCBYY4KRB11","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"172.18.0.1","meta.feature_category":"continuous_integration","meta.client_id":"ip/172.18.0.1","content_length":"641","request_urgency":"low","target_duration_s":5,"response_bytes":27}
{"time":"2023-07-11T20:35:21.049Z","severity":"INFO","duration_s":0.00834,"db_duration_s":0.00266,"view_duration_s":0.00568,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"14.0.1","revision":"c1edb478","platform":"windows","architecture":"amd64","executor":"shell","shell":"pwsh","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"}],"host":"localhost","remote_ip":"172.18.0.1, 127.0.0.1","ua":"gitlab-runner 14.0.1 (refs/pipelines/326100216; go1.13.8; windows/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.005146,"redis_calls":5,"redis_duration_s":0.000799,"redis_read_bytes":609,"redis_write_bytes":378,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000529,"redis_feature_flag_read_bytes":609,"redis_feature_flag_write_bytes":234,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_write_bytes":144,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":3,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011881,"mem_objects":6786,"mem_bytes":597616,"mem_mallocs":1804,"mem_total_bytes":869056,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01H53AMTT9WXSHC7M1RJP80N3Y","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"172.18.0.1","meta.feature_category":"continuous_integration","meta.client_id":"ip/172.18.0.1","content_length":"641","request_urgency":"low","target_duration_s":5,"response_bytes":27}
{"time":"2023-07-11T21:32:21.768Z","severity":"INFO","duration_s":0.06062,"db_duration_s":0.04943,"view_duration_s":0.01119,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"14.0.1","revision":"c1edb478","platform":"windows","architecture":"amd64","executor":"shell","shell":"pwsh","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"}],"host":"localhost","remote_ip":"172.18.0.1, 127.0.0.1","ua":"gitlab-runner 14.0.1 (refs/pipelines/326100216; go1.13.8; windows/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.005181,"redis_calls":8,"redis_duration_s":0.0018410000000000002,"redis_read_bytes":6,"redis_write_bytes":1318,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001198,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1102,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000643,"redis_shared_state_write_bytes":216,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":6,"db_main_count":3,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_duration_s":0.002,"db_ci_duration_s":0.048,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017842,"mem_objects":8245,"mem_bytes":752632,"mem_mallocs":2014,"mem_total_bytes":1082432,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01H53DX7A3H140TBXPB1S4KHX8","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"172.18.0.1","meta.feature_category":"continuous_integration","meta.client_id":"ip/172.18.0.1","content_length":"641","request_urgency":"low","target_duration_s":5,"response_bytes":27}
{"time":"2023-07-11T21:33:21.794Z","severity":"INFO","duration_s":0.01034,"db_duration_s":0.0031,"view_duration_s":0.00724,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"14.0.1","revision":"c1edb478","platform":"windows","architecture":"amd64","executor":"shell","shell":"pwsh","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"}],"host":"localhost","remote_ip":"172.18.0.1, 127.0.0.1","ua":"gitlab-runner 14.0.1 (refs/pipelines/326100216; go1.13.8; windows/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.005159,"redis_calls":5,"redis_duration_s":0.000983,"redis_read_bytes":607,"redis_write_bytes":378,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000581,"redis_feature_flag_read_bytes":607,"redis_feature_flag_write_bytes":234,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_write_bytes":144,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":3,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013492,"mem_objects":6786,"mem_bytes":597616,"mem_mallocs":1804,"mem_total_bytes":869056,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01H53DZ1ZF4JXXT7KPERE7G4HF","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"172.18.0.1","meta.feature_category":"continuous_integration","meta.client_id":"ip/172.18.0.1","content_length":"641","request_urgency":"low","target_duration_s":5,"response_bytes":27}
{"time":"2023-07-11T21:34:21.830Z","severity":"INFO","duration_s":0.016,"db_duration_s":0.00416,"view_duration_s":0.01184,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"14.0.1","revision":"c1edb478","platform":"windows","architecture":"amd64","executor":"shell","shell":"pwsh","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"}],"host":"localhost","remote_ip":"172.18.0.1, 127.0.0.1","ua":"gitlab-runner 14.0.1 (refs/pipelines/326100216; go1.13.8; windows/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.009916,"redis_calls":5,"redis_duration_s":0.001475,"redis_read_bytes":607,"redis_write_bytes":378,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000937,"redis_feature_flag_read_bytes":607,"redis_feature_flag_write_bytes":234,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000538,"redis_shared_state_write_bytes":144,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":3,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023594,"mem_objects":6813,"mem_bytes":598016,"mem_mallocs":1807,"mem_total_bytes":870536,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01H53E0WK89GGAV2RV4EN564P3","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"172.18.0.1","meta.feature_category":"continuous_integration","meta.client_id":"ip/172.18.0.1","content_length":"641","request_urgency":"low","target_duration_s":5,"response_bytes":27}
{"time":"2023-07-11T21:35:21.865Z","severity":"INFO","duration_s":0.00911,"db_duration_s":0.0019,"view_duration_s":0.00721,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"14.0.1","revision":"c1edb478","platform":"windows","architecture":"amd64","executor":"shell","shell":"pwsh","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"}],"host":"localhost","remote_ip":"172.18.0.1, 127.0.0.1","ua":"gitlab-runner 14.0.1 (refs/pipelines/326100216; go1.13.8; windows/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.004832,"redis_calls":5,"redis_duration_s":0.000962,"redis_read_bytes":607,"redis_write_bytes":378,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000683,"redis_feature_flag_read_bytes":607,"redis_feature_flag_write_bytes":234,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_write_bytes":144,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":3,"db_main_count":0,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012984,"mem_objects":6786,"mem_bytes":597616,"mem_mallocs":1804,"mem_total_bytes":869056,"pid":778,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01H53E2Q7SEEQNVVEV7EPVH97C","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"172.18.0.1","meta.feature_category":"continuous_integration","meta.client_id":"ip/172.18.0.1","content_length":"641","request_urgency":"low","target_duration_s":5,"response_bytes":27}
==> /var/log/gitlab/gitlab-rails/database_load_balancing.log <==
{"severity":"DEBUG","time":"2023-07-11T18:30:07.569Z","correlation_id":null,"event":"setup","model":"Ci::ApplicationRecord","start_service_discovery":false}
{"severity":"DEBUG","time":"2023-07-11T18:30:07.570Z","correlation_id":null,"event":"setup","model":"Ci::ApplicationRecord","start_service_discovery":true}
{"severity":"DEBUG","time":"2023-07-11T18:30:26.697Z","correlation_id":null,"event":"setup","model":"ActiveRecord::Base","start_service_discovery":false}
{"severity":"DEBUG","time":"2023-07-11T18:30:26.698Z","correlation_id":null,"event":"setup","model":"Ci::ApplicationRecord","start_service_discovery":false}
{"severity":"DEBUG","time":"2023-07-11T18:30:46.993Z","correlation_id":null,"event":"setup","model":"ActiveRecord::Base","start_service_discovery":false}
{"severity":"DEBUG","time":"2023-07-11T18:30:46.994Z","correlation_id":null,"event":"setup","model":"ActiveRecord::Base","start_service_discovery":true}
{"severity":"DEBUG","time":"2023-07-11T18:30:46.995Z","correlation_id":null,"event":"setup","model":"Ci::ApplicationRecord","start_service_discovery":false}
{"severity":"DEBUG","time":"2023-07-11T18:30:46.995Z","correlation_id":null,"event":"setup","model":"Ci::ApplicationRecord","start_service_discovery":true}
{"severity":"DEBUG","time":"2023-07-11T18:31:22.542Z","correlation_id":null,"event":"setup","model":"ActiveRecord::Base","start_service_discovery":true}
{"severity":"DEBUG","time":"2023-07-11T18:31:22.543Z","correlation_id":null,"event":"setup","model":"Ci::ApplicationRecord","start_service_discovery":true}
==> /var/log/gitlab/gitlab-rails/git_json.log <==
# Logfile created on 2022-03-23 01:30:25 +0000 by logger.rb/v1.4.2
==> /var/log/gitlab/gitlab-rails/graphql_json.log <==
{"severity":"INFO","time":"2023-07-08T09:33:09.904Z","correlation_id":"01H4TDJ5D26KFXVM0RZ6Y0J0YE","meta.caller_id":"graphql:getAllRunnersEE","meta.remote_ip":"172.18.0.1","meta.feature_category":"runner","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","trace_type":"execute_query","query_fingerprint":"getAllRunnersEE/X1oRoXqMpQMgAL-ZyeejBIV938rFqRAsOdhHCMSW19o=/3/DOMxa8P8jhNolIWUFSfJh5TEg2iBSmebcNcHWene8sA=","duration_s":0.45417041600012453,"operation_name":"getAllRunnersEE","operation_fingerprint":"getAllRunnersEE/X1oRoXqMpQMgAL-ZyeejBIV938rFqRAsOdhHCMSW19o=","is_mutation":false,"variables":"{\"membership\"=\u003e\"DESCENDANTS\", \"sort\"=\u003e\"CREATED_DESC\", \"first\"=\u003e20}","query_string":"query getAllRunnersEE($before: String, $after: String, $first: Int, $last: Int, $paused: Boolean, $status: CiRunnerStatus, $upgradeStatus: CiRunnerUpgradeStatus, $type: CiRunnerType, $tagList: [String!], $search: String, $sort: CiRunnerSort) {\n runners(\n before: $before\n after: $after\n first: $first\n last: $last\n paused: $paused\n status: $status\n upgradeStatus: $upgradeStatus\n type: $type\n tagList: $tagList\n search: $search\n sort: $sort\n ) {\n ...AllRunnersConnection\n __typename\n }\n}\n\nfragment AllRunnersConnection on CiRunnerConnection {\n nodes {\n ...ListItem\n adminUrl\n editAdminUrl\n __typename\n }\n pageInfo {\n ...PageInfo\n __typename\n }\n __typename\n}\n\nfragment ListItem on CiRunner {\n ...ListItemShared\n upgradeStatus\n __typename\n}\n\nfragment ListItemShared on CiRunner {\n id\n description\n runnerType\n shortSha\n version\n ipAddress\n paused\n locked\n jobCount\n tagList\n createdAt\n createdBy {\n ...User\n __typename\n }\n contactedAt\n status\n jobExecutionStatus\n userPermissions {\n updateRunner\n deleteRunner\n __typename\n }\n managers {\n count\n __typename\n }\n groups(first: 1) {\n nodes {\n id\n name\n fullName\n webUrl\n __typename\n }\n __typename\n }\n ownerProject {\n id\n name\n nameWithNamespace\n webUrl\n __typename\n }\n __typename\n}\n\nfragment User on User {\n id\n avatarUrl\n name\n username\n webUrl\n __typename\n}\n\nfragment PageInfo on PageInfo {\n hasNextPage\n hasPreviousPage\n startCursor\n endCursor\n __typename\n}\n","query_analysis.duration_s":0.004544535999912114,"query_analysis.depth":5,"query_analysis.complexity":67,"query_analysis.used_fields":["CiRunner.id","CiRunner.description","CiRunner.runnerType","CiRunner.shortSha","CiRunner.version","CiRunner.ipAddress","CiRunner.paused","CiRunner.locked","CiRunner.jobCount","CiRunner.tagList","CiRunner.createdAt","User.id","User.avatarUrl","User.name","User.username","User.webUrl","User.__typename","UserCore.__typename","CiRunner.createdBy","CiRunner.contactedAt","CiRunner.status","CiRunner.jobExecutionStatus","RunnerPermissions.updateRunner","RunnerPermissions.deleteRunner","RunnerPermissions.__typename","CiRunner.userPermissions","CiRunnerManagerConnection.count","CiRunnerManagerConnection.__typename","CiRunner.managers","Group.id","Group.name","Group.fullName","Group.webUrl","Group.__typename","GroupConnection.nodes","GroupConnection.__typename","CiRunner.groups","Project.id","Project.name","Project.nameWithNamespace","Project.webUrl","Project.__typename","CiRunner.ownerProject","CiRunner.__typename","CiRunner.upgradeStatus","CiRunner.adminUrl","CiRunner.editAdminUrl","CiRunnerConnection.nodes","PageInfo.hasNextPage","PageInfo.hasPreviousPage","PageInfo.startCursor","PageInfo.endCursor","PageInfo.__typename","CiRunnerConnection.pageInfo","CiRunnerConnection.__typename","Query.runners"],"query_analysis.used_deprecated_fields":["CiRunner.jobExecutionStatus","CiRunner.managers","CiRunner.upgradeStatus"]}
{"severity":"INFO","time":"2023-07-08T09:35:31.057Z","correlation_id":"01H4TDPFP81EQ8JSMGQV284T7Y","meta.caller_id":"graphql:getCurrentLicense","meta.remote_ip":"172.18.0.1","meta.feature_category":"sm_provisioning","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","trace_type":"execute_query","query_fingerprint":"getCurrentLicense/zERbRVaI6kxVt-bcqnIdm7g4BLuEWCoZN9DIjCd7wbc=/0/RBNvo1WzZ4oRRq0W9-hknpT7T8If536DEMBg9hyq_4o=","duration_s":0.004421761999992668,"operation_name":"getCurrentLicense","operation_fingerprint":"getCurrentLicense/zERbRVaI6kxVt-bcqnIdm7g4BLuEWCoZN9DIjCd7wbc=","is_mutation":false,"variables":"{}","query_string":"query getCurrentLicense {\n currentLicense {\n ...License\n __typename\n }\n}\n\nfragment License on CurrentLicense {\n id\n type\n plan\n name\n email\n company\n startsAt\n expiresAt\n activatedAt\n lastSync\n usersInLicenseCount\n billableUsersCount\n maximumUserCount\n usersOverLicenseCount\n __typename\n}\n","query_analysis.duration_s":0.0008586399999330752,"query_analysis.depth":3,"query_analysis.complexity":24,"query_analysis.used_fields":["SubscriptionFutureEntry.type","SubscriptionFutureEntry.plan","SubscriptionFutureEntry.name","SubscriptionFutureEntry.email","SubscriptionFutureEntry.company","SubscriptionFutureEntry.usersInLicenseCount","SubscriptionFutureEntry.startsAt","SubscriptionFutureEntry.expiresAt","SubscriptionFutureEntry.__typename","SubscriptionFutureEntryConnection.nodes","SubscriptionFutureEntryConnection.__typename","Query.subscriptionFutureEntries"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-07-08T09:35:31.064Z","correlation_id":"01H4TDPFP81EQ8JSMGQV284T7Y","meta.caller_id":"graphql:getPastLicenseHistory","meta.remote_ip":"172.18.0.1","meta.feature_category":"sm_provisioning","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","trace_type":"execute_query","query_fingerprint":"getPastLicenseHistory/WfGLeIhF3IP13NhWd0aNbFSSKIR80O7yIpjg63_oT6k=/0/RBNvo1WzZ4oRRq0W9-hknpT7T8If536DEMBg9hyq_4o=","duration_s":0.0057902979999653326,"operation_name":"getPastLicenseHistory","operation_fingerprint":"getPastLicenseHistory/WfGLeIhF3IP13NhWd0aNbFSSKIR80O7yIpjg63_oT6k=","is_mutation":false,"variables":"{}","query_string":"query getPastLicenseHistory {\n licenseHistoryEntries {\n nodes {\n id\n type\n plan\n name\n email\n company\n usersInLicenseCount\n startsAt\n expiresAt\n activatedAt\n __typename\n }\n __typename\n }\n}\n","query_analysis.duration_s":0.0008586399999330752,"query_analysis.depth":3,"query_analysis.complexity":24,"query_analysis.used_fields":["SubscriptionFutureEntry.type","SubscriptionFutureEntry.plan","SubscriptionFutureEntry.name","SubscriptionFutureEntry.email","SubscriptionFutureEntry.company","SubscriptionFutureEntry.usersInLicenseCount","SubscriptionFutureEntry.startsAt","SubscriptionFutureEntry.expiresAt","SubscriptionFutureEntry.__typename","SubscriptionFutureEntryConnection.nodes","SubscriptionFutureEntryConnection.__typename","Query.subscriptionFutureEntries"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-07-08T09:35:31.070Z","correlation_id":"01H4TDPFP81EQ8JSMGQV284T7Y","meta.caller_id":"graphql:getFutureLicenseHistory","meta.remote_ip":"172.18.0.1","meta.feature_category":"sm_provisioning","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","trace_type":"execute_query","query_fingerprint":"getFutureLicenseHistory/CyB7CLSNF9zWDNlGSnx9QWMoIt8mwsfHRtJ663Qxn0Y=/0/RBNvo1WzZ4oRRq0W9-hknpT7T8If536DEMBg9hyq_4o=","duration_s":0.0037692340001740376,"operation_name":"getFutureLicenseHistory","operation_fingerprint":"getFutureLicenseHistory/CyB7CLSNF9zWDNlGSnx9QWMoIt8mwsfHRtJ663Qxn0Y=","is_mutation":false,"variables":"{}","query_string":"query getFutureLicenseHistory {\n subscriptionFutureEntries {\n nodes {\n ...FutureSubscription\n __typename\n }\n __typename\n }\n}\n\nfragment FutureSubscription on SubscriptionFutureEntry {\n type\n plan\n name\n email\n company\n usersInLicenseCount\n startsAt\n expiresAt\n __typename\n}\n","query_analysis.duration_s":0.0008586399999330752,"query_analysis.depth":3,"query_analysis.complexity":24,"query_analysis.used_fields":["SubscriptionFutureEntry.type","SubscriptionFutureEntry.plan","SubscriptionFutureEntry.name","SubscriptionFutureEntry.email","SubscriptionFutureEntry.company","SubscriptionFutureEntry.usersInLicenseCount","SubscriptionFutureEntry.startsAt","SubscriptionFutureEntry.expiresAt","SubscriptionFutureEntry.__typename","SubscriptionFutureEntryConnection.nodes","SubscriptionFutureEntryConnection.__typename","Query.subscriptionFutureEntries"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-07-08T09:36:17.923Z","correlation_id":"01H4TDQXAJ9D99QN6S71VE1STS","meta.caller_id":"graphql:getVariables","meta.remote_ip":"172.18.0.1","meta.feature_category":"continuous_integration","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","trace_type":"execute_query","query_fingerprint":"getVariables/Bb2TnhQsCvgjT6-l1QkT6lIBUdnDE85F-4rUWzV7c0A=/2/m__gFtU9e510h5DhgY3P6pVREq0ZMZgl9AE1WUeylV8=","duration_s":0.06796718400005375,"operation_name":"getVariables","operation_fingerprint":"getVariables/Bb2TnhQsCvgjT6-l1QkT6lIBUdnDE85F-4rUWzV7c0A=","is_mutation":false,"variables":"{\"sort\"=\u003e\"KEY_ASC\", \"first\"=\u003e100}","query_string":"query getVariables($after: String, $before: String, $first: Int, $last: Int, $sort: CiVariableSort = KEY_ASC) {\n ciVariables(\n after: $after\n before: $before\n first: $first\n last: $last\n sort: $sort\n ) {\n pageInfo {\n ...PageInfo\n __typename\n }\n nodes {\n ...BaseCiVariable\n ... on CiInstanceVariable {\n masked\n protected\n raw\n __typename\n }\n __typename\n }\n __typename\n }\n}\n\nfragment BaseCiVariable on CiVariable {\n __typename\n id\n key\n value\n variableType\n}\n\nfragment PageInfo on PageInfo {\n hasNextPage\n hasPreviousPage\n startCursor\n endCursor\n __typename\n}\n","query_analysis.duration_s":0.009850929000094766,"query_analysis.depth":3,"query_analysis.complexity":36,"query_analysis.used_fields":["PageInfo.hasNextPage","PageInfo.hasPreviousPage","PageInfo.startCursor","PageInfo.endCursor","PageInfo.__typename","CiInstanceVariableConnection.pageInfo","CiVariable.__typename","CiVariable.id","CiVariable.key","CiVariable.value","CiVariable.variableType","CiInstanceVariable.masked","CiInstanceVariable.protected","CiInstanceVariable.raw","CiInstanceVariable.__typename","CiInstanceVariableConnection.nodes","CiInstanceVariableConnection.__typename","Query.ciVariables"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-07-09T01:15:15.117Z","correlation_id":"01H4W3F67J3YS65FD2PZ69RM3V","meta.caller_id":"graphql:getCurrentLicense","meta.remote_ip":"172.18.0.1","meta.feature_category":"sm_provisioning","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","trace_type":"execute_query","query_fingerprint":"getCurrentLicense/zERbRVaI6kxVt-bcqnIdm7g4BLuEWCoZN9DIjCd7wbc=/0/RBNvo1WzZ4oRRq0W9-hknpT7T8If536DEMBg9hyq_4o=","duration_s":0.003517533994454425,"operation_name":"getCurrentLicense","operation_fingerprint":"getCurrentLicense/zERbRVaI6kxVt-bcqnIdm7g4BLuEWCoZN9DIjCd7wbc=","is_mutation":false,"variables":"{}","query_string":"query getCurrentLicense {\n currentLicense {\n ...License\n __typename\n }\n}\n\nfragment License on CurrentLicense {\n id\n type\n plan\n name\n email\n company\n startsAt\n expiresAt\n activatedAt\n lastSync\n usersInLicenseCount\n billableUsersCount\n maximumUserCount\n usersOverLicenseCount\n __typename\n}\n","query_analysis.duration_s":0.0007262479994096793,"query_analysis.depth":3,"query_analysis.complexity":24,"query_analysis.used_fields":["SubscriptionFutureEntry.type","SubscriptionFutureEntry.plan","SubscriptionFutureEntry.name","SubscriptionFutureEntry.email","SubscriptionFutureEntry.company","SubscriptionFutureEntry.usersInLicenseCount","SubscriptionFutureEntry.startsAt","SubscriptionFutureEntry.expiresAt","SubscriptionFutureEntry.__typename","SubscriptionFutureEntryConnection.nodes","SubscriptionFutureEntryConnection.__typename","Query.subscriptionFutureEntries"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-07-09T01:15:15.126Z","correlation_id":"01H4W3F67J3YS65FD2PZ69RM3V","meta.caller_id":"graphql:getPastLicenseHistory","meta.remote_ip":"172.18.0.1","meta.feature_category":"sm_provisioning","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","trace_type":"execute_query","query_fingerprint":"getPastLicenseHistory/WfGLeIhF3IP13NhWd0aNbFSSKIR80O7yIpjg63_oT6k=/0/RBNvo1WzZ4oRRq0W9-hknpT7T8If536DEMBg9hyq_4o=","duration_s":0.0034458919981261715,"operation_name":"getPastLicenseHistory","operation_fingerprint":"getPastLicenseHistory/WfGLeIhF3IP13NhWd0aNbFSSKIR80O7yIpjg63_oT6k=","is_mutation":false,"variables":"{}","query_string":"query getPastLicenseHistory {\n licenseHistoryEntries {\n nodes {\n id\n type\n plan\n name\n email\n company\n usersInLicenseCount\n startsAt\n expiresAt\n activatedAt\n __typename\n }\n __typename\n }\n}\n","query_analysis.duration_s":0.0007262479994096793,"query_analysis.depth":3,"query_analysis.complexity":24,"query_analysis.used_fields":["SubscriptionFutureEntry.type","SubscriptionFutureEntry.plan","SubscriptionFutureEntry.name","SubscriptionFutureEntry.email","SubscriptionFutureEntry.company","SubscriptionFutureEntry.usersInLicenseCount","SubscriptionFutureEntry.startsAt","SubscriptionFutureEntry.expiresAt","SubscriptionFutureEntry.__typename","SubscriptionFutureEntryConnection.nodes","SubscriptionFutureEntryConnection.__typename","Query.subscriptionFutureEntries"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-07-09T01:15:15.129Z","correlation_id":"01H4W3F67J3YS65FD2PZ69RM3V","meta.caller_id":"graphql:getFutureLicenseHistory","meta.remote_ip":"172.18.0.1","meta.feature_category":"sm_provisioning","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","trace_type":"execute_query","query_fingerprint":"getFutureLicenseHistory/CyB7CLSNF9zWDNlGSnx9QWMoIt8mwsfHRtJ663Qxn0Y=/0/RBNvo1WzZ4oRRq0W9-hknpT7T8If536DEMBg9hyq_4o=","duration_s":0.001695855004072655,"operation_name":"getFutureLicenseHistory","operation_fingerprint":"getFutureLicenseHistory/CyB7CLSNF9zWDNlGSnx9QWMoIt8mwsfHRtJ663Qxn0Y=","is_mutation":false,"variables":"{}","query_string":"query getFutureLicenseHistory {\n subscriptionFutureEntries {\n nodes {\n ...FutureSubscription\n __typename\n }\n __typename\n }\n}\n\nfragment FutureSubscription on SubscriptionFutureEntry {\n type\n plan\n name\n email\n company\n usersInLicenseCount\n startsAt\n expiresAt\n __typename\n}\n","query_analysis.duration_s":0.0007262479994096793,"query_analysis.depth":3,"query_analysis.complexity":24,"query_analysis.used_fields":["SubscriptionFutureEntry.type","SubscriptionFutureEntry.plan","SubscriptionFutureEntry.name","SubscriptionFutureEntry.email","SubscriptionFutureEntry.company","SubscriptionFutureEntry.usersInLicenseCount","SubscriptionFutureEntry.startsAt","SubscriptionFutureEntry.expiresAt","SubscriptionFutureEntry.__typename","SubscriptionFutureEntryConnection.nodes","SubscriptionFutureEntryConnection.__typename","Query.subscriptionFutureEntries"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-07-11T20:24:30.498Z","correlation_id":"01H53A0ZE28NHP9SMGRA98HRQN","meta.caller_id":"graphql:getDashboardIssuesCount","meta.remote_ip":"172.18.0.1","meta.feature_category":"team_planning","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","trace_type":"execute_query","query_fingerprint":"getDashboardIssuesCount/ivt4gG24cOlmVqjeJeCwuSZu_DvCHagxXz38gYmdLmc=/6/RYkuBDLdxCDiFQgQDUmjgaofSYyhlgVZBHcknfeLNRU=","duration_s":0.0393290160000106,"operation_name":"getDashboardIssuesCount","operation_fingerprint":"getDashboardIssuesCount/ivt4gG24cOlmVqjeJeCwuSZu_DvCHagxXz38gYmdLmc=","is_mutation":false,"variables":"{\"hideUsers\"=\u003efalse, \"isSignedIn\"=\u003etrue, \"sort\"=\u003e\"CREATED_DESC\", \"state\"=\u003e\"opened\", \"firstPageSize\"=\u003e20, \"assigneeUsernames\"=\u003e\"WizzardMaker\"}","query_string":"query getDashboardIssuesCount($search: String, $assigneeId: String, $assigneeUsernames: [String!], $authorUsername: String, $confidential: Boolean, $labelName: [String], $milestoneTitle: [String], $milestoneWildcardId: MilestoneWildcardId, $myReactionEmoji: String, $types: [IssueType!], $in: [IssuableSearchableField!], $not: NegatedIssueFilterInput, $or: UnionedIssueFilterInput) {\n openedIssues: issues(\n state: opened\n search: $search\n assigneeId: $assigneeId\n assigneeUsernames: $assigneeUsernames\n authorUsername: $authorUsername\n confidential: $confidential\n labelName: $labelName\n milestoneTitle: $milestoneTitle\n milestoneWildcardId: $milestoneWildcardId\n myReactionEmoji: $myReactionEmoji\n types: $types\n in: $in\n not: $not\n or: $or\n ) {\n count\n __typename\n }\n closedIssues: issues(\n state: closed\n search: $search\n assigneeId: $assigneeId\n assigneeUsernames: $assigneeUsernames\n authorUsername: $authorUsername\n confidential: $confidential\n labelName: $labelName\n milestoneTitle: $milestoneTitle\n milestoneWildcardId: $milestoneWildcardId\n myReactionEmoji: $myReactionEmoji\n types: $types\n in: $in\n not: $not\n or: $or\n ) {\n count\n __typename\n }\n allIssues: issues(\n state: all\n search: $search\n assigneeId: $assigneeId\n assigneeUsernames: $assigneeUsernames\n authorUsername: $authorUsername\n confidential: $confidential\n labelName: $labelName\n milestoneTitle: $milestoneTitle\n milestoneWildcardId: $milestoneWildcardId\n myReactionEmoji: $myReactionEmoji\n types: $types\n in: $in\n not: $not\n or: $or\n ) {\n count\n __typename\n }\n}\n","query_analysis.duration_s":0.003177502000653476,"query_analysis.depth":2,"query_analysis.complexity":24,"query_analysis.used_fields":["IssueConnection.count","IssueConnection.__typename","Query.issues"],"query_analysis.used_deprecated_fields":["Query.issues"]}
{"severity":"INFO","time":"2023-07-11T20:24:31.065Z","correlation_id":"01H53A0ZGHJYB39HK8K5SQ0E5X","meta.caller_id":"graphql:getDashboardIssuesEE","meta.remote_ip":"172.18.0.1","meta.feature_category":"team_planning","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","trace_type":"execute_query","query_fingerprint":"getDashboardIssuesEE/IAzApZlUPbk8SeVuSkisGelNFSCgsDZEwCaW6jPxUHw=/6/RYkuBDLdxCDiFQgQDUmjgaofSYyhlgVZBHcknfeLNRU=","duration_s":0.5283423029995902,"operation_name":"getDashboardIssuesEE","operation_fingerprint":"getDashboardIssuesEE/IAzApZlUPbk8SeVuSkisGelNFSCgsDZEwCaW6jPxUHw=","is_mutation":false,"variables":"{\"hideUsers\"=\u003efalse, \"isSignedIn\"=\u003etrue, \"sort\"=\u003e\"CREATED_DESC\", \"state\"=\u003e\"opened\", \"firstPageSize\"=\u003e20, \"assigneeUsernames\"=\u003e\"WizzardMaker\"}","query_string":"query getDashboardIssuesEE($hideUsers: Boolean = false, $isSignedIn: Boolean = false, $search: String, $sort: IssueSort, $state: IssuableState, $assigneeId: String, $assigneeUsernames: [String!], $authorUsername: String, $confidential: Boolean, $labelName: [String], $milestoneTitle: [String], $milestoneWildcardId: MilestoneWildcardId, $myReactionEmoji: String, $types: [IssueType!], $in: [IssuableSearchableField!], $not: NegatedIssueFilterInput, $or: UnionedIssueFilterInput, $afterCursor: String, $beforeCursor: String, $firstPageSize: Int, $lastPageSize: Int) {\n issues(\n search: $search\n sort: $sort\n state: $state\n assigneeId: $assigneeId\n assigneeUsernames: $assigneeUsernames\n authorUsername: $authorUsername\n confidential: $confidential\n labelName: $labelName\n milestoneTitle: $milestoneTitle\n milestoneWildcardId: $milestoneWildcardId\n myReactionEmoji: $myReactionEmoji\n types: $types\n in: $in\n not: $not\n or: $or\n after: $afterCursor\n before: $beforeCursor\n first: $firstPageSize\n last: $lastPageSize\n ) {\n nodes {\n ...IssueFragment\n blockingCount\n healthStatus\n reference(full: true)\n weight\n }\n pageInfo {\n ...PageInfo\n __typename\n }\n __typename\n }\n}\n\nfragment PageInfo on PageInfo {\n hasNextPage\n hasPreviousPage\n startCursor\n endCursor\n __typename\n}\n\nfragment IssueFragment on Issue {\n id\n iid\n confidential\n createdAt\n downvotes\n dueDate\n hidden\n humanTimeEstimate\n mergeRequestsCount\n moved\n state\n title\n updatedAt\n closedAt\n upvotes\n userDiscussionsCount @include(if: $isSignedIn)\n webPath\n webUrl\n type\n assignees @skip(if: $hideUsers) {\n nodes {\n id\n avatarUrl\n name\n username\n webUrl\n }\n __typename\n }\n author @skip(if: $hideUsers) {\n id\n avatarUrl\n name\n username\n webUrl\n }\n labels {\n nodes {\n id\n color\n title\n description\n }\n __typename\n }\n milestone {\n id\n dueDate\n startDate\n webPath\n title\n }\n taskCompletionStatus {\n completedCount\n count\n __typename\n }\n __typename\n}\n","query_analysis.duration_s":0.0044921520002390025,"query_analysis.depth":5,"query_analysis.complexity":86,"query_analysis.used_fields":["Issue.id","Issue.iid","Issue.confidential","Issue.createdAt","Issue.downvotes","Issue.dueDate","Issue.hidden","Issue.humanTimeEstimate","Issue.mergeRequestsCount","Issue.moved","Issue.state","Issue.title","Issue.updatedAt","Issue.closedAt","Issue.upvotes","Issue.userDiscussionsCount","Issue.webPath","Issue.webUrl","Issue.type","UserCore.id","UserCore.avatarUrl","UserCore.name","UserCore.username","UserCore.webUrl","UserCoreConnection.nodes","UserCoreConnection.__typename","Issue.assignees","Issue.author","Label.id","Label.color","Label.title","Label.description","LabelConnection.nodes","LabelConnection.__typename","Issue.labels","Milestone.id","Milestone.dueDate","Milestone.startDate","Milestone.webPath","Milestone.title","Issue.milestone","TaskCompletionStatus.completedCount","TaskCompletionStatus.count","TaskCompletionStatus.__typename","Issue.taskCompletionStatus","Issue.__typename","Issue.blockingCount","Issue.healthStatus","Issue.reference","Issue.weight","IssueConnection.nodes","PageInfo.hasNextPage","PageInfo.hasPreviousPage","PageInfo.startCursor","PageInfo.endCursor","PageInfo.__typename","IssueConnection.pageInfo","IssueConnection.__typename","Query.issues"],"query_analysis.used_deprecated_fields":["Query.issues"]}
==> /var/log/gitlab/gitlab-rails/auth_json.log <==
{"severity":"INFO","time":"2023-07-08T09:22:33.094Z","gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2023-07-08T09:24:04.084Z","gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2023-07-08T09:25:11.960Z","gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2023-07-08T09:25:17.388Z","gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2023-07-11T08:06:06.658Z","gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2023-07-11T08:06:40.549Z","gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2023-07-11T08:06:52.036Z","gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2023-07-11T18:30:08.161Z","gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2023-07-11T18:30:30.453Z","gitlab_throttle_user_allowlist":[]}
{"severity":"INFO","time":"2023-07-11T18:30:49.794Z","gitlab_throttle_user_allowlist":[]}
==> /var/log/gitlab/gitlab-rails/sidekiq_client.log <==
{"severity":"WARN","time":"2023-06-28T10:06:07.762Z","message":"Job arguments to ExternalServiceReactiveCachingWorker do not serialize to JSON safely. This will raise an error in\nSidekiq 7.0. See https://github.com/mperham/sidekiq/wiki/Best-Practices or raise an error today\nby calling `Sidekiq.strict_args!` during Sidekiq initialization.\n"}
{"severity":"WARN","time":"2023-06-28T10:06:07.921Z","message":"Job arguments to ExternalServiceReactiveCachingWorker do not serialize to JSON safely. This will raise an error in\nSidekiq 7.0. See https://github.com/mperham/sidekiq/wiki/Best-Practices or raise an error today\nby calling `Sidekiq.strict_args!` during Sidekiq initialization.\n"}
{"severity":"WARN","time":"2023-06-28T10:06:07.963Z","message":"Job arguments to ExternalServiceReactiveCachingWorker do not serialize to JSON safely. This will raise an error in\nSidekiq 7.0. See https://github.com/mperham/sidekiq/wiki/Best-Practices or raise an error today\nby calling `Sidekiq.strict_args!` during Sidekiq initialization.\n"}
{"severity":"WARN","time":"2023-06-28T10:06:08.237Z","message":"Job arguments to ExternalServiceReactiveCachingWorker do not serialize to JSON safely. This will raise an error in\nSidekiq 7.0. See https://github.com/mperham/sidekiq/wiki/Best-Practices or raise an error today\nby calling `Sidekiq.strict_args!` during Sidekiq initialization.\n"}
{"severity":"INFO","time":"2023-06-29T11:10:44.455Z","message":"Cron Jobs - deleted job with name: free_user_cap_data_remediation"}
{"severity":"WARN","time":"2023-06-29T12:10:08.243Z","message":"Job arguments to ExternalServiceReactiveCachingWorker do not serialize to JSON safely. This will raise an error in\nSidekiq 7.0. See https://github.com/mperham/sidekiq/wiki/Best-Practices or raise an error today\nby calling `Sidekiq.strict_args!` during Sidekiq initialization.\n"}
{"severity":"WARN","time":"2023-06-29T12:22:58.534Z","message":"Job arguments to ExternalServiceReactiveCachingWorker do not serialize to JSON safely. This will raise an error in\nSidekiq 7.0. See https://github.com/mperham/sidekiq/wiki/Best-Practices or raise an error today\nby calling `Sidekiq.strict_args!` during Sidekiq initialization.\n"}
{"severity":"WARN","time":"2023-06-29T13:15:06.115Z","message":"Job arguments to ExternalServiceReactiveCachingWorker do not serialize to JSON safely. This will raise an error in\nSidekiq 7.0. See https://github.com/mperham/sidekiq/wiki/Best-Practices or raise an error today\nby calling `Sidekiq.strict_args!` during Sidekiq initialization.\n"}
{"severity":"WARN","time":"2023-07-05T22:01:19.517Z","message":"Job arguments to ExternalServiceReactiveCachingWorker do not serialize to JSON safely. This will raise an error in\nSidekiq 7.0. See https://github.com/mperham/sidekiq/wiki/Best-Practices or raise an error today\nby calling `Sidekiq.strict_args!` during Sidekiq initialization.\n"}
{"severity":"WARN","time":"2023-07-09T01:15:01.679Z","message":"Job arguments to ExternalServiceReactiveCachingWorker do not serialize to JSON safely. This will raise an error in\nSidekiq 7.0. See https://github.com/mperham/sidekiq/wiki/Best-Practices or raise an error today\nby calling `Sidekiq.strict_args!` during Sidekiq initialization.\n"}
==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2023-06-29-09-44-55.log <==
-- add_column(:application_settings, :pipeline_limit_per_project_user_sha, :integer, {:default=>0, :null=>false})
-> 0.0029s
== 20220511191503 AddPipelineCreateLimitPerProjectUserShaToApplicationSettings: migrated (0.0029s)
== 20220513043344 RescheduleExpireOAuthTokens: migrating ======================
== 20220513043344 RescheduleExpireOAuthTokens: migrated (0.4513s) =============
== 20220517144749 RemoveVulnerabilityApprovalRules: migrating =================
== 20220517144749 RemoveVulnerabilityApprovalRules: migrated (0.0944s) ========
==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2023-06-29-09-25-10.log <==
-- indexes(:packages_build_infos)
-> 0.0016s
-- execute("SET statement_timeout TO 0")
-> 0.0002s
-- remove_index(:packages_build_infos, {:algorithm=>:concurrently, :name=>"index_packages_build_infos_package_id_pipeline_id"})
-> 0.0457s
-- execute("RESET statement_timeout")
-> 0.0003s
== 20220413075921 UpdateIndexOnPackagesBuildInfos: migrated (0.1727s) =========
==> /var/log/gitlab/gitlab-rails/exceptions_json.log <==
{"severity":"ERROR","time":"2023-07-06T07:53:09.272Z","meta.caller_id":"GitlabServicePingWorker","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","meta.client_id":"ip/","exception.class":"NoMethodError","exception.message":"undefined method `starts_at' for nil:NilClass","exception.backtrace":["ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:42:in `block in value'","lib/gitlab/utils/usage_data.rb:200:in `block in alt_usage_data'","lib/gitlab/utils/usage_data.rb:48:in `with_metadata'","lib/gitlab/utils/usage_data.rb:198:in `alt_usage_data'","ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:40:in `value'","lib/gitlab/usage/metric.rb:21:in `block in with_value'","lib/gitlab/usage/metric.rb:37:in `with_availability'","lib/gitlab/usage/metric.rb:21:in `with_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `call'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `compute_instrumental_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:19:in `block in build'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `map'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `build'","lib/gitlab/usage/service_ping_report.rb:27:in `with_instrumentation_classes'","lib/gitlab/usage/service_ping_report.rb:13:in `block in for'","lib/gitlab/usage/service_ping_report.rb:12:in `for'","lib/service_ping/build_payload.rb:12:in `raw_payload'","lib/service_ping/build_payload.rb:15:in `filtered_usage_data'","lib/service_ping/build_payload.rb:6:in `execute'","app/workers/gitlab_service_ping_worker.rb:40:in `usage_data'","app/workers/gitlab_service_ping_worker.rb:35:in `block in perform'","lib/gitlab/exclusive_lease_helpers.rb:38:in `in_lock'","app/workers/gitlab_service_ping_worker.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/defer_jobs.rb:33:in `call'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:29:in `call'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/sidekiq_middleware/worker_context.rb:11:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/application_context.rb:57:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'","lib/gitlab/with_request_store.rb:17:in `enabling_request_store'","lib/gitlab/with_request_store.rb:10:in `with_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:83:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:82:in `call'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'"],"user.username":null,"tags.program":"sidekiq","tags.locale":"en","tags.feature_category":"service_ping","tags.correlation_id":"c04cd818b6092142aa8d341ec120d3d9","extra.sidekiq":{"retry":3,"queue":"default","version":0,"queue_namespace":"cronjob","dead":false,"args":[],"class":"GitlabServicePingWorker","jid":"b1dc7d026207a53732b2ed08","created_at":1688442424.3723645,"meta.caller_id":"Cronjob","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:f08b08648429384a6800590e6650412b8bf3ae483020a60c2e1d227e13e464e8","size_limiter":"validated","enqueued_at":1688629949.6904507,"error_message":"Unsuccessful response code: 406","error_class":"ServicePing::SubmitService::SubmissionError","failed_at":1688442523.669093,"retry_count":2,"retried_at":1688529036.9532511}}
{"severity":"ERROR","time":"2023-07-06T07:53:09.276Z","meta.caller_id":"GitlabServicePingWorker","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","meta.client_id":"ip/","exception.class":"NoMethodError","exception.message":"undefined method `expires_at' for nil:NilClass","exception.backtrace":["ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:42:in `block in value'","lib/gitlab/utils/usage_data.rb:200:in `block in alt_usage_data'","lib/gitlab/utils/usage_data.rb:48:in `with_metadata'","lib/gitlab/utils/usage_data.rb:198:in `alt_usage_data'","ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:40:in `value'","lib/gitlab/usage/metric.rb:21:in `block in with_value'","lib/gitlab/usage/metric.rb:37:in `with_availability'","lib/gitlab/usage/metric.rb:21:in `with_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `call'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `compute_instrumental_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:19:in `block in build'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `map'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `build'","lib/gitlab/usage/service_ping_report.rb:27:in `with_instrumentation_classes'","lib/gitlab/usage/service_ping_report.rb:13:in `block in for'","lib/gitlab/usage/service_ping_report.rb:12:in `for'","lib/service_ping/build_payload.rb:12:in `raw_payload'","lib/service_ping/build_payload.rb:15:in `filtered_usage_data'","lib/service_ping/build_payload.rb:6:in `execute'","app/workers/gitlab_service_ping_worker.rb:40:in `usage_data'","app/workers/gitlab_service_ping_worker.rb:35:in `block in perform'","lib/gitlab/exclusive_lease_helpers.rb:38:in `in_lock'","app/workers/gitlab_service_ping_worker.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/defer_jobs.rb:33:in `call'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:29:in `call'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/sidekiq_middleware/worker_context.rb:11:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/application_context.rb:57:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'","lib/gitlab/with_request_store.rb:17:in `enabling_request_store'","lib/gitlab/with_request_store.rb:10:in `with_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:83:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:82:in `call'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'"],"user.username":null,"tags.program":"sidekiq","tags.locale":"en","tags.feature_category":"service_ping","tags.correlation_id":"c04cd818b6092142aa8d341ec120d3d9","extra.sidekiq":{"retry":3,"queue":"default","version":0,"queue_namespace":"cronjob","dead":false,"args":[],"class":"GitlabServicePingWorker","jid":"b1dc7d026207a53732b2ed08","created_at":1688442424.3723645,"meta.caller_id":"Cronjob","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:f08b08648429384a6800590e6650412b8bf3ae483020a60c2e1d227e13e464e8","size_limiter":"validated","enqueued_at":1688629949.6904507,"error_message":"Unsuccessful response code: 406","error_class":"ServicePing::SubmitService::SubmissionError","failed_at":1688442523.669093,"retry_count":2,"retried_at":1688529036.9532511}}
{"severity":"ERROR","time":"2023-07-06T07:53:09.281Z","meta.caller_id":"GitlabServicePingWorker","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","meta.client_id":"ip/","exception.class":"NoMethodError","exception.message":"undefined method `plan' for nil:NilClass","exception.backtrace":["ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:42:in `block in value'","lib/gitlab/utils/usage_data.rb:200:in `block in alt_usage_data'","lib/gitlab/utils/usage_data.rb:48:in `with_metadata'","lib/gitlab/utils/usage_data.rb:198:in `alt_usage_data'","ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:40:in `value'","lib/gitlab/usage/metric.rb:21:in `block in with_value'","lib/gitlab/usage/metric.rb:37:in `with_availability'","lib/gitlab/usage/metric.rb:21:in `with_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `call'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `compute_instrumental_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:19:in `block in build'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `map'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `build'","lib/gitlab/usage/service_ping_report.rb:27:in `with_instrumentation_classes'","lib/gitlab/usage/service_ping_report.rb:13:in `block in for'","lib/gitlab/usage/service_ping_report.rb:12:in `for'","lib/service_ping/build_payload.rb:12:in `raw_payload'","lib/service_ping/build_payload.rb:15:in `filtered_usage_data'","lib/service_ping/build_payload.rb:6:in `execute'","app/workers/gitlab_service_ping_worker.rb:40:in `usage_data'","app/workers/gitlab_service_ping_worker.rb:35:in `block in perform'","lib/gitlab/exclusive_lease_helpers.rb:38:in `in_lock'","app/workers/gitlab_service_ping_worker.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/defer_jobs.rb:33:in `call'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:29:in `call'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/sidekiq_middleware/worker_context.rb:11:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/application_context.rb:57:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'","lib/gitlab/with_request_store.rb:17:in `enabling_request_store'","lib/gitlab/with_request_store.rb:10:in `with_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:83:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:82:in `call'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'"],"user.username":null,"tags.program":"sidekiq","tags.locale":"en","tags.feature_category":"service_ping","tags.correlation_id":"c04cd818b6092142aa8d341ec120d3d9","extra.sidekiq":{"retry":3,"queue":"default","version":0,"queue_namespace":"cronjob","dead":false,"args":[],"class":"GitlabServicePingWorker","jid":"b1dc7d026207a53732b2ed08","created_at":1688442424.3723645,"meta.caller_id":"Cronjob","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:f08b08648429384a6800590e6650412b8bf3ae483020a60c2e1d227e13e464e8","size_limiter":"validated","enqueued_at":1688629949.6904507,"error_message":"Unsuccessful response code: 406","error_class":"ServicePing::SubmitService::SubmissionError","failed_at":1688442523.669093,"retry_count":2,"retried_at":1688529036.9532511}}
{"severity":"ERROR","time":"2023-07-06T07:53:09.285Z","meta.caller_id":"GitlabServicePingWorker","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","meta.client_id":"ip/","exception.class":"NoMethodError","exception.message":"undefined method `trial?' for nil:NilClass","exception.backtrace":["ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:42:in `block in value'","lib/gitlab/utils/usage_data.rb:200:in `block in alt_usage_data'","lib/gitlab/utils/usage_data.rb:48:in `with_metadata'","lib/gitlab/utils/usage_data.rb:198:in `alt_usage_data'","ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:40:in `value'","lib/gitlab/usage/metric.rb:21:in `block in with_value'","lib/gitlab/usage/metric.rb:37:in `with_availability'","lib/gitlab/usage/metric.rb:21:in `with_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `call'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `compute_instrumental_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:19:in `block in build'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `map'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `build'","lib/gitlab/usage/service_ping_report.rb:27:in `with_instrumentation_classes'","lib/gitlab/usage/service_ping_report.rb:13:in `block in for'","lib/gitlab/usage/service_ping_report.rb:12:in `for'","lib/service_ping/build_payload.rb:12:in `raw_payload'","lib/service_ping/build_payload.rb:15:in `filtered_usage_data'","lib/service_ping/build_payload.rb:6:in `execute'","app/workers/gitlab_service_ping_worker.rb:40:in `usage_data'","app/workers/gitlab_service_ping_worker.rb:35:in `block in perform'","lib/gitlab/exclusive_lease_helpers.rb:38:in `in_lock'","app/workers/gitlab_service_ping_worker.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/defer_jobs.rb:33:in `call'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:29:in `call'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/sidekiq_middleware/worker_context.rb:11:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/application_context.rb:57:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'","lib/gitlab/with_request_store.rb:17:in `enabling_request_store'","lib/gitlab/with_request_store.rb:10:in `with_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:83:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:82:in `call'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'"],"user.username":null,"tags.program":"sidekiq","tags.locale":"en","tags.feature_category":"service_ping","tags.correlation_id":"c04cd818b6092142aa8d341ec120d3d9","extra.sidekiq":{"retry":3,"queue":"default","version":0,"queue_namespace":"cronjob","dead":false,"args":[],"class":"GitlabServicePingWorker","jid":"b1dc7d026207a53732b2ed08","created_at":1688442424.3723645,"meta.caller_id":"Cronjob","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:f08b08648429384a6800590e6650412b8bf3ae483020a60c2e1d227e13e464e8","size_limiter":"validated","enqueued_at":1688629949.6904507,"error_message":"Unsuccessful response code: 406","error_class":"ServicePing::SubmitService::SubmissionError","failed_at":1688442523.669093,"retry_count":2,"retried_at":1688529036.9532511}}
{"severity":"ERROR","time":"2023-07-06T07:53:09.291Z","meta.caller_id":"GitlabServicePingWorker","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","meta.client_id":"ip/","exception.class":"NoMethodError","exception.message":"undefined method `subscription_id' for nil:NilClass","exception.backtrace":["ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:42:in `block in value'","lib/gitlab/utils/usage_data.rb:200:in `block in alt_usage_data'","lib/gitlab/utils/usage_data.rb:48:in `with_metadata'","lib/gitlab/utils/usage_data.rb:198:in `alt_usage_data'","ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:40:in `value'","lib/gitlab/usage/metric.rb:21:in `block in with_value'","lib/gitlab/usage/metric.rb:37:in `with_availability'","lib/gitlab/usage/metric.rb:21:in `with_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `call'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `compute_instrumental_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:19:in `block in build'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `map'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `build'","lib/gitlab/usage/service_ping_report.rb:27:in `with_instrumentation_classes'","lib/gitlab/usage/service_ping_report.rb:13:in `block in for'","lib/gitlab/usage/service_ping_report.rb:12:in `for'","lib/service_ping/build_payload.rb:12:in `raw_payload'","lib/service_ping/build_payload.rb:15:in `filtered_usage_data'","lib/service_ping/build_payload.rb:6:in `execute'","app/workers/gitlab_service_ping_worker.rb:40:in `usage_data'","app/workers/gitlab_service_ping_worker.rb:35:in `block in perform'","lib/gitlab/exclusive_lease_helpers.rb:38:in `in_lock'","app/workers/gitlab_service_ping_worker.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/defer_jobs.rb:33:in `call'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:29:in `call'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/sidekiq_middleware/worker_context.rb:11:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/application_context.rb:57:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'","lib/gitlab/with_request_store.rb:17:in `enabling_request_store'","lib/gitlab/with_request_store.rb:10:in `with_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:83:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:82:in `call'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'"],"user.username":null,"tags.program":"sidekiq","tags.locale":"en","tags.feature_category":"service_ping","tags.correlation_id":"c04cd818b6092142aa8d341ec120d3d9","extra.sidekiq":{"retry":3,"queue":"default","version":0,"queue_namespace":"cronjob","dead":false,"args":[],"class":"GitlabServicePingWorker","jid":"b1dc7d026207a53732b2ed08","created_at":1688442424.3723645,"meta.caller_id":"Cronjob","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:f08b08648429384a6800590e6650412b8bf3ae483020a60c2e1d227e13e464e8","size_limiter":"validated","enqueued_at":1688629949.6904507,"error_message":"Unsuccessful response code: 406","error_class":"ServicePing::SubmitService::SubmissionError","failed_at":1688442523.669093,"retry_count":2,"retried_at":1688529036.9532511}}
{"severity":"ERROR","time":"2023-07-06T07:53:09.299Z","meta.caller_id":"GitlabServicePingWorker","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","meta.client_id":"ip/","exception.class":"NoMethodError","exception.message":"undefined method `daily_billable_users_count' for nil:NilClass","exception.backtrace":["ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:42:in `block in value'","lib/gitlab/utils/usage_data.rb:200:in `block in alt_usage_data'","lib/gitlab/utils/usage_data.rb:48:in `with_metadata'","lib/gitlab/utils/usage_data.rb:198:in `alt_usage_data'","ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:40:in `value'","lib/gitlab/usage/metric.rb:21:in `block in with_value'","lib/gitlab/usage/metric.rb:37:in `with_availability'","lib/gitlab/usage/metric.rb:21:in `with_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `call'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `compute_instrumental_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:19:in `block in build'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `map'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `build'","lib/gitlab/usage/service_ping_report.rb:27:in `with_instrumentation_classes'","lib/gitlab/usage/service_ping_report.rb:13:in `block in for'","lib/gitlab/usage/service_ping_report.rb:12:in `for'","lib/service_ping/build_payload.rb:12:in `raw_payload'","lib/service_ping/build_payload.rb:15:in `filtered_usage_data'","lib/service_ping/build_payload.rb:6:in `execute'","app/workers/gitlab_service_ping_worker.rb:40:in `usage_data'","app/workers/gitlab_service_ping_worker.rb:35:in `block in perform'","lib/gitlab/exclusive_lease_helpers.rb:38:in `in_lock'","app/workers/gitlab_service_ping_worker.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/defer_jobs.rb:33:in `call'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:29:in `call'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/sidekiq_middleware/worker_context.rb:11:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/application_context.rb:57:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'","lib/gitlab/with_request_store.rb:17:in `enabling_request_store'","lib/gitlab/with_request_store.rb:10:in `with_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:83:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:82:in `call'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'"],"user.username":null,"tags.program":"sidekiq","tags.locale":"en","tags.feature_category":"service_ping","tags.correlation_id":"c04cd818b6092142aa8d341ec120d3d9","extra.sidekiq":{"retry":3,"queue":"default","version":0,"queue_namespace":"cronjob","dead":false,"args":[],"class":"GitlabServicePingWorker","jid":"b1dc7d026207a53732b2ed08","created_at":1688442424.3723645,"meta.caller_id":"Cronjob","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:f08b08648429384a6800590e6650412b8bf3ae483020a60c2e1d227e13e464e8","size_limiter":"validated","enqueued_at":1688629949.6904507,"error_message":"Unsuccessful response code: 406","error_class":"ServicePing::SubmitService::SubmissionError","failed_at":1688442523.669093,"retry_count":2,"retried_at":1688529036.9532511}}
{"severity":"ERROR","time":"2023-07-06T07:53:09.303Z","meta.caller_id":"GitlabServicePingWorker","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","meta.client_id":"ip/","exception.class":"NoMethodError","exception.message":"undefined method `licensee_name' for nil:NilClass","exception.backtrace":["ee/lib/gitlab/usage/metrics/instrumentations/licensee_metrics.rb:10:in `block in \u003cclass:LicenseeMetrics\u003e'","lib/gitlab/usage/metrics/instrumentations/generic_metric.rb:37:in `block in value'","lib/gitlab/utils/usage_data.rb:200:in `block in alt_usage_data'","lib/gitlab/utils/usage_data.rb:48:in `with_metadata'","lib/gitlab/utils/usage_data.rb:198:in `alt_usage_data'","lib/gitlab/usage/metrics/instrumentations/generic_metric.rb:36:in `value'","lib/gitlab/usage/metric.rb:21:in `block in with_value'","lib/gitlab/usage/metric.rb:37:in `with_availability'","lib/gitlab/usage/metric.rb:21:in `with_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `call'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `compute_instrumental_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:19:in `block in build'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `map'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `build'","lib/gitlab/usage/service_ping_report.rb:27:in `with_instrumentation_classes'","lib/gitlab/usage/service_ping_report.rb:13:in `block in for'","lib/gitlab/usage/service_ping_report.rb:12:in `for'","lib/service_ping/build_payload.rb:12:in `raw_payload'","lib/service_ping/build_payload.rb:15:in `filtered_usage_data'","lib/service_ping/build_payload.rb:6:in `execute'","app/workers/gitlab_service_ping_worker.rb:40:in `usage_data'","app/workers/gitlab_service_ping_worker.rb:35:in `block in perform'","lib/gitlab/exclusive_lease_helpers.rb:38:in `in_lock'","app/workers/gitlab_service_ping_worker.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/defer_jobs.rb:33:in `call'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:29:in `call'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/sidekiq_middleware/worker_context.rb:11:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/application_context.rb:57:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'","lib/gitlab/with_request_store.rb:17:in `enabling_request_store'","lib/gitlab/with_request_store.rb:10:in `with_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:83:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:82:in `call'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'"],"user.username":null,"tags.program":"sidekiq","tags.locale":"en","tags.feature_category":"service_ping","tags.correlation_id":"c04cd818b6092142aa8d341ec120d3d9","extra.sidekiq":{"retry":3,"queue":"default","version":0,"queue_namespace":"cronjob","dead":false,"args":[],"class":"GitlabServicePingWorker","jid":"b1dc7d026207a53732b2ed08","created_at":1688442424.3723645,"meta.caller_id":"Cronjob","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:f08b08648429384a6800590e6650412b8bf3ae483020a60c2e1d227e13e464e8","size_limiter":"validated","enqueued_at":1688629949.6904507,"error_message":"Unsuccessful response code: 406","error_class":"ServicePing::SubmitService::SubmissionError","failed_at":1688442523.669093,"retry_count":2,"retried_at":1688529036.9532511}}
{"severity":"ERROR","time":"2023-07-06T07:53:09.308Z","meta.caller_id":"GitlabServicePingWorker","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","meta.client_id":"ip/","exception.class":"NoMethodError","exception.message":"undefined method `sha256' for nil:NilClass","exception.backtrace":["ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:42:in `block in value'","lib/gitlab/utils/usage_data.rb:200:in `block in alt_usage_data'","lib/gitlab/utils/usage_data.rb:48:in `with_metadata'","lib/gitlab/utils/usage_data.rb:198:in `alt_usage_data'","ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:40:in `value'","lib/gitlab/usage/metric.rb:21:in `block in with_value'","lib/gitlab/usage/metric.rb:37:in `with_availability'","lib/gitlab/usage/metric.rb:21:in `with_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `call'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `compute_instrumental_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:19:in `block in build'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `map'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `build'","lib/gitlab/usage/service_ping_report.rb:27:in `with_instrumentation_classes'","lib/gitlab/usage/service_ping_report.rb:13:in `block in for'","lib/gitlab/usage/service_ping_report.rb:12:in `for'","lib/service_ping/build_payload.rb:12:in `raw_payload'","lib/service_ping/build_payload.rb:15:in `filtered_usage_data'","lib/service_ping/build_payload.rb:6:in `execute'","app/workers/gitlab_service_ping_worker.rb:40:in `usage_data'","app/workers/gitlab_service_ping_worker.rb:35:in `block in perform'","lib/gitlab/exclusive_lease_helpers.rb:38:in `in_lock'","app/workers/gitlab_service_ping_worker.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/defer_jobs.rb:33:in `call'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:29:in `call'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/sidekiq_middleware/worker_context.rb:11:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/application_context.rb:57:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'","lib/gitlab/with_request_store.rb:17:in `enabling_request_store'","lib/gitlab/with_request_store.rb:10:in `with_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:83:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:82:in `call'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'"],"user.username":null,"tags.program":"sidekiq","tags.locale":"en","tags.feature_category":"service_ping","tags.correlation_id":"c04cd818b6092142aa8d341ec120d3d9","extra.sidekiq":{"retry":3,"queue":"default","version":0,"queue_namespace":"cronjob","dead":false,"args":[],"class":"GitlabServicePingWorker","jid":"b1dc7d026207a53732b2ed08","created_at":1688442424.3723645,"meta.caller_id":"Cronjob","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:f08b08648429384a6800590e6650412b8bf3ae483020a60c2e1d227e13e464e8","size_limiter":"validated","enqueued_at":1688629949.6904507,"error_message":"Unsuccessful response code: 406","error_class":"ServicePing::SubmitService::SubmissionError","failed_at":1688442523.669093,"retry_count":2,"retried_at":1688529036.9532511}}
{"severity":"ERROR","time":"2023-07-06T07:53:09.312Z","meta.caller_id":"GitlabServicePingWorker","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","meta.client_id":"ip/","exception.class":"NoMethodError","exception.message":"undefined method `add_ons' for nil:NilClass","exception.backtrace":["ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:42:in `block in value'","lib/gitlab/utils/usage_data.rb:200:in `block in alt_usage_data'","lib/gitlab/utils/usage_data.rb:48:in `with_metadata'","lib/gitlab/utils/usage_data.rb:198:in `alt_usage_data'","ee/lib/gitlab/usage/metrics/instrumentations/license_metric.rb:40:in `value'","lib/gitlab/usage/metric.rb:21:in `block in with_value'","lib/gitlab/usage/metric.rb:37:in `with_availability'","lib/gitlab/usage/metric.rb:21:in `with_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `call'","lib/gitlab/usage/service_ping/instrumented_payload.rb:36:in `compute_instrumental_value'","lib/gitlab/usage/service_ping/instrumented_payload.rb:19:in `block in build'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `map'","lib/gitlab/usage/service_ping/instrumented_payload.rb:18:in `build'","lib/gitlab/usage/service_ping_report.rb:27:in `with_instrumentation_classes'","lib/gitlab/usage/service_ping_report.rb:13:in `block in for'","lib/gitlab/usage/service_ping_report.rb:12:in `for'","lib/service_ping/build_payload.rb:12:in `raw_payload'","lib/service_ping/build_payload.rb:15:in `filtered_usage_data'","lib/service_ping/build_payload.rb:6:in `execute'","app/workers/gitlab_service_ping_worker.rb:40:in `usage_data'","app/workers/gitlab_service_ping_worker.rb:35:in `block in perform'","lib/gitlab/exclusive_lease_helpers.rb:38:in `in_lock'","app/workers/gitlab_service_ping_worker.rb:31:in `perform'","lib/gitlab/sidekiq_middleware/defer_jobs.rb:33:in `call'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:29:in `call'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/sidekiq_middleware/worker_context.rb:11:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/application_context.rb:57:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'","lib/gitlab/with_request_store.rb:17:in `enabling_request_store'","lib/gitlab/with_request_store.rb:10:in `with_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:83:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:82:in `call'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'"],"user.username":null,"tags.program":"sidekiq","tags.locale":"en","tags.feature_category":"service_ping","tags.correlation_id":"c04cd818b6092142aa8d341ec120d3d9","extra.sidekiq":{"retry":3,"queue":"default","version":0,"queue_namespace":"cronjob","dead":false,"args":[],"class":"GitlabServicePingWorker","jid":"b1dc7d026207a53732b2ed08","created_at":1688442424.3723645,"meta.caller_id":"Cronjob","correlation_id":"c04cd818b6092142aa8d341ec120d3d9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_ping","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:f08b08648429384a6800590e6650412b8bf3ae483020a60c2e1d227e13e464e8","size_limiter":"validated","enqueued_at":1688629949.6904507,"error_message":"Unsuccessful response code: 406","error_class":"ServicePing::SubmitService::SubmissionError","failed_at":1688442523.669093,"retry_count":2,"retried_at":1688529036.9532511}}
{"severity":"ERROR","time":"2023-07-08T21:26:33.916Z","correlation_id":"2aa5d9c6010f3c590faa206fc7c5f9c2","meta.caller_id":"ExternalServiceReactiveCachingWorker","meta.remote_ip":"172.18.0.1","meta.feature_category":"groups_and_projects","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","meta.related_class":"VersionCheck","meta.root_caller_id":"Dashboard::GroupsController#index","exception.class":"Gitlab::HTTP::BlockedUrlError","exception.message":"URL is blocked: Host cannot be resolved or invalid","exception.backtrace":["lib/gitlab/http_connection_adapter.rb:58:in `rescue in validate_url_with_proxy!'","lib/gitlab/http_connection_adapter.rb:50:in `validate_url_with_proxy!'","lib/gitlab/http_connection_adapter.rb:27:in `connection'","lib/gitlab/http.rb:65:in `perform_request'","lib/gitlab/http.rb:85:in `try_get'","lib/version_check.rb:71:in `calculate_reactive_cache'","app/models/concerns/reactive_caching.rb:94:in `block (2 levels) in exclusively_update_reactive_cache!'","app/models/concerns/reactive_caching.rb:153:in `enqueuing_update'","app/models/concerns/reactive_caching.rb:93:in `block in exclusively_update_reactive_cache!'","app/models/concerns/reactive_caching.rb:143:in `locking_reactive_cache'","app/models/concerns/reactive_caching.rb:89:in `exclusively_update_reactive_cache!'","app/workers/concerns/reactive_cacheable_worker.rb:35:in `perform'","lib/gitlab/sidekiq_middleware/defer_jobs.rb:33:in `call'","lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:29:in `call'","lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform'","lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'","lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call'","lib/gitlab/application_context.rb:118:in `block in use'","lib/gitlab/application_context.rb:118:in `use'","lib/gitlab/application_context.rb:57:in `with_context'","lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call'","lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'","lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call'","lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'","lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'","lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'","lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'","lib/gitlab/with_request_store.rb:17:in `enabling_request_store'","lib/gitlab/with_request_store.rb:10:in `with_request_store'","lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:83:in `block in call'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `block in instrument'","lib/gitlab/metrics/background_transaction.rb:33:in `run'","lib/gitlab/sidekiq_middleware/server_metrics.rb:110:in `instrument'","lib/gitlab/sidekiq_middleware/server_metrics.rb:82:in `call'","lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call'","lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job'","lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call'","lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call'","lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call'"],"exception.cause_class":"Gitlab::UrlBlocker::BlockedUrlError","user.username":"WizzardMaker","tags.program":"sidekiq","tags.locale":"en","tags.feature_category":"groups_and_projects","tags.correlation_id":"2aa5d9c6010f3c590faa206fc7c5f9c2","extra.sidekiq":{"retry":3,"queue":"default","version":0,"class":"ExternalServiceReactiveCachingWorker","args":["VersionCheck","[FILTERED]"],"jid":"6db771bce31a00d4c8e3683f","created_at":1688808373.193478,"correlation_id":"2aa5d9c6010f3c590faa206fc7c5f9c2","meta.caller_id":"ExternalServiceReactiveCachingWorker","meta.remote_ip":"172.18.0.1","meta.feature_category":"groups_and_projects","meta.user":"WizzardMaker","meta.user_id":5,"meta.client_id":"user/5","meta.related_class":"VersionCheck","meta.root_caller_id":"Dashboard::GroupsController#index","worker_data_consistency":"always","size_limiter":"validated","scheduled_at":1688851573.1934164,"idempotency_key":"resque:gitlab:duplicate:default:28ead719eb8caefc84a596757ef3850b6aa8799ae42215a3ad988ecafc7925c7","enqueued_at":1688851577.8812397}}
==> /var/log/gitlab/grafana/state <==
==> /var/log/gitlab/grafana/current <==
2023-06-29_11:29:02.97286 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2023-06-29T11:29:02.972856Z","queryType":"arrow"}
2023-06-29_11:29:02.97286 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2023-06-29T11:29:02.972860Z","queryType":"annotations"}
2023-06-29_11:29:02.97287 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2023-06-29T11:29:02.972864Z","queryType":"table_static"}
2023-06-29_11:29:02.97287 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2023-06-29T11:29:02.972867Z","queryType":"random_walk_with_error"}
2023-06-29_11:29:02.97287 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2023-06-29T11:29:02.972871Z","queryType":"server_error_500"}
2023-06-29_11:29:02.97287 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2023-06-29T11:29:02.972874Z","queryType":"logs"}
2023-06-29_11:29:02.97288 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2023-06-29T11:29:02.972877Z","queryType":"node_graph"}
2023-06-29_11:29:02.97288 {"@level":"debug","@message":"datasource: registering query type fallback handler","@timestamp":"2023-06-29T11:29:02.972881Z"}
2023-06-29_11:29:03.34040 t=2023-06-29T11:29:03+0000 lvl=eror msg="Failed to read plugin provisioning files from directory" logger=provisioning.plugins path=/var/opt/gitlab/grafana/provisioning/plugins error="open /var/opt/gitlab/grafana/provisioning/plugins: no such file or directory"
2023-06-29_11:29:03.34978 t=2023-06-29T11:29:03+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=127.0.0.1:3000 protocol=http subUrl=/-/grafana socket=
==> /var/log/gitlab/redis-exporter/current <==
2023-07-11_18:28:45.94744 time="2023-07-11T18:28:45Z" level=info msg="Redis Metrics Exporter 1.46.0 build date: sha1: Go: go1.19.8 GOOS: linux GOARCH: amd64"
2023-07-11_18:28:45.96961 time="2023-07-11T18:28:45Z" level=info msg="Providing metrics at localhost:9121/metrics"
==> /var/log/gitlab/gitlab-shell/gitlab-shell.log <==
{"content_length_bytes":89,"correlation_id":"01H4N34S5J8AGKHXN3QVBFQ7N4","duration_ms":62,"level":"info","method":"GET","msg":"Finished HTTP request","status":200,"time":"2023-07-06T07:54:53Z","url":"http://unix/api/v4/internal/check"}
{"content_length_bytes":89,"correlation_id":"01H4N3BF8TJNK3HHV407MM5J14","duration_ms":51,"level":"info","method":"GET","msg":"Finished HTTP request","status":200,"time":"2023-07-06T07:58:32Z","url":"http://unix/api/v4/internal/check"}
==> /var/log/gitlab/sidekiq/state <==
==> /var/log/gitlab/sidekiq/current <==
{"severity":"INFO","time":"2023-07-11T22:01:01.670Z","retry":0,"queue":"default","version":0,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"941f28f219f16d7ea8a4d6c8","created_at":"2023-07-11T22:01:01.558Z","meta.caller_id":"Cronjob","correlation_id":"fe76912e93ef7a88f3dfa17846e78244","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2023-07-11T22:01:01.559Z","job_size_bytes":2,"pid":724,"message":"ScheduleMergeRequestCleanupRefsWorker JID-941f28f219f16d7ea8a4d6c8: done: 0.096376 sec","job_status":"done","scheduling_latency_s":0.014952,"redis_calls":2,"redis_duration_s":0.053012,"redis_read_bytes":2,"redis_write_bytes":186,"redis_queues_calls":2,"redis_queues_duration_s":0.053012,"redis_queues_read_bytes":2,"redis_queues_write_bytes":186,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00512,"mem_objects":2949,"mem_bytes":385160,"mem_mallocs":1073,"mem_total_bytes":503120,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":0.096376,"completed_at":"2023-07-11T22:01:01.670Z","load_balancing_strategy":"primary","db_duration_s":0.0}
{"severity":"INFO","time":"2023-07-11T22:01:01.680Z","retry":0,"queue":"default","version":0,"queue_namespace":"cronjob","args":[],"class":"Metrics::GlobalMetricsUpdateWorker","jid":"83684f006ebb7e40fece2029","created_at":"2023-07-11T22:01:01.574Z","meta.caller_id":"Cronjob","correlation_id":"1392eefb7e2fab6af166e9db743a2f14","meta.root_caller_id":"Cronjob","meta.feature_category":"metrics","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:default:0ef60401f7cd15675ace9e5ac4456efd82434f42e048584516e1e7a85b24e5ba","size_limiter":"validated","enqueued_at":"2023-07-11T22:01:01.575Z","job_size_bytes":2,"pid":724,"message":"Metrics::GlobalMetricsUpdateWorker JID-83684f006ebb7e40fece2029: done: 0.101888 sec","job_status":"done","scheduling_latency_s":0.002502,"redis_calls":4,"redis_duration_s":0.055722,"redis_read_bytes":4,"redis_write_bytes":494,"redis_queues_calls":2,"redis_queues_duration_s":0.000264,"redis_queues_read_bytes":2,"redis_queues_write_bytes":186,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.055458,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":308,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.004702,"mem_objects":3042,"mem_bytes":468024,"mem_mallocs":1086,"mem_total_bytes":589704,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":0.101888,"completed_at":"2023-07-11T22:01:01.680Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0}
{"severity":"INFO","time":"2023-07-11T22:01:01.772Z","retry":0,"queue":"default","version":0,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3f7a16719584c9ab41de1358","created_at":"2023-07-11T22:01:01.682Z","meta.caller_id":"Cronjob","correlation_id":"e5d6d230bcc1ba22f8b77d3d440e1ad9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2023-07-11T22:01:01.682Z","job_size_bytes":2,"pid":724,"message":"Database::BatchedBackgroundMigrationWorker JID-3f7a16719584c9ab41de1358: start","job_status":"start","scheduling_latency_s":0.089611}
{"severity":"INFO","time":"2023-07-11T22:01:01.808Z","retry":0,"queue":"default","version":0,"queue_namespace":"cronjob","args":[],"class":"Geo::SidekiqCronConfigWorker","jid":"e12828e650b98527bf69253a","created_at":"2023-07-11T22:01:01.522Z","meta.caller_id":"Cronjob","correlation_id":"fd2974abef2d7b37505b5164b0639350","meta.root_caller_id":"Cronjob","meta.feature_category":"geo_replication","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:ee3c9c9863d06b67487cd0dcfc522a28b6dc54e34bf5abd078467931033818a0","size_limiter":"validated","enqueued_at":"2023-07-11T22:01:01.523Z","job_size_bytes":2,"pid":724,"message":"Geo::SidekiqCronConfigWorker JID-e12828e650b98527bf69253a: done: 0.256185 sec","job_status":"done","scheduling_latency_s":0.0289,"redis_calls":163,"redis_duration_s":0.186317,"redis_read_bytes":59785,"redis_write_bytes":10827,"redis_queues_calls":163,"redis_queues_duration_s":0.186317,"redis_queues_read_bytes":59785,"redis_queues_write_bytes":10827,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.099912,"mem_objects":87463,"mem_bytes":85084800,"mem_mallocs":26338,"mem_total_bytes":88583320,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":0.256185,"completed_at":"2023-07-11T22:01:01.808Z","load_balancing_strategy":"primary","db_duration_s":0.0}
{"severity":"INFO","time":"2023-07-11T22:01:01.681Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"skipping migration execution for database that shares database configuration with another database","retry":0}
{"severity":"INFO","time":"2023-07-11T22:01:01.682Z","retry":0,"queue":"default","version":0,"queue_namespace":"cronjob","args":[],"class":"Search::IndexCurationWorker","jid":"998cb532332d4bf81893c266","created_at":"2023-07-11T22:01:01.611Z","meta.caller_id":"Cronjob","correlation_id":"9f71fa8756d0bca0b61ea48562d01f92","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:f1b35de73fa09aee87da69fc70f7f498516e5b000ec47c894d20a533c302983e","size_limiter":"validated","enqueued_at":"2023-07-11T22:01:01.612Z","job_size_bytes":2,"pid":724,"message":"Search::IndexCurationWorker JID-998cb532332d4bf81893c266: start","job_status":"start","scheduling_latency_s":0.069551}
{"severity":"INFO","time":"2023-07-11T22:01:01.814Z","retry":0,"queue":"default","version":0,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"227edf01fb5b6aa3a43e8c7e","created_at":"2023-07-11T22:01:01.586Z","meta.caller_id":"Cronjob","correlation_id":"c20ea674c4db7f18e0f618c2b7b76967","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2023-07-11T22:01:01.586Z","job_size_bytes":2,"pid":724,"message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-227edf01fb5b6aa3a43e8c7e: done: 0.22427 sec","job_status":"done","scheduling_latency_s":0.003477,"redis_calls":2,"redis_duration_s":0.001883,"redis_read_bytes":2,"redis_write_bytes":186,"redis_queues_calls":2,"redis_queues_duration_s":0.001883,"redis_queues_read_bytes":2,"redis_queues_write_bytes":186,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.002449,"mem_objects":1387,"mem_bytes":111880,"mem_mallocs":301,"mem_total_bytes":167360,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":0.22427,"completed_at":"2023-07-11T22:01:01.814Z","load_balancing_strategy":"primary","db_duration_s":0.0}
{"severity":"INFO","time":"2023-07-11T22:01:01.810Z","retry":0,"queue":"default","version":0,"queue_namespace":"cronjob","args":[],"class":"ElasticIndexBulkCronWorker","jid":"401bab93bcb7ddac9bf5f3a6","created_at":"2023-07-11T22:01:01.601Z","meta.caller_id":"Cronjob","correlation_id":"73330aea1deff41d29b7f099f12ec377","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:default:f252f68f3cc1cae1877f9a0e1f5b889102a68a9e335a3a5a9e683c7bdf0507f5","size_limiter":"validated","enqueued_at":"2023-07-11T22:01:01.602Z","job_size_bytes":2,"pid":724,"message":"ElasticIndexBulkCronWorker JID-401bab93bcb7ddac9bf5f3a6: done: 0.204303 sec","job_status":"done","scheduling_latency_s":0.004572,"redis_calls":2,"redis_duration_s":0.000283,"redis_read_bytes":2,"redis_write_bytes":186,"redis_queues_calls":2,"redis_queues_duration_s":0.000283,"redis_queues_read_bytes":2,"redis_queues_write_bytes":186,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":1,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.096,"db_main_duration_s":0.096,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0055,"mem_objects":3289,"mem_bytes":399504,"mem_mallocs":1294,"mem_total_bytes":531064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":0.204303,"completed_at":"2023-07-11T22:01:01.810Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.095865}
{"severity":"INFO","time":"2023-07-11T22:01:01.832Z","retry":0,"queue":"default","version":0,"queue_namespace":"cronjob","args":[],"class":"Search::IndexCurationWorker","jid":"998cb532332d4bf81893c266","created_at":"2023-07-11T22:01:01.611Z","meta.caller_id":"Cronjob","correlation_id":"9f71fa8756d0bca0b61ea48562d01f92","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","worker_data_consistency":"always","idempotency_key":"resque:gitlab:duplicate:default:f1b35de73fa09aee87da69fc70f7f498516e5b000ec47c894d20a533c302983e","size_limiter":"validated","enqueued_at":"2023-07-11T22:01:01.612Z","job_size_bytes":2,"pid":724,"message":"Search::IndexCurationWorker JID-998cb532332d4bf81893c266: done: 0.150505 sec","job_status":"done","scheduling_latency_s":0.069551,"redis_calls":2,"redis_duration_s":0.000258,"redis_read_bytes":2,"redis_write_bytes":186,"redis_queues_calls":2,"redis_queues_duration_s":0.000258,"redis_queues_read_bytes":2,"redis_queues_write_bytes":186,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.001994,"mem_objects":1432,"mem_bytes":113040,"mem_mallocs":321,"mem_total_bytes":170320,"worker_id":"sidekiq_0","rate_limiting_gates":[],"duration_s":0.150505,"completed_at":"2023-07-11T22:01:01.832Z","load_balancing_strategy":"primary","db_duration_s":0.0}
{"severity":"INFO","time":"2023-07-11T22:20:04.906Z","retry":0,"queue":"default","version":0,"queue_namespace":"cronjob","class":"Ci::StuckBuilds::DropRunningWorker","args":[],"jid":"2fa5e3fb6a0ebe746cd22fbc","created_at":"2023-07-11T22:00:03.228Z","meta.caller_id":"Cronjob","correlation_id":"750cd4cd9b944f1e4558ec9454827f34","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","size_limiter":"validated","scheduled_at":"2023-07-11T22:20:03.228Z","idempotency_key":"resque:gitlab:duplicate:default:ea2a97d1a67ae4af54a74ae3b0ab2d60e87ef116206ede972bdbbdafa0a53cd4","enqueued_at":"2023-07-11T22:20:04.905Z","job_size_bytes":2,"pid":724,"message":"Ci::StuckBuilds::DropRunningWorker JID-2fa5e3fb6a0ebe746cd22fbc: start","job_status":"start","scheduling_latency_s":0.001329,"enqueue_latency_s":1.676986}
==> /var/log/gitlab/gitlab-workhorse/state <==
==> /var/log/gitlab/gitlab-workhorse/current <==
{"correlation_id":"01H53GTTKP4FKPPQJ0PB2JERJ2","duration_ms":10001,"error":"badgateway: failed to receive response: context canceled","level":"error","method":"GET","msg":"","time":"2023-07-11T22:23:38Z","uri":"/help"}
{"content_type":"text/plain","correlation_id":"01H53GTTKP4FKPPQJ0PB2JERJ2","duration_ms":10001,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":499,"system":"http","time":"2023-07-11T22:23:38Z","ttfb_ms":10001,"uri":"/help","user_agent":"curl/8.0.1-DEV","written_bytes":24}
{"correlation_id":"01H53GWZ9PMXWWA3VKX7V9SSB9","duration_ms":10001,"error":"badgateway: failed to receive response: context canceled","level":"error","method":"GET","msg":"","time":"2023-07-11T22:24:49Z","uri":"/help"}
{"content_type":"text/plain","correlation_id":"01H53GWZ9PMXWWA3VKX7V9SSB9","duration_ms":10001,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":499,"system":"http","time":"2023-07-11T22:24:49Z","ttfb_ms":10001,"uri":"/help","user_agent":"curl/8.0.1-DEV","written_bytes":24}
{"correlation_id":"01H53GWM5HD7S2VN0QBMCRS7B1","duration_ms":43399,"error":"badgateway: failed to receive response: context canceled","level":"error","method":"GET","msg":"","time":"2023-07-11T22:25:11Z","uri":"/s4-plugins/netengine/-/milestones?state=opened"}
{"content_type":"text/plain","correlation_id":"01H53GWM5HD7S2VN0QBMCRS7B1","duration_ms":43399,"host":"localhost:8010","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"http://**REDACTED**/s4-plugins/netengine/-/milestones","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":499,"system":"http","time":"2023-07-11T22:25:11Z","ttfb_ms":43399,"uri":"/s4-plugins/netengine/-/milestones?state=opened","user_agent":"Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://webmaster.petalsearch.com/site/petalbot)","written_bytes":24}
{"correlation_id":"01H53GZ44FGPQN8Y61H6MBF8JD","duration_ms":10000,"error":"badgateway: failed to receive response: context canceled","level":"error","method":"GET","msg":"","time":"2023-07-11T22:25:59Z","uri":"/help"}
{"content_type":"text/plain","correlation_id":"01H53GZ44FGPQN8Y61H6MBF8JD","duration_ms":10000,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":499,"system":"http","time":"2023-07-11T22:25:59Z","ttfb_ms":10000,"uri":"/help","user_agent":"curl/8.0.1-DEV","written_bytes":24}
{"correlation_id":"01H53H18VFPMDMN5ZD1SPYT5GW","duration_ms":10000,"error":"badgateway: failed to receive response: context canceled","level":"error","method":"GET","msg":"","time":"2023-07-11T22:27:10Z","uri":"/help"}
{"content_type":"text/plain","correlation_id":"01H53H18VFPMDMN5ZD1SPYT5GW","duration_ms":10001,"host":"localhost","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","status":499,"system":"http","time":"2023-07-11T22:27:10Z","ttfb_ms":10001,"uri":"/help","user_agent":"curl/8.0.1-DEV","written_bytes":24}
==> /var/log/gitlab/postgres-exporter/state <==
==> /var/log/gitlab/postgres-exporter/current <==
2023-07-11_18:29:52.86548 ts=2023-07-11T18:29:52.865Z caller=server.go:74 level=info msg="Established new database connection" fingerprint=/var/opt/gitlab/postgresql:5432
2023-07-11_18:29:52.86656 ts=2023-07-11T18:29:52.866Z caller=collector.go:196 level=error msg="collector failed" name=database duration_seconds=0.001010681 err="pq: the database system is starting up"
2023-07-11_18:29:52.86752 ts=2023-07-11T18:29:52.867Z caller=collector.go:196 level=error msg="collector failed" name=bgwriter duration_seconds=0.001891113 err="pq: the database system is starting up"
2023-07-11_18:29:53.86694 ts=2023-07-11T18:29:53.866Z caller=postgres_exporter.go:714 level=error err="Error opening connection to database (host=/var/opt/gitlab/postgresql%20user=gitlab-psql%20database=gitlabhq_production): pq: the database system is starting up"
2023-07-11_18:30:07.86626 ts=2023-07-11T18:30:07.866Z caller=server.go:74 level=info msg="Established new database connection" fingerprint=/var/opt/gitlab/postgresql:5432
2023-07-11_18:30:07.86722 ts=2023-07-11T18:30:07.867Z caller=collector.go:196 level=error msg="collector failed" name=database duration_seconds=0.000900216 err="pq: the database system is starting up"
2023-07-11_18:30:07.86787 ts=2023-07-11T18:30:07.867Z caller=collector.go:196 level=error msg="collector failed" name=bgwriter duration_seconds=0.001421489 err="pq: the database system is starting up"
2023-07-11_18:30:08.86735 ts=2023-07-11T18:30:08.867Z caller=postgres_exporter.go:714 level=error err="Error opening connection to database (host=/var/opt/gitlab/postgresql%20user=gitlab-psql%20database=gitlabhq_production): pq: the database system is starting up"
2023-07-11_18:30:22.86635 ts=2023-07-11T18:30:22.866Z caller=server.go:74 level=info msg="Established new database connection" fingerprint=/var/opt/gitlab/postgresql:5432
2023-07-11_18:30:22.96948 ts=2023-07-11T18:30:22.969Z caller=postgres_exporter.go:645 level=info msg="Semantic version changed" server=/var/opt/gitlab/postgresql:5432 from=0.0.0 to=13.11.0
==> /var/log/gitlab/puma/puma_stdout.log <==
{"timestamp":"2023-07-09T10:23:56.167Z","pid":737,"message":"- Worker 7 (PID: 9326) booted in 0.59s, phase: 0"}
{"timestamp":"2023-07-09T12:03:56.407Z","pid":737,"message":"- Worker 4 (PID: 12257) booted in 0.71s, phase: 0"}
{"timestamp":"2023-07-09T12:18:54.549Z","pid":737,"message":"- Worker 3 (PID: 12711) booted in 0.45s, phase: 0"}
{"timestamp":"2023-07-11T07:52:57.203Z","pid":737,"message":"! Terminating timed out worker (worker failed to check in within 60 seconds): 26698"}
{"timestamp":"2023-07-11T07:53:32.425Z","pid":737,"message":"! Terminating timed out worker (worker failed to check in within 60 seconds): 1532"}
{"timestamp":"2023-07-11T07:53:37.591Z","pid":737,"message":"! Terminating timed out worker (worker failed to check in within 60 seconds): 9326"}
{"timestamp":"2023-07-11T07:53:41.723Z","pid":737,"message":"! Terminating timed out worker (worker failed to check in within 60 seconds): 31307"}
=== puma startup: 2023-07-11 08:07:35 +0000 ===
=== puma startup: 2023-07-11 18:31:22 +0000 ===
{"timestamp":"2023-07-11T18:31:22.543Z","pid":654,"message":"- Worker 0 (PID: 778) booted in 0.35s, phase: 0"}
==> /var/log/gitlab/puma/state <==
==> /var/log/gitlab/puma/current <==
2023-07-11_18:31:22.04702 {"timestamp":"2023-07-11T18:31:22.046Z","pid":654,"message":"* Listening on unix:///var/opt/gitlab/gitlab-rails/sockets/gitlab.socket"}
2023-07-11_18:31:22.04722 {"timestamp":"2023-07-11T18:31:22.047Z","pid":654,"message":"* Listening on http://127.0.0.1:8080"}
2023-07-11_18:31:22.04731 {"timestamp":"2023-07-11T18:31:22.047Z","pid":654,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}
2023-07-11_18:31:22.04740 {"timestamp":"2023-07-11T18:31:22.047Z","pid":654,"message":"! #\u003cThread:0x00007fa29cc27d88 /opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/rack-timeout-0.6.3/lib/rack/timeout/support/scheduler.rb:73 sleep\u003e - /opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/rack-timeout-0.6.3/lib/rack/timeout/support/scheduler.rb:91:in `sleep'"}
2023-07-11_18:31:22.04748 {"timestamp":"2023-07-11T18:31:22.047Z","pid":654,"message":"! #\u003cThread:0x00007fa290014980 /opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/sentry-ruby-5.8.0/lib/sentry/session_flusher.rb:81 sleep\u003e - /opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/sentry-ruby-5.8.0/lib/sentry/session_flusher.rb:83:in `sleep'"}
2023-07-11_18:31:22.04765 {"timestamp":"2023-07-11T18:31:22.047Z","pid":654,"message":"Use Ctrl-C to stop"}
2023-07-11_18:31:22.04768 {"timestamp":"2023-07-11T18:31:22.047Z","pid":654,"message":"! WARNING: Detected running cluster mode with 1 worker."}
2023-07-11_18:31:22.04769 {"timestamp":"2023-07-11T18:31:22.047Z","pid":654,"message":"! Running Puma in cluster mode with a single worker is often a misconfiguration."}
2023-07-11_18:31:22.04773 {"timestamp":"2023-07-11T18:31:22.047Z","pid":654,"message":"! Consider running Puma in single-mode (workers = 0) in order to reduce memory overhead."}
2023-07-11_18:31:22.04776 {"timestamp":"2023-07-11T18:31:22.047Z","pid":654,"message":"! Set the `silence_single_worker_warning` option to silence this warning message."}
==> /var/log/gitlab/puma/puma_stderr.log <==
source=rack-timeout id=01H5244293JYVR32HYVQ38Z11D timeout=600000ms service=600000ms state=timed_out at=error
source=rack-timeout id=01H52466XB2DY2QBBT25CXP73D timeout=600000ms service=600000ms state=timed_out at=error
source=rack-timeout id=01H5248AKH1MWEVY2W00YH4A4G timeout=600000ms service=600000ms state=timed_out at=error
source=rack-timeout id=01H5248BFVZ43C333XK2Q4HZBJ timeout=600000ms service=600001ms state=timed_out at=error
=== puma startup: 2023-07-11 18:31:22 +0000 ===
source=rack-timeout id=01H53FJ1D4B4WGV8CH06PHWFTB timeout=600000ms service=600000ms state=timed_out at=error
source=rack-timeout id=01H53FJ7AQ1EDTG485BSPFTYJB timeout=600000ms service=600000ms state=timed_out at=error
source=rack-timeout id=01H53FM679YTC4PAYG5TDTW10Q timeout=600000ms service=600000ms state=timed_out at=error
source=rack-timeout id=01H53FPAV7YXP23EZPZ2QQN63R timeout=600000ms service=600000ms state=timed_out at=error
source=rack-timeout id=01H53FRFET717RYF58GAAJASH6 timeout=600000ms service=600000ms state=timed_out at=error
==> /var/log/gitlab/gitaly/gitaly_ruby_json.log <==
{"type":"gitaly-ruby","grpc.start_time":"2023-06-29T11:05:53Z","grpc.time_ms":0.163,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1184,"correlation_id":"66e07d6e399b72a0eb2e3fcea59b72bb","time":"2023-06-29T11:05:53.289Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-06-29T11:05:53Z","grpc.time_ms":0.208,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1185,"correlation_id":"67f8add28fc5c0c270c281e2e8cb4df8","time":"2023-06-29T11:05:53.289Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-06-29T11:06:08Z","grpc.time_ms":0.131,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1185,"correlation_id":"8a1903573351a7b19fc615d5dc46222d","time":"2023-06-29T11:06:08.291Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-06-29T11:06:08Z","grpc.time_ms":0.136,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1184,"correlation_id":"16921497dfdf06eb90ac1d4843d9393d","time":"2023-06-29T11:06:08.291Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-06-29T11:06:23Z","grpc.time_ms":0.115,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1184,"correlation_id":"2314b7a0c064989cb2d5d94cfc4da93e","time":"2023-06-29T11:06:23.293Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-06-29T11:06:23Z","grpc.time_ms":0.14,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1185,"correlation_id":"2faad52d605c268fd796f148ac680c39","time":"2023-06-29T11:06:23.293Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-06-29T11:06:38Z","grpc.time_ms":0.123,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1185,"correlation_id":"19a0602cf2fc9b215f5923605cc7dccd","time":"2023-06-29T11:06:38.295Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-06-29T11:06:38Z","grpc.time_ms":0.125,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1184,"correlation_id":"c772357d69a08e12262a1ae51a86dfb0","time":"2023-06-29T11:06:38.295Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-06-29T11:06:53Z","grpc.time_ms":0.108,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1185,"correlation_id":"4f9f6afa89b5ae1b4350b0eebd996d92","time":"2023-06-29T11:06:53.297Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-06-29T11:06:53Z","grpc.time_ms":0.148,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1184,"correlation_id":"57d754a199edf258d5cbd474ba08da95","time":"2023-06-29T11:06:53.297Z"}
==> /var/log/gitlab/alertmanager/state <==
==> /var/log/gitlab/alertmanager/current <==
2023-07-11_18:28:48.99360 ts=2023-07-11T18:28:48.993Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=master, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)"
2023-07-11_18:28:48.99367 ts=2023-07-11T18:28:48.993Z caller=main.go:241 level=info build_context="(go=go1.19.8, user=GitLab-Omnibus, date=)"
2023-07-11_18:28:48.99621 ts=2023-07-11T18:28:48.996Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.18.0.2 port=9094
2023-07-11_18:28:49.00556 ts=2023-07-11T18:28:49.005Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s
2023-07-11_18:28:49.04502 ts=2023-07-11T18:28:49.044Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/var/opt/gitlab/alertmanager/alertmanager.yml
2023-07-11_18:28:49.05822 ts=2023-07-11T18:28:49.058Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/var/opt/gitlab/alertmanager/alertmanager.yml
2023-07-11_18:28:49.10531 ts=2023-07-11T18:28:49.063Z caller=tls_config.go:232 level=info msg="Listening on" address=127.0.0.1:9093
2023-07-11_18:28:49.10533 ts=2023-07-11T18:28:49.105Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=127.0.0.1:9093
2023-07-11_18:28:51.00583 ts=2023-07-11T18:28:51.005Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000209024s
2023-07-11_18:28:59.00810 ts=2023-07-11T18:28:59.008Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002507878s
==> /var/log/gitlab/postgresql/state <==
==> /var/log/gitlab/postgresql/current <==
2023-07-11_18:30:19.25702 FATAL: the database system is starting up
2023-07-11_18:30:20.99848 LOG: database system was not properly shut down; automatic recovery in progress
2023-07-11_18:30:21.09308 LOG: invalid record length at 0/480111B0: wanted 24, got 0
2023-07-11_18:30:21.09310 LOG: redo is not required
2023-07-11_18:30:21.52190 FATAL: the database system is starting up
2023-07-11_18:30:21.55122 FATAL: the database system is starting up
2023-07-11_18:30:21.55227 FATAL: the database system is starting up
2023-07-11_18:30:21.68676 FATAL: the database system is starting up
2023-07-11_18:30:21.69289 FATAL: the database system is starting up
2023-07-11_18:30:21.70278 LOG: database system is ready to accept connections
==> /var/log/gitlab/nginx/gitlab_registry_error.log <==
==> /var/log/gitlab/nginx/gitlab_registry_access.log <==
==> /var/log/gitlab/nginx/access.log <==
==> /var/log/gitlab/nginx/gitlab_error.log <==
2022/03/22 22:14:56 [crit] 650#0: *493 connect() to unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket failed (2: No such file or directory) while connecting to upstream, client: 172.18.0.1, server: **REDACTED**, request: "POST /api/v4/jobs/request HTTP/1.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/api/v4/jobs/request", host: "localhost:8010"
2022/03/22 22:21:04 [crit] 645#0: *349 connect() to unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket failed (2: No such file or directory) while connecting to upstream, client: 172.18.0.1, server: **REDACTED**, request: "POST /api/v4/jobs/request HTTP/1.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/api/v4/jobs/request", host: "localhost:8010"
2022/06/24 14:11:05 [crit] 399#0: *1723197 connect() to unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket failed (2: No such file or directory) while connecting to upstream, client: 172.18.0.1, server: **REDACTED**, request: "POST /api/v4/jobs/request HTTP/1.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/api/v4/jobs/request", host: "localhost:8010"
2022/06/24 14:17:47 [crit] 399#0: *186 connect() to unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket failed (2: No such file or directory) while connecting to upstream, client: 172.18.0.1, server: **REDACTED**, request: "POST /api/v4/jobs/request HTTP/1.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/api/v4/jobs/request", host: "localhost:8010"
2022/09/04 18:37:54 [crit] 383#0: *278 connect() to unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket failed (2: No such file or directory) while connecting to upstream, client: 172.18.0.1, server: **REDACTED**, request: "POST /api/v4/jobs/request HTTP/1.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/api/v4/jobs/request", host: "localhost:8010"
2023/02/19 00:25:26 [crit] 413#0: *21656 connect() to unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket failed (2: No such file or directory) while connecting to upstream, client: 127.0.0.1, server: **REDACTED**, request: "GET /help HTTP/1.1", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/help", host: "localhost"
2023/02/19 00:25:26 [crit] 413#0: *21658 connect() to unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket failed (2: No such file or directory) while connecting to upstream, client: 172.18.0.1, server: **REDACTED**, request: "POST /api/v4/jobs/request HTTP/1.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/api/v4/jobs/request", host: "localhost:8010"
2023/06/29 11:07:10 [crit] 654#0: *3404 connect() to unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket failed (2: No such file or directory) while connecting to upstream, client: 172.18.0.1, server: **REDACTED**, request: "POST /api/v4/jobs/request HTTP/1.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/api/v4/jobs/request", host: "localhost:8010"
==> /var/log/gitlab/nginx/current <==
==> /var/log/gitlab/nginx/gitlab_access.log <==
127.0.0.1 - - [11/Jul/2023:22:17:47 +0000] "GET /help HTTP/1.1" 499 0 "" "curl/8.0.1-DEV" -
127.0.0.1 - - [11/Jul/2023:22:25:59 +0000] "GET /help HTTP/1.1" 499 0 "" "curl/8.0.1-DEV" -
127.0.0.1 - - [11/Jul/2023:22:27:10 +0000] "GET /help HTTP/1.1" 499 0 "" "curl/8.0.1-DEV" -
==> /var/log/gitlab/nginx/error.log <==
==> /var/log/gitlab/gitlab-kas/state <==
==> /var/log/gitlab/gitlab-kas/current <==
2023-07-11_18:28:38.49500 {"level":"info","time":"2023-07-11T18:28:38.467Z","msg":"API endpoint is up","net_network":"tcp","net_address":"127.0.0.1:8153"}
2023-07-11_18:28:38.49514 {"level":"info","time":"2023-07-11T18:28:38.467Z","msg":"Private API endpoint is up","net_network":"tcp","net_address":"127.0.0.1:8155"}
2023-07-11_18:28:38.49521 {"level":"info","time":"2023-07-11T18:28:38.467Z","msg":"Agentk API endpoint is up","net_network":"tcp","net_address":"127.0.0.1:8150","is_websocket":true}
2023-07-11_18:28:38.52969 {"level":"info","time":"2023-07-11T18:28:38.529Z","msg":"Kubernetes API endpoint is up","mod_name":"kubernetes_api","net_network":"tcp","net_address":"127.0.0.1:8154"}
2023-07-11_18:28:38.52980 {"level":"info","time":"2023-07-11T18:28:38.529Z","msg":"Observability endpoint is up","mod_name":"observability","net_network":"tcp","net_address":"127.0.0.1:8151"}
==> /var/log/gitlab/gitlab-exporter/state <==
==> /var/log/gitlab/gitlab-exporter/current <==
2023-07-11_22:00:11.13900 127.0.0.1 - - [11/Jul/2023:22:00:10 UTC] "GET /database HTTP/1.1" 200 1795
2023-07-11_22:00:11.13917 - -> /database
2023-07-11_22:00:15.08677 127.0.0.1 - - [11/Jul/2023:22:00:15 UTC] "GET /ruby HTTP/1.1" 200 1074
2023-07-11_22:00:15.08682 - -> /ruby
2023-07-11_22:00:17.69650 127.0.0.1 - - [11/Jul/2023:22:00:17 UTC] "GET /sidekiq HTTP/1.1" 200 145992
2023-07-11_22:00:17.69654 - -> /sidekiq
2023-07-11_22:00:30.08460 127.0.0.1 - - [11/Jul/2023:22:00:30 UTC] "GET /ruby HTTP/1.1" 200 1076
2023-07-11_22:00:30.08468 - -> /ruby
2023-07-11_22:00:32.69745 127.0.0.1 - - [11/Jul/2023:22:00:32 UTC] "GET /sidekiq HTTP/1.1" 200 145992
2023-07-11_22:00:32.69750 - -> /sidekiq
==> /var/log/gitlab/redis/state <==
==> /var/log/gitlab/redis/current <==
2023-07-11_21:53:36.32608 5548:C 11 Jul 2023 21:53:36.326 * DB saved on disk
2023-07-11_21:53:36.32637 5548:C 11 Jul 2023 21:53:36.326 * RDB: 1 MB of memory used by copy-on-write
2023-07-11_21:53:36.39912 276:M 11 Jul 2023 21:53:36.399 * Background saving terminated with success
2023-07-11_21:58:37.00012 276:M 11 Jul 2023 21:58:37.000 * 10 changes in 300 seconds. Saving...
2023-07-11_21:58:37.00045 276:M 11 Jul 2023 21:58:37.000 * Background saving started by pid 5657
2023-07-11_21:58:37.20708 5657:C 11 Jul 2023 21:58:37.207 * DB saved on disk
2023-07-11_21:58:37.20735 5657:C 11 Jul 2023 21:58:37.207 * RDB: 1 MB of memory used by copy-on-write
2023-07-11_21:58:37.30126 276:M 11 Jul 2023 21:58:37.301 * Background saving terminated with success
2023-07-11_22:03:38.04207 276:M 11 Jul 2023 22:03:38.042 * 10 changes in 300 seconds. Saving...
2023-07-11_22:03:38.04267 276:M 11 Jul 2023 22:03:38.042 * Background saving started by pid 5756
==> /var/log/gitlab/registry/state <==
==> /var/log/gitlab/registry/current <==
2022-09-04_18:38:22.98877 time="2022-09-04T18:38:22.988Z" level=info msg="listening on 127.0.0.1:5000" environment=production go_version=go1.17.6 instance_id=ec415078-682a-4ab6-b5a4-a56a16da1a27 service=registry version=v3.31.0-gitlab
2022-09-04_18:39:13.53518 time="2022-09-04T18:39:13.535Z" level=info msg="attempting to stop server gracefully..." database_drain_timeout=0s http_drain_timeout=0s quit_signal=terminated
2022-09-04_18:39:13.53519 time="2022-09-04T18:39:13.535Z" level=info msg="graceful shutdown successful" database_drain_timeout=0s http_drain_timeout=0s quit_signal=terminated
2022-09-04_18:39:13.57765 time="2022-09-04T18:39:13.577Z" level=info msg="redis not configured" environment=production go_version=go1.17.6 instance_id=342e3e5e-682c-46fa-9714-b3c69c24466b service=registry version=v3.31.0-gitlab
2022-09-04_18:39:13.57776 time="2022-09-04T18:39:13.577Z" level=info msg="storage backend redirection enabled" environment=production go_version=go1.17.6 instance_id=342e3e5e-682c-46fa-9714-b3c69c24466b service=registry version=v3.31.0-gitlab
2022-09-04_18:39:13.57782 time="2022-09-04T18:39:13.577Z" level=info msg="using inmemory blob descriptor cache" environment=production go_version=go1.17.6 instance_id=342e3e5e-682c-46fa-9714-b3c69c24466b service=registry version=v3.31.0-gitlab
2022-09-04_18:39:13.57807 time="2022-09-04T18:39:13.577Z" level=info msg="Starting upload purge in 21m0s" environment=production go_version=go1.17.6 instance_id=342e3e5e-682c-46fa-9714-b3c69c24466b service=registry version=v3.31.0-gitlab
2022-09-04_18:39:13.58212 time="2022-09-04T18:39:13.582Z" level=info msg="listening on 127.0.0.1:5000" environment=production go_version=go1.17.6 instance_id=342e3e5e-682c-46fa-9714-b3c69c24466b service=registry version=v3.31.0-gitlab
2022-09-04_18:51:55.52492 time="2022-09-04T18:51:55.524Z" level=info msg="attempting to stop server gracefully..." database_drain_timeout=0s http_drain_timeout=0s quit_signal=terminated
2022-09-04_18:51:55.52495 time="2022-09-04T18:51:55.524Z" level=info msg="graceful shutdown successful" database_drain_timeout=0s http_drain_timeout=0s quit_signal=terminated
==> /var/log/gitlab/prometheus/state <==
==> /var/log/gitlab/prometheus/current <==
2023-07-11_20:29:11.72739 ts=2023-07-11T20:29:11.727Z caller=head.go:1269 level=info component=tsdb msg="Head GC completed" caller=truncateMemory duration=18.426754ms
2023-07-11_20:29:11.73490 ts=2023-07-11T20:29:11.734Z caller=checkpoint.go:100 level=info component=tsdb msg="Creating checkpoint" from_segment=8754 to_segment=8755 mint=1689098400000
2023-07-11_20:29:12.30741 ts=2023-07-11T20:29:12.307Z caller=head.go:1241 level=info component=tsdb msg="WAL checkpoint complete" first=8754 last=8755 duration=577.516699ms
2023-07-11_21:29:10.79747 ts=2023-07-11T21:29:10.797Z caller=compact.go:519 level=info component=tsdb msg="write block" mint=1689100147021 maxt=1689105600000 ulid=01H53DQAV0ZJJZWC64T5P2QPP0 duration=2.092399828s
2023-07-11_21:29:11.03373 ts=2023-07-11T21:29:11.033Z caller=db.go:1548 level=info component=tsdb msg="Deleting obsolete block" block=01H3WAC0VARPZN61C3Q3YJ72BK
2023-07-11_21:29:11.09169 ts=2023-07-11T21:29:11.091Z caller=head.go:1269 level=info component=tsdb msg="Head GC completed" caller=truncateMemory duration=57.881811ms
2023-07-11_21:29:21.66373 ts=2023-07-11T21:29:21.663Z caller=compact.go:460 level=info component=tsdb msg="compact blocks" count=3 mint=1688947200241 maxt=1688968800000 ulid=01H53DQD8K8G92TD47KB1MAKGP sources="[01H4YVVTADRA6PGYNK8VK07BE7 01H4Z2QJ1ESM7BRERC7NWV9AK3 01H51ZRNBPA1RE9YVSRT237CGT]" duration=10.485422146s
2023-07-11_21:29:21.83660 ts=2023-07-11T21:29:21.836Z caller=db.go:1548 level=info component=tsdb msg="Deleting obsolete block" block=01H4Z2QJ1ESM7BRERC7NWV9AK3
2023-07-11_21:29:21.87637 ts=2023-07-11T21:29:21.876Z caller=db.go:1548 level=info component=tsdb msg="Deleting obsolete block" block=01H51ZRNBPA1RE9YVSRT237CGT
2023-07-11_21:29:21.92215 ts=2023-07-11T21:29:21.922Z caller=db.go:1548 level=info component=tsdb msg="Deleting obsolete block" block=01H4YVVTADRA6PGYNK8VK07BE7
==> /var/log/gitlab/logrotate/current <==
2023-07-11_21:37:56.01420 warning: Potentially dangerous mode on /var/opt/gitlab/logrotate/logrotate.d/gitlab-workhorse: 0777
2023-07-11_21:37:56.01422 error: Ignoring /var/opt/gitlab/logrotate/logrotate.d/gitlab-workhorse because it is writable by group or others.
2023-07-11_21:37:56.02076 warning: Potentially dangerous mode on /var/opt/gitlab/logrotate/logrotate.d/gitlab-pages: 0777
2023-07-11_21:37:56.02081 error: Ignoring /var/opt/gitlab/logrotate/logrotate.d/gitlab-pages because it is writable by group or others.
2023-07-11_21:37:56.02825 warning: Potentially dangerous mode on /var/opt/gitlab/logrotate/logrotate.d/gitlab-kas: 0777
2023-07-11_21:37:56.02828 error: Ignoring /var/opt/gitlab/logrotate/logrotate.d/gitlab-kas because it is writable by group or others.
2023-07-11_21:37:56.03536 warning: Potentially dangerous mode on /var/opt/gitlab/logrotate/logrotate.d/gitaly: 0777
2023-07-11_21:37:56.03539 error: Ignoring /var/opt/gitlab/logrotate/logrotate.d/gitaly because it is writable by group or others.
2023-07-11_21:37:56.04305 warning: Potentially dangerous mode on /var/opt/gitlab/logrotate/logrotate.d/mailroom: 0777
2023-07-11_21:37:56.04307 error: Ignoring /var/opt/gitlab/logrotate/logrotate.d/mailroom because it is writable by group or others.
==> /var/log/gitlab/gitlab-rails/production.log <==
Rack::Timeout::RequestTimeoutException (Request ran for longer than 600000ms ):
lib/gitlab/database/postgresql_adapter/empty_query_ping.rb:15:in `exec'
lib/gitlab/database/postgresql_adapter/empty_query_ping.rb:15:in `block in active?'
lib/gitlab/database/postgresql_adapter/empty_query_ping.rb:14:in `active?'
lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
lib/gitlab/database/load_balancing/connection_proxy.rb:96:in `method_missing'
app/models/concerns/cacheable_attributes.rb:19:in `current_without_cache'
app/models/concerns/cacheable_attributes.rb:55:in `current'
lib/gitlab/current_settings.rb:60:in `uncached_application_settings'
lib/gitlab/current_settings.rb:42:in `ensure_application_settings!'
lib/gitlab/current_settings.rb:15:in `block in current_application_settings'
lib/gitlab/safe_request_store.rb:12:in `fetch'
lib/gitlab/current_settings.rb:15:in `current_application_settings'
lib/gitlab/throttle.rb:13:in `settings'
lib/gitlab/rack_attack/request.rb:95:in `throttle_unauthenticated_web?'
lib/gitlab/rack_attack.rb:104:in `block in configure_throttles'
lib/gitlab/middleware/read_only/controller.rb:50:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'
lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
lib/gitlab/middleware/request_context.rb:15:in `call'
lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call'
config/initializers/fix_local_cache_middleware.rb:11:in `call'
lib/gitlab/middleware/compressed_json.rb:44:in `call'
lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'
lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:79:in `call'
lib/gitlab/middleware/release_env.rb:13:in `call'
</pre>
</details>
<details>
<summary>
`top` Output
</summary>
```
top - 23:53:10 up 5:28, 0 users, load average: 58.57, 58.41, 58.29
Tasks: 121 total, 2 running, 119 sleeping, 0 stopped, 0 zombie
%Cpu(s): 1.6 us, 7.2 sy, 0.0 ni, 73.4 id, 16.9 wa, 0.0 hi, 0.7 si, 0.0 st
MiB Mem : 9949.1 total, 1996.1 free, 5232.7 used, 2720.4 buff/cache
MiB Swap: 32768.0 total, 32768.0 free, 0.0 used. 4367.1 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
778 git 20 0 2207896 1.2g 18868 R 100.0 12.1 100:07.25 ruby
446 gitlab-+ 20 0 5313072 309052 71256 S 4.3 3.0 6:17.68 prometheus
724 git 20 0 3610264 1.2g 28220 S 1.3 12.0 17:39.80 ruby
276 gitlab-+ 20 0 70428 15812 6052 S 0.7 0.2 2:36.00 redis-server
444 gitlab-+ 20 0 1802924 52112 21084 S 0.3 0.5 0:12.69 alertmanager
1 root 20 0 4912 3488 3148 S 0.0 0.0 0:00.04 wrapper
24 root 20 0 2808 1312 1212 S 0.0 0.0 0:00.12 runsvdir
30 root 20 0 2656 1068 980 S 0.0 0.0 0:00.00 runsv
31 root 20 0 2804 1072 980 S 0.0 0.0 0:00.00 svlogd
32 root 20 0 15424 8568 7012 S 0.0 0.1 0:00.01 sshd
269 root 20 0 2656 1060 972 S 0.0 0.0 0:00.00 runsv
270 root 20 0 2656 1072 984 S 0.0 0.0 0:00.00 runsv
271 root 20 0 2656 1040 952 S 0.0 0.0 0:00.00 runsv
272 root 20 0 2804 1056 972 S 0.0 0.0 0:00.00 svlogd
273 root 20 0 2804 1080 996 S 0.0 0.0 0:00.00 svlogd
274 root 20 0 2804 1080 992 S 0.0 0.0 0:00.00 svlogd
310 git 20 0 1328484 19756 7152 S 0.0 0.2 0:01.88 gitaly-wrapper
316 git 20 0 2161168 286540 157828 S 0.0 2.8 0:31.64 gitaly
351 root 20 0 2656 1084 992 S 0.0 0.0 0:00.00 runsv
352 root 20 0 2804 1112 1020 S 0.0 0.0 0:00.00 svlogd
353 gitlab-+ 20 0 49020 28676 26816 D 0.0 0.3 0:00.36 postgres
378 root 20 0 2656 1064 972 S 0.0 0.0 0:00.00 runsv
379 root 20 0 2656 1060 972 S 0.0 0.0 0:00.00 runsv
```
</details>
`gitlab-rake gitlab:check` does not return during the outage periods, but does not return any errors/problems during working time periods
### Details of package version
<details>
<summary>Provide the package version installation details</summary>
<pre>
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-============================================================
un gitlab-ce <none> <none> (no description available)
ii gitlab-ee 16.1.2-ee.0 amd64 GitLab Enterprise Edition (including NGINX, Postgres, Redis)
un gitlab-fips <none> <none> (no description available)
</pre>
</details>
### Environment details
* Operating System: WSL Windows Docker
* Installation Target, remove incorrect values:
* Docker 24.0.0.2
* Installation Type, remove incorrect values:
* Upgrade from version 14.9.0-ee (Upgraded with all necessary according to the upgrade manual)
* Is this a single or multiple node installation? Single Node
* Resources
* CPU: 16 Cores
* Memory total: 10GB with 32GB swap
### Configuration details
<details>
<summary> Provide the relevant sections of `/etc/gitlab/gitlab.rb` </summary>
<pre>
<!--
Paste the details but omit lines starting with `#`. Relevant lines can be retrieved by
running `grep -v -e '^#' -e '^$' /etc/gitlab/gitlab.rb`. Be certain to sanitize sensitive
configuration such as passwords and secrets.
-->
external_url '"**REDACTED**'
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = ""**REDACTED**"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = ""**REDACTED**"
gitlab_rails['smtp_password'] = ""**REDACTED**"
gitlab_rails['smtp_domain'] = ""**REDACTED**"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = false
gitlab_rails['smtp_openssl_verify_mode'] = 'peer'
gitlab_rails['gitlab_email_enabled'] = true
gitlab_rails['gitlab_default_theme'] = 2
gitlab_rails['lfs_enabled'] = true
gitlab_rails['omniauth_enabled'] = nil
gitlab_rails['omniauth_allow_single_sign_on'] = ['github']
gitlab_rails['omniauth_providers'] = [
{
"name" => "github",
"app_id" => ""**REDACTED**",
"app_secret" => "**REDACTED**",
"args" => { "scope" => "user:email"}
}
]
gitlab_rails['env'] = {
'GITLAB_RAILS_RACK_TIMEOUT' => 600
}
puma['per_worker_max_memory_mb'] = 2048
letsencrypt['enable'] = false
</pre>
</details>https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/8056unrecognized configuration parameter "wal_keep_segments" after upgrade to ver...2023-09-20T14:26:15ZTamino E.unrecognized configuration parameter "wal_keep_segments" after upgrade to version 16### Summary
I have a gitlab instance running in docker. So far it is running on version 15.11.10. Now I wanted to upgrade to version 16 but the container dies after a short time with the error message:
```
PG::ConnectionBad: could not ...### Summary
I have a gitlab instance running in docker. So far it is running on version 15.11.10. Now I wanted to upgrade to version 16 but the container dies after a short time with the error message:
```
PG::ConnectionBad: could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
```
After looking at the Postgres log, I saw the following error:
```
2023-07-02_11:46:31.32233 2023-07-02 11:46:31.322 GMT [384] FATAL: configuration file "/var/opt/gitlab/postgresql/data/runtime.conf" contains errors
2023-07-02_11:46:32.32822 2023-07-02 11:46:32.328 GMT [385] LOG: unrecognized configuration parameter "wal_keep_segments" in file "/var/opt/gitlab/postgresql/data/runtime.conf" line 30
```
I had a look at the file `runtime.conf`, there is a field `wal_keep_segments`. But this field was renamed to `wal_keep_size` some time ago. (https://postgresqlco.nf/doc/en/param/wal_keep_segments/)
I haven't found a way to change this in the `gitlab.rb` file. Here everything concerning the postgres is commented out. I can provide this if needed.
Maybe I am doing something wrong. Thanks already
### Steps to reproduce
I have updated a gitlab instance from version 15 to 16. I don't know if it's that easy to recreate. The instance is a bit older, maybe something has disintegrated here over time.
### What is the current *bug* behavior?
The Gitlab container dies
### What is the expected *correct* behavior?
The gitlab container does not die but gitlab starts
### Relevant logs
<details>
<summary> Relevant logs </summary>
<pre>
Running handlers:
[2023-07-02T11:33:02+00:00] ERROR: Running exception handlers
There was an error running gitlab-ctl reconfigure:
rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash_hide_env[migrate gitlab-rails database] (gitlab::database_migrations line 20) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" ----
STDOUT: rake aborted!
ActiveRecord::ConnectionNotEstablished: could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:96:in `method_missing'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:115:in `configure_database'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:101:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'
Caused by:
PG::ConnectionBad: could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:96:in `method_missing'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:115:in `configure_database'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:101:in `block (3 levels) in <top (required)>'
Running handlers complete
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'
Tasks: TOP => gitlab:db:configure
(See full trace by running task with --trace)
STDERR:
---- End output of "bash" ----
Ran "bash" returned 1
[2023-07-02T11:33:02+00:00] ERROR: Exception handlers complete
Infra Phase failed. 168 resources updated in 01 minutes 42 seconds
[2023-07-02T11:33:02+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2023-07-02T11:33:02+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-07-02T11:33:02+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2023-07-02T11:33:02+00:00] FATAL: ---------------------------------------------------------------------------------------
[2023-07-02T11:33:02+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash_hide_env[migrate gitlab-rails database] (gitlab::database_migrations line 20) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" ----
STDOUT: rake aborted!
ActiveRecord::ConnectionNotEstablished: could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:96:in `method_missing'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:115:in `configure_database'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:101:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'
Caused by:
PG::ConnectionBad: could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:96:in `method_missing'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:115:in `configure_database'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:101:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'
Tasks: TOP => gitlab:db:configure
(See full trace by running task with --trace)
STDERR:
---- End output of "bash" ----
Ran "bash" returned 1
</pre>
</details>
### Environment details
* Operating System: `linux ubuntu 22 with docker`
* Installation Target, remove incorrect values:
* Bare Metal Machine
* Installation Type, remove incorrect values:
* Upgrade from version 15 to 16
* Is there any other software running on the machine: yes, there are running other containers
* Is this a single or multiple node installation? single
* Resources
* CPU: amd 12 core 24 threads cpu
* Memory total: 32 gb
### Configuration details
<details>
<summary> Provide the relevant sections of `/etc/gitlab/gitlab.rb` </summary>
<pre>
nginx['listen_port'] = 80
nginx['listen_https'] = false
nginx['proxy_set_headers'] = {
"Host" => "$http_host_with_default",
"X-Forwarded-For" => "$proxy_add_x_forwarded_for",
"X-Forwarded-Proto" => "https",
"X-Forwarded-Ssl" => "on",
}
nginx['real_ip_trusted_addresses'] = ['10.100.0.0/15']
nginx['real_ip_header'] = 'X-Real-IP'
nginx['real_ip_recursive'] = 'on'
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = ####
gitlab_rails['smtp_port'] = ####
gitlab_rails['smtp_user_name'] = ####
gitlab_rails['smtp_password'] = ####
gitlab_rails['smtp_domain'] = ####
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true
gitlab_rails['smtp_tls'] = false
gitlab_rails['smtp_openssl_verify_mode'] = 'peer'
gitlab_rails['gitlab_default_projects_features_issues'] = false
gitlab_rails['gitlab_default_projects_features_wiki'] = false
gitlab_rails['gitlab_default_projects_features_container_registry'] = false
</pre>
</details>https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7781gitlab-ce 15.10.3 pg-upgrade = empty Postgesql DB2023-05-25T18:01:31ZSven Mertengitlab-ce 15.10.3 pg-upgrade = empty Postgesql DB### Summary
After a pg-upgrade on a Gitlab-ce 15.10.3 to Upgrade the internal Postgres 12.12 to 13.8 i got an empty gitlabhq_production DB.
### Steps to reproduce
gitlab-ctl pg-upgrade
### What is the current *bug* behavior?
empty...### Summary
After a pg-upgrade on a Gitlab-ce 15.10.3 to Upgrade the internal Postgres 12.12 to 13.8 i got an empty gitlabhq_production DB.
### Steps to reproduce
gitlab-ctl pg-upgrade
### What is the current *bug* behavior?
empty gitlabhq_production aft upgrading the internal PG version from 12.12 to 13.8.
### What is the expected *correct* behavior?
migrated Data from the running System and Pg 13.8.
### Relevant logs
<details>
<summary> Relevant logs </summary>
output of the gitlab-psql [gitlab-psql](/uploads/95aa64266ab1c8050961b14cdc3107b5/gitlab-psql)
the pg-upgrade console log [pg-upgrade](/uploads/4cc4b3d8cbe3df4eacc70e61650b5823/pg-upgrade)
and a postgres/current log after pg-upgrade [postgres.current](/uploads/2d25c4d296502c2023dfc4256a22fe5e/postgres.current)
[gitlab-exporter log](/uploads/91c7133c003fb5075e0f47efde7a4421/gitlab-exporter)
</details>
### Details of package version
<details>
<summary>Provide the package version installation details</summary>
<pre>
`gitlab-ce 15.10.3-ce.0 amd64 GitLab Community Edition (including NGINX, Postgres, Redis)`
</pre>
</details>
### Environment details
* Operating System: Debian Bullseye
* Installation Target, remove incorrect values:
* VM: VM in VMWare
* Installation Type, remove incorrect values:
* Other: pg-upgrade on Gitlab-ce 15.10.3 to upgrade the internal pg Version from 12.12 to 13.8
* Is there any other software running on the machine: `no`
* Is this a single or multiple node installation?
* Single
* Resources
* CPU: `4`
* Memory total: `8GB`
### Configuration details
<details>
<summary> Provide the relevant sections of `/etc/gitlab/gitlab.rb` </summary>
[gitlab.rb](/uploads/b2ab5115c6d0e2915f5877d973e0bfa0/gitlab.rb)
</details>https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7707Unable to register or view Kubernetes agents2023-03-29T21:41:21ZWhitUnable to register or view Kubernetes agents<!---
Please read this!
Before you create a new issue, please make sure that you already ran
`sudo gitlab-ctl reconfigure`
Also make sure to search in https://gitlab.com/gitlab-org/omnibus-gitlab/issues,
to verify the issue you're abo...<!---
Please read this!
Before you create a new issue, please make sure that you already ran
`sudo gitlab-ctl reconfigure`
Also make sure to search in https://gitlab.com/gitlab-org/omnibus-gitlab/issues,
to verify the issue you're about to submit isn't a duplicate.
--->
### Summary
On a fresh installation of Gitlab Omnibus, I am unable to register agents or see the Kubernetes Agents screen.
### Steps to reproduce
1. Navigate to a project
2. Navigate to "Infrastructure -> Kubernetes clusters" in the project sidebar
3. Attempt to register an agent and receive a GRPC timeout error
![Screenshot_2023-03-28_at_11.27.20_AM](/uploads/623b2b436dc5f94a5a8200dcfbe6989a/Screenshot_2023-03-28_at_11.27.20_AM.png)
4. See an error on the page anytime you navigate to "Infrastructure -> Kubernetes clusters"
![Screenshot_2023-03-28_at_11.21.00_AM](/uploads/517bef5d33969cab8040ca1c15fe8992/Screenshot_2023-03-28_at_11.21.00_AM.png)
### What is the current *bug* behavior?
Any attempt on any project to create a Kubernetes agent or list the Kubernetes agents after attempting to create one displays an error
### What is the expected *correct* behavior?
I should be able to create, register, and manage Kubernetes agents
### Relevant logs
<details>
<summary> Relevant logs </summary>
These are the logs from tailing while trying to create an agent named "i-am-a-test-agent-3"
Despite the fact that I cannot see agents and get an error while trying to create them, if I try to create the same agent a second time, it says the name already exists.
<pre>
==> /var/log/gitlab/postgres-exporter/state <==
==> /var/log/gitlab/postgres-exporter/current <==
2023-03-27_16:30:37.77429 ts=2023-03-27T16:30:37.772Z caller=server.go:74 level=info msg="Established new database connection" fingerprint=/var/opt/gitlab/postgresql:5432
2023-03-27_16:30:40.04563 ts=2023-03-27T16:30:40.044Z caller=postgres_exporter.go:645 level=info msg="Semantic version changed" server=/var/opt/gitlab/postgresql:5432 from=0.0.0 to=13.8.0
2023-03-27_19:58:23.96118 ts=2023-03-27T19:58:23.960Z caller=main.go:135 level=info msg="Listening on address" address=localhost:9187
2023-03-27_19:58:23.96607 ts=2023-03-27T19:58:23.962Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false
2023-03-27_19:58:52.74643 ts=2023-03-27T19:58:52.745Z caller=server.go:74 level=info msg="Established new database connection" fingerprint=/var/opt/gitlab/postgresql:5432
2023-03-27_19:58:52.76365 ts=2023-03-27T19:58:52.763Z caller=postgres_exporter.go:645 level=info msg="Semantic version changed" server=/var/opt/gitlab/postgresql:5432 from=0.0.0 to=13.8.0
2023-03-27_21:00:53.57206 ts=2023-03-27T21:00:53.571Z caller=main.go:135 level=info msg="Listening on address" address=localhost:9187
2023-03-27_21:00:53.57587 ts=2023-03-27T21:00:53.574Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false
2023-03-27_21:01:22.74685 ts=2023-03-27T21:01:22.746Z caller=server.go:74 level=info msg="Established new database connection" fingerprint=/var/opt/gitlab/postgresql:5432
2023-03-27_21:01:22.76075 ts=2023-03-27T21:01:22.760Z caller=postgres_exporter.go:645 level=info msg="Semantic version changed" server=/var/opt/gitlab/postgresql:5432 from=0.0.0 to=13.8.0
==> /var/log/gitlab/prometheus/state <==
==> /var/log/gitlab/prometheus/current <==
2023-03-28_15:00:03.47429 ts=2023-03-28T15:00:03.474Z caller=compact.go:519 level=info component=tsdb msg="write block" mint=1680004800661 maxt=1680012000000 ulid=01GWMBQBBJ7VAJG2KXGZYXE9SJ duration=2.463203321s
2023-03-28_15:00:03.55304 ts=2023-03-28T15:00:03.552Z caller=head.go:844 level=info component=tsdb msg="Head GC completed" duration=68.769859ms
2023-03-28_15:00:03.56119 ts=2023-03-28T15:00:03.560Z caller=checkpoint.go:100 level=info component=tsdb msg="Creating checkpoint" from_segment=73 to_segment=74 mint=1680012000000
2023-03-28_15:00:06.78796 ts=2023-03-28T15:00:06.787Z caller=head.go:1013 level=info component=tsdb msg="WAL checkpoint complete" first=73 last=74 duration=3.227401526s
2023-03-28_15:00:17.82347 ts=2023-03-28T15:00:17.823Z caller=compact.go:460 level=info component=tsdb msg="compact blocks" count=5 mint=1679940000000 maxt=1680004800000 ulid=01GWMBQH07C1M121S4HMZFQH3Z sources="[01GWK2H5HBZBGP6D7KPMEXCZPW 01GWKY02TJ24FQ9BXC0PV9XDFB 01GWKQ45KA5V61C7Z51JRHF3FF 01GWKXZWVF9260Q5CY2K1K4KEZ 01GWM4VM3CJM6WX9F6JPHT5BH6]" duration=11.032384908s
2023-03-28_15:00:17.96709 ts=2023-03-28T15:00:17.966Z caller=db.go:1294 level=info component=tsdb msg="Deleting obsolete block" block=01GWK2H5HBZBGP6D7KPMEXCZPW
2023-03-28_15:00:18.08788 ts=2023-03-28T15:00:18.087Z caller=db.go:1294 level=info component=tsdb msg="Deleting obsolete block" block=01GWKY02TJ24FQ9BXC0PV9XDFB
2023-03-28_15:00:18.21093 ts=2023-03-28T15:00:18.210Z caller=db.go:1294 level=info component=tsdb msg="Deleting obsolete block" block=01GWM4VM3CJM6WX9F6JPHT5BH6
2023-03-28_15:00:18.41326 ts=2023-03-28T15:00:18.413Z caller=db.go:1294 level=info component=tsdb msg="Deleting obsolete block" block=01GWKXZWVF9260Q5CY2K1K4KEZ
2023-03-28_15:00:18.53163 ts=2023-03-28T15:00:18.531Z caller=db.go:1294 level=info component=tsdb msg="Deleting obsolete block" block=01GWKQ45KA5V61C7Z51JRHF3FF
==> /var/log/gitlab/registry/current <==
2023-03-22_00:00:04.18288 time="2023-03-22T00:00:04.180Z" level=info msg="router info" config_http_addr="127.0.0.1:5000" config_http_host= config_http_net= config_http_prefix= config_http_relative_urls=false correlation_id=01GW39V62KF6NXCFG2D6C68W11 go_version=go1.18.7 method=GET path=/v2/ root_repo= router=gorilla/mux version=v3.66.0-gitlab
2023-03-22_00:00:04.18297 time="2023-03-22T00:00:04.182Z" level=info msg="authorized request" auth_user_name= auth_user_type= correlation_id=01GW39V62KF6NXCFG2D6C68W11 go_version=go1.18.7 root_repo= version=v3.66.0-gitlab
2023-03-22_00:00:04.18301 {"content_type":"application/json","correlation_id":"01GW39V62KF6NXCFG2D6C68W11","duration_ms":2,"host":"127.0.0.1:5000","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:41280","remote_ip":"127.0.0.1","status":200,"system":"http","time":"2023-03-22T00:00:04.182Z","ttfb_ms":2,"uri":"/v2/","user_agent":"GitLab/15.9.3-ee","written_bytes":2}
2023-03-22_00:00:04.23630 time="2023-03-22T00:00:04.236Z" level=info msg="router info" config_http_addr="127.0.0.1:5000" config_http_host= config_http_net= config_http_prefix= config_http_relative_urls=false correlation_id=01GW39V64BJH5MTWCM2B87E3MH go_version=go1.18.7 method=GET path=/gitlab/v1/ root_repo= router=gorilla/mux version=v3.66.0-gitlab
2023-03-22_00:00:04.23746 time="2023-03-22T00:00:04.237Z" level=info msg="authorized request" auth_user_name= auth_user_type= correlation_id=01GW39V64BJH5MTWCM2B87E3MH go_version=go1.18.7 version=v3.66.0-gitlab
2023-03-22_00:00:04.23779 {"content_type":"","correlation_id":"01GW39V64BJH5MTWCM2B87E3MH","duration_ms":1,"host":"127.0.0.1:5000","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:41290","remote_ip":"127.0.0.1","status":404,"system":"http","time":"2023-03-22T00:00:04.237Z","ttfb_ms":1,"uri":"/gitlab/v1/","user_agent":"GitLab/15.9.3-ee","written_bytes":0}
2023-03-22_15:27:35.60159 time="2023-03-22T15:27:35.601Z" level=info msg="attempting to stop server gracefully..." database_drain_timeout=0s http_drain_timeout=0s quit_signal=terminated
2023-03-22_15:27:35.60247 time="2023-03-22T15:27:35.601Z" level=info msg="graceful shutdown successful" database_drain_timeout=0s http_drain_timeout=0s quit_signal=terminated
==> /var/log/gitlab/puma/state <==
==> /var/log/gitlab/puma/puma_stderr.log <==
==> /var/log/gitlab/puma/puma_stdout.log <==
==> /var/log/gitlab/puma/current <==
2023-03-27_21:00:59.30714 {"timestamp":"2023-03-27T21:00:59.306Z","pid":1500,"message":"* Environment: production"}
2023-03-27_21:00:59.30714 {"timestamp":"2023-03-27T21:00:59.306Z","pid":1500,"message":"* Master PID: 1500"}
2023-03-27_21:00:59.30715 {"timestamp":"2023-03-27T21:00:59.306Z","pid":1500,"message":"* Workers: 4"}
2023-03-27_21:00:59.30716 {"timestamp":"2023-03-27T21:00:59.306Z","pid":1500,"message":"* Restarts: (✔) hot (✖) phased"}
2023-03-27_21:00:59.30716 {"timestamp":"2023-03-27T21:00:59.306Z","pid":1500,"message":"* Preloading application"}
2023-03-27_21:03:22.37689 {"timestamp":"2023-03-27T21:03:22.376Z","pid":1500,"message":"* Listening on unix:///var/opt/gitlab/gitlab-rails/sockets/gitlab.socket"}
2023-03-27_21:03:22.37752 {"timestamp":"2023-03-27T21:03:22.377Z","pid":1500,"message":"* Listening on http://127.0.0.1:8080"}
2023-03-27_21:03:22.37779 {"timestamp":"2023-03-27T21:03:22.377Z","pid":1500,"message":"! WARNING: Detected 1 Thread(s) started in app boot:"}
2023-03-27_21:03:22.37810 {"timestamp":"2023-03-27T21:03:22.377Z","pid":1500,"message":"! #\u003cThread:0x00007efc05c0c670 /opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/rack-timeout-0.6.3/lib/rack/timeout/support/scheduler.rb:73 sleep\u003e - /opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/rack-timeout-0.6.3/lib/rack/timeout/support/scheduler.rb:91:in `sleep'"}
2023-03-27_21:03:22.37850 {"timestamp":"2023-03-27T21:03:22.378Z","pid":1500,"message":"Use Ctrl-C to stop"}
==> /var/log/gitlab/sidekiq/state <==
==> /var/log/gitlab/sidekiq/current <==
{"severity":"INFO","time":"2023-03-28T16:31:11.363Z","retry":0,"queue":"cronjob:elastic_index_initial_bulk_cron","version":0,"queue_namespace":"cronjob","args":["10"],"class":"ElasticIndexInitialBulkCronWorker","jid":"a6e188a1b711b3c29c4904e5","created_at":"2023-03-28T16:31:10.974Z","meta.caller_id":"ElasticIndexInitialBulkCronWorker","correlation_id":"f65c467529c2278276713c976f63d47e","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","meta.client_id":"ip/","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:cronjob:elastic_index_initial_bulk_cron:8a43c915cea5dbaf1b19e6c75ce378344601b69fae652c0ec91a49e707f813b4","size_limiter":"validated","enqueued_at":"2023-03-28T16:31:10.977Z","job_size_bytes":4,"pid":1625,"message":"ElasticIndexInitialBulkCronWorker JID-a6e188a1b711b3c29c4904e5: done: 0.323452 sec","job_status":"done","scheduling_latency_s":0.062521,"redis_calls":6,"redis_duration_s":0.101244,"redis_read_bytes":4,"redis_write_bytes":681,"redis_queues_calls":2,"redis_queues_duration_s":0.069653,"redis_queues_read_bytes":2,"redis_queues_write_bytes":218,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.031591,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":463,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":1,"db_main_count":1,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.072,"db_main_duration_s":0.072,"db_main_replica_duration_s":0.0,"cpu_s":0.019825,"mem_objects":1801,"mem_bytes":281072,"mem_mallocs":370,"mem_total_bytes":353112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"extra.elastic_index_initial_bulk_cron_worker.records_count":0,"extra.elastic_index_initial_bulk_cron_worker.shard_number":10,"duration_s":0.323452,"completed_at":"2023-03-28T16:31:11.363Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.071905}
{"severity":"INFO","time":"2023-03-28T16:31:11.346Z","retry":0,"queue":"cronjob:elastic_index_initial_bulk_cron","version":0,"queue_namespace":"cronjob","args":["12"],"class":"ElasticIndexInitialBulkCronWorker","jid":"82e35c817311ce1927a47514","created_at":"2023-03-28T16:31:11.072Z","meta.caller_id":"ElasticIndexInitialBulkCronWorker","correlation_id":"f65c467529c2278276713c976f63d47e","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","meta.client_id":"ip/","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:cronjob:elastic_index_initial_bulk_cron:cb835d06037e108881056ced28f6f0d9e854e555c1b19a303eedb47f929a6f38","size_limiter":"validated","enqueued_at":"2023-03-28T16:31:11.085Z","job_size_bytes":4,"pid":1625,"message":"ElasticIndexInitialBulkCronWorker JID-82e35c817311ce1927a47514: done: 0.240881 sec","job_status":"done","scheduling_latency_s":0.020604,"redis_calls":6,"redis_duration_s":0.06439399999999999,"redis_read_bytes":4,"redis_write_bytes":681,"redis_queues_calls":2,"redis_queues_duration_s":0.02743,"redis_queues_read_bytes":2,"redis_queues_write_bytes":218,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.036964,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":463,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":1,"db_main_count":1,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.082,"db_main_duration_s":0.082,"db_main_replica_duration_s":0.0,"cpu_s":0.019717,"mem_objects":1801,"mem_bytes":281520,"mem_mallocs":372,"mem_total_bytes":353560,"worker_id":"sidekiq_0","rate_limiting_gates":[],"extra.elastic_index_initial_bulk_cron_worker.records_count":0,"extra.elastic_index_initial_bulk_cron_worker.shard_number":12,"duration_s":0.240881,"completed_at":"2023-03-28T16:31:11.346Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.081934}
{"severity":"INFO","time":"2023-03-28T16:31:11.352Z","retry":0,"queue":"cronjob:elastic_index_initial_bulk_cron","version":0,"queue_namespace":"cronjob","args":["11"],"class":"ElasticIndexInitialBulkCronWorker","jid":"514e6f4a46ae79c725e1dd1e","created_at":"2023-03-28T16:31:11.013Z","meta.caller_id":"ElasticIndexInitialBulkCronWorker","correlation_id":"f65c467529c2278276713c976f63d47e","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","meta.client_id":"ip/","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:cronjob:elastic_index_initial_bulk_cron:f3d9b29e941800fc4e9f8741bf1a3b9fd9aee3c149cc020d29c7f05bdb74efba","size_limiter":"validated","enqueued_at":"2023-03-28T16:31:11.041Z","job_size_bytes":4,"pid":1625,"message":"ElasticIndexInitialBulkCronWorker JID-514e6f4a46ae79c725e1dd1e: done: 0.271295 sec","job_status":"done","scheduling_latency_s":0.039241,"redis_calls":6,"redis_duration_s":0.190376,"redis_read_bytes":4,"redis_write_bytes":681,"redis_queues_calls":2,"redis_queues_duration_s":0.03856,"redis_queues_read_bytes":2,"redis_queues_write_bytes":218,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.151816,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":463,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":1,"db_main_count":1,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_duration_s":0.033,"db_main_replica_duration_s":0.0,"cpu_s":0.021622,"mem_objects":1801,"mem_bytes":281072,"mem_mallocs":370,"mem_total_bytes":353112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"extra.elastic_index_initial_bulk_cron_worker.records_count":0,"extra.elastic_index_initial_bulk_cron_worker.shard_number":11,"duration_s":0.271295,"completed_at":"2023-03-28T16:31:11.352Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032601}
{"severity":"INFO","time":"2023-03-28T16:31:11.267Z","retry":0,"queue":"cronjob:elastic_index_initial_bulk_cron","version":0,"queue_namespace":"cronjob","args":["14"],"class":"ElasticIndexInitialBulkCronWorker","jid":"25fb666bbe15e6b25f340a68","created_at":"2023-03-28T16:31:11.151Z","meta.caller_id":"ElasticIndexInitialBulkCronWorker","correlation_id":"f65c467529c2278276713c976f63d47e","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","meta.client_id":"ip/","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:cronjob:elastic_index_initial_bulk_cron:ac40d5fddda5259f14234221a79cdc010d7db78ef29d8812667f5e49ea6b2639","size_limiter":"validated","enqueued_at":"2023-03-28T16:31:11.182Z","job_size_bytes":4,"pid":1625,"message":"ElasticIndexInitialBulkCronWorker JID-25fb666bbe15e6b25f340a68: start","job_status":"start","scheduling_latency_s":0.084147}
{"severity":"INFO","time":"2023-03-28T16:31:11.392Z","retry":0,"queue":"cronjob:elastic_index_initial_bulk_cron","version":0,"queue_namespace":"cronjob","args":["6"],"class":"ElasticIndexInitialBulkCronWorker","jid":"7d3f10860cad2a6764846063","created_at":"2023-03-28T16:31:10.642Z","meta.caller_id":"ElasticIndexInitialBulkCronWorker","correlation_id":"f65c467529c2278276713c976f63d47e","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","meta.client_id":"ip/","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:cronjob:elastic_index_initial_bulk_cron:a4436ee3f346e9a3b9977be8610fcd1f4e35acefdee10bd484c866a7d118e3ea","size_limiter":"validated","enqueued_at":"2023-03-28T16:31:10.678Z","job_size_bytes":3,"pid":1625,"message":"ElasticIndexInitialBulkCronWorker JID-7d3f10860cad2a6764846063: done: 0.601062 sec","job_status":"done","scheduling_latency_s":0.112263,"redis_calls":6,"redis_duration_s":0.095617,"redis_read_bytes":4,"redis_write_bytes":678,"redis_queues_calls":2,"redis_queues_duration_s":0.001312,"redis_queues_read_bytes":2,"redis_queues_write_bytes":218,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.094305,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":460,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":1,"db_main_count":1,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_duration_s":0.032,"db_main_replica_duration_s":0.0,"cpu_s":0.020332,"mem_objects":1802,"mem_bytes":281088,"mem_mallocs":370,"mem_total_bytes":353168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"extra.elastic_index_initial_bulk_cron_worker.records_count":0,"extra.elastic_index_initial_bulk_cron_worker.shard_number":6,"duration_s":0.601062,"completed_at":"2023-03-28T16:31:11.392Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032}
{"severity":"INFO","time":"2023-03-28T16:31:11.443Z","retry":0,"queue":"cronjob:elastic_index_initial_bulk_cron","version":0,"queue_namespace":"cronjob","args":["15"],"class":"ElasticIndexInitialBulkCronWorker","jid":"a34ce313720681a89eaacdb8","created_at":"2023-03-28T16:31:11.227Z","meta.caller_id":"ElasticIndexInitialBulkCronWorker","correlation_id":"f65c467529c2278276713c976f63d47e","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","meta.client_id":"ip/","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:cronjob:elastic_index_initial_bulk_cron:0af5f4477b54dfac13d3139066f926fb8a307f83e6a3be92109a25481214c9c5","size_limiter":"validated","enqueued_at":"2023-03-28T16:31:11.230Z","job_size_bytes":4,"pid":1625,"message":"ElasticIndexInitialBulkCronWorker JID-a34ce313720681a89eaacdb8: done: 0.13887 sec","job_status":"done","scheduling_latency_s":0.074561,"redis_calls":6,"redis_duration_s":0.026105999999999997,"redis_read_bytes":4,"redis_write_bytes":681,"redis_queues_calls":2,"redis_queues_duration_s":0.001223,"redis_queues_read_bytes":2,"redis_queues_write_bytes":218,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.024883,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":463,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":1,"db_main_count":1,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_duration_s":0.035,"db_main_replica_duration_s":0.0,"cpu_s":0.01983,"mem_objects":1800,"mem_bytes":260592,"mem_mallocs":369,"mem_total_bytes":332592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"extra.elastic_index_initial_bulk_cron_worker.records_count":0,"extra.elastic_index_initial_bulk_cron_worker.shard_number":15,"duration_s":0.13887,"completed_at":"2023-03-28T16:31:11.443Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03526}
{"severity":"INFO","time":"2023-03-28T16:31:11.397Z","retry":0,"queue":"cronjob:elastic_index_initial_bulk_cron","version":0,"queue_namespace":"cronjob","args":["8"],"class":"ElasticIndexInitialBulkCronWorker","jid":"c24d7f8d7ad937ffa07a9804","created_at":"2023-03-28T16:31:10.842Z","meta.caller_id":"ElasticIndexInitialBulkCronWorker","correlation_id":"f65c467529c2278276713c976f63d47e","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","meta.client_id":"ip/","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:cronjob:elastic_index_initial_bulk_cron:37c6d711679df274bf2f140ac4bf7445ca404afc6b50537f8a0c34ff8f488678","size_limiter":"validated","enqueued_at":"2023-03-28T16:31:10.869Z","job_size_bytes":3,"pid":1625,"message":"ElasticIndexInitialBulkCronWorker JID-c24d7f8d7ad937ffa07a9804: done: 0.459084 sec","job_status":"done","scheduling_latency_s":0.068968,"redis_calls":6,"redis_duration_s":0.196289,"redis_read_bytes":4,"redis_write_bytes":678,"redis_queues_calls":2,"redis_queues_duration_s":0.061416,"redis_queues_read_bytes":2,"redis_queues_write_bytes":218,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.134873,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":460,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":1,"db_main_count":1,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_duration_s":0.045,"db_main_replica_duration_s":0.0,"cpu_s":0.019109,"mem_objects":1802,"mem_bytes":281088,"mem_mallocs":370,"mem_total_bytes":353168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"extra.elastic_index_initial_bulk_cron_worker.records_count":0,"extra.elastic_index_initial_bulk_cron_worker.shard_number":8,"duration_s":0.459084,"completed_at":"2023-03-28T16:31:11.397Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044809}
{"severity":"INFO","time":"2023-03-28T16:31:11.462Z","retry":0,"queue":"cronjob:elastic_index_initial_bulk_cron","version":0,"queue_namespace":"cronjob","args":["13"],"class":"ElasticIndexInitialBulkCronWorker","jid":"5fb9d64b3ec0e95640bc3e63","created_at":"2023-03-28T16:31:11.101Z","meta.caller_id":"ElasticIndexInitialBulkCronWorker","correlation_id":"f65c467529c2278276713c976f63d47e","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","meta.client_id":"ip/","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:cronjob:elastic_index_initial_bulk_cron:3d3d6d1021dcbd22c7490b10d518b7a873933b520ebd16624f8d91a818367cad","size_limiter":"validated","enqueued_at":"2023-03-28T16:31:11.122Z","job_size_bytes":4,"pid":1625,"message":"ElasticIndexInitialBulkCronWorker JID-5fb9d64b3ec0e95640bc3e63: done: 0.282879 sec","job_status":"done","scheduling_latency_s":0.057488,"redis_calls":6,"redis_duration_s":0.152503,"redis_read_bytes":4,"redis_write_bytes":681,"redis_queues_calls":2,"redis_queues_duration_s":0.069484,"redis_queues_read_bytes":2,"redis_queues_write_bytes":218,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.083019,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":463,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":1,"db_main_count":1,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_duration_s":0.03,"db_main_replica_duration_s":0.0,"cpu_s":0.020492,"mem_objects":1801,"mem_bytes":281072,"mem_mallocs":370,"mem_total_bytes":353112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"extra.elastic_index_initial_bulk_cron_worker.records_count":0,"extra.elastic_index_initial_bulk_cron_worker.shard_number":13,"duration_s":0.282879,"completed_at":"2023-03-28T16:31:11.462Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029797}
{"severity":"INFO","time":"2023-03-28T16:31:11.482Z","retry":0,"queue":"cronjob:elastic_index_initial_bulk_cron","version":0,"queue_namespace":"cronjob","args":["9"],"class":"ElasticIndexInitialBulkCronWorker","jid":"299ecd8b858ac7d7c302dae7","created_at":"2023-03-28T16:31:10.913Z","meta.caller_id":"ElasticIndexInitialBulkCronWorker","correlation_id":"f65c467529c2278276713c976f63d47e","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","meta.client_id":"ip/","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:cronjob:elastic_index_initial_bulk_cron:8baec2a811006d88bfd4d1c775f3a0eff8d414fc0e73124d197c0655e5c6040d","size_limiter":"validated","enqueued_at":"2023-03-28T16:31:10.948Z","job_size_bytes":3,"pid":1625,"message":"ElasticIndexInitialBulkCronWorker JID-299ecd8b858ac7d7c302dae7: done: 0.47435 sec","job_status":"done","scheduling_latency_s":0.059123,"redis_calls":6,"redis_duration_s":0.152009,"redis_read_bytes":4,"redis_write_bytes":678,"redis_queues_calls":2,"redis_queues_duration_s":0.058998,"redis_queues_read_bytes":2,"redis_queues_write_bytes":218,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.093011,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":460,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":1,"db_main_count":1,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_duration_s":0.038,"db_main_replica_duration_s":0.0,"cpu_s":0.022572,"mem_objects":1802,"mem_bytes":281088,"mem_mallocs":370,"mem_total_bytes":353168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"extra.elastic_index_initial_bulk_cron_worker.records_count":0,"extra.elastic_index_initial_bulk_cron_worker.shard_number":9,"duration_s":0.47435,"completed_at":"2023-03-28T16:31:11.482Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.083325}
{"severity":"INFO","time":"2023-03-28T16:31:11.507Z","retry":0,"queue":"cronjob:elastic_index_initial_bulk_cron","version":0,"queue_namespace":"cronjob","args":["14"],"class":"ElasticIndexInitialBulkCronWorker","jid":"25fb666bbe15e6b25f340a68","created_at":"2023-03-28T16:31:11.151Z","meta.caller_id":"ElasticIndexInitialBulkCronWorker","correlation_id":"f65c467529c2278276713c976f63d47e","meta.root_caller_id":"Cronjob","meta.feature_category":"global_search","meta.client_id":"ip/","worker_data_consistency":"sticky","wal_locations":{},"wal_location_source":"primary","idempotency_key":"resque:gitlab:duplicate:cronjob:elastic_index_initial_bulk_cron:ac40d5fddda5259f14234221a79cdc010d7db78ef29d8812667f5e49ea6b2639","size_limiter":"validated","enqueued_at":"2023-03-28T16:31:11.182Z","job_size_bytes":4,"pid":1625,"message":"ElasticIndexInitialBulkCronWorker JID-25fb666bbe15e6b25f340a68: done: 0.240942 sec","job_status":"done","scheduling_latency_s":0.084147,"redis_calls":6,"redis_duration_s":0.033337000000000006,"redis_read_bytes":4,"redis_write_bytes":681,"redis_queues_calls":2,"redis_queues_duration_s":0.02415,"redis_queues_read_bytes":2,"redis_queues_write_bytes":218,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.009187,"redis_shared_state_read_bytes":2,"redis_shared_state_write_bytes":463,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":1,"db_main_count":1,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_duration_s":0.012,"db_main_replica_duration_s":0.0,"cpu_s":0.018582,"mem_objects":1801,"mem_bytes":281072,"mem_mallocs":370,"mem_total_bytes":353112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"extra.elastic_index_initial_bulk_cron_worker.records_count":0,"extra.elastic_index_initial_bulk_cron_worker.shard_number":14,"duration_s":0.240942,"completed_at":"2023-03-28T16:31:11.507Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01147}
==> /var/log/gitlab/logrotate/state <==
==> /var/log/gitlab/logrotate/current <==
==> /var/log/gitlab/node-exporter/state <==
==> /var/log/gitlab/node-exporter/current <==
2023-03-27_21:00:53.73809 ts=2023-03-27T21:00:53.736Z caller=node_exporter.go:117 level=info collector=thermal_zone
2023-03-27_21:00:53.73810 ts=2023-03-27T21:00:53.736Z caller=node_exporter.go:117 level=info collector=time
2023-03-27_21:00:53.73810 ts=2023-03-27T21:00:53.736Z caller=node_exporter.go:117 level=info collector=timex
2023-03-27_21:00:53.73811 ts=2023-03-27T21:00:53.736Z caller=node_exporter.go:117 level=info collector=udp_queues
2023-03-27_21:00:53.73812 ts=2023-03-27T21:00:53.736Z caller=node_exporter.go:117 level=info collector=uname
2023-03-27_21:00:53.73815 ts=2023-03-27T21:00:53.736Z caller=node_exporter.go:117 level=info collector=vmstat
2023-03-27_21:00:53.73818 ts=2023-03-27T21:00:53.736Z caller=node_exporter.go:117 level=info collector=xfs
2023-03-27_21:00:53.73818 ts=2023-03-27T21:00:53.736Z caller=node_exporter.go:117 level=info collector=zfs
2023-03-27_21:00:53.74972 ts=2023-03-27T21:00:53.749Z caller=tls_config.go:232 level=info msg="Listening on" address=127.0.0.1:9100
2023-03-27_21:00:53.74978 ts=2023-03-27T21:00:53.749Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=127.0.0.1:9100
==> /var/log/gitlab/postgresql/state <==
==> /var/log/gitlab/postgresql/current <==
2023-03-27_21:00:55.86098 LOG: database system is ready to accept connections
2023-03-28_16:20:26.70025 LOG: no match in usermap "gitlab" for user "gitlab" authenticated as "root"
2023-03-28_16:20:26.70043 FATAL: Peer authentication failed for user "gitlab"
2023-03-28_16:20:26.70045 DETAIL: Connection matched pg_hba.conf line 70: "local all all peer map=gitlab"
2023-03-28_16:20:27.49449 LOG: no match in usermap "gitlab" for user "gitlab" authenticated as "root"
2023-03-28_16:20:27.49461 FATAL: Peer authentication failed for user "gitlab"
2023-03-28_16:20:27.49463 DETAIL: Connection matched pg_hba.conf line 70: "local all all peer map=gitlab"
2023-03-28_16:20:27.64181 LOG: no match in usermap "gitlab" for user "gitlab" authenticated as "root"
2023-03-28_16:20:27.64187 FATAL: Peer authentication failed for user "gitlab"
2023-03-28_16:20:27.64189 DETAIL: Connection matched pg_hba.conf line 70: "local all all peer map=gitlab"
==> /var/log/gitlab/redis/state <==
==> /var/log/gitlab/redis/current <==
2023-03-28_16:24:53.03558 1529:M 28 Mar 2023 16:24:53.035 * 10 changes in 300 seconds. Saving...
2023-03-28_16:24:53.03665 1529:M 28 Mar 2023 16:24:53.036 * Background saving started by pid 26734
2023-03-28_16:24:53.14436 26734:C 28 Mar 2023 16:24:53.144 * DB saved on disk
2023-03-28_16:24:53.14583 26734:C 28 Mar 2023 16:24:53.145 * RDB: 0 MB of memory used by copy-on-write
2023-03-28_16:24:53.23851 1529:M 28 Mar 2023 16:24:53.238 * Background saving terminated with success
2023-03-28_16:29:54.03427 1529:M 28 Mar 2023 16:29:54.034 * 10 changes in 300 seconds. Saving...
2023-03-28_16:29:54.03535 1529:M 28 Mar 2023 16:29:54.035 * Background saving started by pid 26940
2023-03-28_16:29:54.13344 26940:C 28 Mar 2023 16:29:54.133 * DB saved on disk
2023-03-28_16:29:54.13483 26940:C 28 Mar 2023 16:29:54.134 * RDB: 0 MB of memory used by copy-on-write
2023-03-28_16:29:54.13616 1529:M 28 Mar 2023 16:29:54.136 * Background saving terminated with success
==> /var/log/gitlab/gitlab-workhorse/state <==
==> /var/log/gitlab/gitlab-workhorse/current <==
{"content_type":"","correlation_id":"01GWMGYMN6EXK5R7Y09Q28MV8R","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:22Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
{"content_type":"","correlation_id":"01GWMGYNME4C0K1X3YBBMJZVDZ","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:23Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
{"content_type":"","correlation_id":"01GWMGYPKPNYH2X5GVT0ZV3TPN","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:24Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
{"content_type":"","correlation_id":"01GWMGYQJZF7CGZ09TYGYYMD5R","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:25Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
{"content_type":"","correlation_id":"01GWMGYRJ7CEYTVYKSDXYCY0R1","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:26Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
{"content_type":"","correlation_id":"01GWMGYSHHMD3KQ7CKNWAQ3PYQ","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:27Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
{"content_type":"","correlation_id":"01GWMGYTGRZAXVDD34BD0JCVMG","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:28Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
{"content_type":"","correlation_id":"01GWMGYVG4G67VDETFBG12V6RN","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:29Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
{"content_type":"","correlation_id":"01GWMGYWFBP6042XN8HNESR1K4","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:30Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
{"content_type":"","correlation_id":"01GWMGYXEKGJ3Q9DSVEFW8664H","duration_ms":3,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:31Z","ttfb_ms":3,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
==> /var/log/gitlab/redis-exporter/state <==
==> /var/log/gitlab/redis-exporter/current <==
2023-03-27_16:29:52.03641 time="2023-03-27T16:29:52Z" level=info msg="Redis Metrics Exporter 1.46.0 build date: sha1: Go: go1.18.7 GOOS: linux GOARCH: amd64"
2023-03-27_16:29:52.03894 time="2023-03-27T16:29:52Z" level=info msg="Providing metrics at localhost:9121/metrics"
2023-03-27_19:58:23.88543 time="2023-03-27T19:58:23Z" level=info msg="Redis Metrics Exporter 1.46.0 build date: sha1: Go: go1.18.7 GOOS: linux GOARCH: amd64"
2023-03-27_19:58:23.89598 time="2023-03-27T19:58:23Z" level=info msg="Providing metrics at localhost:9121/metrics"
2023-03-27_21:00:53.53773 time="2023-03-27T21:00:53Z" level=info msg="Redis Metrics Exporter 1.46.0 build date: sha1: Go: go1.18.7 GOOS: linux GOARCH: amd64"
2023-03-27_21:00:53.54333 time="2023-03-27T21:00:53Z" level=info msg="Providing metrics at localhost:9121/metrics"
==> /var/log/gitlab/gitlab-shell/gitlab-shell.log <==
==> /var/log/gitlab/nginx/access.log <==
==> /var/log/gitlab/nginx/state <==
==> /var/log/gitlab/nginx/gitlab_access.log <==
xx.xx.xx.xxx - - [28/Mar/2023:16:31:22 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
xx.xx.xx.xxx - - [28/Mar/2023:16:31:23 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
xx.xx.xx.xxx - - [28/Mar/2023:16:31:24 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
xx.xx.xx.xxx - - [28/Mar/2023:16:31:25 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
xx.xx.xx.xxx - - [28/Mar/2023:16:31:26 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
xx.xx.xx.xxx - - [28/Mar/2023:16:31:27 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
xx.xx.xx.xxx - - [28/Mar/2023:16:31:28 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
xx.xx.xx.xxx - - [28/Mar/2023:16:31:29 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
xx.xx.xx.xxx - - [28/Mar/2023:16:31:30 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
xx.xx.xx.xxx - - [28/Mar/2023:16:31:31 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
==> /var/log/gitlab/nginx/gitlab_registry_access.log <==
==> /var/log/gitlab/nginx/gitlab_error.log <==
==> /var/log/gitlab/nginx/gitlab_registry_error.log <==
==> /var/log/gitlab/nginx/error.log <==
==> /var/log/gitlab/nginx/current <==
==> /var/log/gitlab/alertmanager/state <==
==> /var/log/gitlab/alertmanager/current <==
2023-03-27_21:00:54.20796 ts=2023-03-27T21:00:54.207Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=master, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)"
2023-03-27_21:00:54.20942 ts=2023-03-27T21:00:54.207Z caller=main.go:241 level=info build_context="(go=go1.18.7, user=GitLab-Omnibus, date=)"
2023-03-27_21:00:54.21957 ts=2023-03-27T21:00:54.218Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=xx.xx.xx.xxy port=9094
2023-03-27_21:00:54.23387 ts=2023-03-27T21:00:54.233Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s
2023-03-27_21:00:54.40268 ts=2023-03-27T21:00:54.399Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/var/opt/gitlab/alertmanager/alertmanager.yml
2023-03-27_21:00:54.40700 ts=2023-03-27T21:00:54.406Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/var/opt/gitlab/alertmanager/alertmanager.yml
2023-03-27_21:00:54.43119 ts=2023-03-27T21:00:54.431Z caller=tls_config.go:232 level=info msg="Listening on" address=127.0.0.1:9093
2023-03-27_21:00:54.43137 ts=2023-03-27T21:00:54.431Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=127.0.0.1:9093
2023-03-27_21:00:56.23489 ts=2023-03-27T21:00:56.234Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000847854s
2023-03-27_21:01:04.23711 ts=2023-03-27T21:01:04.236Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003135958s
==> /var/log/gitlab/gitlab-kas/state <==
==> /var/log/gitlab/gitlab-kas/current <==
2023-03-27_19:58:24.63393 {"level":"info","time":"2023-03-27T19:58:24.630Z","msg":"API endpoint is up","net_network":"tcp","net_address":"127.0.0.1:8153"}
2023-03-27_19:58:24.63726 {"level":"info","time":"2023-03-27T19:58:24.634Z","msg":"Private API endpoint is up","net_network":"tcp","net_address":"127.0.0.1:8155"}
2023-03-27_19:58:24.63728 {"level":"info","time":"2023-03-27T19:58:24.632Z","msg":"Observability endpoint is up","mod_name":"observability","net_network":"tcp","net_address":"127.0.0.1:8151"}
2023-03-27_19:58:24.63729 {"level":"info","time":"2023-03-27T19:58:24.632Z","msg":"Kubernetes API endpoint is up","mod_name":"kubernetes_api","net_network":"tcp","net_address":"127.0.0.1:8154"}
2023-03-27_19:58:24.63730 {"level":"info","time":"2023-03-27T19:58:24.636Z","msg":"Agentk API endpoint is up","net_network":"tcp","net_address":"127.0.0.1:8150","is_websocket":true}
2023-03-27_21:00:54.28542 {"level":"info","time":"2023-03-27T21:00:54.284Z","msg":"Private API endpoint is up","net_network":"tcp","net_address":"127.0.0.1:8155"}
2023-03-27_21:00:54.28768 {"level":"info","time":"2023-03-27T21:00:54.285Z","msg":"Observability endpoint is up","mod_name":"observability","net_network":"tcp","net_address":"127.0.0.1:8151"}
2023-03-27_21:00:54.28770 {"level":"info","time":"2023-03-27T21:00:54.285Z","msg":"API endpoint is up","net_network":"tcp","net_address":"127.0.0.1:8153"}
2023-03-27_21:00:54.28772 {"level":"info","time":"2023-03-27T21:00:54.286Z","msg":"Kubernetes API endpoint is up","mod_name":"kubernetes_api","net_network":"tcp","net_address":"127.0.0.1:8154"}
2023-03-27_21:00:54.28802 {"level":"info","time":"2023-03-27T21:00:54.287Z","msg":"Agentk API endpoint is up","net_network":"tcp","net_address":"127.0.0.1:8150","is_websocket":true}
==> /var/log/gitlab/gitaly/state <==
==> /var/log/gitlab/gitaly/gitaly_hooks.log <==
==> /var/log/gitlab/gitaly/gitaly_ruby_json.log <==
{"type":"gitaly-ruby","grpc.start_time":"2023-03-22T15:43:54Z","grpc.time_ms":0.329,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1584,"correlation_id":"a787d7cfcc0d9f783762f96e64cd489b","time":"2023-03-22T15:43:54.139Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-03-22T15:43:54Z","grpc.time_ms":0.329,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1583,"correlation_id":"90fab96aaec3a7acd6beb4ca6e8eb3d7","time":"2023-03-22T15:43:54.144Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-03-22T15:44:09Z","grpc.time_ms":0.329,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1584,"correlation_id":"99aba45314de0ed215646e5d5e506130","time":"2023-03-22T15:44:09.142Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-03-22T15:44:09Z","grpc.time_ms":0.471,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1583,"correlation_id":"bf8d6ec7075273f79994047288d156e5","time":"2023-03-22T15:44:09.147Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-03-22T15:44:24Z","grpc.time_ms":0.411,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1584,"correlation_id":"1984c4e3fea7064f15ebb7f623534f31","time":"2023-03-22T15:44:24.145Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-03-22T15:44:24Z","grpc.time_ms":0.329,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1583,"correlation_id":"a620a32b841161f49b2ff8d6e1cc00db","time":"2023-03-22T15:44:24.150Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-03-22T15:44:39Z","grpc.time_ms":0.491,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1584,"correlation_id":"1c42b30148211bf46cd8f379500a5da6","time":"2023-03-22T15:44:39.149Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-03-22T15:44:39Z","grpc.time_ms":0.51,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1583,"correlation_id":"6af889160dc724bb4b5ca202ec069b43","time":"2023-03-22T15:44:39.153Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-03-22T15:44:54Z","grpc.time_ms":0.328,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1584,"correlation_id":"eb8362fe81fa27240e37081db6c9c3b1","time":"2023-03-22T15:44:54.153Z"}
{"type":"gitaly-ruby","grpc.start_time":"2023-03-22T15:44:54Z","grpc.time_ms":0.295,"grpc.code":"OK","grpc.method":"Check","grpc.service":"grpc.health.v1.Health","pid":1583,"correlation_id":"0327b51a45ff06e02081c5f710e609b3","time":"2023-03-22T15:44:54.155Z"}
==> /var/log/gitlab/gitaly/current <==
{"level":"info","msg":"finished tempdir cleaner walk","storage":"default","time":"2023-03-28T14:01:06.579Z","time_ms":0}
{"level":"info","msg":"finished tempdir cleaner walk","storage":"default","time":"2023-03-28T15:01:06.579Z","time_ms":0}
{"correlation_id":"01GWMDB3B4FNB5PZJ53ME5ZKW0","grpc.code":"OK","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-web","grpc.meta.deadline_type":"regular","grpc.meta.method_type":"unary","grpc.method":"RepositoryExists","grpc.request.deadline":"2023-03-28T15:28:28.010","grpc.request.fullMethod":"/gitaly.RepositoryService/RepositoryExists","grpc.request.glProjectPath":"infrastructure/k8s-monorepo","grpc.request.glRepository":"project-5","grpc.request.payload_bytes":136,"grpc.request.repoPath":"@hashed/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git","grpc.request.repoStorage":"default","grpc.response.payload_bytes":2,"grpc.service":"gitaly.RepositoryService","grpc.start_time":"2023-03-28T15:28:18.321","grpc.time_ms":0.294,"level":"info","limit.concurrency_queue_ms":0,"msg":"finished unary call with code OK","peer.address":"@","pid":1588,"remote_ip":"10.11.2.63","span.kind":"server","system":"grpc","time":"2023-03-28T15:28:18.322Z","user_id":"2","username":"myuser"}
{"command.count":1,"command.cpu_time_ms":5,"command.inblock":392,"command.majflt":1,"command.maxrss":308164,"command.minflt":159,"command.oublock":0,"command.real_time_ms":123,"command.spawn_token_wait_ms":0,"command.system_time_ms":5,"command.user_time_ms":0,"correlation_id":"01GWMDB3B4FNB5PZJ53ME5ZKW0","grpc.code":"OK","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-web","grpc.meta.deadline_type":"regular","grpc.meta.method_type":"unary","grpc.method":"HasLocalBranches","grpc.request.deadline":"2023-03-28T15:28:28.003","grpc.request.fullMethod":"/gitaly.RepositoryService/HasLocalBranches","grpc.request.glProjectPath":"infrastructure/k8s-monorepo","grpc.request.glRepository":"project-5","grpc.request.payload_bytes":136,"grpc.request.repoPath":"@hashed/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git","grpc.request.repoStorage":"default","grpc.response.payload_bytes":2,"grpc.service":"gitaly.RepositoryService","grpc.start_time":"2023-03-28T15:28:18.334","grpc.time_ms":703.455,"level":"info","limit.concurrency_queue_ms":0,"msg":"finished unary call with code OK","peer.address":"@","pid":1588,"remote_ip":"10.11.2.63","span.kind":"server","system":"grpc","time":"2023-03-28T15:28:19.038Z","user_id":"2","username":"myuser"}
{"command.count":1,"command.cpu_time_ms":4,"command.inblock":0,"command.majflt":0,"command.maxrss":308164,"command.minflt":160,"command.oublock":0,"command.real_time_ms":5,"command.spawn_token_wait_ms":0,"command.system_time_ms":0,"command.user_time_ms":4,"correlation_id":"01GWMDB3B4FNB5PZJ53ME5ZKW0","grpc.code":"OK","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-web","grpc.meta.deadline_type":"regular","grpc.meta.method_type":"unary","grpc.method":"FindDefaultBranchName","grpc.request.deadline":"2023-03-28T15:28:29.008","grpc.request.fullMethod":"/gitaly.RefService/FindDefaultBranchName","grpc.request.glProjectPath":"infrastructure/k8s-monorepo","grpc.request.glRepository":"project-5","grpc.request.payload_bytes":136,"grpc.request.repoPath":"@hashed/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git","grpc.request.repoStorage":"default","grpc.response.payload_bytes":17,"grpc.service":"gitaly.RefService","grpc.start_time":"2023-03-28T15:28:19.080","grpc.time_ms":17.78,"level":"info","limit.concurrency_queue_ms":0,"msg":"finished unary call with code OK","peer.address":"@","pid":1588,"remote_ip":"10.11.2.63","span.kind":"server","system":"grpc","time":"2023-03-28T15:28:19.098Z","user_id":"2","username":"myuser"}
{"catfile.duration_ms":4,"catfile.flush_count":2,"catfile.flush_ms":0,"catfile.read_info_count":1,"catfile.read_info_ms":4,"catfile.read_object_count":1,"catfile.read_object_ms":0,"catfile.request_info_count":1,"catfile.request_info_ms":0,"catfile.request_object_count":1,"catfile.request_object_ms":0,"command.count":1,"command.cpu_time_ms":4,"command.inblock":0,"command.majflt":0,"command.maxrss":308164,"command.minflt":158,"command.oublock":0,"command.real_time_ms":4,"command.spawn_token_wait_ms":0,"command.system_time_ms":4,"command.user_time_ms":0,"correlation_id":"01GWMDB3B4FNB5PZJ53ME5ZKW0","grpc.code":"OK","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-web","grpc.meta.deadline_type":"regular","grpc.meta.method_type":"server_stream","grpc.method":"GetTreeEntries","grpc.request.deadline":"2023-03-28T15:28:49.077","grpc.request.fullMethod":"/gitaly.CommitService/GetTreeEntries","grpc.request.glProjectPath":"infrastructure/k8s-monorepo","grpc.request.glRepository":"project-5","grpc.request.payload_bytes":155,"grpc.request.repoPath":"@hashed/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git","grpc.request.repoStorage":"default","grpc.response.payload_bytes":970,"grpc.service":"gitaly.CommitService","grpc.start_time":"2023-03-28T15:28:19.177","grpc.time_ms":14.738,"level":"info","limit.concurrency_queue_ms":0,"msg":"finished streaming call with code OK","peer.address":"@","pid":1588,"remote_ip":"10.11.2.63","span.kind":"server","system":"grpc","time":"2023-03-28T15:28:19.192Z","user_id":"2","username":"myuser"}
{"catfile.duration_ms":5,"catfile.flush_count":2,"catfile.flush_ms":0,"catfile.read_info_count":1,"catfile.read_info_ms":4,"catfile.read_object_count":1,"catfile.read_object_ms":1,"catfile.request_info_count":1,"catfile.request_info_ms":0,"catfile.request_object_count":1,"catfile.request_object_ms":0,"command.count":1,"command.cpu_time_ms":4,"command.inblock":0,"command.majflt":0,"command.maxrss":308164,"command.minflt":156,"command.oublock":0,"command.real_time_ms":5,"command.spawn_token_wait_ms":0,"command.system_time_ms":0,"command.user_time_ms":4,"correlation_id":"01GWMDB8HHYTGCKRCB8HRX6SJT","grpc.code":"OK","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-web","grpc.meta.deadline_type":"regular","grpc.meta.method_type":"server_stream","grpc.method":"GetTreeEntries","grpc.request.deadline":"2023-03-28T15:28:55.024","grpc.request.fullMethod":"/gitaly.CommitService/GetTreeEntries","grpc.request.glProjectPath":"infrastructure/k8s-monorepo","grpc.request.glRepository":"project-5","grpc.request.payload_bytes":168,"grpc.request.repoPath":"@hashed/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git","grpc.request.repoStorage":"default","grpc.response.payload_bytes":136,"grpc.service":"gitaly.CommitService","grpc.start_time":"2023-03-28T15:28:25.325","grpc.time_ms":15.161,"level":"info","limit.concurrency_queue_ms":0,"msg":"finished streaming call with code OK","peer.address":"@","pid":1588,"remote_ip":"10.11.2.63","span.kind":"server","system":"grpc","time":"2023-03-28T15:28:25.340Z","user_id":"2","username":"myuser"}
{"level":"info","msg":"finished tempdir cleaner walk","storage":"default","time":"2023-03-28T16:01:06.580Z","time_ms":0}
{"catfile.duration_ms":5,"catfile.flush_count":2,"catfile.flush_ms":0,"catfile.read_info_count":1,"catfile.read_info_ms":4,"catfile.read_object_count":1,"catfile.read_object_ms":1,"catfile.request_info_count":1,"catfile.request_info_ms":0,"catfile.request_object_count":1,"catfile.request_object_ms":0,"command.count":1,"command.cpu_time_ms":4,"command.inblock":0,"command.majflt":0,"command.maxrss":308164,"command.minflt":152,"command.oublock":0,"command.real_time_ms":5,"command.spawn_token_wait_ms":0,"command.system_time_ms":4,"command.user_time_ms":0,"correlation_id":"01GWMFWA1WSQ5ZM1KXK3Y9P74Y","grpc.code":"OK","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-web","grpc.meta.deadline_type":"regular","grpc.meta.method_type":"server_stream","grpc.method":"GetTreeEntries","grpc.request.deadline":"2023-03-28T16:13:11.019","grpc.request.fullMethod":"/gitaly.CommitService/GetTreeEntries","grpc.request.glProjectPath":"infrastructure/k8s-monorepo","grpc.request.glRepository":"project-5","grpc.request.payload_bytes":168,"grpc.request.repoPath":"@hashed/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git","grpc.request.repoStorage":"default","grpc.response.payload_bytes":136,"grpc.service":"gitaly.CommitService","grpc.start_time":"2023-03-28T16:12:41.219","grpc.time_ms":15.583,"level":"info","limit.concurrency_queue_ms":0,"msg":"finished streaming call with code OK","peer.address":"@","pid":1588,"remote_ip":"10.11.2.63","span.kind":"server","system":"grpc","time":"2023-03-28T16:12:41.235Z","user_id":"2","username":"myuser"}
{"catfile.duration_ms":5,"catfile.flush_count":2,"catfile.flush_ms":0,"catfile.read_info_count":1,"catfile.read_info_ms":4,"catfile.read_object_count":1,"catfile.read_object_ms":1,"catfile.request_info_count":1,"catfile.request_info_ms":0,"catfile.request_object_count":1,"catfile.request_object_ms":0,"command.count":1,"command.cpu_time_ms":3,"command.inblock":0,"command.majflt":0,"command.maxrss":308164,"command.minflt":155,"command.oublock":0,"command.real_time_ms":3,"command.spawn_token_wait_ms":0,"command.system_time_ms":0,"command.user_time_ms":3,"correlation_id":"01GWMGB4BJ4KK7QRTNXG8RMP4C","grpc.code":"OK","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-web","grpc.meta.deadline_type":"regular","grpc.meta.method_type":"server_stream","grpc.method":"GetTreeEntries","grpc.request.deadline":"2023-03-28T16:21:15.001","grpc.request.fullMethod":"/gitaly.CommitService/GetTreeEntries","grpc.request.glProjectPath":"infrastructure/k8s-monorepo","grpc.request.glRepository":"project-5","grpc.request.payload_bytes":168,"grpc.request.repoPath":"@hashed/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git","grpc.request.repoStorage":"default","grpc.response.payload_bytes":136,"grpc.service":"gitaly.CommitService","grpc.start_time":"2023-03-28T16:20:45.902","grpc.time_ms":12.883,"level":"info","limit.concurrency_queue_ms":0,"msg":"finished streaming call with code OK","peer.address":"@","pid":1588,"remote_ip":"10.11.2.63","span.kind":"server","system":"grpc","time":"2023-03-28T16:20:45.915Z","user_id":"2","username":"myuser"}
==> /var/log/gitlab/gitlab-rails/exceptions_json.log <==
==> /var/log/gitlab/gitlab-rails/elasticsearch.log <==
==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2023-03-21-18-47-00.log <==
==> /var/log/gitlab/gitlab-rails/auth_json.log <==
==> /var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-2023-03-22-15-41-16.log <==
==> /var/log/gitlab/gitlab-rails/graphql_json.log <==
{"severity":"INFO","time":"2023-03-28T15:31:33.779Z","correlation_id":"01GWMDH3FQ5C8PFHNA2WYX60B1","trace_type":"execute_query","query_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=/1/nS4lEGu0cp_ZoEJvQCDvAGfheNVnaOdVNG5OisHnXzs=","duration_s":0.11650621399166994,"operation_name":"createClusterAgent","operation_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=","is_mutation":true,"variables":"{\"input\"=\u003e{\"name\"=\u003e\"k3s-monorepo-agent\", \"projectPath\"=\u003e\"infrastructure/k8s-monorepo\"}}","query_string":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n","meta.caller_id":"graphql:createClusterAgent","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","query_analysis.duration_s":0.009876397991320118,"query_analysis.depth":6,"query_analysis.complexity":31,"query_analysis.used_fields":["ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","CreateClusterAgentPayload.clusterAgent","CreateClusterAgentPayload.errors","CreateClusterAgentPayload.__typename","Mutation.createClusterAgent"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-03-28T16:12:38.626Z","correlation_id":"01GWMFWA1WSQ5ZM1KXK3Y9P74Y","trace_type":"execute_query","query_fingerprint":"agentConfigurations/NsK33nkL6xInoaphnn74WZnvH7Y70rAdFNe4lvbfYrk=/1/QKkrzvVW-Pxj9iW4I0BpCWXXbKja8svLMfEaNczycyA=","duration_s":0.009548381989588961,"operation_name":"agentConfigurations","operation_fingerprint":"agentConfigurations/NsK33nkL6xInoaphnn74WZnvH7Y70rAdFNe4lvbfYrk=","is_mutation":false,"variables":"{\"projectPath\"=\u003e\"infrastructure/k8s-monorepo\"}","query_string":"query agentConfigurations($projectPath: ID!) {\n project(fullPath: $projectPath) {\n id\n agentConfigurations {\n nodes {\n agentName\n __typename\n }\n __typename\n }\n clusterAgents {\n nodes {\n id\n name\n __typename\n }\n __typename\n }\n __typename\n }\n}\n","meta.caller_id":"graphql:agentConfigurations","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","query_analysis.duration_s":0.016062858994700946,"query_analysis.depth":7,"query_analysis.complexity":62,"query_analysis.used_fields":["Project.id","ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","ClusterAgentConnection.nodes","PageInfo.hasNextPage","PageInfo.hasPreviousPage","PageInfo.startCursor","PageInfo.endCursor","PageInfo.__typename","ClusterAgentConnection.pageInfo","ClusterAgentConnection.count","ClusterAgentConnection.__typename","Project.clusterAgents","TreeEntry.id","TreeEntry.name","TreeEntry.path","TreeEntry.webPath","TreeEntry.__typename","TreeEntryConnection.nodes","TreeEntryConnection.__typename","Tree.trees","Tree.__typename","Repository.tree","Repository.__typename","Project.repository","Project.__typename","Query.project"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-03-28T16:12:38.631Z","correlation_id":"01GWMFWA1WSQ5ZM1KXK3Y9P74Y","trace_type":"execute_query","query_fingerprint":"getAgents/KAaYeJVrCDjNpOKCG6kKacYVwAJ2IudnrZkZ9XmOoTk=/4/RvdwoDZ2QSrlOJ2MTlOw_Pfg0F_x-Zj5X3TNp_SlDXc=","duration_s":0.0016185370041057467,"operation_name":"getAgents","operation_fingerprint":"getAgents/KAaYeJVrCDjNpOKCG6kKacYVwAJ2IudnrZkZ9XmOoTk=","is_mutation":false,"variables":"{\"defaultBranchName\"=\u003e\"main\", \"projectPath\"=\u003e\"infrastructure/k8s-monorepo\", \"first\"=\u003e25, \"last\"=\u003enil}","query_string":"query getAgents($defaultBranchName: String!, $projectPath: ID!, $first: Int, $last: Int, $afterAgent: String, $beforeAgent: String) {\n project(fullPath: $projectPath) {\n id\n clusterAgents(\n first: $first\n last: $last\n before: $beforeAgent\n after: $afterAgent\n ) {\n nodes {\n ...ClusterAgentFragment\n __typename\n }\n pageInfo {\n ...PageInfo\n __typename\n }\n count\n __typename\n }\n repository {\n tree(path: \".gitlab/agents\", ref: $defaultBranchName) {\n trees {\n nodes {\n id\n name\n path\n webPath\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n}\n\nfragment PageInfo on PageInfo {\n hasNextPage\n hasPreviousPage\n startCursor\n endCursor\n __typename\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n","meta.caller_id":"graphql:getAgents","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","query_analysis.duration_s":0.016062858994700946,"query_analysis.depth":7,"query_analysis.complexity":62,"query_analysis.used_fields":["Project.id","ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","ClusterAgentConnection.nodes","PageInfo.hasNextPage","PageInfo.hasPreviousPage","PageInfo.startCursor","PageInfo.endCursor","PageInfo.__typename","ClusterAgentConnection.pageInfo","ClusterAgentConnection.count","ClusterAgentConnection.__typename","Project.clusterAgents","TreeEntry.id","TreeEntry.name","TreeEntry.path","TreeEntry.webPath","TreeEntry.__typename","TreeEntryConnection.nodes","TreeEntryConnection.__typename","Tree.trees","Tree.__typename","Repository.tree","Repository.__typename","Project.repository","Project.__typename","Query.project"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-03-28T16:20:43.694Z","correlation_id":"01GWMGB4BJ4KK7QRTNXG8RMP4C","trace_type":"execute_query","query_fingerprint":"agentConfigurations/NsK33nkL6xInoaphnn74WZnvH7Y70rAdFNe4lvbfYrk=/1/QKkrzvVW-Pxj9iW4I0BpCWXXbKja8svLMfEaNczycyA=","duration_s":0.0016127370035974309,"operation_name":"agentConfigurations","operation_fingerprint":"agentConfigurations/NsK33nkL6xInoaphnn74WZnvH7Y70rAdFNe4lvbfYrk=","is_mutation":false,"variables":"{\"projectPath\"=\u003e\"infrastructure/k8s-monorepo\"}","query_string":"query agentConfigurations($projectPath: ID!) {\n project(fullPath: $projectPath) {\n id\n agentConfigurations {\n nodes {\n agentName\n __typename\n }\n __typename\n }\n clusterAgents {\n nodes {\n id\n name\n __typename\n }\n __typename\n }\n __typename\n }\n}\n","meta.caller_id":"graphql:agentConfigurations","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","query_analysis.duration_s":0.016576346999499947,"query_analysis.depth":7,"query_analysis.complexity":62,"query_analysis.used_fields":["Project.id","ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","ClusterAgentConnection.nodes","PageInfo.hasNextPage","PageInfo.hasPreviousPage","PageInfo.startCursor","PageInfo.endCursor","PageInfo.__typename","ClusterAgentConnection.pageInfo","ClusterAgentConnection.count","ClusterAgentConnection.__typename","Project.clusterAgents","TreeEntry.id","TreeEntry.name","TreeEntry.path","TreeEntry.webPath","TreeEntry.__typename","TreeEntryConnection.nodes","TreeEntryConnection.__typename","Tree.trees","Tree.__typename","Repository.tree","Repository.__typename","Project.repository","Project.__typename","Query.project"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-03-28T16:20:43.697Z","correlation_id":"01GWMGB4BJ4KK7QRTNXG8RMP4C","trace_type":"execute_query","query_fingerprint":"getAgents/KAaYeJVrCDjNpOKCG6kKacYVwAJ2IudnrZkZ9XmOoTk=/4/RvdwoDZ2QSrlOJ2MTlOw_Pfg0F_x-Zj5X3TNp_SlDXc=","duration_s":0.0013559950020862743,"operation_name":"getAgents","operation_fingerprint":"getAgents/KAaYeJVrCDjNpOKCG6kKacYVwAJ2IudnrZkZ9XmOoTk=","is_mutation":false,"variables":"{\"defaultBranchName\"=\u003e\"main\", \"projectPath\"=\u003e\"infrastructure/k8s-monorepo\", \"first\"=\u003e25, \"last\"=\u003enil}","query_string":"query getAgents($defaultBranchName: String!, $projectPath: ID!, $first: Int, $last: Int, $afterAgent: String, $beforeAgent: String) {\n project(fullPath: $projectPath) {\n id\n clusterAgents(\n first: $first\n last: $last\n before: $beforeAgent\n after: $afterAgent\n ) {\n nodes {\n ...ClusterAgentFragment\n __typename\n }\n pageInfo {\n ...PageInfo\n __typename\n }\n count\n __typename\n }\n repository {\n tree(path: \".gitlab/agents\", ref: $defaultBranchName) {\n trees {\n nodes {\n id\n name\n path\n webPath\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n}\n\nfragment PageInfo on PageInfo {\n hasNextPage\n hasPreviousPage\n startCursor\n endCursor\n __typename\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n","meta.caller_id":"graphql:getAgents","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","query_analysis.duration_s":0.016576346999499947,"query_analysis.depth":7,"query_analysis.complexity":62,"query_analysis.used_fields":["Project.id","ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","ClusterAgentConnection.nodes","PageInfo.hasNextPage","PageInfo.hasPreviousPage","PageInfo.startCursor","PageInfo.endCursor","PageInfo.__typename","ClusterAgentConnection.pageInfo","ClusterAgentConnection.count","ClusterAgentConnection.__typename","Project.clusterAgents","TreeEntry.id","TreeEntry.name","TreeEntry.path","TreeEntry.webPath","TreeEntry.__typename","TreeEntryConnection.nodes","TreeEntryConnection.__typename","Tree.trees","Tree.__typename","Repository.tree","Repository.__typename","Project.repository","Project.__typename","Query.project"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-03-28T16:26:52.800Z","correlation_id":"01GWMGPCQKFX128336D49T41H5","trace_type":"execute_query","query_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=/1/uy2vq6AkbYHSt-sEmzFrZ3GQdGLuLNaUnyo9KdU-Utc=","duration_s":0.09887804900063202,"operation_name":"createClusterAgent","operation_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=","is_mutation":true,"variables":"{\"input\"=\u003e{\"name\"=\u003e\"I-am-a-test-agent-name\", \"projectPath\"=\u003e\"infrastructure/k8s-monorepo\"}}","query_string":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n","meta.caller_id":"graphql:createClusterAgent","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","query_analysis.duration_s":0.007821002000127919,"query_analysis.depth":6,"query_analysis.complexity":31,"query_analysis.used_fields":["ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","CreateClusterAgentPayload.clusterAgent","CreateClusterAgentPayload.errors","CreateClusterAgentPayload.__typename","Mutation.createClusterAgent"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-03-28T16:27:19.613Z","correlation_id":"01GWMGQ4S5YWW5NNR1ARHHMCKQ","trace_type":"execute_query","query_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=/1/MBF1udFABlnJxI-UmIDdssC26t4YGO2AcLWqA87lXBQ=","duration_s":2.2567087759962305,"operation_name":"createClusterAgent","operation_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=","is_mutation":true,"variables":"{\"input\"=\u003e{\"name\"=\u003e\"i-am-a-test-agent\", \"projectPath\"=\u003e\"infrastructure/k8s-monorepo\"}}","query_string":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n","meta.caller_id":"graphql:createClusterAgent","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","query_analysis.duration_s":0.016775429001427256,"query_analysis.depth":6,"query_analysis.complexity":31,"query_analysis.used_fields":["ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","CreateClusterAgentPayload.clusterAgent","CreateClusterAgentPayload.errors","CreateClusterAgentPayload.__typename","Mutation.createClusterAgent"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-03-28T16:28:58.767Z","correlation_id":"01GWMGT7SQTWFHT039BS8XXX42","trace_type":"execute_query","query_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=/1/MBF1udFABlnJxI-UmIDdssC26t4YGO2AcLWqA87lXBQ=","duration_s":0.07471416899352334,"operation_name":"createClusterAgent","operation_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=","is_mutation":true,"variables":"{\"input\"=\u003e{\"name\"=\u003e\"i-am-a-test-agent\", \"projectPath\"=\u003e\"infrastructure/k8s-monorepo\"}}","query_string":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n","meta.caller_id":"graphql:createClusterAgent","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","query_analysis.duration_s":0.008053875004407018,"query_analysis.depth":6,"query_analysis.complexity":31,"query_analysis.used_fields":["ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","CreateClusterAgentPayload.clusterAgent","CreateClusterAgentPayload.errors","CreateClusterAgentPayload.__typename","Mutation.createClusterAgent"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-03-28T16:29:20.859Z","correlation_id":"01GWMGTXBK8S0TEXKJ9HS9BQFS","trace_type":"execute_query","query_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=/1/MBF1udFABlnJxI-UmIDdssC26t4YGO2AcLWqA87lXBQ=","duration_s":0.09165865699469578,"operation_name":"createClusterAgent","operation_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=","is_mutation":true,"variables":"{\"input\"=\u003e{\"name\"=\u003e\"i-am-a-test-agent\", \"projectPath\"=\u003e\"infrastructure/k8s-monorepo\"}}","query_string":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n","meta.caller_id":"graphql:createClusterAgent","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","query_analysis.duration_s":0.009142016991972923,"query_analysis.depth":6,"query_analysis.complexity":31,"query_analysis.used_fields":["ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","CreateClusterAgentPayload.clusterAgent","CreateClusterAgentPayload.errors","CreateClusterAgentPayload.__typename","Mutation.createClusterAgent"],"query_analysis.used_deprecated_fields":[]}
{"severity":"INFO","time":"2023-03-28T16:29:45.268Z","correlation_id":"01GWMGVK856R6FEX2B02EED8TR","trace_type":"execute_query","query_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=/1/-75Bixa39ggSSiLIW7nKoy9pHl3EmGdBzHM5OsvypPY=","duration_s":2.1181385529926047,"operation_name":"createClusterAgent","operation_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=","is_mutation":true,"variables":"{\"input\"=\u003e{\"name\"=\u003e\"i-am-a-test-agent-2\", \"projectPath\"=\u003e\"infrastructure/k8s-monorepo\"}}","query_string":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n","meta.caller_id":"graphql:createClusterAgent","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","query_analysis.duration_s":0.005402105991379358,"query_analysis.depth":6,"query_analysis.complexity":31,"query_analysis.used_fields":["ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","CreateClusterAgentPayload.clusterAgent","CreateClusterAgentPayload.errors","CreateClusterAgentPayload.__typename","Mutation.createClusterAgent"],"query_analysis.used_deprecated_fields":[]}
==> /var/log/gitlab/gitlab-rails/importer.log <==
==> /var/log/gitlab/gitlab-rails/audit_json.log <==
==> /var/log/gitlab/gitlab-rails/database_load_balancing.log <==
==> /var/log/gitlab/gitlab-rails/git_json.log <==
==> /var/log/gitlab/gitlab-rails/sidekiq_client.log <==
==> /var/log/gitlab/gitlab-rails/service_measurement.log <==
==> /var/log/gitlab/gitlab-rails/application.log <==
==> /var/log/gitlab/gitlab-rails/grpc.log <==
==> /var/log/gitlab/gitlab-rails/application_json.log <==
{"severity":"INFO","time":"2023-03-28T13:20:04.501Z","correlation_id":"e911c6c9e2e103edf9cca40c0474fb22","message":"Ci::StuckBuilds::DropRunningService: Cleaning running, timed-out builds"}
{"severity":"INFO","time":"2023-03-28T13:40:05.805Z","correlation_id":"6d8a381b59d970207fce4e6d2dc2cd68","message":"Ci::StuckBuilds::DropScheduledService: Cleaning scheduled, timed-out builds"}
{"severity":"INFO","time":"2023-03-28T14:00:09.243Z","correlation_id":"b18d38692636dd6d191f02a1ef5a5236","message":"Ci::StuckBuilds::DropPendingService: Cleaning pending timed-out builds"}
{"severity":"INFO","time":"2023-03-28T14:20:12.867Z","correlation_id":"db23b697a8ab5ae475eef336c07e5a2d","message":"Ci::StuckBuilds::DropRunningService: Cleaning running, timed-out builds"}
{"severity":"INFO","time":"2023-03-28T14:40:13.772Z","correlation_id":"e83c0bf124abd2f699f94657057b9a24","message":"Ci::StuckBuilds::DropScheduledService: Cleaning scheduled, timed-out builds"}
{"severity":"INFO","time":"2023-03-28T15:00:07.452Z","correlation_id":"96881a807dc96a9c63f32acc993d2410","message":"Ci::StuckBuilds::DropPendingService: Cleaning pending timed-out builds"}
{"severity":"INFO","time":"2023-03-28T15:20:14.195Z","correlation_id":"3556265910dbf979d530fcd3216d1d7b","message":"Ci::StuckBuilds::DropRunningService: Cleaning running, timed-out builds"}
{"severity":"INFO","time":"2023-03-28T15:40:09.134Z","correlation_id":"2cb4a4fd53260ec6375f365a4e44bf5a","message":"Ci::StuckBuilds::DropScheduledService: Cleaning scheduled, timed-out builds"}
{"severity":"INFO","time":"2023-03-28T16:00:03.510Z","correlation_id":"b99bcebd57975fda1b7d2591333f525b","message":"Ci::StuckBuilds::DropPendingService: Cleaning pending timed-out builds"}
{"severity":"INFO","time":"2023-03-28T16:20:07.946Z","correlation_id":"2ca26f16a92a984955f96845de12606b","message":"Ci::StuckBuilds::DropRunningService: Cleaning running, timed-out builds"}
==> /var/log/gitlab/gitlab-rails/api_json.log <==
{"time":"2023-03-28T12:36:09.320Z","severity":"INFO","duration_s":0.17613,"db_duration_s":0.08224,"view_duration_s":0.09389,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"15.10.0","revision":"456e3482","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null,"service_variables":"[FILTERED]","service_multiple_aliases":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"},{"key":"system_id","value":"s_51a726a890af"},{"key":"last_update","value":"082796e3f228a6ddadde9647ed6e72f8"}],"host":"gitlab.obfuscated.domain","remote_ip":"xx.xx.xx.xxx, 127.0.0.1","ua":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.023244,"redis_calls":15,"redis_duration_s":0.026901,"redis_read_bytes":228,"redis_write_bytes":1217,"redis_cache_calls":2,"redis_cache_duration_s":0.001655,"redis_cache_read_bytes":193,"redis_cache_write_bytes":262,"redis_shared_state_calls":13,"redis_shared_state_duration_s":0.025246,"redis_shared_state_read_bytes":35,"redis_shared_state_write_bytes":955,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_replica_count":0,"db_primary_count":6,"db_main_count":6,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.083,"db_main_duration_s":0.083,"db_main_replica_duration_s":0.0,"cpu_s":0.102535,"mem_objects":10613,"mem_bytes":1031584,"mem_mallocs":3017,"mem_total_bytes":1456104,"pid":1723,"worker_id":"puma_1","rate_limiting_gates":[],"correlation_id":"01GWM3FXRQ2KADBD8607FZQQR7","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"xx.xx.xx.xxx","meta.feature_category":"continuous_integration","meta.client_id":"runner/3","content_length":"773","request_urgency":"low","target_duration_s":5}
{"time":"2023-03-28T13:36:09.199Z","severity":"INFO","duration_s":0.24687,"db_duration_s":0.12416,"view_duration_s":0.12271,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"15.10.0","revision":"456e3482","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null,"service_variables":"[FILTERED]","service_multiple_aliases":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"},{"key":"system_id","value":"s_51a726a890af"},{"key":"last_update","value":"063bb5494ae2874fa32c21147fb509fb"}],"host":"gitlab.obfuscated.domain","remote_ip":"xx.xx.xx.xxx, 127.0.0.1","ua":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.116675,"redis_calls":28,"redis_duration_s":0.020086,"redis_read_bytes":358,"redis_write_bytes":3622,"redis_cache_calls":15,"redis_cache_duration_s":0.011007,"redis_cache_read_bytes":323,"redis_cache_write_bytes":2511,"redis_shared_state_calls":13,"redis_shared_state_duration_s":0.009079,"redis_shared_state_read_bytes":35,"redis_shared_state_write_bytes":1111,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_replica_count":0,"db_primary_count":13,"db_main_count":13,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.147,"db_main_duration_s":0.147,"db_main_replica_duration_s":0.0,"cpu_s":0.210273,"mem_objects":20441,"mem_bytes":2498775,"mem_mallocs":6073,"mem_total_bytes":3316415,"pid":1725,"worker_id":"puma_2","rate_limiting_gates":[],"correlation_id":"01GWM6XS3R24C1Z0N6GFE9WX0N","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"xx.xx.xx.xxx","meta.feature_category":"continuous_integration","meta.client_id":"runner/4","content_length":"773","request_urgency":"low","target_duration_s":5}
{"time":"2023-03-28T13:36:10.143Z","severity":"INFO","duration_s":0.18474,"db_duration_s":0.04361,"view_duration_s":0.14113,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"15.10.0","revision":"456e3482","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null,"service_variables":"[FILTERED]","service_multiple_aliases":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"},{"key":"system_id","value":"s_51a726a890af"},{"key":"last_update","value":"898c577635657e408e5114f25ee3a5e1"}],"host":"gitlab.obfuscated.domain","remote_ip":"xx.xx.xx.xxx, 127.0.0.1","ua":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.124627,"redis_calls":22,"redis_duration_s":0.061214000000000005,"redis_read_bytes":1564,"redis_write_bytes":1887,"redis_cache_calls":9,"redis_cache_duration_s":0.040056,"redis_cache_read_bytes":1529,"redis_cache_write_bytes":776,"redis_shared_state_calls":13,"redis_shared_state_duration_s":0.021158,"redis_shared_state_read_bytes":35,"redis_shared_state_write_bytes":1111,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_replica_count":0,"db_primary_count":7,"db_main_count":7,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.078,"db_main_duration_s":0.078,"db_main_replica_duration_s":0.0,"cpu_s":0.191564,"mem_objects":16922,"mem_bytes":2256327,"mem_mallocs":5589,"mem_total_bytes":2933207,"pid":1721,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01GWM6XT317ZYN9GWFKF46TNQX","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"xx.xx.xx.xxx","meta.feature_category":"continuous_integration","meta.client_id":"runner/5","content_length":"773","request_urgency":"low","target_duration_s":5}
{"time":"2023-03-28T13:36:11.608Z","severity":"INFO","duration_s":0.6559,"db_duration_s":0.56451,"view_duration_s":0.09139,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"15.10.0","revision":"456e3482","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null,"service_variables":"[FILTERED]","service_multiple_aliases":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"},{"key":"system_id","value":"s_51a726a890af"},{"key":"last_update","value":"52846c8f133c6fd50becfd15e32a42df"}],"host":"gitlab.obfuscated.domain","remote_ip":"xx.xx.xx.xxx, 127.0.0.1","ua":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.112546,"redis_calls":26,"redis_duration_s":0.02353,"redis_read_bytes":2376,"redis_write_bytes":2202,"redis_cache_calls":13,"redis_cache_duration_s":0.014006,"redis_cache_read_bytes":2341,"redis_cache_write_bytes":1091,"redis_shared_state_calls":13,"redis_shared_state_duration_s":0.009524,"redis_shared_state_read_bytes":35,"redis_shared_state_write_bytes":1111,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_replica_count":0,"db_primary_count":7,"db_main_count":7,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.588,"db_main_duration_s":0.588,"db_main_replica_duration_s":0.0,"cpu_s":0.184394,"mem_objects":17680,"mem_bytes":2247127,"mem_mallocs":5962,"mem_total_bytes":2954327,"pid":1727,"worker_id":"puma_3","rate_limiting_gates":[],"correlation_id":"01GWM6XV298XZBQ0V5XD60DSXF","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"xx.xx.xx.xxx","meta.feature_category":"continuous_integration","meta.client_id":"runner/3","content_length":"773","request_urgency":"low","target_duration_s":5}
{"time":"2023-03-28T14:36:10.803Z","severity":"INFO","duration_s":0.19423,"db_duration_s":0.07544,"view_duration_s":0.11879,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"15.10.0","revision":"456e3482","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null,"service_variables":"[FILTERED]","service_multiple_aliases":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"},{"key":"system_id","value":"s_51a726a890af"},{"key":"last_update","value":"adcf0d85f417a17f4dba4d8e618d6807"}],"host":"gitlab.obfuscated.domain","remote_ip":"xx.xx.xx.xxx, 127.0.0.1","ua":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.090563,"redis_calls":28,"redis_duration_s":0.022012,"redis_read_bytes":358,"redis_write_bytes":3619,"redis_cache_calls":15,"redis_cache_duration_s":0.01145,"redis_cache_read_bytes":323,"redis_cache_write_bytes":2508,"redis_shared_state_calls":13,"redis_shared_state_duration_s":0.010562,"redis_shared_state_read_bytes":35,"redis_shared_state_write_bytes":1111,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_replica_count":0,"db_primary_count":13,"db_main_count":13,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.098,"db_main_duration_s":0.098,"db_main_replica_duration_s":0.0,"cpu_s":0.195377,"mem_objects":19207,"mem_bytes":2539895,"mem_mallocs":5907,"mem_total_bytes":3308175,"pid":1727,"worker_id":"puma_3","rate_limiting_gates":[],"correlation_id":"01GWMABPCDBN29R2DJ2GP5EYEF","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"xx.xx.xx.xxx","meta.feature_category":"continuous_integration","meta.client_id":"runner/4","content_length":"773","request_urgency":"low","target_duration_s":5}
{"time":"2023-03-28T14:36:12.312Z","severity":"INFO","duration_s":0.70462,"db_duration_s":0.58838,"view_duration_s":0.11624,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"15.10.0","revision":"456e3482","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null,"service_variables":"[FILTERED]","service_multiple_aliases":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"},{"key":"system_id","value":"s_51a726a890af"},{"key":"last_update","value":"71751ef1e5550f21fbeb5e04cd06fe50"}],"host":"gitlab.obfuscated.domain","remote_ip":"xx.xx.xx.xxx, 127.0.0.1","ua":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.08904,"redis_calls":22,"redis_duration_s":0.041337,"redis_read_bytes":1561,"redis_write_bytes":1887,"redis_cache_calls":9,"redis_cache_duration_s":0.017606,"redis_cache_read_bytes":1526,"redis_cache_write_bytes":776,"redis_shared_state_calls":13,"redis_shared_state_duration_s":0.023731,"redis_shared_state_read_bytes":35,"redis_shared_state_write_bytes":1111,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_replica_count":0,"db_primary_count":7,"db_main_count":7,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.611,"db_main_duration_s":0.611,"db_main_replica_duration_s":0.0,"cpu_s":0.16435,"mem_objects":16949,"mem_bytes":2257495,"mem_mallocs":5613,"mem_total_bytes":2935455,"pid":1723,"worker_id":"puma_1","rate_limiting_gates":[],"correlation_id":"01GWMABQBNSVBT598RA0QF2EZH","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"xx.xx.xx.xxx","meta.feature_category":"continuous_integration","meta.client_id":"runner/5","content_length":"773","request_urgency":"low","target_duration_s":5}
{"time":"2023-03-28T14:36:12.740Z","severity":"INFO","duration_s":0.19614,"db_duration_s":0.12019,"view_duration_s":0.07595,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"15.10.0","revision":"456e3482","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null,"service_variables":"[FILTERED]","service_multiple_aliases":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"},{"key":"system_id","value":"s_51a726a890af"},{"key":"last_update","value":"d3ca7b998c55f26e2ce47e57f3968eaa"}],"host":"gitlab.obfuscated.domain","remote_ip":"xx.xx.xx.xxx, 127.0.0.1","ua":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.023786,"redis_calls":15,"redis_duration_s":0.009717,"redis_read_bytes":228,"redis_write_bytes":1217,"redis_cache_calls":2,"redis_cache_duration_s":0.001573,"redis_cache_read_bytes":193,"redis_cache_write_bytes":262,"redis_shared_state_calls":13,"redis_shared_state_duration_s":0.008144,"redis_shared_state_read_bytes":35,"redis_shared_state_write_bytes":955,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_replica_count":0,"db_primary_count":6,"db_main_count":6,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.121,"db_main_duration_s":0.121,"db_main_replica_duration_s":0.0,"cpu_s":0.10484,"mem_objects":10607,"mem_bytes":908704,"mem_mallocs":3011,"mem_total_bytes":1332984,"pid":1723,"worker_id":"puma_1","rate_limiting_gates":[],"correlation_id":"01GWMABRAYZYZ8Z9M4JPGJ7RZH","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"xx.xx.xx.xxx","meta.feature_category":"continuous_integration","meta.client_id":"runner/3","content_length":"773","request_urgency":"low","target_duration_s":5}
{"time":"2023-03-28T15:36:12.848Z","severity":"INFO","duration_s":0.54516,"db_duration_s":0.42043,"view_duration_s":0.12473,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"15.10.0","revision":"456e3482","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null,"service_variables":"[FILTERED]","service_multiple_aliases":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"},{"key":"system_id","value":"s_51a726a890af"},{"key":"last_update","value":"8052a232ef98eaf202072bee76768a64"}],"host":"gitlab.obfuscated.domain","remote_ip":"xx.xx.xx.xxx, 127.0.0.1","ua":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.073337,"redis_calls":28,"redis_duration_s":0.021764,"redis_read_bytes":358,"redis_write_bytes":3619,"redis_cache_calls":15,"redis_cache_duration_s":0.010858,"redis_cache_read_bytes":323,"redis_cache_write_bytes":2508,"redis_shared_state_calls":13,"redis_shared_state_duration_s":0.010906,"redis_shared_state_read_bytes":35,"redis_shared_state_write_bytes":1111,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_replica_count":0,"db_primary_count":13,"db_main_count":13,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.432,"db_main_duration_s":0.432,"db_main_replica_duration_s":0.0,"cpu_s":0.191506,"mem_objects":19518,"mem_bytes":2294455,"mem_mallocs":5930,"mem_total_bytes":3075175,"pid":1723,"worker_id":"puma_1","rate_limiting_gates":[],"correlation_id":"01GWMDSKNW8ZTJFTJ1DTD7J8V8","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"xx.xx.xx.xxx","meta.feature_category":"continuous_integration","meta.client_id":"runner/4","content_length":"773","request_urgency":"low","target_duration_s":5}
{"time":"2023-03-28T15:36:13.571Z","severity":"INFO","duration_s":0.26158,"db_duration_s":0.20178,"view_duration_s":0.0598,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"15.10.0","revision":"456e3482","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null,"service_variables":"[FILTERED]","service_multiple_aliases":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"},{"key":"system_id","value":"s_51a726a890af"},{"key":"last_update","value":"f32d002a71e5b2da676c202da8ad02c3"}],"host":"gitlab.obfuscated.domain","remote_ip":"xx.xx.xx.xxx, 127.0.0.1","ua":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.082347,"redis_calls":22,"redis_duration_s":0.011141000000000002,"redis_read_bytes":1561,"redis_write_bytes":1887,"redis_cache_calls":9,"redis_cache_duration_s":0.004695,"redis_cache_read_bytes":1526,"redis_cache_write_bytes":776,"redis_shared_state_calls":13,"redis_shared_state_duration_s":0.006446,"redis_shared_state_read_bytes":35,"redis_shared_state_write_bytes":1111,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_replica_count":0,"db_primary_count":7,"db_main_count":7,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.224,"db_main_duration_s":0.224,"db_main_replica_duration_s":0.0,"cpu_s":0.123969,"mem_objects":16933,"mem_bytes":1929815,"mem_mallocs":5597,"mem_total_bytes":2607135,"pid":1721,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01GWMDSMN318FR5XSWPWDVJ8NR","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"xx.xx.xx.xxx","meta.feature_category":"continuous_integration","meta.client_id":"runner/5","content_length":"773","request_urgency":"low","target_duration_s":5}
{"time":"2023-03-28T15:36:14.368Z","severity":"INFO","duration_s":0.11244,"db_duration_s":0.02844,"view_duration_s":0.084,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"15.10.0","revision":"456e3482","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null,"service_variables":"[FILTERED]","service_multiple_aliases":null},"config":{"gpus":""}}},{"key":"token","value":"[FILTERED]"},{"key":"system_id","value":"s_51a726a890af"},{"key":"last_update","value":"6275db44a0e6e7b9c7c3231d08f2823f"}],"host":"gitlab.obfuscated.domain","remote_ip":"xx.xx.xx.xxx, 127.0.0.1","ua":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.025818,"redis_calls":15,"redis_duration_s":0.011567000000000001,"redis_read_bytes":228,"redis_write_bytes":1217,"redis_cache_calls":2,"redis_cache_duration_s":0.001423,"redis_cache_read_bytes":193,"redis_cache_write_bytes":262,"redis_shared_state_calls":13,"redis_shared_state_duration_s":0.010144,"redis_shared_state_read_bytes":35,"redis_shared_state_write_bytes":955,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_replica_count":0,"db_primary_count":6,"db_main_count":6,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_duration_s":0.029,"db_main_replica_duration_s":0.0,"cpu_s":0.103736,"mem_objects":10605,"mem_bytes":867744,"mem_mallocs":3009,"mem_total_bytes":1291944,"pid":1721,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"01GWMDSNMCTNZ4Z7QVKVYVSYQ7","meta.caller_id":"POST /api/:version/jobs/request","meta.remote_ip":"xx.xx.xx.xxx","meta.feature_category":"continuous_integration","meta.client_id":"runner/3","content_length":"773","request_urgency":"low","target_duration_s":5}
==> /var/log/gitlab/gitlab-rails/backup_json.log <==
==> /var/log/gitlab/gitlab-rails/production.log <==
==> /var/log/gitlab/gitlab-rails/migrations.log <==
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"POST","path":"/api/graphql","format":"*/*","controller":"GraphqlController","action":"execute","status":200,"time":"2023-03-28T16:29:20.866Z","params":[{"key":"_json","value":[{"operationName":"createClusterAgent","variables":"[FILTERED]","query":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n"}]},{"key":"graphql","value":{"_json":[{"operationName":"createClusterAgent","variables":"[FILTERED]","query":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n"}]}}],"correlation_id":"01GWMGTXBK8S0TEXKJ9HS9BQFS","meta.caller_id":"GraphqlController#execute","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","graphql":[{"depth":6,"complexity":31,"used_fields":["ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","CreateClusterAgentPayload.clusterAgent","CreateClusterAgentPayload.errors","CreateClusterAgentPayload.__typename","Mutation.createClusterAgent"],"used_deprecated_fields":[],"variables":"{\"input\"=>{\"name\"=>\"i-am-a-test-agent\", \"projectPath\"=>\"infrastructure/k8s-monorepo\"}}","operation_name":"createClusterAgent"}],"remote_ip":"10.11.2.63","user_id":2,"username":"myuser","ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605.1.15","queue_duration_s":0.062803,"request_urgency":"low","target_duration_s":5,"redis_calls":15,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.010382,"redis_read_bytes":1726,"redis_write_bytes":2563,"redis_cache_calls":10,"redis_cache_duration_s":0.006437,"redis_cache_read_bytes":1543,"redis_cache_write_bytes":826,"redis_sessions_calls":4,"redis_sessions_allowed_cross_slot_calls":1,"redis_sessions_duration_s":0.002572,"redis_sessions_read_bytes":183,"redis_sessions_write_bytes":1631,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.001373,"redis_shared_state_write_bytes":106,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_replica_count":0,"db_primary_count":9,"db_main_count":9,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_duration_s":0.044,"db_main_replica_duration_s":0.0,"cpu_s":0.195853,"mem_objects":28082,"mem_bytes":2734743,"mem_mallocs":8026,"mem_total_bytes":3858023,"pid":1723,"worker_id":"puma_1","rate_limiting_gates":[],"db_duration_s":0.02944,"view_duration_s":0.00065,"duration_s":0.15025}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-03-28T16:29:34.436Z","params":[],"redis_calls":4,"redis_duration_s":0.00212,"redis_read_bytes":810,"redis_write_bytes":335,"redis_cache_calls":4,"redis_cache_duration_s":0.00212,"redis_cache_read_bytes":810,"redis_cache_write_bytes":335,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_main_replica_duration_s":0.0,"cpu_s":0.177847,"mem_objects":2900,"mem_bytes":1879328,"mem_mallocs":10856,"mem_total_bytes":1995328,"pid":1725,"worker_id":"puma_2","rate_limiting_gates":[],"correlation_id":"332e2f95-8ca1-44eb-97e2-65032d953fe5","db_duration_s":0.0,"view_duration_s":0.00108,"duration_s":0.1646}
{"method":"POST","path":"/api/graphql","format":"*/*","controller":"GraphqlController","action":"execute","status":200,"time":"2023-03-28T16:29:45.276Z","params":[{"key":"_json","value":[{"operationName":"createClusterAgent","variables":"[FILTERED]","query":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n"}]},{"key":"graphql","value":{"_json":[{"operationName":"createClusterAgent","variables":"[FILTERED]","query":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n"}]}}],"correlation_id":"01GWMGVK856R6FEX2B02EED8TR","meta.caller_id":"GraphqlController#execute","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","graphql":[{"depth":6,"complexity":31,"used_fields":["ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","CreateClusterAgentPayload.clusterAgent","CreateClusterAgentPayload.errors","CreateClusterAgentPayload.__typename","Mutation.createClusterAgent"],"used_deprecated_fields":[],"variables":"{\"input\"=>{\"name\"=>\"i-am-a-test-agent-2\", \"projectPath\"=>\"infrastructure/k8s-monorepo\"}}","operation_name":"createClusterAgent"}],"remote_ip":"10.11.2.63","user_id":2,"username":"myuser","ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605.1.15","queue_duration_s":0.056022,"request_urgency":"low","target_duration_s":5,"redis_calls":16,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.011825,"redis_read_bytes":1844,"redis_write_bytes":2708,"redis_cache_calls":11,"redis_cache_duration_s":0.006982,"redis_cache_read_bytes":1661,"redis_cache_write_bytes":971,"redis_sessions_calls":4,"redis_sessions_allowed_cross_slot_calls":1,"redis_sessions_duration_s":0.003965,"redis_sessions_read_bytes":183,"redis_sessions_write_bytes":1631,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000878,"redis_shared_state_write_bytes":106,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_replica_count":0,"db_primary_count":12,"db_main_count":12,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.061,"db_main_duration_s":0.061,"db_main_replica_duration_s":0.0,"cpu_s":0.176685,"mem_objects":25391,"mem_bytes":2892655,"mem_mallocs":7741,"mem_total_bytes":3908295,"pid":1727,"worker_id":"puma_3","rate_limiting_gates":[],"db_duration_s":0.05124,"view_duration_s":0.00104,"duration_s":2.1524}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-03-28T16:29:49.538Z","params":[],"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_main_replica_duration_s":0.0,"cpu_s":0.277952,"mem_objects":2161,"mem_bytes":1721920,"mem_mallocs":10473,"mem_total_bytes":1808360,"pid":1725,"worker_id":"puma_2","rate_limiting_gates":[],"correlation_id":"4f6912a2-025a-4740-bcbe-bbb14795dbf1","db_duration_s":0.0,"view_duration_s":0.00158,"duration_s":0.26784}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-03-28T16:30:04.494Z","params":[],"redis_calls":4,"redis_duration_s":0.003628,"redis_read_bytes":810,"redis_write_bytes":335,"redis_cache_calls":4,"redis_cache_duration_s":0.003628,"redis_cache_read_bytes":810,"redis_cache_write_bytes":335,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_main_replica_duration_s":0.0,"cpu_s":0.2356,"mem_objects":2903,"mem_bytes":1940768,"mem_mallocs":10859,"mem_total_bytes":2056888,"pid":1721,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"b29cafe9-e18b-4252-886f-56c13335c08b","db_duration_s":0.0,"view_duration_s":0.00111,"duration_s":0.21511}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-03-28T16:30:19.492Z","params":[],"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_main_replica_duration_s":0.0,"cpu_s":0.23526,"mem_objects":2161,"mem_bytes":1721920,"mem_mallocs":10473,"mem_total_bytes":1808360,"pid":1721,"worker_id":"puma_0","rate_limiting_gates":[],"correlation_id":"757e54e7-44e8-4e10-9268-41e68f9e64a4","db_duration_s":0.0,"view_duration_s":0.00108,"duration_s":0.22123}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-03-28T16:30:34.523Z","params":[],"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_main_replica_duration_s":0.0,"cpu_s":0.263053,"mem_objects":2211,"mem_bytes":1721920,"mem_mallocs":10473,"mem_total_bytes":1810360,"pid":1727,"worker_id":"puma_3","rate_limiting_gates":[],"correlation_id":"52269b9b-f220-4f64-b2ec-c55b48054a36","db_duration_s":0.0,"view_duration_s":0.00234,"duration_s":0.25348}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-03-28T16:30:49.527Z","params":[],"redis_calls":4,"redis_duration_s":0.003783,"redis_read_bytes":810,"redis_write_bytes":335,"redis_cache_calls":4,"redis_cache_duration_s":0.003783,"redis_cache_read_bytes":810,"redis_cache_write_bytes":335,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_main_replica_duration_s":0.0,"cpu_s":0.271063,"mem_objects":2903,"mem_bytes":1940768,"mem_mallocs":10859,"mem_total_bytes":2056888,"pid":1727,"worker_id":"puma_3","rate_limiting_gates":[],"correlation_id":"75f6d0db-8e10-4334-8efb-93f2f0846c49","db_duration_s":0.0,"view_duration_s":0.00158,"duration_s":0.2504}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-03-28T16:31:04.508Z","params":[],"redis_calls":4,"redis_duration_s":0.003743,"redis_read_bytes":810,"redis_write_bytes":335,"redis_cache_calls":4,"redis_cache_duration_s":0.003743,"redis_cache_read_bytes":810,"redis_cache_write_bytes":335,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_main_replica_duration_s":0.0,"cpu_s":0.251466,"mem_objects":2904,"mem_bytes":1940816,"mem_mallocs":10860,"mem_total_bytes":2056976,"pid":1723,"worker_id":"puma_1","rate_limiting_gates":[],"correlation_id":"3c80ac42-5bb7-4bab-8e17-8d64a01f5133","db_duration_s":0.0,"view_duration_s":0.00119,"duration_s":0.23199}
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-03-28T16:31:19.529Z","params":[],"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_main_replica_duration_s":0.0,"cpu_s":0.274401,"mem_objects":2161,"mem_bytes":1721920,"mem_mallocs":10473,"mem_total_bytes":1808360,"pid":1723,"worker_id":"puma_1","rate_limiting_gates":[],"correlation_id":"d16db252-4d1f-49ef-b6e0-094da2133674","db_duration_s":0.0,"view_duration_s":0.00143,"duration_s":0.26311}
==> /var/log/gitlab/crond/current <==
2023-03-21_18:49:24.86654 time="2023-03-21T18:49:24Z" level=info msg="start runner with 0 jobs\n"
2023-03-21_19:18:12.10027 time="2023-03-21T19:18:11Z" level=info msg="got signal: terminated"
2023-03-21_19:18:12.10034 time="2023-03-21T19:18:11Z" level=info msg="stop runner"
2023-03-21_19:18:12.10035 time="2023-03-21T19:18:11Z" level=info msg=terminated
2023-03-21_19:19:09.43717 time="2023-03-21T19:19:09Z" level=info msg="starting go-crond version 22.9.1 (5bd4127; go1.18.7) "
2023-03-21_19:19:09.44190 time="2023-03-21T19:19:09Z" level=info msg="{\"ShowVersion\":false,\"ShowOnlyVersion\":false,\"ShowHelp\":false,\"Cron\":{\"DefaultUser\":\"root\",\"IncludeCronD\":[\"/var/opt/gitlab/crond\"],\"Auto\":false,\"RunParts\":null,\"RunParts1m\":null,\"RunParts15m\":null,\"RunPartsHourly\":null,\"RunPartsDaily\":null,\"RunPartsWeekly\":null,\"RunPartsMonthly\":null,\"AllowUnprivileged\":false,\"WorkDir\":\"/\",\"EnableUserSwitching\":false},\"Log\":{\"Verbose\":false,\"Json\":false},\"Server\":{\"Bind\":\"\",\"ReadTimeout\":5000000000,\"WriteTimeout\":10000000000,\"Metrics\":false},\"Args\":{\"Crontabs\":null}}"
2023-03-21_19:19:09.44215 time="2023-03-21T19:19:09Z" level=info msg="start runner with 0 jobs\n"
2023-03-22_15:27:26.56402 time="2023-03-22T15:27:26Z" level=info msg="got signal: terminated"
2023-03-22_15:27:26.58130 time="2023-03-22T15:27:26Z" level=info msg="stop runner"
2023-03-22_15:27:26.58133 time="2023-03-22T15:27:26Z" level=info msg=terminated
==> /var/log/gitlab/gitlab-exporter/state <==
==> /var/log/gitlab/gitlab-exporter/current <==
2023-03-28_16:31:14.59031 127.0.0.1 - - [28/Mar/2023:16:31:14 UTC] "GET /sidekiq HTTP/1.1" 200 124371
2023-03-28_16:31:14.59043 - -> /sidekiq
2023-03-28_16:31:15.18698 127.0.0.1 - - [28/Mar/2023:16:31:14 UTC] "GET /database HTTP/1.1" 200 3105
2023-03-28_16:31:15.18723 - -> /database
2023-03-28_16:31:27.90614 127.0.0.1 - - [28/Mar/2023:16:31:27 UTC] "GET /ruby HTTP/1.1" 200 1080
2023-03-28_16:31:27.90632 - -> /ruby
2023-03-28_16:31:29.61437 127.0.0.1 - - [28/Mar/2023:16:31:29 UTC] "GET /sidekiq HTTP/1.1" 200 124371
2023-03-28_16:31:29.61455 - -> /sidekiq
2023-03-28_16:31:30.15430 127.0.0.1 - - [28/Mar/2023:16:31:29 UTC] "GET /database HTTP/1.1" 200 3105
2023-03-28_16:31:30.15440 - -> /database
==> /var/log/gitlab/gitlab-workhorse/current <==
{"content_type":"","correlation_id":"01GWMGYYDW3VWFSCY3Y0A3NZ26","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:32Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
==> /var/log/gitlab/nginx/gitlab_access.log <==
xx.xx.xx.xxx - - [28/Mar/2023:16:31:32 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
==> /var/log/gitlab/gitlab-rails/graphql_json.log <==
{"severity":"INFO","time":"2023-03-28T16:31:33.261Z","correlation_id":"01GWMGYWQZKBW6KTKZ3ATTW3YS","trace_type":"execute_query","query_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=/1/OlZofyHd_90AUTER8aeBjp2soAi2xRXJ-c2EmWJ8zjI=","duration_s":2.091781398994499,"operation_name":"createClusterAgent","operation_fingerprint":"createClusterAgent/HNLU1l1QuzbKV8bFfaypdezQJAHNT7OY_nODAs-Jtmw=","is_mutation":true,"variables":"{\"input\"=\u003e{\"name\"=\u003e\"i-am-a-test-agent-3\", \"projectPath\"=\u003e\"infrastructure/k8s-monorepo\"}}","query_string":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n","meta.caller_id":"graphql:createClusterAgent","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","query_analysis.duration_s":0.0049185540119651705,"query_analysis.depth":6,"query_analysis.complexity":31,"query_analysis.used_fields":["ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","CreateClusterAgentPayload.clusterAgent","CreateClusterAgentPayload.errors","CreateClusterAgentPayload.__typename","Mutation.createClusterAgent"],"query_analysis.used_deprecated_fields":[]}
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"POST","path":"/api/graphql","format":"*/*","controller":"GraphqlController","action":"execute","status":200,"time":"2023-03-28T16:31:33.267Z","params":[{"key":"_json","value":[{"operationName":"createClusterAgent","variables":"[FILTERED]","query":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n"}]},{"key":"graphql","value":{"_json":[{"operationName":"createClusterAgent","variables":"[FILTERED]","query":"mutation createClusterAgent($input: CreateClusterAgentInput!) {\n createClusterAgent(input: $input) {\n clusterAgent {\n ...ClusterAgentFragment\n __typename\n }\n errors\n __typename\n }\n}\n\nfragment ClusterAgentFragment on ClusterAgent {\n id\n name\n webPath\n connections {\n nodes {\n metadata {\n version\n __typename\n }\n __typename\n }\n __typename\n }\n tokens {\n nodes {\n id\n lastUsedAt\n __typename\n }\n __typename\n }\n __typename\n}\n"}]}}],"correlation_id":"01GWMGYWQZKBW6KTKZ3ATTW3YS","meta.caller_id":"GraphqlController#execute","meta.remote_ip":"10.11.2.63","meta.feature_category":"kubernetes_management","meta.user":"myuser","meta.user_id":2,"meta.client_id":"user/2","graphql":[{"depth":6,"complexity":31,"used_fields":["ClusterAgent.id","ClusterAgent.name","ClusterAgent.webPath","AgentMetadata.version","AgentMetadata.__typename","ConnectedAgent.metadata","ConnectedAgent.__typename","ConnectedAgentConnection.nodes","ConnectedAgentConnection.__typename","ClusterAgent.connections","ClusterAgentToken.id","ClusterAgentToken.lastUsedAt","ClusterAgentToken.__typename","ClusterAgentTokenConnection.nodes","ClusterAgentTokenConnection.__typename","ClusterAgent.tokens","ClusterAgent.__typename","CreateClusterAgentPayload.clusterAgent","CreateClusterAgentPayload.errors","CreateClusterAgentPayload.__typename","Mutation.createClusterAgent"],"used_deprecated_fields":[],"variables":"{\"input\"=>{\"name\"=>\"i-am-a-test-agent-3\", \"projectPath\"=>\"infrastructure/k8s-monorepo\"}}","operation_name":"createClusterAgent"}],"remote_ip":"10.11.2.63","user_id":2,"username":"myuser","ua":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605.1.15","queue_duration_s":0.048493,"request_urgency":"low","target_duration_s":5,"redis_calls":11,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.009644,"redis_read_bytes":916,"redis_write_bytes":2229,"redis_cache_calls":6,"redis_cache_duration_s":0.002613,"redis_cache_read_bytes":733,"redis_cache_write_bytes":492,"redis_sessions_calls":4,"redis_sessions_allowed_cross_slot_calls":1,"redis_sessions_duration_s":0.00608,"redis_sessions_read_bytes":183,"redis_sessions_write_bytes":1631,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000951,"redis_shared_state_write_bytes":106,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_replica_count":0,"db_primary_count":12,"db_main_count":12,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_duration_s":0.026,"db_main_replica_duration_s":0.0,"cpu_s":0.175883,"mem_objects":23825,"mem_bytes":2519135,"mem_mallocs":7228,"mem_total_bytes":3472135,"pid":1727,"worker_id":"puma_3","rate_limiting_gates":[],"db_duration_s":0.01647,"view_duration_s":0.00104,"duration_s":2.12746}
==> /var/log/gitlab/gitlab-workhorse/current <==
{"content_type":"application/json; charset=utf-8","correlation_id":"01GWMGYWQZKBW6KTKZ3ATTW3YS","duration_ms":2207,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"https://gitlab.obfuscated.domain/infrastructure/k8s-monorepo/-/clusters","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/graphql\\z","status":200,"system":"http","time":"2023-03-28T16:31:33Z","ttfb_ms":2207,"uri":"/api/graphql","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605.1.15","written_bytes":506}
==> /var/log/gitlab/nginx/gitlab_access.log <==
10.11.2.63 - - [28/Mar/2023:16:31:33 +0000] "POST /api/graphql HTTP/2.0" 200 276 "https://gitlab.obfuscated.domain/infrastructure/k8s-monorepo/-/clusters" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.3 Safari/605.1.15" 1.83
==> /var/log/gitlab/gitlab-workhorse/current <==
{"content_type":"","correlation_id":"01GWMGYZD6E0XZHE9CW88F3B5M","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:33Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
==> /var/log/gitlab/nginx/gitlab_access.log <==
xx.xx.xx.xxx - - [28/Mar/2023:16:31:33 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
==> /var/log/gitlab/gitlab-rails/production_json.log <==
{"method":"GET","path":"/-/metrics","format":"html","controller":"MetricsController","action":"index","status":200,"time":"2023-03-28T16:31:34.495Z","params":[],"redis_calls":4,"redis_duration_s":0.008367,"redis_read_bytes":810,"redis_write_bytes":335,"redis_cache_calls":4,"redis_cache_duration_s":0.008367,"redis_cache_read_bytes":810,"redis_cache_write_bytes":335,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_replica_count":0,"db_primary_count":0,"db_main_count":0,"db_main_replica_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_main_cached_count":0,"db_main_replica_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_main_wal_count":0,"db_main_replica_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_main_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_duration_s":0.0,"db_main_replica_duration_s":0.0,"cpu_s":0.23289,"mem_objects":2903,"mem_bytes":1940792,"mem_mallocs":10859,"mem_total_bytes":2056912,"pid":1725,"worker_id":"puma_2","rate_limiting_gates":[],"correlation_id":"dde6a09c-05b7-4392-951e-154bfe641a36","db_duration_s":0.0,"view_duration_s":0.00126,"duration_s":0.2134}
==> /var/log/gitlab/gitlab-workhorse/current <==
{"content_type":"","correlation_id":"01GWMGZ0CEXXD6XZH625VM4PKP","duration_ms":0,"host":"gitlab.obfuscated.domain","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"^/api/v4/jobs/request\\z","status":204,"system":"http","time":"2023-03-28T16:31:34Z","ttfb_ms":0,"uri":"/api/v4/jobs/request","user_agent":"gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)","written_bytes":0}
==> /var/log/gitlab/nginx/gitlab_access.log <==
xx.xx.xx.xxx - - [28/Mar/2023:16:31:34 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-runner 15.10.0 (15-10-stable; go1.19.6; linux/amd64)" -
</pre>
</details>
### Details of package version
<details>
<summary>Provide the package version installation details</summary>
<pre>
||/ Name Version Architecture Description
+++-======================-====================-============-============================================================
un gitlab-ce <none> <none> (no description available)
un gitlab-ci-multi-runner <none> <none> (no description available)
ii gitlab-ee 15.10.0-ee.0 amd64 GitLab Enterprise Edition (including NGINX, Postgres, Redis)
un gitlab-fips <none> <none> (no description available)
rc gitlab-runner 11.2.0+dfsg-2ubuntu1 amd64 GitLab Runner - runs continuous integration (CI) jobs
</pre>
</details>
### Environment details
* Operating System: `Ubuntu 20.04.6 LTS`
* Installation Target, remove incorrect values:
* VM: Other `Proxmox VE 6.4.X`
* Installation Type, remove incorrect values:
* New Installation
* Is there any other software running on the machine: No
* Is this a single or multiple node installation? Single
* Resources
* CPU: 4
* Memory total: 8GB
### Configuration details
<details>
<summary> Provide the relevant sections of `/etc/gitlab/gitlab.rb` </summary>
<pre>
external_url 'https://gitlab.obfuscated.domain'
gitlab_rails['env'] = {
"no_proxy" => "0,127.0.1.1,127.0.0.1,localhost,169.254.169.254"
}
gitlab_rails['gitlab_kas_enabled'] = true
gitlab_rails['gitlab_kas_external_url'] = 'wss://gitlab.obfuscated.domain/-/kubernetes-agent/'
gitlab_rails['gitlab_kas_internal_url'] = 'grpc://localhost:8153'
gitlab_kas['enable'] = true
gitlab_kas['gitlab_address'] = 'https://gitlab.obfuscated.domain'
</pre>
</details>https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7218upgrading gitlab-ce v14.10.5 to v15.0.5 - RuntimeError compiling Redis cookbook2022-10-04T15:05:57ZGeorgios Tzikasupgrading gitlab-ce v14.10.5 to v15.0.5 - RuntimeError compiling Redis cookbook### Summary
I have gitlab-ce v14.10.5 package installed in a Debian stretch VM.
VM is managed with Puppet, using vox-pupuli puppet module v4.0.1 for gitlab-omnibus.
During the upgrade to v15.0.5-ce, a `Could not connect to Redis at /va...### Summary
I have gitlab-ce v14.10.5 package installed in a Debian stretch VM.
VM is managed with Puppet, using vox-pupuli puppet module v4.0.1 for gitlab-omnibus.
During the upgrade to v15.0.5-ce, a `Could not connect to Redis at /var/opt/gitlab/redis/redis.socket: No such file or directory` error occured, executing `/opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket INFO` command in `gitlab-ctl reconfigure`. The upgrade ends with `error exit status 1`:
```
gitlab preinstall: Automatically backing up /etc/gitlab
Could not find '/etc/gitlab/config_backup' directory. Creating.
Running configuration backup
Creating configuration backup archive: gitlab_config_1663152512_2022_09_14.tar
/etc/gitlab/
/etc/gitlab/gitlab.rb
/etc/gitlab/gitlab-secrets.json
/etc/gitlab/initial_root_password
/etc/gitlab/gitlab-secrets.json.bak
/etc/gitlab/trusted-certs/
Configuration backup archive complete: /etc/gitlab/config_backup/gitlab_config_1
663152512_2022_09_14.tar
Keeping all older configuration backups
Unpacking gitlab-ce (15.0.5-ce.0) over (14.10.5-ce.0) ...
Setting up gitlab-ce (15.0.5-ce.0) ...
Checking PostgreSQL executables:Starting Cinc Client, version 15.17.4
resolving cookbooks for run list: ["gitlab::config", "postgresql::bin"]
Synchronizing Cookbooks:
- gitlab (0.0.1)
- postgresql (0.1.0)
- logrotate (0.1.0)
- package (0.1.0)
- redis (0.1.0)
- monitoring (0.1.0)
- registry (0.1.0)
- mattermost (0.1.0)
- consul (0.1.0)
- praefect (0.1.0)
- gitlab-kas (0.1.0)
- nginx (0.1.0)
- gitaly (0.1.0)
- letsencrypt (0.1.0)
- gitlab-pages (0.1.0)
- runit (5.1.3)
- acme (4.1.3)
- crond (0.1.0)
Installing Cookbook Gems:
Compiling Cookbooks...
Converging 4 resources
Recipe: postgresql::bin
* ruby_block[check_postgresql_version] action run (skipped due to not_if)
* ruby_block[check_postgresql_version_is_deprecated] action run (skipped due t
o not_if)
* ruby_block[Link postgresql bin files to the correct version] action run
- execute the ruby block Link postgresql bin files to the correct version
* template[/opt/gitlab/etc/gitlab-psql-rc] action create (up to date)
Running handlers:
Running handlers complete
Cinc Client finished, 1/4 resources updated in 20 seconds
Checking PostgreSQL executables: OK
Shutting down all GitLab services except those needed for migrations
ok: down: gitlab-kas: 0s, normally up
ok: down: gitlab-workhorse: 0s, normally up
ok: down: logrotate: 0s, normally up
ok: down: nginx: 0s, normally up
ok: down: puma: 1s, normally up
ok: down: sidekiq: 0s, normally up
Ensuring the required services are running
ok: run: postgresql: (pid 15745) 89755s
ok: run: redis: (pid 15755) 89754s
ok: run: gitaly: (pid 15677) 89758s
run: postgresql: (pid 15745) 89755s; run: log: (pid 1689) 135518s
run: redis: (pid 15755) 89754s; run: log: (pid 1688) 135518s
run: gitaly: (pid 15677) 89758s; run: log: (pid 1687) 135518s
Reconfigure GitLab to apply migrations
Starting Cinc Client, version 15.17.4
resolving cookbooks for run list: ["gitlab"]
Synchronizing Cookbooks:
- package (0.1.0)
- redis (0.1.0)
- gitlab (0.0.1)
- logrotate (0.1.0)
- postgresql (0.1.0)
- consul (0.1.0)
- registry (0.1.0)
- monitoring (0.1.0)
- gitlab-kas (0.1.0)
- gitlab-pages (0.1.0)
- letsencrypt (0.1.0)
- nginx (0.1.0)
- runit (5.1.3)
- acme (4.1.3)
- mattermost (0.1.0)
- gitaly (0.1.0)
- praefect (0.1.0)
- crond (0.1.0)
Installing Cookbook Gems:
Compiling Cookbooks...
...
...
...
Recipe: redis::enable
* redis_service[redis] action create
* account[user and group for redis] action create
* group[user and group for redis] action create (up to date)
* linux_user[user and group for redis] action create (up to date)
(up to date)
* group[Socket group] action create (up to date)
* directory[/var/opt/gitlab/redis] action create (up to date)
* directory[/var/log/gitlab/redis] action create (up to date)
* template[/var/opt/gitlab/redis/redis.conf] action create (up to date)
Recipe: <Dynamically Defined Resource>
* service[redis] action nothing (skipped due to action :nothing)
* runit_service[redis] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothi
ng)
* ruby_block[restart_log_service] action nothing (skipped due to action :n
othing)
* ruby_block[reload_log_service] action nothing (skipped due to action :no
thing)
* directory[/opt/gitlab/sv/redis] action create (up to date)
* template[/opt/gitlab/sv/redis/run] action create
- update content in file /opt/gitlab/sv/redis/run from da365d to 593ded
--- /opt/gitlab/sv/redis/run 2022-09-13 00:01:08.279412831 +0300
+++ /opt/gitlab/sv/redis/.chef-run20220914-8872-1s5m2x1 2022-09-14 13:54
:29.755844148 +0300
@@ -2,5 +2,9 @@
exec 2>&1
umask 077
-exec chpst -P -U gitlab-redis:gitlab-redis -u gitlab-redis:gitlab-redis
/opt/gitlab/embedded/bin/redis-server /var/opt/gitlab/redis/redis.conf
+exec chpst -P \
+ -U gitlab-redis:gitlab-redis \
+ -u gitlab-redis:gitlab-redis \
+ /opt/gitlab/embedded/bin/redis-server \
+ /var/opt/gitlab/redis/redis.confCould not connect to Redis
* directory[/opt/gitlab/sv/redis/log] action create (up to date)
* directory[/opt/gitlab/sv/redis/log/main] action create (up to date)
* template[/opt/gitlab/sv/redis/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_redis] action nothing (skipped due
to action :nothing)
* link[/var/log/gitlab/redis/config] action create (up to date)
* template[/opt/gitlab/sv/redis/log/run] action create (up to date)
* directory[/opt/gitlab/sv/redis/env] action create (up to date)
* ruby_block[Delete unmanaged env files for redis service] action run (ski
pped due to only_if)
* template[/opt/gitlab/sv/redis/check] action create (skipped due to only_
if)
* template[/opt/gitlab/sv/redis/finish] action create (skipped due to only
_if)
* directory[/opt/gitlab/sv/redis/control] action create (up to date)
* link[/opt/gitlab/init/redis] action create (up to date)
* file[/opt/gitlab/sv/redis/down] action nothing (skipped due to action :n
othing)
* ruby_block[restart_service] action run
* ruby_block[restart_service] action nothing (skipped due to action :not
hing)
* ruby_block[restart_log_service] action nothing (skipped due to action
:nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :
nothing)
* directory[/opt/gitlab/sv/redis] action create (up to date)
* template[/opt/gitlab/sv/redis/run] action create (up to date)
* directory[/opt/gitlab/sv/redis/log] action create (up to date)
* directory[/opt/gitlab/sv/redis/log/main] action create (up to date)
* template[/opt/gitlab/sv/redis/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_redis] action nothing (skipped du
e to action :nothing)
* link[/var/log/gitlab/redis/config] action create (up to date)
* template[/opt/gitlab/sv/redis/log/run] action create (up to date)
* directory[/opt/gitlab/sv/redis/env] action create (up to date)
* ruby_block[Delete unmanaged env files for redis service] action run (s
kipped due to only_if)
* template[/opt/gitlab/sv/redis/check] action create (skipped due to onl
y_if)
* template[/opt/gitlab/sv/redis/finish] action create (skipped due to on
ly_if)
* directory[/opt/gitlab/sv/redis/control] action create (up to date)
* link[/opt/gitlab/init/redis] action create (up to date)
* file[/opt/gitlab/sv/redis/down] action nothing (skipped due to action
:nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/redis] action create (up to date)
* ruby_block[wait for redis service socket] action run (skipped due to n
ot_if)
- execute the ruby block restart_service
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/redis] action create (up to date)
* ruby_block[wait for redis service socket] action run (skipped due to not
_if)
* ruby_block[warn pending redis restart] action run
==========================================================================
======
Error executing action `run` on resource 'ruby_block[warn pending redis re
start]'
==========================================================================
======
RuntimeError
------------
Execution of the command `/opt/gitlab/embedded/bin/redis-cli -s /var/opt/g
itlab/redis/redis.socket INFO` failed with a non-zero exit code (1)
stdout:
stderr: Could not connect to Redis at /var/opt/gitlab/redis/redis.socket:
No such file or directory
Cookbook Trace:
---------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/v
ersion_helper.rb:16:in `version'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/redis_help
er.rb:65:in `running_version'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/redis/resources/service.rb:
76:in `block (3 levels) in class_from_file'
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/redis/resources/servic
e.rb
66: ruby_block 'warn pending redis restart' do
67: block do
68: message = <<~MESSAGE
69: The version of the running redis service is different than wh
at is installed.
70: Please restart redis to start the new version.
71:
72: sudo gitlab-ctl restart redis
73: MESSAGE
74: LoggingHelper.warning(message)
75: end
76: only_if { new_resource.redis_helper.running_version != new_resour
ce.redis_helper.installed_version }
77: end
78: end
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/redis/resourc
es/service.rb:66:in `block in class_from_file'
ruby_block("warn pending redis restart") do
action [:run]
default_guard_interpreter :default
declared_type :ruby_block
cookbook_name "redis"
block #<Proc:0x000055dd90c2a9c0 /opt/gitlab/embedded/cookbooks/cache/coo
kbooks/redis/resources/service.rb:67>
block_name "warn pending redis restart"
only_if { #code block }
end
System Info:
------------
chef_version=15.17.4
platform=debian
platform_version=9
ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
============================================================================
====
Error executing action `create` on resource 'redis_service[redis]'
============================================================================
====
RuntimeError
------------
ruby_block[warn pending redis restart] (/opt/gitlab/embedded/cookbooks/cache
/cookbooks/redis/resources/service.rb line 66) had an error: RuntimeError: Execu
tion of the command `/opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis
/redis.socket INFO` failed with a non-zero exit code (1)
stdout:
stderr: Could not connect to Redis at /var/opt/gitlab/redis/redis.socket: No
such file or directory
Cookbook Trace:
---------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/ver
sion_helper.rb:16:in `version'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/redis_helper
.rb:65:in `running_version'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/redis/resources/service.rb:76
:in `block (3 levels) in class_from_file'
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/redis/recipes/enable.rb
19: redis_service 'redis' do
20: socket_group AccountHelper.new(node).gitlab_group
21: end
22:
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/redis/recipes/e
nable.rb:19:in `from_file'
redis_service("redis") do
action [:create]
updated true
updated_by_last_action true
default_guard_interpreter :default
declared_type :redis_service
cookbook_name "redis"
recipe_name "enable"
socket_group "git"
...
end
System Info:
------------
chef_version=15.17.4
platform=debian
platform_version=9
ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Recipe: gitlab::gitlab-rails
* execute[clear the gitlab-rails cache] action run
- execute /opt/gitlab/bin/gitlab-rake cache:clear
Running handlers:
There was an error running gitlab-ctl reconfigure:
redis_service[redis] (redis::enable line 19) had an error: RuntimeError: ruby_bl
ock[warn pending redis restart] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/
redis/resources/service.rb line 66) had an error: RuntimeError: Execution of the
command `/opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.sock
et INFO` failed with a non-zero exit code (1)
stdout:
stderr: Could not connect to Redis at /var/opt/gitlab/redis/redis.socket: No suc
h file or directory
Deprecations:
Your OS, debian-9, will be deprecated soon.
Starting with GitLab 15.2, packages will not be built for it.
Switch or upgrade to a supported OS, see https://docs.gitlab.com/omnibus/package
-information/deprecated_os.html for more information.
Notes:
Found old initial root password file at /etc/gitlab/initial_root_password and de
leted it.
Running handlers complete
Cinc Client failed. 10 resources updated in 02 minutes 55 seconds
Deprecations:
Your OS, debian-9, will be deprecated soon.
Starting with GitLab 15.2, packages will not be built for it.
Switch or upgrade to a supported OS, see https://docs.gitlab.com/omnibus/package
-information/deprecated_os.html for more information.
Notes:
Found old initial root password file at /etc/gitlab/initial_root_password and de
leted it.
===
There was an error running gitlab-ctl reconfigure. Please check the output above
for more
details.
===
dpkg: error processing package gitlab-ce (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
gitlab-ce
E: Sub-process /usr/bin/dpkg returned an error code (1)
```
To overcome this issue and conclude the upgrade, I proceed with the following actions:
* gitlab-ctl upgrade
* dpkg --configure -a
* gitlab-ctl restart
* gitlab-ctl reconfigure
Afterwards, I try upgrading to v15.1.6 from v15.0.5, which is completed without any errors.
The above error *is reproducible* in my setup. I have downgraded and retried the upgrade many times and this error occurs every single time.
I have also reproduced the error in a different identical VM.
### Steps to reproduce
* Provision an identical VM, with the specification provided below.
* Install gitlab-ce v14.10.5 package
* apt upgrade to gitlab-ce v15.0.5
### What is the current *bug* behavior?
Seems that GitLab cannot connect to Redis to check running and installed version, so the upgrade procedure breaks with the error posted above.
### What is the expected *correct* behavior?
GitLab should be able to connect to Redis and check running and installed version. These two should match, Redis version does not change in the newer package.
The upgrade procedure should finish with no errors.
### Relevant logs
Upgrade procedure started *13:42* (UTC+3) and finished with error *13:57* (UTC+3).
Relevant redis logs:
```
2022-09-14_10:33:27.34552 15755:M 14 Sep 2022 13:33:27.345 * Background saving terminated with success
2022-09-14_10:38:28.01886 15755:M 14 Sep 2022 13:38:28.018 * 10 changes in 300 seconds. Saving...
2022-09-14_10:38:28.01953 15755:M 14 Sep 2022 13:38:28.019 * Background saving started by pid 27729
2022-09-14_10:38:28.20158 27729:C 14 Sep 2022 13:38:28.201 * DB saved on disk
2022-09-14_10:38:28.20272 27729:C 14 Sep 2022 13:38:28.202 * RDB: 2 MB of memory used by copy-on-write
2022-09-14_10:38:28.22109 15755:M 14 Sep 2022 13:38:28.220 * Background saving terminated with success
2022-09-14_10:43:29.03144 15755:M 14 Sep 2022 13:43:29.031 * 10 changes in 300 seconds. Saving...
2022-09-14_10:43:29.03201 15755:M 14 Sep 2022 13:43:29.031 * Background saving started by pid 29830
2022-09-14_10:43:29.43759 29830:C 14 Sep 2022 13:43:29.437 * DB saved on disk
2022-09-14_10:43:29.43821 29830:C 14 Sep 2022 13:43:29.438 * RDB: 3 MB of memory used by copy-on-write
2022-09-14_10:43:29.53561 15755:M 14 Sep 2022 13:43:29.535 * Background saving terminated with success
2022-09-14_10:48:30.07676 15755:M 14 Sep 2022 13:48:30.076 * 10 changes in 300 seconds. Saving...
2022-09-14_10:48:30.21661 15755:M 14 Sep 2022 13:48:30.077 * Background saving started by pid 30764
2022-09-14_10:48:30.91666 30764:C 14 Sep 2022 13:48:30.916 * DB saved on disk
2022-09-14_10:48:30.91737 30764:C 14 Sep 2022 13:48:30.917 * RDB: 4 MB of memory used by copy-on-write
2022-09-14_10:48:30.98297 15755:M 14 Sep 2022 13:48:30.982 * Background saving terminated with success
2022-09-14_10:53:31.02475 15755:M 14 Sep 2022 13:53:31.024 * 10 changes in 300 seconds. Saving...
2022-09-14_10:53:31.06732 15755:M 14 Sep 2022 13:53:31.025 * Background saving started by pid 8716
2022-09-14_10:53:31.66489 8716:C 14 Sep 2022 13:53:31.664 * DB saved on disk
2022-09-14_10:53:31.66590 8716:C 14 Sep 2022 13:53:31.665 * RDB: 4 MB of memory used by copy-on-write
2022-09-14_10:53:31.73120 15755:M 14 Sep 2022 13:53:31.731 * Background saving terminated with success
2022-09-14_10:54:30.04444 15755:signal-handler (1663152870) Received SIGTERM scheduling shutdown...
2022-09-14_10:54:30.14028 15755:M 14 Sep 2022 13:54:30.140 # User requested shutdown...
2022-09-14_10:54:30.14030 15755:M 14 Sep 2022 13:54:30.140 * Saving the final RDB snapshot before exiting.
2022-09-14_10:54:30.41088 15755:M 14 Sep 2022 13:54:30.410 * DB saved on disk
2022-09-14_10:54:30.41091 15755:M 14 Sep 2022 13:54:30.410 * Removing the pid file.
2022-09-14_10:54:30.41092 15755:M 14 Sep 2022 13:54:30.410 * Removing the unix socket file.
2022-09-14_10:54:30.41094 15755:M 14 Sep 2022 13:54:30.410 # Redis is now ready to exit, bye bye...
2022-09-14_10:54:31.57548 9135:C 14 Sep 2022 13:54:31.575 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2022-09-14_10:54:31.57551 9135:C 14 Sep 2022 13:54:31.575 # Redis version=6.2.6, bits=64, commit=4930d19e, modified=1, pid=9135, just started
2022-09-14_10:54:31.57551 9135:C 14 Sep 2022 13:54:31.575 # Configuration loaded
2022-09-14_10:54:31.57660 9135:M 14 Sep 2022 13:54:31.576 * monotonic clock: POSIX clock_gettime
2022-09-14_10:54:31.62428 _._
2022-09-14_10:54:31.62430 _.-``__ ''-._
2022-09-14_10:54:31.62430 _.-`` `. `_. ''-._ Redis 6.2.6 (4930d19e/1) 64 bit
2022-09-14_10:54:31.62431 .-`` .-```. ```\/ _.,_ ''-._
2022-09-14_10:54:31.62431 ( ' , .-` | `, ) Running in standalone mode
2022-09-14_10:54:31.62432 |`-._`-...-` __...-.``-._|'` _.-'| Port: 0
2022-09-14_10:54:31.62432 | `-._ `._ / _.-' | PID: 9135
2022-09-14_10:54:31.62433 `-._ `-._ `-./ _.-' _.-'
2022-09-14_10:54:31.62433 |`-._`-._ `-.__.-' _.-'_.-'|
2022-09-14_10:54:31.62434 | `-._`-._ _.-'_.-' | https://redis.io
2022-09-14_10:54:31.62434 `-._ `-._`-.__.-'_.-' _.-'
2022-09-14_10:54:31.62434 |`-._`-._ `-.__.-' _.-'_.-'|
2022-09-14_10:54:31.62435 | `-._`-._ _.-'_.-' |
2022-09-14_10:54:31.62435 `-._ `-._`-.__.-'_.-' _.-'
2022-09-14_10:54:31.62436 `-._ `-.__.-' _.-'
2022-09-14_10:54:31.62436 `-._ _.-'
2022-09-14_10:54:31.62438 `-.__.-'
2022-09-14_10:54:31.62439
2022-09-14_10:54:31.62440 9135:M 14 Sep 2022 13:54:31.624 # Server initialized
2022-09-14_10:54:31.62441 9135:M 14 Sep 2022 13:54:31.624 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2022-09-14_10:54:31.62496 9135:M 14 Sep 2022 13:54:31.624 * Loading RDB produced by version 6.2.6
2022-09-14_10:54:31.62498 9135:M 14 Sep 2022 13:54:31.624 * RDB age 1 seconds
2022-09-14_10:54:31.62498 9135:M 14 Sep 2022 13:54:31.624 * RDB memory usage when created 2.08 Mb
2022-09-14_10:54:31.63736 9135:M 14 Sep 2022 13:54:31.637 # Done loading RDB, keys loaded: 632, keys expired: 0.
2022-09-14_10:54:31.63738 9135:M 14 Sep 2022 13:54:31.637 * DB loaded from disk: 0.012 seconds
2022-09-14_10:54:31.63738 9135:M 14 Sep 2022 13:54:31.637 * The server is now ready to accept connections at /var/opt/gitlab/redis/redis.socket
2022-09-14_10:59:32.02587 9135:M 14 Sep 2022 13:59:32.025 * 10 changes in 300 seconds. Saving...
2022-09-14_10:59:32.02590 9135:M 14 Sep 2022 13:59:32.025 * Background saving started by pid 10104
2022-09-14_10:59:32.54139 10104:C 14 Sep 2022 13:59:32.541 * DB saved on disk
2022-09-14_10:59:32.54212 10104:C 14 Sep 2022 13:59:32.542 * RDB: 0 MB of memory used by copy-on-write
2022-09-14_10:59:32.62800 9135:M 14 Sep 2022 13:59:32.627 * Background saving terminated with success
```
### Details of package version
```
[#]> dpkg-query -l "gitlab-*"
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-=================================
ii gitlab-ce 14.10.5-ce.0 amd64 GitLab Community Edition (includi
un gitlab-ee <none> <none> (no description available)
un gitlab-fips <none> <none> (no description available)
```
### Environment details
* Operating System: `Debian “stretch” 9.13`
* Installation Target, remove incorrect values:
* VM: Other: `QEMU VM instance running on private cloud`
* Installation Type, remove incorrect values:
* Upgrade from version `gitlab-ce v14.10.5`
* Is there any other software running on the machine: `no`
* Is this a single or multiple node installation? `single`
* Resources
* CPU: 8 vCPUs
* Memory total: 8G
### Configuration details
Relevant sections of `/etc/gitlab/gitlab.rb`:
```
## Url on which GitLab will be reachable.
external_url 'https://my.gitlab.changeme.gr'
############################
# gitlab.yml configuration #
############################
gitlab_rails['ldap_enabled'] = true
gitlab_rails['ldap_servers'] = {"ldapadmin"=>{"active_directory"=>false, "allow_username_or_email_login"=>true, "base"=>"CHANGEME", "block_auto_created_users"=>false, "host"=>"CHANGEME", "label"=>"CHANGEME", "method"=>"tls", "port"=>389, "uid"=>"uid"}}
gitlab_rails['monitoring_whitelist'] = ["X.X.X.X", "X:X:X:X::XXX", "X.X.X.X", "X:X:X:X::XXX"]
################
# GitLab Nginx #
################
nginx['listen_addresses'] = ["X.X.X.X", "[X:X:X:X::XXX]"]
nginx['redirect_http_to_https'] = true
nginx['ssl_certificate'] = "/path/to/dir"
nginx['ssl_certificate_key'] = "/path/to/dir"
######################
# GitLab Letsencrypt #
######################
letsencrypt['enable'] = false
##############
# Prometheus #
##############
prometheus_monitoring['enable'] = false
```https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7209Package Registry Internal Server Error for Large Files2022-11-28T04:31:43ZOkan EcevitPackage Registry Internal Server Error for Large FilesHello everyone, I tried to upload a file larger then 1GB to the Package Registry of a Project.
Unfortunately it throws only an 'internal server error' message.
The only configuration in the gitlab.rb file is
```
nginx['client_max_body...Hello everyone, I tried to upload a file larger then 1GB to the Package Registry of a Project.
Unfortunately it throws only an 'internal server error' message.
The only configuration in the gitlab.rb file is
```
nginx['client_max_body_size'] = 0
```
The request to upload a file
````
curl --header "PRIVATE-TOKEN:<token>" --upload-file large.tar.xz "https://<link>/api/v4/projects/54/packages/generic/test/0.0.1/large.tar.xz?select=package_file"
````
the response
````
{message: 500 internal server error}
````
I checked
````
/var/log/gitlab/gitlab-rails/production.log
````
there is only one entry about the upload
````
Started PUT "/api/v4/projects/54/packages/generic/test/0.0.1/large.tar.xz?select=package_file" for <IP> at 2022-09-26 11:27:10 +0200
````
Do you have an idea?
Could you help me?https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7185Error while upgrading from 15.2.2-ee to 15.3.3-ee2022-09-27T17:14:30ZRoberto CornacchiaError while upgrading from 15.2.2-ee to 15.3.3-ee### Summary
During the upgrade from 15.2.2-ee to 15.3.3-ee (docker), there was an error:
```
gitlab | Running handlers:
gitlab | [2022-09-14T20:16:20+00:00] ERROR: Running exception handlers
gitlab | There was an error running ...### Summary
During the upgrade from 15.2.2-ee to 15.3.3-ee (docker), there was an error:
```
gitlab | Running handlers:
gitlab | [2022-09-14T20:16:20+00:00] ERROR: Running exception handlers
gitlab | There was an error running gitlab-ctl reconfigure:
gitlab |
gitlab | runit_service[gitlab-exporter] (monitoring::gitlab-exporter line 75) had an error: Mixlib::ShellOut::ShellCommandFailed: ruby_block[reload_log_service] (monitoring::gitlab-exporter line 74) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
gitlab | ---- Begin output of /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log ----
gitlab | STDOUT: kill: run: /opt/gitlab/service/gitlab-exporter/log: (pid 806) 31s, got TERM
gitlab | STDERR:
gitlab | ---- End output of /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log ----
gitlab | Ran /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log returned 1
gitlab |
gitlab | Running handlers complete
gitlab | [2022-09-14T20:16:20+00:00] ERROR: Exception handlers complete
gitlab | Infra Phase failed. 279 resources updated in 17 minutes 48 seconds
gitlab | [2022-09-14T20:16:20+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/chef-stacktrace.out
gitlab | [2022-09-14T20:16:20+00:00] FATAL: ---------------------------------------------------------------------------------------
gitlab | [2022-09-14T20:16:20+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
gitlab | [2022-09-14T20:16:20+00:00] FATAL: ---------------------------------------------------------------------------------------
gitlab | [2022-09-14T20:16:20+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: runit_service[gitlab-exporter] (monitoring::gitlab-exporter line 75) had an error: Mixlib::ShellOut::ShellCommandFailed: ruby_block[reload_log_service] (monitoring::gitlab-exporter line 74) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
gitlab | ---- Begin output of /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log ----
gitlab | STDOUT: kill: run: /opt/gitlab/service/gitlab-exporter/log: (pid 806) 31s, got TERM
gitlab | STDERR:
gitlab | ---- End output of /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log ----
gitlab | Ran /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log returned 1
```
The system seems to work, and all background migrations are reported as completed.
Still, I'd like to know if something is broken.
### Relevant logs
[reconfigure.log](/uploads/86f93de3ebce2623a3d92d80bc505077/reconfigure.log)
<details>
<summary> /opt/gitlab/embedded/cookbooks/cache/chef-stacktrace.out </summary>
<pre>
Generated at 2022-09-14 20:16:20 +0000
Mixlib::ShellOut::ShellCommandFailed: runit_service[gitlab-exporter] (monitoring::gitlab-exporter line 75) had an error: Mixlib::ShellOut::ShellCommandFailed: ruby_block[reload_log_service] (monitoring::gitlab-exporter line 74) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log ----
STDOUT: kill: run: /opt/gitlab/service/gitlab-exporter/log: (pid 806) 31s, got TERM
STDERR:
---- End output of /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log ----
Ran /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log returned 1
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout.rb:300:in `invalid!'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout.rb:287:in `error!'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `tap'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `safe_sv_shellout!'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:177:in `reload_log_service'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:1342:in `method_missing'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb:77:in `block (3 levels) in <class:RunitService>'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider/ruby_block.rb:31:in `block (2 levels) in <class:RubyBlock>'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/mixin/why_run.rb:51:in `add_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:293:in `converge_by'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider/ruby_block.rb:30:in `block in <class:RubyBlock>'
(eval):2:in `block in action_run'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:304:in `instance_eval'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:304:in `compile_and_converge_action'
(eval):2:in `action_run'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:245:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:601:in `block in run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:628:in `with_umask'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:600:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:74:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:168:in `run_delayed_notification'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:155:in `block in run_delayed_notifications'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:154:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:154:in `run_delayed_notifications'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:144:in `converge'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:305:in `compile_and_converge_action'
(eval):2:in `action_create'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb:296:in `block in <class:RunitService>'
(eval):2:in `block in action_enable'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:304:in `instance_eval'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:304:in `compile_and_converge_action'
(eval):2:in `action_enable'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:245:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:601:in `block in run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:628:in `with_umask'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:600:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:74:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:108:in `block in run_all_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:108:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:108:in `run_all_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:132:in `block in converge'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/resource_list.rb:96:in `block in execute_each_resource'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/stepable_iterator.rb:114:in `call_iterator_block'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/stepable_iterator.rb:103:in `iterate'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/stepable_iterator.rb:54:in `each_with_index'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/resource_list.rb:94:in `execute_each_resource'
/opt/gitlab/embedded/lib/ruby/2.7.0/forwardable.rb:235:in `execute_each_resource'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:130:in `converge'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/client.rb:686:in `block in converge'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/client.rb:681:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/client.rb:681:in `converge'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/client.rb:705:in `converge_and_save'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/client.rb:285:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/application.rb:305:in `run_with_graceful_exit_option'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/application.rb:281:in `block in run_chef_client'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/local_mode.rb:42:in `with_server_connectivity'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/application.rb:264:in `run_chef_client'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/application/base.rb:352:in `run_application'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/application.rb:67:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-bin-17.10.0/bin/cinc-client:25:in `<top (required)>'
/opt/gitlab/embedded/bin/cinc-client:23:in `load'
/opt/gitlab/embedded/bin/cinc-client:23:in `<main>'
>>>> Caused by Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log ----
STDOUT: kill: run: /opt/gitlab/service/gitlab-exporter/log: (pid 806) 31s, got TERM
STDERR:
---- End output of /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log ----
Ran /opt/gitlab/embedded/bin/sv force-reload /opt/gitlab/service/gitlab-exporter/log returned 1
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout.rb:300:in `invalid!'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout.rb:287:in `error!'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `tap'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `safe_sv_shellout!'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:177:in `reload_log_service'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:1342:in `method_missing'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb:77:in `block (3 levels) in <class:RunitService>'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider/ruby_block.rb:31:in `block (2 levels) in <class:RubyBlock>'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/mixin/why_run.rb:51:in `add_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:293:in `converge_by'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider/ruby_block.rb:30:in `block in <class:RubyBlock>'
(eval):2:in `block in action_run'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:304:in `instance_eval'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:304:in `compile_and_converge_action'
(eval):2:in `action_run'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:245:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:601:in `block in run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:628:in `with_umask'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:600:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:74:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:168:in `run_delayed_notification'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:155:in `block in run_delayed_notifications'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:154:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:154:in `run_delayed_notifications'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:144:in `converge'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:305:in `compile_and_converge_action'
(eval):2:in `action_create'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb:296:in `block in <class:RunitService>'
(eval):2:in `block in action_enable'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:304:in `instance_eval'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:304:in `compile_and_converge_action'
(eval):2:in `action_enable'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/provider.rb:245:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:601:in `block in run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:628:in `with_umask'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource.rb:600:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:74:in `run_action'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:108:in `block in run_all_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:108:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:108:in `run_all_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:132:in `block in converge'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/resource_list.rb:96:in `block in execute_each_resource'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/stepable_iterator.rb:114:in `call_iterator_block'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/stepable_iterator.rb:103:in `iterate'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/stepable_iterator.rb:54:in `each_with_index'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/resource_collection/resource_list.rb:94:in `execute_each_resource'
/opt/gitlab/embedded/lib/ruby/2.7.0/forwardable.rb:235:in `execute_each_resource'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/runner.rb:130:in `converge'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/client.rb:686:in `block in converge'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/client.rb:681:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/client.rb:681:in `converge'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/client.rb:705:in `converge_and_save'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/client.rb:285:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/application.rb:305:in `run_with_graceful_exit_option'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/application.rb:281:in `block in run_chef_client'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/local_mode.rb:42:in `with_server_connectivity'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/application.rb:264:in `run_chef_client'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/application/base.rb:352:in `run_application'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-17.10.0/lib/chef/application.rb:67:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-bin-17.10.0/bin/cinc-client:25:in `<top (required)>'
/opt/gitlab/embedded/bin/cinc-client:23:in `load'
/opt/gitlab/embedded/bin/cinc-client:23:in `<main>'
</pre>
</details>
### Environment details
* Operating System: `docker deployment on Rocky Linux 8.6`
* Installation Target, remove incorrect values:
* Other: `docker in VM`
* Installation Type, remove incorrect values:
* Upgrade from version `15.2.2`
* Is there any other software running on the machine: `no`
* Is this a single or multiple node installation? `single`
* Resources
* CPU: `4 cores i7`
* Memory total: `8 GB`https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6926update-permissions doesn't fix gitaly pid2023-12-03T21:23:04ZArnold Hendriksupdate-permissions doesn't fix gitaly pidI restored a raw (ie filesystem copy) backup of a gitlab omnibus docker on my mac, which ruined all permissions settings.
I used `update-permissions` to get back to a working installation, which mostly worked fine but gitaly was not abl...I restored a raw (ie filesystem copy) backup of a gitlab omnibus docker on my mac, which ruined all permissions settings.
I used `update-permissions` to get back to a working installation, which mostly worked fine but gitaly was not able to start:
```
{"error":"open /var/opt/gitlab/gitaly/gitaly.pid: permission denied","level":"fatal","msg":"find process","pid":3499,"time":"2022-07-13T14:56:03.048Z","wrapper":3499}
```
which I could confirm (unfortunately i didn't check /var/opt/gitlab/gitaly/ itself)
```
root@a3de00be717f:/# ls -l /var/opt/gitlab/gitaly/gitaly.pid
-rw------- 1 root root 4 Jul 2 20:45 /var/opt/gitlab/gitaly/gitaly.pid
```
after a `chown git /var/opt/gitlab/gitaly/` it came back up, so I think this is just a minor detail missed by update-permissionshttps://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7074GItLab: Exception during reconfigure after upgrade version from 13.6.7 to 15.0.12022-09-07T04:40:26ZManoharan MGItLab: Exception during reconfigure after upgrade version from 13.6.7 to 15.0.1```
gitlab-ctl reconfigure
Starting Cinc Client, version 15.17.4
resolving cookbooks for run list: ["gitlab"]
Synchronizing Cookbooks:
- gitlab (0.0.1)
- logrotate (0.1.0)
- postgresql (0.1.0)
- redis (0.1.0)
- monitoring (0.1...```
gitlab-ctl reconfigure
Starting Cinc Client, version 15.17.4
resolving cookbooks for run list: ["gitlab"]
Synchronizing Cookbooks:
- gitlab (0.0.1)
- logrotate (0.1.0)
- postgresql (0.1.0)
- redis (0.1.0)
- monitoring (0.1.0)
- registry (0.1.0)
- mattermost (0.1.0)
- consul (0.1.0)
- gitlab-kas (0.1.0)
- gitaly (0.1.0)
- praefect (0.1.0)
- gitlab-pages (0.1.0)
- package (0.1.0)
- nginx (0.1.0)
- letsencrypt (0.1.0)
- runit (5.1.3)
- acme (4.1.3)
- crond (0.1.0)
Installing Cookbook Gems:
Compiling Cookbooks...
Recipe: gitlab::default
* directory[/etc/gitlab] action create (up to date)
Converging 296 resources
* directory[/etc/gitlab] action create (up to date)
* directory[Create /var/opt/gitlab] action create (up to date)
* directory[Create /var/log/gitlab] action create (up to date)
* directory[/opt/gitlab/embedded/etc] action create (up to date)
* template[/opt/gitlab/embedded/etc/gitconfig] action create (up to date)
Recipe: gitlab::web-server
* account[Webserver user and group] action create
* group[Webserver user and group] action create (up to date)
* linux_user[Webserver user and group] action create (up to date)
(up to date)
Recipe: gitlab::users
* directory[/var/opt/gitlab] action create (up to date)
* account[GitLab user and group] action create
* group[GitLab user and group] action create (up to date)
* linux_user[GitLab user and group] action create (up to date)
(up to date)
* template[/var/opt/gitlab/.gitconfig] action create (up to date)
* directory[/var/opt/gitlab/.bundle] action create (up to date)
Recipe: gitlab::gitlab-shell
* storage_directory[/var/opt/gitlab/.ssh] action create
* ruby_block[directory resource: /var/opt/gitlab/.ssh] action run (skipped due to not_if)
(up to date)
* directory[/var/log/gitlab/gitlab-shell/] action create (up to date)
* directory[/var/opt/gitlab/gitlab-shell] action create (up to date)
* templatesymlink[Create a config.yml and create a symlink to Rails root] action create
* template[/var/opt/gitlab/gitlab-shell/config.yml] action create (up to date)
* link[Link /opt/gitlab/embedded/service/gitlab-shell/config.yml to /var/opt/gitlab/gitlab-shell/config.yml] action create (up to date)
(up to date)
* link[/opt/gitlab/embedded/service/gitlab-shell/.gitlab_shell_secret] action create (up to date)
* file[/var/opt/gitlab/.ssh/authorized_keys] action create_if_missing (up to date)
Recipe: gitlab::gitlab-rails
* storage_directory[/var/opt/gitlab/git-data] action create
* ruby_block[directory resource: /var/opt/gitlab/git-data] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/git-data/repositories] action create
* ruby_block[directory resource: /var/opt/gitlab/git-data/repositories] action run (skipped due to not_if)
(up to date)
Recipe: gitlab::rails_pages_shared_path
* storage_directory[/var/opt/gitlab/gitlab-rails/shared] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-rails/shared/pages] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/pages] action run (skipped due to not_if)
(up to date)
Recipe: gitlab::gitlab-rails
* storage_directory[/var/opt/gitlab/gitlab-rails/shared/artifacts] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/artifacts] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-rails/shared/external-diffs] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/external-diffs] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-rails/shared/lfs-objects] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/lfs-objects] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-rails/shared/packages] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/packages] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-rails/shared/dependency_proxy] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/dependency_proxy] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-rails/shared/terraform_state] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/terraform_state] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-rails/shared/ci_secure_files] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/ci_secure_files] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-rails/shared/encrypted_settings] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/encrypted_settings] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-rails/uploads] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/uploads] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-ci/builds] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-ci/builds] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-rails/shared/cache] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/cache] action run (skipped due to not_if)
(up to date)
* storage_directory[/var/opt/gitlab/gitlab-rails/shared/tmp] action create
* ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/tmp] action run (skipped due to not_if)
(up to date)
* storage_directory[/opt/gitlab/embedded/service/gitlab-rails/public] action create (skipped due to only_if)
* directory[create /var/opt/gitlab/gitlab-rails/etc] action create (up to date)
* directory[create /opt/gitlab/etc/gitlab-rails] action create (up to date)
* directory[create /var/opt/gitlab/gitlab-rails/working] action create (up to date)
* directory[create /var/opt/gitlab/gitlab-rails/tmp] action create (up to date)
* directory[create /var/opt/gitlab/gitlab-rails/upgrade-status] action create (up to date)
* directory[create /var/log/gitlab/gitlab-rails] action create (up to date)
* storage_directory[/home/gitlab/backups] action create
* ruby_block[directory resource: /home/gitlab/backups] action run (skipped due to not_if)
(up to date)
* directory[/var/opt/gitlab/gitlab-rails] action create (up to date)
* directory[/var/opt/gitlab/gitlab-ci] action create (up to date)
* file[/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key] action create (skipped due to only_if)
* template[/opt/gitlab/etc/gitlab-rails-rc] action create (up to date)
* file[/opt/gitlab/embedded/service/gitlab-rails/.secret] action delete (up to date)
* file[/var/opt/gitlab/gitlab-rails/etc/secret] action delete (up to date)
* templatesymlink[Create a database.yml and create a symlink to Rails root] action create
* template[/var/opt/gitlab/gitlab-rails/etc/database.yml] action create (up to date)
* link[Link /opt/gitlab/embedded/service/gitlab-rails/config/database.yml to /var/opt/gitlab/gitlab-rails/etc/database.yml] action create (up to date)
(up to date)
* templatesymlink[Create a secrets.yml and create a symlink to Rails root] action create
* template[/var/opt/gitlab/gitlab-rails/etc/secrets.yml] action create (up to date)
* link[Link /opt/gitlab/embedded/service/gitlab-rails/config/secrets.yml to /var/opt/gitlab/gitlab-rails/etc/secrets.yml] action create (up to date)
(up to date)
* templatesymlink[Create a resque.yml and create a symlink to Rails root] action create
* template[/var/opt/gitlab/gitlab-rails/etc/resque.yml] action create (up to date)
* link[Link /opt/gitlab/embedded/service/gitlab-rails/config/resque.yml to /var/opt/gitlab/gitlab-rails/etc/resque.yml] action create (up to date)
(up to date)
* templatesymlink[Create a cable.yml and create a symlink to Rails root] action create
* template[/var/opt/gitlab/gitlab-rails/etc/cable.yml] action create (up to date)
* link[Link /opt/gitlab/embedded/service/gitlab-rails/config/cable.yml to /var/opt/gitlab/gitlab-rails/etc/cable.yml] action create (up to date)
(up to date)
* templatesymlink[Create a redis.cache.yml and create a symlink to Rails root] action create (skipped due to not_if)
* file[/opt/gitlab/embedded/service/gitlab-rails/config/redis.cache.yml] action delete (up to date)
* file[/var/opt/gitlab/gitlab-rails/etc/redis.cache.yml] action delete (up to date)
* templatesymlink[Create a redis.queues.yml and create a symlink to Rails root] action create (skipped due to not_if)
* file[/opt/gitlab/embedded/service/gitlab-rails/config/redis.queues.yml] action delete (up to date)
* file[/var/opt/gitlab/gitlab-rails/etc/redis.queues.yml] action delete (up to date)
* templatesymlink[Create a redis.shared_state.yml and create a symlink to Rails root] action create (skipped due to not_if)
* file[/opt/gitlab/embedded/service/gitlab-rails/config/redis.shared_state.yml] action delete (up to date)
* file[/var/opt/gitlab/gitlab-rails/etc/redis.shared_state.yml] action delete (up to date)
* templatesymlink[Create a redis.trace_chunks.yml and create a symlink to Rails root] action create (skipped due to not_if)
* file[/opt/gitlab/embedded/service/gitlab-rails/config/redis.trace_chunks.yml] action delete (up to date)
* file[/var/opt/gitlab/gitlab-rails/etc/redis.trace_chunks.yml] action delete (up to date)
* templatesymlink[Create a redis.rate_limiting.yml and create a symlink to Rails root] action create (skipped due to not_if)
* file[/opt/gitlab/embedded/service/gitlab-rails/config/redis.rate_limiting.yml] action delete (up to date)
* file[/var/opt/gitlab/gitlab-rails/etc/redis.rate_limiting.yml] action delete (up to date)
* templatesymlink[Create a redis.sessions.yml and create a symlink to Rails root] action create (skipped due to not_if)
* file[/opt/gitlab/embedded/service/gitlab-rails/config/redis.sessions.yml] action delete (up to date)
* file[/var/opt/gitlab/gitlab-rails/etc/redis.sessions.yml] action delete (up to date)
* templatesymlink[Create a smtp_settings.rb and create a symlink to Rails root] action create
* template[/var/opt/gitlab/gitlab-rails/etc/smtp_settings.rb] action create (up to date)
* link[Link /opt/gitlab/embedded/service/gitlab-rails/config/initializers/smtp_settings.rb to /var/opt/gitlab/gitlab-rails/etc/smtp_settings.rb] action create (up to date)
(up to date)
* templatesymlink[Create a gitlab.yml and create a symlink to Rails root] action create
* template[/var/opt/gitlab/gitlab-rails/etc/gitlab.yml] action create
- update content in file /var/opt/gitlab/gitlab-rails/etc/gitlab.yml from 592aac to 6b022d
- suppressed sensitive resource
* link[Link /opt/gitlab/embedded/service/gitlab-rails/config/gitlab.yml to /var/opt/gitlab/gitlab-rails/etc/gitlab.yml] action create (up to date)
* templatesymlink[Create a gitlab_workhorse_secret and create a symlink to Rails root] action create
* template[/var/opt/gitlab/gitlab-rails/etc/gitlab_workhorse_secret] action create (up to date)
* link[Link /opt/gitlab/embedded/service/gitlab-rails/.gitlab_workhorse_secret to /var/opt/gitlab/gitlab-rails/etc/gitlab_workhorse_secret] action create (up to date)
(up to date)
* templatesymlink[Create a gitlab_shell_secret and create a symlink to Rails root] action create
* template[/var/opt/gitlab/gitlab-rails/etc/gitlab_shell_secret] action create (up to date)
* link[Link /opt/gitlab/embedded/service/gitlab-rails/.gitlab_shell_secret to /var/opt/gitlab/gitlab-rails/etc/gitlab_shell_secret] action create (up to date)
(up to date)
* templatesymlink[Create a gitlab_incoming_email_secret and create a symlink to Rails root] action create (skipped due to only_if)
* templatesymlink[Create a gitlab_service_desk_email_secret and create a symlink to Rails root] action create (skipped due to only_if)
* templatesymlink[Create a gitlab_pages_secret and create a symlink to Rails root] action create
* template[/var/opt/gitlab/gitlab-rails/etc/gitlab_pages_secret] action create (up to date)
* link[Link /opt/gitlab/embedded/service/gitlab-rails/.gitlab_pages_secret to /var/opt/gitlab/gitlab-rails/etc/gitlab_pages_secret] action create (up to date)
(up to date)
* templatesymlink[Create a gitlab_kas_secret and create a symlink to Rails root] action create
* template[/var/opt/gitlab/gitlab-rails/etc/gitlab_kas_secret] action create (up to date)
* link[Link /opt/gitlab/embedded/service/gitlab-rails/.gitlab_kas_secret to /var/opt/gitlab/gitlab-rails/etc/gitlab_kas_secret] action create (up to date)
(up to date)
* link[/opt/gitlab/embedded/service/gitlab-rails/config/initializers/relative_url.rb] action delete (up to date)
* file[/var/opt/gitlab/gitlab-rails/etc/relative_url.rb] action delete (up to date)
* env_dir[/opt/gitlab/etc/gitlab-rails/env] action create
* directory[/opt/gitlab/etc/gitlab-rails/env] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/HOME] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/RAILS_ENV] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/LD_PRELOAD] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/BUNDLE_GEMFILE] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/SIDEKIQ_MEMORY_KILLER_MAX_RSS] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/PATH] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/ICU_DATA] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/PYTHONPATH] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/EXECJS_RUNTIME] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/TZ] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/SSL_CERT_DIR] action create (up to date)
* file[/opt/gitlab/etc/gitlab-rails/env/SSL_CERT_FILE] action create (up to date)
(up to date)
* link[/opt/gitlab/embedded/service/gitlab-rails/tmp] action create (up to date)
* link[/opt/gitlab/embedded/service/gitlab-rails/public/uploads] action create (up to date)
* link[/opt/gitlab/embedded/service/gitlab-rails/log] action create (up to date)
* link[/var/log/gitlab/gitlab-rails/sidekiq.log] action delete (skipped due to only_if)
* file[/opt/gitlab/embedded/service/gitlab-rails/db/structure.sql] action create (up to date)
* remote_file[/var/opt/gitlab/gitlab-rails/VERSION] action create/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-15.17.4/lib/chef/provider/remote_file/local_file.rb:43: warning: URI.unescape is obsolete
(up to date)
* remote_file[/var/opt/gitlab/gitlab-rails/REVISION] action create/opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/chef-15.17.4/lib/chef/provider/remote_file/local_file.rb:43: warning: URI.unescape is obsolete
(up to date)
* version_file[Create version file for Rails] action create
* file[/var/opt/gitlab/gitlab-rails/RUBY_VERSION] action create (up to date)
(up to date)
* execute[clear the gitlab-rails cache] action nothing (skipped due to action :nothing)
* file[/var/opt/gitlab/gitlab-rails/config.ru] action delete (up to date)
Recipe: gitlab::selinux
* execute[semodule -i /opt/gitlab/embedded/selinux/rhel/7/gitlab-7.2.0-ssh-keygen.pp] action runDisabled
(skipped due to not_if)
* execute[semodule -i /opt/gitlab/embedded/selinux/rhel/7/gitlab-10.5.0-ssh-authorized-keys.pp] action runDisabled
(skipped due to not_if)
* execute[semodule -i /opt/gitlab/embedded/selinux/rhel/7/gitlab-13.5.0-gitlab-shell.pp] action runDisabled
(skipped due to not_if)
* bash[Set proper security context on ssh files for selinux] action nothing (skipped due to action :nothing)
Recipe: gitlab::add_trusted_certs
* directory[/etc/gitlab/trusted-certs] action create (up to date)
* directory[/opt/gitlab/embedded/ssl/certs] action create (up to date)
* file[/opt/gitlab/embedded/ssl/certs/README] action create (up to date)
* ruby_block[Move existing certs and link to /opt/gitlab/embedded/ssl/certs] action run (skipped due to only_if)
Recipe: gitlab::default
* service[create a temporary puma service] action nothing (skipped due to action :nothing)
* service[create a temporary sidekiq service] action nothing (skipped due to action :nothing)
* service[create a temporary mailroom service] action nothing (skipped due to action :nothing)
Recipe: package::runit_systemd
* directory[/usr/lib/systemd/system] action create (up to date)
* template[/usr/lib/systemd/system/gitlab-runsvdir.service] action create (up to date)
* file[/etc/systemd/system/default.target.wants/gitlab-runsvdir.service] action delete (up to date)
* file[/etc/systemd/system/basic.target.wants/gitlab-runsvdir.service] action delete (up to date)
* execute[systemctl daemon-reload] action nothing (skipped due to action :nothing)
* execute[systemctl enable gitlab-runsvdir] action nothing (skipped due to action :nothing)
* execute[systemctl start gitlab-runsvdir] action nothing (skipped due to action :nothing)
Recipe: package::sysctl
* execute[reload all sysctl conf] action nothing (skipped due to action :nothing)
Recipe: logrotate::folders_and_configs
* directory[/var/opt/gitlab/logrotate] action create (up to date)
* directory[/var/opt/gitlab/logrotate/logrotate.d] action create (up to date)
* directory[/var/log/gitlab/logrotate] action create (up to date)
* template[/var/opt/gitlab/logrotate/logrotate.conf] action create (up to date)
* template[/var/opt/gitlab/logrotate/logrotate.d/nginx] action create (up to date)
* template[/var/opt/gitlab/logrotate/logrotate.d/puma] action create (up to date)
* template[/var/opt/gitlab/logrotate/logrotate.d/gitlab-rails] action create (up to date)
* template[/var/opt/gitlab/logrotate/logrotate.d/gitlab-shell] action create (up to date)
* template[/var/opt/gitlab/logrotate/logrotate.d/gitlab-workhorse] action create (up to date)
* template[/var/opt/gitlab/logrotate/logrotate.d/gitlab-pages] action create (up to date)
* template[/var/opt/gitlab/logrotate/logrotate.d/gitlab-kas] action create (up to date)
* template[/var/opt/gitlab/logrotate/logrotate.d/gitaly] action create (up to date)
* template[/var/opt/gitlab/logrotate/logrotate.d/mailroom] action create (up to date)
Recipe: logrotate::enable
* service[logrotate] action nothing (skipped due to action :nothing)
* runit_service[logrotate] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/logrotate] action create (up to date)
* template[/opt/gitlab/sv/logrotate/run] action create (up to date)
* directory[/opt/gitlab/sv/logrotate/log] action create (up to date)
* directory[/opt/gitlab/sv/logrotate/log/main] action create (up to date)
* template[/opt/gitlab/sv/logrotate/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_logrotate] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/logrotate/config] action create (up to date)
* template[/opt/gitlab/sv/logrotate/log/run] action create (up to date)
* directory[/opt/gitlab/sv/logrotate/env] action create (up to date)
* ruby_block[Delete unmanaged env files for logrotate service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/logrotate/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/logrotate/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/logrotate/control] action create (up to date)
* template[/opt/gitlab/sv/logrotate/control/t] action create (up to date)
* link[/opt/gitlab/init/logrotate] action create (up to date)
* file[/opt/gitlab/sv/logrotate/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/logrotate] action create (up to date)
* ruby_block[wait for logrotate service socket] action run (skipped due to not_if)
(up to date)
Recipe: redis::enable
* redis_service[redis] action create
* account[user and group for redis] action create
* group[user and group for redis] action create (up to date)
* linux_user[user and group for redis] action create (up to date)
(up to date)
* group[Socket group] action create (up to date)
* directory[/var/opt/gitlab/redis] action create (up to date)
* directory[/var/log/gitlab/redis] action create (up to date)
* template[/var/opt/gitlab/redis/redis.conf] action create (up to date)
Recipe: <Dynamically Defined Resource>
* service[redis] action nothing (skipped due to action :nothing)
* runit_service[redis] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/redis] action create (up to date)
* template[/opt/gitlab/sv/redis/run] action create (up to date)
* directory[/opt/gitlab/sv/redis/log] action create (up to date)
* directory[/opt/gitlab/sv/redis/log/main] action create (up to date)
* template[/opt/gitlab/sv/redis/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_redis] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/redis/config] action create (up to date)
* template[/opt/gitlab/sv/redis/log/run] action create (up to date)
* directory[/opt/gitlab/sv/redis/env] action create (up to date)
* ruby_block[Delete unmanaged env files for redis service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/redis/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/redis/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/redis/control] action create (up to date)
* link[/opt/gitlab/init/redis] action create (up to date)
* file[/opt/gitlab/sv/redis/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/redis] action create (up to date)
* ruby_block[wait for redis service socket] action run (skipped due to not_if)
(up to date)
* ruby_block[warn pending redis restart] action run (skipped due to only_if)
(up to date)
Recipe: redis::enable
* template[/opt/gitlab/etc/gitlab-redis-cli-rc] action create (up to date)
Recipe: gitaly::enable
* directory[/var/opt/gitlab/gitaly] action create (up to date)
* directory[/var/opt/gitlab/gitaly/run] action create (up to date)
* directory[/var/log/gitlab/gitaly] action create (up to date)
* directory[/var/opt/gitlab/gitaly/internal_sockets] action delete (up to date)
* env_dir[/opt/gitlab/etc/gitaly/env] action create
* directory[/opt/gitlab/etc/gitaly/env] action create (up to date)
* file[/opt/gitlab/etc/gitaly/env/HOME] action create (up to date)
* file[/opt/gitlab/etc/gitaly/env/PATH] action create (up to date)
* file[/opt/gitlab/etc/gitaly/env/TZ] action create (up to date)
* file[/opt/gitlab/etc/gitaly/env/PYTHONPATH] action create (up to date)
* file[/opt/gitlab/etc/gitaly/env/ICU_DATA] action create (up to date)
* file[/opt/gitlab/etc/gitaly/env/SSL_CERT_DIR] action create (up to date)
* file[/opt/gitlab/etc/gitaly/env/GITALY_PID_FILE] action create (up to date)
* file[/opt/gitlab/etc/gitaly/env/WRAPPER_JSON_LOGGING] action create (up to date)
(up to date)
* template[Create Gitaly config.toml] action create (up to date)
* service[gitaly] action nothing (skipped due to action :nothing)
* runit_service[gitaly] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/gitaly] action create (up to date)
* template[/opt/gitlab/sv/gitaly/run] action create (up to date)
* directory[/opt/gitlab/sv/gitaly/log] action create (up to date)
* directory[/opt/gitlab/sv/gitaly/log/main] action create (up to date)
* template[/opt/gitlab/sv/gitaly/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_gitaly] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/gitaly/config] action create (up to date)
* template[/opt/gitlab/sv/gitaly/log/run] action create (up to date)
* directory[/opt/gitlab/sv/gitaly/env] action create (up to date)
* ruby_block[Delete unmanaged env files for gitaly service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/gitaly/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/gitaly/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/gitaly/control] action create (up to date)
* link[/opt/gitlab/init/gitaly] action create (up to date)
* file[/opt/gitlab/sv/gitaly/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/gitaly] action create (up to date)
* ruby_block[wait for gitaly service socket] action run (skipped due to not_if)
(up to date)
* version_file[Create version file for Gitaly] action create
* file[/var/opt/gitlab/gitaly/VERSION] action create (up to date)
(up to date)
* version_file[Create Ruby version file for Gitaly] action create
* file[/var/opt/gitlab/gitaly/RUBY_VERSION] action create (up to date)
(up to date)
* consul_service[gitaly] action delete
* file[/var/opt/gitlab/consul/config.d/gitaly-service.json] action delete (up to date)
(up to date)
Recipe: postgresql::bin
* ruby_block[check_postgresql_version] action run (skipped due to not_if)
* ruby_block[check_postgresql_version_is_deprecated] action run (skipped due to not_if)
* ruby_block[Link postgresql bin files to the correct version] action run (skipped due to only_if)
* template[/opt/gitlab/etc/gitlab-psql-rc] action create (up to date)
Recipe: postgresql::user
* account[Postgresql user and group] action create
* group[Postgresql user and group] action create (up to date)
* linux_user[Postgresql user and group] action create (up to date)
(up to date)
* directory[/var/opt/gitlab/postgresql] action create (up to date)
* file[/var/opt/gitlab/postgresql/.profile] action create (up to date)
Recipe: postgresql::sysctl
* gitlab_sysctl[kernel.shmmax] action create
* directory[create /etc/sysctl.d for kernel.shmmax] action create (up to date)
* file[create /opt/gitlab/embedded/etc/90-omnibus-gitlab-kernel.shmmax.conf kernel.shmmax] action create (up to date)
* link[/etc/sysctl.d/90-omnibus-gitlab-kernel.shmmax.conf] action create (up to date)
* execute[load sysctl conf kernel.shmmax] action nothing (skipped due to action :nothing)
(up to date)
* gitlab_sysctl[kernel.shmall] action create
* directory[create /etc/sysctl.d for kernel.shmall] action create (up to date)
* file[create /opt/gitlab/embedded/etc/90-omnibus-gitlab-kernel.shmall.conf kernel.shmall] action create (up to date)
* link[/etc/sysctl.d/90-omnibus-gitlab-kernel.shmall.conf] action create (up to date)
* execute[load sysctl conf kernel.shmall] action nothing (skipped due to action :nothing)
(up to date)
* gitlab_sysctl[kernel.sem] action create
* directory[create /etc/sysctl.d for kernel.sem] action create (up to date)
* file[create /opt/gitlab/embedded/etc/90-omnibus-gitlab-kernel.sem.conf kernel.sem] action create (up to date)
* link[/etc/sysctl.d/90-omnibus-gitlab-kernel.sem.conf] action create (up to date)
* execute[load sysctl conf kernel.sem] action nothing (skipped due to action :nothing)
(up to date)
Recipe: postgresql::enable
* directory[/var/opt/gitlab/postgresql] action create (up to date)
* directory[/var/opt/gitlab/postgresql/data] action create (up to date)
* directory[/var/log/gitlab/postgresql] action create (up to date)
* directory[/var/opt/gitlab/postgresql/data] action create (up to date)
* execute[/opt/gitlab/embedded/bin/initdb -D /var/opt/gitlab/postgresql/data -E UTF8] action run (skipped due to not_if)
* file[/var/opt/gitlab/postgresql/data/server.crt] action create (up to date)
* file[/var/opt/gitlab/postgresql/data/server.key] action create (up to date)
* postgresql_config[gitlab] action create
* template[/var/opt/gitlab/postgresql/data/postgresql.conf] action create (up to date)
* template[/var/opt/gitlab/postgresql/data/runtime.conf] action create (up to date)
* template[/var/opt/gitlab/postgresql/data/pg_hba.conf] action create (up to date)
* template[/var/opt/gitlab/postgresql/data/pg_ident.conf] action create (up to date)
(up to date)
Recipe: postgresql::standalone
* service[postgresql] action nothing (skipped due to action :nothing)
* runit_service[postgresql] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/postgresql] action create (up to date)
* template[/opt/gitlab/sv/postgresql/run] action create (up to date)
* directory[/opt/gitlab/sv/postgresql/log] action create (up to date)
* directory[/opt/gitlab/sv/postgresql/log/main] action create (up to date)
* template[/opt/gitlab/sv/postgresql/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_postgresql] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/postgresql/config] action create (up to date)
* template[/opt/gitlab/sv/postgresql/log/run] action create (up to date)
* directory[/opt/gitlab/sv/postgresql/env] action create (up to date)
* ruby_block[Delete unmanaged env files for postgresql service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/postgresql/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/postgresql/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/postgresql/control] action create (up to date)
* template[/opt/gitlab/sv/postgresql/control/t] action create (up to date)
* link[/opt/gitlab/init/postgresql] action create (up to date)
* file[/opt/gitlab/sv/postgresql/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/postgresql] action create (up to date)
* ruby_block[wait for postgresql service socket] action run (skipped due to not_if)
* directory[/opt/gitlab/service/postgresql/supervise] action create (up to date)
* directory[/opt/gitlab/service/postgresql/log/supervise] action create (up to date)
* file[/opt/gitlab/service/postgresql/supervise/ok] action touch (skipped due to only_if)
* file[/opt/gitlab/service/postgresql/log/supervise/ok] action touch (skipped due to only_if)
* file[/opt/gitlab/service/postgresql/supervise/status] action touch (skipped due to only_if)
* file[/opt/gitlab/service/postgresql/log/supervise/status] action touch (skipped due to only_if)
* file[/opt/gitlab/service/postgresql/supervise/control] action touch (skipped due to only_if)
* file[/opt/gitlab/service/postgresql/log/supervise/control] action touch (skipped due to only_if)
(up to date)
* database_objects[postgresql] action create
* postgresql_user[gitlab] action create
* execute[create gitlab postgresql user] action run (skipped due to not_if)
(up to date)
* postgresql_user[gitlab_replicator] action create
* execute[create gitlab_replicator postgresql user] action run (skipped due to not_if)
* execute[set options for gitlab_replicator postgresql user] action run (skipped due to not_if)
(up to date)
* postgresql_database[gitlabhq_production] action create
* execute[create database gitlabhq_production] action run (skipped due to not_if)
(up to date)
* postgresql_extension[pg_trgm] action enable
* postgresql_query[enable pg_trgm extension] action run (skipped due to only_if)
(up to date)
* postgresql_extension[btree_gist] action enable
* postgresql_query[enable btree_gist extension] action run (skipped due to only_if)
(up to date)
(up to date)
* version_file[Create version file for PostgreSQL] action create
* file[/var/opt/gitlab/postgresql/VERSION] action create (up to date)
(up to date)
* ruby_block[warn pending postgresql restart] action run (skipped due to only_if)
* execute[reload postgresql] action nothing (skipped due to action :nothing)
* execute[start postgresql] action nothing (skipped due to action :nothing)
Recipe: praefect::disable
* service[praefect] action nothing (skipped due to action :nothing)
* runit_service[praefect] action disable
* ruby_block[disable praefect] action run (skipped due to only_if)
(up to date)
* consul_service[praefect] action delete
* file[/var/opt/gitlab/consul/config.d/praefect-service.json] action delete (up to date)
(up to date)
Recipe: gitlab-kas::enable
* directory[/var/opt/gitlab/gitlab-kas] action create (up to date)
* directory[/var/log/gitlab/gitlab-kas] action create (up to date)
* directory[/opt/gitlab/etc/gitlab-kas] action create (up to date)
* ruby_block[websocket TLS termination] action run (skipped due to only_if)
* version_file[Create version file for Gitlab KAS] action create
* file[/var/opt/gitlab/gitlab-kas/VERSION] action create (up to date)
(up to date)
* file[/var/opt/gitlab/gitlab-kas/authentication_secret_file] action create (up to date)
* file[/var/opt/gitlab/gitlab-kas/private_api_authentication_secret_file] action create (up to date)
* file[/var/opt/gitlab/gitlab-kas/redis_password_file] action create (skipped due to only_if)
* template[/var/opt/gitlab/gitlab-kas/gitlab-kas-config.yml] action create (up to date)
* env_dir[/opt/gitlab/etc/gitlab-kas/env] action create
* directory[/opt/gitlab/etc/gitlab-kas/env] action create (up to date)
* file[/opt/gitlab/etc/gitlab-kas/env/SSL_CERT_DIR] action create (up to date)
* file[/opt/gitlab/etc/gitlab-kas/env/OWN_PRIVATE_API_URL] action create (up to date)
(up to date)
* service[gitlab-kas] action nothing (skipped due to action :nothing)
* runit_service[gitlab-kas] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/gitlab-kas] action create (up to date)
* template[/opt/gitlab/sv/gitlab-kas/run] action create (up to date)
* directory[/opt/gitlab/sv/gitlab-kas/log] action create (up to date)
* directory[/opt/gitlab/sv/gitlab-kas/log/main] action create (up to date)
* template[/opt/gitlab/sv/gitlab-kas/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_gitlab-kas] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/gitlab-kas/config] action create (up to date)
* template[/opt/gitlab/sv/gitlab-kas/log/run] action create (up to date)
* directory[/opt/gitlab/sv/gitlab-kas/env] action create (up to date)
* ruby_block[Delete unmanaged env files for gitlab-kas service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/gitlab-kas/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/gitlab-kas/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/gitlab-kas/control] action create (up to date)
* link[/opt/gitlab/init/gitlab-kas] action create (up to date)
* file[/opt/gitlab/sv/gitlab-kas/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/gitlab-kas] action create (up to date)
* ruby_block[wait for gitlab-kas service socket] action run (skipped due to not_if)
(up to date)
Recipe: gitlab::database_migrations
* ruby_block[check remote PG version] action nothing (skipped due to action :nothing)
* rails_migration[gitlab-rails] action run
* bash[migrate gitlab-rails database] action run (skipped due to not_if)
(up to date)
Recipe: crond::disable
* service[crond] action nothing (skipped due to action :nothing)
* runit_service[crond] action disable
* ruby_block[disable crond] action run (skipped due to only_if)
(up to date)
Recipe: gitlab::puma
* directory[/var/log/gitlab/puma] action create (up to date)
* directory[/opt/gitlab/var/puma] action create (up to date)
* directory[/var/opt/gitlab/gitlab-rails/sockets] action create (up to date)
* puma_config[/var/opt/gitlab/gitlab-rails/etc/puma.rb] action create
* directory[/var/opt/gitlab/gitlab-rails/etc] action create (up to date)
* template[/var/opt/gitlab/gitlab-rails/etc/puma.rb] action create (up to date)
(up to date)
* service[puma] action nothing (skipped due to action :nothing)
* runit_service[puma] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/puma] action create (up to date)
* template[/opt/gitlab/sv/puma/run] action create (up to date)
* directory[/opt/gitlab/sv/puma/log] action create (up to date)
* directory[/opt/gitlab/sv/puma/log/main] action create (up to date)
* template[/opt/gitlab/sv/puma/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_puma] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/puma/config] action create (up to date)
* template[/opt/gitlab/sv/puma/log/run] action create (up to date)
* directory[/opt/gitlab/sv/puma/env] action create (up to date)
* ruby_block[Delete unmanaged env files for puma service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/puma/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/puma/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/puma/control] action create (up to date)
* template[/opt/gitlab/sv/puma/control/t] action create (up to date)
* template[/opt/gitlab/sv/puma/control/h] action create (up to date)
* link[/opt/gitlab/init/puma] action create (up to date)
* file[/opt/gitlab/sv/puma/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/puma] action create (up to date)
* ruby_block[wait for puma service socket] action run (skipped due to not_if)
(up to date)
* consul_service[rails] action delete
* file[/var/opt/gitlab/consul/config.d/rails-service.json] action delete (up to date)
(up to date)
* gitlab_sysctl[net.core.somaxconn] action create
* directory[create /etc/sysctl.d for net.core.somaxconn] action create (up to date)
* file[create /opt/gitlab/embedded/etc/90-omnibus-gitlab-net.core.somaxconn.conf net.core.somaxconn] action create (up to date)
* link[/etc/sysctl.d/90-omnibus-gitlab-net.core.somaxconn.conf] action create (up to date)
* execute[load sysctl conf net.core.somaxconn] action nothing (skipped due to action :nothing)
(up to date)
Recipe: gitlab::sidekiq
* sidekiq_service[sidekiq] action enable
* directory[/var/log/gitlab/sidekiq] action create (up to date)
Recipe: <Dynamically Defined Resource>
* service[sidekiq] action nothing (skipped due to action :nothing)
* runit_service[sidekiq] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/sidekiq] action create (up to date)
* template[/opt/gitlab/sv/sidekiq/run] action create (up to date)
* directory[/opt/gitlab/sv/sidekiq/log] action create (up to date)
* directory[/opt/gitlab/sv/sidekiq/log/main] action create (up to date)
* template[/opt/gitlab/sv/sidekiq/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_sidekiq] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/sidekiq/config] action create (up to date)
* template[/opt/gitlab/sv/sidekiq/log/run] action create (up to date)
* directory[/opt/gitlab/sv/sidekiq/env] action create (up to date)
* ruby_block[Delete unmanaged env files for sidekiq service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/sidekiq/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/sidekiq/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/sidekiq/control] action create (up to date)
* link[/opt/gitlab/init/sidekiq] action create (up to date)
* file[/opt/gitlab/sv/sidekiq/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/sidekiq] action create (up to date)
* ruby_block[wait for sidekiq service socket] action run (skipped due to not_if)
(up to date)
(up to date)
Recipe: gitlab::sidekiq
* consul_service[sidekiq] action delete
* file[/var/opt/gitlab/consul/config.d/sidekiq-service.json] action delete (up to date)
(up to date)
Recipe: gitlab::gitlab-workhorse
* directory[/var/opt/gitlab/gitlab-workhorse] action create (up to date)
* directory[/var/opt/gitlab/gitlab-workhorse/sockets] action create (up to date)
* directory[/var/log/gitlab/gitlab-workhorse] action create (up to date)
* directory[/opt/gitlab/etc/gitlab-workhorse] action create (up to date)
* env_dir[/opt/gitlab/etc/gitlab-workhorse/env] action create
* directory[/opt/gitlab/etc/gitlab-workhorse/env] action create (up to date)
* file[/opt/gitlab/etc/gitlab-workhorse/env/PATH] action create (up to date)
* file[/opt/gitlab/etc/gitlab-workhorse/env/HOME] action create (up to date)
* file[/opt/gitlab/etc/gitlab-workhorse/env/SSL_CERT_DIR] action create (up to date)
(up to date)
* service[gitlab-workhorse] action nothing (skipped due to action :nothing)
* runit_service[gitlab-workhorse] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/gitlab-workhorse] action create (up to date)
* template[/opt/gitlab/sv/gitlab-workhorse/run] action create (up to date)
* directory[/opt/gitlab/sv/gitlab-workhorse/log] action create (up to date)
* directory[/opt/gitlab/sv/gitlab-workhorse/log/main] action create (up to date)
* template[/opt/gitlab/sv/gitlab-workhorse/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_gitlab-workhorse] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/gitlab-workhorse/config] action create (up to date)
* template[/opt/gitlab/sv/gitlab-workhorse/log/run] action create (up to date)
* directory[/opt/gitlab/sv/gitlab-workhorse/env] action create (up to date)
* ruby_block[Delete unmanaged env files for gitlab-workhorse service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/gitlab-workhorse/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/gitlab-workhorse/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/gitlab-workhorse/control] action create (up to date)
* link[/opt/gitlab/init/gitlab-workhorse] action create (up to date)
* file[/opt/gitlab/sv/gitlab-workhorse/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/gitlab-workhorse] action create (up to date)
* ruby_block[wait for gitlab-workhorse service socket] action run (skipped due to not_if)
(up to date)
* consul_service[workhorse] action delete
* file[/var/opt/gitlab/consul/config.d/workhorse-service.json] action delete (up to date)
(up to date)
* version_file[Create version file for Workhorse] action create
* file[/var/opt/gitlab/gitlab-workhorse/VERSION] action create (up to date)
(up to date)
* template[/var/opt/gitlab/gitlab-workhorse/config.toml] action create (up to date)
Recipe: gitlab::mailroom_disable
* service[mailroom] action nothing (skipped due to action :nothing)
* runit_service[mailroom] action disable
* ruby_block[disable mailroom] action run (skipped due to only_if)
(up to date)
Recipe: gitlab::nginx
* directory[/var/opt/gitlab/nginx] action create (up to date)
* directory[/var/opt/gitlab/nginx/conf] action create (up to date)
* directory[/var/log/gitlab/nginx] action create (up to date)
* link[/var/opt/gitlab/nginx/logs] action create (up to date)
* template[/var/opt/gitlab/nginx/conf/gitlab-http.conf] action create (up to date)
* template[/var/opt/gitlab/nginx/conf/gitlab-smartcard-http.conf] action delete (up to date)
* template[/var/opt/gitlab/nginx/conf/gitlab-health.conf] action create (up to date)
* template[/var/opt/gitlab/nginx/conf/gitlab-pages.conf] action delete (up to date)
* template[/var/opt/gitlab/nginx/conf/gitlab-registry.conf] action delete (up to date)
* template[/var/opt/gitlab/nginx/conf/gitlab-mattermost-http.conf] action delete (up to date)
* template[/var/opt/gitlab/nginx/conf/nginx-status.conf] action create (up to date)
* consul_service[nginx] action delete
* file[/var/opt/gitlab/consul/config.d/nginx-service.json] action delete (up to date)
(up to date)
* template[/var/opt/gitlab/nginx/conf/nginx.conf] action create (up to date)
Recipe: nginx::enable
* service[nginx] action nothing (skipped due to action :nothing)
* runit_service[nginx] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/nginx] action create (up to date)
* template[/opt/gitlab/sv/nginx/run] action create (up to date)
* directory[/opt/gitlab/sv/nginx/log] action create (up to date)
* directory[/opt/gitlab/sv/nginx/log/main] action create (up to date)
* template[/opt/gitlab/sv/nginx/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_nginx] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/nginx/config] action create (up to date)
* template[/opt/gitlab/sv/nginx/log/run] action create (up to date)
* directory[/opt/gitlab/sv/nginx/env] action create (up to date)
* ruby_block[Delete unmanaged env files for nginx service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/nginx/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/nginx/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/nginx/control] action create (up to date)
* link[/opt/gitlab/init/nginx] action create (up to date)
* file[/opt/gitlab/sv/nginx/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/nginx] action create (up to date)
* ruby_block[wait for nginx service socket] action run (skipped due to not_if)
(up to date)
* execute[reload nginx] action nothing (skipped due to action :nothing)
Recipe: gitlab::remote-syslog_disable
* service[remote-syslog] action nothing (skipped due to action :nothing)
* runit_service[remote-syslog] action disable
* ruby_block[disable remote-syslog] action run (skipped due to only_if)
(up to date)
Recipe: gitlab::storage-check_disable
* service[storage-check] action nothing (skipped due to action :nothing)
* runit_service[storage-check] action disable
* ruby_block[disable storage-check] action run (skipped due to only_if)
(up to date)
Recipe: gitlab-pages::disable
* service[gitlab-pages] action nothing (skipped due to action :nothing)
* runit_service[gitlab-pages] action disable
* ruby_block[disable gitlab-pages] action run (skipped due to only_if)
(up to date)
Recipe: registry::disable
* service[registry] action nothing (skipped due to action :nothing)
* runit_service[registry] action disable
* ruby_block[disable registry] action run (skipped due to only_if)
(up to date)
Recipe: mattermost::disable
* service[mattermost] action nothing (skipped due to action :nothing)
* runit_service[mattermost] action disable
* ruby_block[disable mattermost] action run (skipped due to only_if)
(up to date)
Recipe: letsencrypt::disable
* crond_job[letsencrypt-renew] action delete
* file[/var/opt/gitlab/crond/letsencrypt-renew] action delete (up to date)
(up to date)
Recipe: gitlab::gitlab-healthcheck
* template[/opt/gitlab/etc/gitlab-healthcheck-rc] action create (up to date)
Recipe: monitoring::user
* account[Prometheus user and group] action create
* group[Prometheus user and group] action create (up to date)
* linux_user[Prometheus user and group] action create (up to date)
(up to date)
Recipe: monitoring::node-exporter
* directory[/var/log/gitlab/node-exporter] action create (up to date)
* directory[/opt/gitlab/etc/node-exporter/env] action create (up to date)
* env_dir[/opt/gitlab/etc/node-exporter/env] action create
* directory[/opt/gitlab/etc/node-exporter/env] action create (up to date)
* file[/opt/gitlab/etc/node-exporter/env/SSL_CERT_DIR] action create (up to date)
(up to date)
* directory[/var/opt/gitlab/node-exporter/textfile_collector] action create (up to date)
* service[node-exporter] action nothing (skipped due to action :nothing)
* runit_service[node-exporter] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/node-exporter] action create (up to date)
* template[/opt/gitlab/sv/node-exporter/run] action create (up to date)
* directory[/opt/gitlab/sv/node-exporter/log] action create (up to date)
* directory[/opt/gitlab/sv/node-exporter/log/main] action create (up to date)
* template[/opt/gitlab/sv/node-exporter/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_node-exporter] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/node-exporter/config] action create (up to date)
* template[/opt/gitlab/sv/node-exporter/log/run] action create (up to date)
* directory[/opt/gitlab/sv/node-exporter/env] action create (up to date)
* ruby_block[Delete unmanaged env files for node-exporter service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/node-exporter/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/node-exporter/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/node-exporter/control] action create (up to date)
* link[/opt/gitlab/init/node-exporter] action create (up to date)
* file[/opt/gitlab/sv/node-exporter/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/node-exporter] action create (up to date)
* ruby_block[wait for node-exporter service socket] action run (skipped due to not_if)
(up to date)
* consul_service[node-exporter] action delete
* file[/var/opt/gitlab/consul/config.d/node-exporter-service.json] action delete (up to date)
(up to date)
Recipe: monitoring::gitlab-exporter
* directory[/var/opt/gitlab/gitlab-exporter] action create (up to date)
* directory[/var/log/gitlab/gitlab-exporter] action create (up to date)
* env_dir[/opt/gitlab/etc/gitlab-exporter/env] action create
* directory[/opt/gitlab/etc/gitlab-exporter/env] action create (up to date)
* file[/opt/gitlab/etc/gitlab-exporter/env/LD_PRELOAD] action create (up to date)
* file[/opt/gitlab/etc/gitlab-exporter/env/MALLOC_CONF] action create (up to date)
* file[/opt/gitlab/etc/gitlab-exporter/env/RUBY_GC_HEAP_INIT_SLOTS] action create (up to date)
* file[/opt/gitlab/etc/gitlab-exporter/env/RUBY_GC_HEAP_FREE_SLOTS_MIN_RATIO] action create (up to date)
* file[/opt/gitlab/etc/gitlab-exporter/env/RUBY_GC_HEAP_FREE_SLOTS_MAX_RATIO] action create (up to date)
* file[/opt/gitlab/etc/gitlab-exporter/env/SSL_CERT_DIR] action create (up to date)
* file[/opt/gitlab/etc/gitlab-exporter/env/SSL_CERT_FILE] action create (up to date)
(up to date)
* template[/var/opt/gitlab/gitlab-exporter/gitlab-exporter.yml] action create (up to date)
* version_file[Create version file for GitLab-Exporter] action create
* file[/var/opt/gitlab/gitlab-exporter/RUBY_VERSION] action create (up to date)
(up to date)
* service[gitlab-exporter] action nothing (skipped due to action :nothing)
* runit_service[gitlab-exporter] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/gitlab-exporter] action create (up to date)
* template[/opt/gitlab/sv/gitlab-exporter/run] action create (up to date)
* directory[/opt/gitlab/sv/gitlab-exporter/log] action create (up to date)
* directory[/opt/gitlab/sv/gitlab-exporter/log/main] action create (up to date)
* template[/opt/gitlab/sv/gitlab-exporter/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_gitlab-exporter] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/gitlab-exporter/config] action create (up to date)
* template[/opt/gitlab/sv/gitlab-exporter/log/run] action create (up to date)
* directory[/opt/gitlab/sv/gitlab-exporter/env] action create (up to date)
* ruby_block[Delete unmanaged env files for gitlab-exporter service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/gitlab-exporter/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/gitlab-exporter/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/gitlab-exporter/control] action create (up to date)
* link[/opt/gitlab/init/gitlab-exporter] action create (up to date)
* file[/opt/gitlab/sv/gitlab-exporter/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/gitlab-exporter] action create (up to date)
* ruby_block[wait for gitlab-exporter service socket] action run (skipped due to not_if)
(up to date)
* consul_service[gitlab-exporter] action delete
* file[/var/opt/gitlab/consul/config.d/gitlab-exporter-service.json] action delete (up to date)
(up to date)
Recipe: monitoring::redis-exporter
* directory[/var/log/gitlab/redis-exporter] action create (up to date)
* directory[/opt/gitlab/etc/redis-exporter/env] action create (up to date)
* env_dir[/opt/gitlab/etc/redis-exporter/env] action create
* directory[/opt/gitlab/etc/redis-exporter/env] action create (up to date)
* file[/opt/gitlab/etc/redis-exporter/env/SSL_CERT_DIR] action create (up to date)
(up to date)
* service[redis-exporter] action nothing (skipped due to action :nothing)
* runit_service[redis-exporter] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/redis-exporter] action create (up to date)
* template[/opt/gitlab/sv/redis-exporter/run] action create (up to date)
* directory[/opt/gitlab/sv/redis-exporter/log] action create (up to date)
* directory[/opt/gitlab/sv/redis-exporter/log/main] action create (up to date)
* template[/opt/gitlab/sv/redis-exporter/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_redis-exporter] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/redis-exporter/config] action create (up to date)
* template[/opt/gitlab/sv/redis-exporter/log/run] action create (up to date)
* directory[/opt/gitlab/sv/redis-exporter/env] action create (up to date)
* ruby_block[Delete unmanaged env files for redis-exporter service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/redis-exporter/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/redis-exporter/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/redis-exporter/control] action create (up to date)
* link[/opt/gitlab/init/redis-exporter] action create (up to date)
* file[/opt/gitlab/sv/redis-exporter/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/redis-exporter] action create (up to date)
* ruby_block[wait for redis-exporter service socket] action run (skipped due to not_if)
(up to date)
* consul_service[redis-exporter] action delete
* file[/var/opt/gitlab/consul/config.d/redis-exporter-service.json] action delete (up to date)
(up to date)
Recipe: monitoring::prometheus
* directory[/var/opt/gitlab/prometheus] action create (up to date)
* directory[/var/opt/gitlab/prometheus/rules] action create (up to date)
* directory[/var/log/gitlab/prometheus] action create (up to date)
* directory[/opt/gitlab/etc/prometheus/env] action create (up to date)
* env_dir[/opt/gitlab/etc/prometheus/env] action create
* directory[/opt/gitlab/etc/prometheus/env] action create (up to date)
* file[/opt/gitlab/etc/prometheus/env/SSL_CERT_DIR] action create (up to date)
(up to date)
* execute[reload prometheus] action nothing (skipped due to action :nothing)
* file[Prometheus config] action create (up to date)
* service[prometheus] action nothing (skipped due to action :nothing)
* runit_service[prometheus] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/prometheus] action create (up to date)
* template[/opt/gitlab/sv/prometheus/run] action create (up to date)
* directory[/opt/gitlab/sv/prometheus/log] action create (up to date)
* directory[/opt/gitlab/sv/prometheus/log/main] action create (up to date)
* template[/opt/gitlab/sv/prometheus/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_prometheus] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/prometheus/config] action create (up to date)
* template[/opt/gitlab/sv/prometheus/log/run] action create (up to date)
* directory[/opt/gitlab/sv/prometheus/env] action create (up to date)
* ruby_block[Delete unmanaged env files for prometheus service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/prometheus/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/prometheus/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/prometheus/control] action create (up to date)
* link[/opt/gitlab/init/prometheus] action create (up to date)
* file[/opt/gitlab/sv/prometheus/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/prometheus] action create (up to date)
* ruby_block[wait for prometheus service socket] action run (skipped due to not_if)
(up to date)
* consul_service[prometheus] action delete
* file[/var/opt/gitlab/consul/config.d/prometheus-service.json] action delete (up to date)
(up to date)
* template[/var/opt/gitlab/prometheus/rules/gitlab.rules] action create (up to date)
* template[/var/opt/gitlab/prometheus/rules/node.rules] action create (up to date)
Recipe: monitoring::alertmanager
* directory[/var/opt/gitlab/alertmanager] action create (up to date)
* directory[/var/log/gitlab/alertmanager] action create (up to date)
* directory[/opt/gitlab/etc/alertmanager/env] action create (up to date)
* env_dir[/opt/gitlab/etc/alertmanager/env] action create
* directory[/opt/gitlab/etc/alertmanager/env] action create (up to date)
* file[/opt/gitlab/etc/alertmanager/env/SSL_CERT_DIR] action create (up to date)
(up to date)
* file[Alertmanager config] action create (up to date)
* service[alertmanager] action nothing (skipped due to action :nothing)
* runit_service[alertmanager] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/alertmanager] action create (up to date)
* template[/opt/gitlab/sv/alertmanager/run] action create (up to date)
* directory[/opt/gitlab/sv/alertmanager/log] action create (up to date)
* directory[/opt/gitlab/sv/alertmanager/log/main] action create (up to date)
* template[/opt/gitlab/sv/alertmanager/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_alertmanager] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/alertmanager/config] action create (up to date)
* template[/opt/gitlab/sv/alertmanager/log/run] action create (up to date)
* directory[/opt/gitlab/sv/alertmanager/env] action create (up to date)
* ruby_block[Delete unmanaged env files for alertmanager service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/alertmanager/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/alertmanager/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/alertmanager/control] action create (up to date)
* link[/opt/gitlab/init/alertmanager] action create (up to date)
* file[/opt/gitlab/sv/alertmanager/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/alertmanager] action create (up to date)
* ruby_block[wait for alertmanager service socket] action run (skipped due to not_if)
(up to date)
Recipe: monitoring::postgres-exporter
* directory[/var/log/gitlab/postgres-exporter] action create (up to date)
* directory[/var/opt/gitlab/postgres-exporter] action create (up to date)
* env_dir[/opt/gitlab/etc/postgres-exporter/env] action create
* directory[/opt/gitlab/etc/postgres-exporter/env] action create (up to date)
* file[/opt/gitlab/etc/postgres-exporter/env/SSL_CERT_DIR] action create (up to date)
* file[/opt/gitlab/etc/postgres-exporter/env/DATA_SOURCE_NAME] action create (up to date)
(up to date)
* service[postgres-exporter] action nothing (skipped due to action :nothing)
* runit_service[postgres-exporter] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/postgres-exporter] action create (up to date)
* template[/opt/gitlab/sv/postgres-exporter/run] action create (up to date)
* directory[/opt/gitlab/sv/postgres-exporter/log] action create (up to date)
* directory[/opt/gitlab/sv/postgres-exporter/log/main] action create (up to date)
* template[/opt/gitlab/sv/postgres-exporter/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_postgres-exporter] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/postgres-exporter/config] action create (up to date)
* template[/opt/gitlab/sv/postgres-exporter/log/run] action create (up to date)
* directory[/opt/gitlab/sv/postgres-exporter/env] action create (up to date)
* ruby_block[Delete unmanaged env files for postgres-exporter service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/postgres-exporter/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/postgres-exporter/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/postgres-exporter/control] action create (up to date)
* link[/opt/gitlab/init/postgres-exporter] action create (up to date)
* file[/opt/gitlab/sv/postgres-exporter/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/postgres-exporter] action create (up to date)
* ruby_block[wait for postgres-exporter service socket] action run (skipped due to not_if)
(up to date)
* template[/var/opt/gitlab/postgres-exporter/queries.yaml] action create (up to date)
* consul_service[postgres-exporter] action delete
* file[/var/opt/gitlab/consul/config.d/postgres-exporter-service.json] action delete (up to date)
(up to date)
Recipe: monitoring::grafana
* directory[/var/log/gitlab/grafana] action create (up to date)
* directory[/var/opt/gitlab/grafana] action create (up to date)
* directory[/var/opt/gitlab/grafana/provisioning] action create (up to date)
* directory[/var/opt/gitlab/grafana/provisioning/dashboards] action create (up to date)
* directory[/var/opt/gitlab/grafana/provisioning/datasources] action create (up to date)
* directory[/var/opt/gitlab/grafana/provisioning/notifiers] action create (up to date)
* file[/var/opt/gitlab/grafana/CVE_reset_status] action delete (up to date)
* link[/var/opt/gitlab/grafana/conf] action create (up to date)
* link[/var/opt/gitlab/grafana/public] action create (up to date)
* directory[/opt/gitlab/etc/grafana/env] action create (up to date)
* ruby_block[populate Grafana configuration options] action run
- execute the ruby block populate Grafana configuration options
* env_dir[/opt/gitlab/etc/grafana/env] action create
* directory[/opt/gitlab/etc/grafana/env] action create (up to date)
* file[/opt/gitlab/etc/grafana/env/SSL_CERT_DIR] action create (up to date)
(up to date)
* template[/var/opt/gitlab/grafana/grafana.ini] action create (up to date)
* file[/var/opt/gitlab/grafana/provisioning/dashboards/gitlab_dashboards.yml] action create (up to date)
* file[/var/opt/gitlab/grafana/provisioning/datasources/gitlab_datasources.yml] action create (up to date)
* service[grafana] action nothing (skipped due to action :nothing)
* runit_service[grafana] action enable
* ruby_block[restart_service] action nothing (skipped due to action :nothing)
* ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
* ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/sv/grafana] action create (up to date)
* template[/opt/gitlab/sv/grafana/run] action create (up to date)
* directory[/opt/gitlab/sv/grafana/log] action create (up to date)
* directory[/opt/gitlab/sv/grafana/log/main] action create (up to date)
* template[/opt/gitlab/sv/grafana/log/config] action create (up to date)
* ruby_block[verify_chown_persisted_on_grafana] action nothing (skipped due to action :nothing)
* link[/var/log/gitlab/grafana/config] action create (up to date)
* template[/opt/gitlab/sv/grafana/log/run] action create (up to date)
* directory[/opt/gitlab/sv/grafana/env] action create (up to date)
* ruby_block[Delete unmanaged env files for grafana service] action run (skipped due to only_if)
* template[/opt/gitlab/sv/grafana/check] action create (skipped due to only_if)
* template[/opt/gitlab/sv/grafana/finish] action create (skipped due to only_if)
* directory[/opt/gitlab/sv/grafana/control] action create (up to date)
* link[/opt/gitlab/init/grafana] action create (up to date)
* file[/opt/gitlab/sv/grafana/down] action nothing (skipped due to action :nothing)
* directory[/opt/gitlab/service] action create (up to date)
* link[/opt/gitlab/service/grafana] action create (up to date)
* ruby_block[wait for grafana service socket] action run (skipped due to not_if)
(up to date)
Recipe: gitlab::database_reindexing_disable
* crond_job[database-reindexing] action delete
* file[/var/opt/gitlab/crond/database-reindexing] action delete (up to date)
(up to date)
Recipe: gitlab::puma
* runit_service[puma] action restart (up to date)
Recipe: gitlab::sidekiq
* sidekiq_service[sidekiq] action restart
Recipe: <Dynamically Defined Resource>
* service[sidekiq] action nothing (skipped due to action :nothing)
* runit_service[sidekiq] action restart
================================================================================
Error executing action `restart` on resource 'runit_service[sidekiq]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
STDOUT: timeout: run: /opt/gitlab/service/sidekiq: (pid 30767) 145658s, got TERM
STDERR:
---- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq returned 1
Cookbook Trace:
---------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `tap'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `safe_sv_shellout!'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:164:in `restart_service'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb:358:in `block in <class:RunitService>'
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/sidekiq_service.rb
50: runit_service new_resource.name do
51: action :restart
52: end
53: end
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/sidekiq_service.rb:50:in `block in class_from_file'
runit_service("sidekiq") do
action [:restart]
default_guard_interpreter :default
declared_type :runit_service
cookbook_name "gitlab"
service_name "sidekiq"
options {:env_dir=>"/opt/gitlab/sv/sidekiq/env"}
run_template_name "sidekiq"
log_template_name "sidekiq"
check_script_template_name "sidekiq"
finish_script_template_name "sidekiq"
control_template_names {}
end
System Info:
------------
chef_version=15.17.4
platform=centos
platform_version=7.7.1908
ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
================================================================================
Error executing action `restart` on resource 'sidekiq_service[sidekiq]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
runit_service[sidekiq] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/sidekiq_service.rb line 50) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
STDOUT: timeout: run: /opt/gitlab/service/sidekiq: (pid 30767) 145658s, got TERM
STDERR:
---- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq returned 1
Cookbook Trace:
---------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `tap'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:136:in `safe_sv_shellout!'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/helpers.rb:164:in `restart_service'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/runit/libraries/provider_runit_service.rb:358:in `block in <class:RunitService>'
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/sidekiq.rb
21: sidekiq_service 'sidekiq' do
22: rails_app 'gitlab-rails'
23: user account_helper.gitlab_user
24: group account_helper.gitlab_group
25: end
26:
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/sidekiq.rb:21:in `from_file'
sidekiq_service("sidekiq") do
action [:enable]
default_guard_interpreter :default
declared_type :sidekiq_service
cookbook_name "gitlab"
recipe_name "sidekiq"
rails_app "gitlab-rails"
user "git"
group "git"
end
System Info:
------------
chef_version=15.17.4
platform=centos
platform_version=7.7.1908
ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client
Recipe: gitlab::gitlab-rails
* execute[clear the gitlab-rails cache] action run
- execute /opt/gitlab/bin/gitlab-rake cache:clear
```
**Running handlers:**
```
There was an error running gitlab-ctl reconfigure:
sidekiq_service[sidekiq] (gitlab::sidekiq line 21) had an error: Mixlib::ShellOut::ShellCommandFailed: runit_service[sidekiq] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/sidekiq_service.rb line 50) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
STDOUT: timeout: run: /opt/gitlab/service/sidekiq: (pid 30767) 145658s, got TERM
STDERR:
---- End output of /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq ----
Ran /opt/gitlab/embedded/bin/sv restart /opt/gitlab/service/sidekiq returned 1
Running handlers complete
Cinc Client failed. 4 resources updated in 02 minutes 08 seconds
```https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6753Mattermost 502 Error when updating to version 14.9.12022-09-10T09:29:03ZJuan CastrillonMattermost 502 Error when updating to version 14.9.1### Summary
We updated a gitlab omnibus installation (`gitlab-ce`) from 14.5.2 to the latest 14.9.1. Everything works fine with gitlab, but when accessing mattermost, it returns a 502 error.
### Steps to reproduce
Update gitlab-ce fro...### Summary
We updated a gitlab omnibus installation (`gitlab-ce`) from 14.5.2 to the latest 14.9.1. Everything works fine with gitlab, but when accessing mattermost, it returns a 502 error.
### Steps to reproduce
Update gitlab-ce from 14.5.2 to 14.9.1 with `sudo yum install gitlab-ce`
### What is the current *bug* behavior?
Some postgres error appears to happen, making mattermost fail its database migration and therefore failing to serve it.
### What is the expected *correct* behavior?
The update is done and mattermost is online after a while
### Relevant logs
<details>
<summary> Relevant logs </summary>
<pre>
`/var/log/gitlab/postgresql/current` (This set of errors repeats every 5 minutes):
```
2022-03-28_12:54:11.79145 ERROR: duplicate key value violates unique constraint "db_lock_pkey"
2022-03-28_12:54:11.79148 DETAIL: Key (id)=(mm-lock-key) already exists.
2022-03-28_12:54:11.79148 STATEMENT: INSERT INTO db_lock (id, expireat) VALUES ($1, $2)
2022-03-28_12:54:11.79160 ERROR: current transaction is aborted, commands ignored until end of transaction block
2022-03-28_12:54:11.79161 STATEMENT: SELECT expireat FROM db_lock WHERE id = $1
```
`/var/log/gitlab/mattermost/mattermost.log` (Is worth noting that in the morning of the 24th, I tried to restart mattermost with `gitlab-ctl`. That explains the gap in the logs, however, it seems to be stuck in that point in time and dont show any more) :
```
{"timestamp":"2022-03-23 17:18:30.414 Z","level":"fatal","msg":"Failed to apply database migrations.","caller":"sqlstore/store.go:157","error":"driver: postgres, message: failed to execute migration, command: executing_query, originalError: pq: canceling statement due to statement timeout, query: \n\nDO $$\n<<migrate_root_id>>\nDECLARE \n parentid_exist boolean := false;\n alter_fileids boolean := false;\n alter_props boolean := false;\nBEGIN\nSELECT count(*) != 0 INTO parentid_exist\n FROM information_schema.columns\n WHERE table_name = 'posts'\n AND column_name = 'parentid';\nSELECT count(*) != 0 INTO alter_fileids\n FROM information_schema.columns\n WHERE table_name = 'posts'\n AND column_name = 'fileids'\n AND data_type = 'character varying'\n AND character_maximum_length != 300;\nSELECT count(*) != 0 INTO alter_props\n FROM information_schema.columns\n WHERE table_name = 'posts'\n AND column_name = 'props'\n AND data_type != 'jsonb';\nIF alter_fileids OR alter_props THEN\n IF parentid_exist THEN\n UPDATE posts SET rootid = parentid WHERE rootid = '' AND rootid != parentid;\n ALTER TABLE posts ALTER COLUMN fileids TYPE varchar(300), ALTER COLUMN props TYPE jsonb USING props::jsonb, DROP COLUMN ParentId;\n ELSE\n ALTER TABLE posts ALTER COLUMN fileids TYPE varchar(300), ALTER COLUMN props TYPE jsonb USING props::jsonb;\n END IF;\nEND IF;\nEND migrate_root_id $$;\n\nCREATE INDEX IF NOT EXISTS idx_posts_root_id_delete_at ON posts(rootid, deleteat);\n\nDROP INDEX IF EXISTS idx_posts_root_id;\n\n"}
{"timestamp":"2022-03-23 17:18:30.687 Z","level":"info","msg":"Server is initializing...","caller":"app/server.go:237","go_version":"go1.16.7"}
{"timestamp":"2022-03-23 17:18:30.687 Z","level":"info","msg":"Starting websocket hubs","caller":"app/web_hub.go:93","number_of_hubs":12}
{"timestamp":"2022-03-23 17:18:30.687 Z","level":"info","msg":"Pinging SQL","caller":"sqlstore/store.go:215","database":"master"}
{"timestamp":"2022-03-23 17:19:10.349 Z","level":"info","msg":"Server is initializing...","caller":"app/server.go:237","go_version":"go1.16.7"}
{"timestamp":"2022-03-23 17:19:10.349 Z","level":"info","msg":"Starting websocket hubs","caller":"app/web_hub.go:93","number_of_hubs":12}
{"timestamp":"2022-03-23 17:19:10.349 Z","level":"info","msg":"Pinging SQL","caller":"sqlstore/store.go:215","database":"master"}
{"timestamp":"2022-03-24 07:35:06.196 Z","level":"info","msg":"Server is initializing...","caller":"app/server.go:237","go_version":"go1.16.7"}
{"timestamp":"2022-03-24 07:35:06.196 Z","level":"info","msg":"Starting websocket hubs","caller":"app/web_hub.go:93","number_of_hubs":12}
{"timestamp":"2022-03-24 07:35:06.196 Z","level":"info","msg":"Pinging SQL","caller":"sqlstore/store.go:215","database":"master"}
```
</pre>
</details>
### Details of package version
<details>
<summary>Provide the package version installation details</summary>
<pre>
gitlab-ce-14.9.1-ce.0.el7.x86_64
</pre>
</details>
### Environment details
* Operating System: `CentOS Linux release 7.6.1810 (Core)`
* Installation Target, remove incorrect values:
* VM: Digital Ocean
* Installation Type, remove incorrect values:
* Upgrade from version `14.5.2` to `14.9.1`
* Is there any other software running on the machine: `No`
* Is this a single or multiple node installation? Single node
* Resources
* CPU: `6vCPUs`
* Memory total: `16GB`
### Configuration details
<details>
<summary> Provide the relevant sections of `/etc/gitlab/gitlab.rb` </summary>
<pre>
<!--
Paste the details but omit lines starting with `#`. Relevant lines can be retrieved by
running `grep -v -e '^#' -e '^$' /etc/gitlab/gitlab.rb`. Be certain to sanitize sensitive
configuration such as passwords and secrets.
-->
</pre>
</details>https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6612Document database upgrade steps with official container when using podman2024-01-08T19:04:50ZJuan Carlos PerezDocument database upgrade steps with official container when using podman**Original Title:** Problem with postgresql upgrading gitlab from 8.17.7-ee.0 to 9.5.10 (PODMAN)
Hi!
I'm upgrading gitlab from 8.17.7 to 9.5.10 and i got some problems with postgresql. I use PODMAN.
The specific problem is creating a...**Original Title:** Problem with postgresql upgrading gitlab from 8.17.7-ee.0 to 9.5.10 (PODMAN)
Hi!
I'm upgrading gitlab from 8.17.7 to 9.5.10 and i got some problems with postgresql. I use PODMAN.
The specific problem is creating a directory so i created manually that directory but error persists.
LOG:
```
Creating temporary data directory:Error creating new directory: /var/opt/gitlab/postgresql/data.9.6.3
STDOUT:
**STDERR: su: System error
Creating temporary data directory: NOT OK**
== Fatal error ==
Please check the output
== Reverting ==
ok: down: postgresql: 1s, normally up
ok: run: postgresql: (pid 881) 1s
== Reverted ==
== Reverted to 9.2.18. Please check output for what went wrong ==
Toggling deploy page:rm -f /opt/gitlab/embedded/service/gitlab-rails/public/index.html
Toggling deploy page: OK
Toggling services:ok: run: gitaly: (pid 434) 32s
ok: run: gitlab-monitor: (pid 444) 32s
ok: run: logrotate: (pid 446) 32s
ok: run: node-exporter: (pid 441) 32s
ok: run: postgres-exporter: (pid 429) 32s
ok: run: prometheus: (pid 432) 32s
ok: run: redis-exporter: (pid 437) 32s
ok: run: sidekiq: (pid 426) 32s
ok: run: sshd: (pid 903) 0s
Toggling services: OK
Upgrading the existing database to 9.6 failed and was reverted.
Please check the output, and open an issue at:
https://gitlab.com/gitlab-org/omnibus-gitlab/issues
If you would like to restart the instance without attempting to
upgrade, add the following to your docker command:
-e GITLAB_SKIP_PG_UPGRADE=true
```
Please help :smile:
Thankshttps://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6563PG::UndefinedTable: ERROR: relation "analytics_instance_statistics_measureme...2022-11-11T11:58:05ZCarlo MaciasPG::UndefinedTable: ERROR: relation "analytics_instance_statistics_measurements" does not existI made a mistake in upgrading and now i got an error
> PG::UndefinedTable: ERROR: relation "analytics_instance_statistics_measurements" does not exist
> LINE 8: WHERE a.attrelid = '"analytics_instance_statistics_measurem...
> ...I made a mistake in upgrading and now i got an error
> PG::UndefinedTable: ERROR: relation "analytics_instance_statistics_measurements" does not exist
> LINE 8: WHERE a.attrelid = '"analytics_instance_statistics_measurem...
> ^
> ):
> 23: .gl-card
> 24: .gl-card-body
> 25: %h2.gl-mt-0
> 26: = number_with_delimiter @license.daily_billable_users_count
> 27: .text-uppercase.gl-mb-4= _('Billable Users')
> 28: %p
> 29: = _('This is the number of %{billable_users_link_start}billable users%{link_end} on your installation, and this is the minimum number you need to purchase when you renew your license.').html_safe % { billable_users_link_start: billable_users_link_start, link_end: link_end }
>
> app/models/analytics/usage_trends/measurement.rb:26:in `block in <class:Measurement>'
> app/models/analytics/usage_trends/measurement.rb:55:in `find_latest_or_fallback'
> lib/gitlab/metrics/instrumentation.rb:160:in `block in find_latest_or_fallback'
can you please help me on how to fix this? I need to renew my premium license and i can't go to /admin/license ?https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6308Create GitLab server on Ubuntu2023-04-22T16:24:04Zaguerax9Create GitLab server on Ubuntu<!---
Please read this!
Before you create a new issue, please make sure that you already ran
`sudo gitlab-ctl reconfigure`
Also make sure to search in https://gitlab.com/gitlab-org/omnibus-gitlab/issues,
to verify the issue you're abo...<!---
Please read this!
Before you create a new issue, please make sure that you already ran
`sudo gitlab-ctl reconfigure`
Also make sure to search in https://gitlab.com/gitlab-org/omnibus-gitlab/issues,
to verify the issue you're about to submit isn't a duplicate.
--->
### Summary
<!-- Describe in short the bug you've encountered -->
I'm trying to create a GitLab account for the company I work for.
We want to host GitLab ourselves so I decided to create a GitLab server on Ubuntu Linux.
### Steps to reproduce
<!-- How one can reproduce the issue - this is very important -->
By following the Omnibus GitLab installation tutorial on a Ubuntu machine
without a Docker Image or anything else and try to access to the GitLab by using
the external url provided during the installation.
### What is the current *bug* behavior?
<!-- What actually happens -->
I have installed GitLab by following the Omnibus GitLab installation Tutorial and It works well.
I can access to the GitLab by using my IP address (as root). The problem is that I can't access to
the GitLab by using the url provided during the installation. I can only access by using my IP address.
Maybe I made errors during the DNS choice step. For the DNS choice, I gave the url from which I want to access to GitLab (http://gitlab-server.example.com).
### What is the expected *correct* behavior?
<!-- What you should see instead -->
I expect to access GitLab by using the url and not the IP address in order that clients can access to GitLab too.
### Relevant logs
<details>
<summary> Relevant logs </summary>
<pre>
<!--
Paste any relevant logs.
Live log output can be found with `sudo gitlab-ctl tail`.
Log files live in `/var/log/gitlab` by default.
-->
</pre>
</details>
### Details of package version
<details>
<summary>Provide the package version installation details</summary>
<pre>
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-============================================================
ii gitlab-ce 14.1.2-ce.0 amd64 GitLab Community Edition (including NGINX, Postgres, Redis)
rc gitlab-ee 14.1.1-ee.0 amd64 GitLab Enterprise Edition (including NGINX, Postgres, Redis)
<!--
For Debian, Ubuntu based systems, paste the output of:
`dpkg-query -l "gitlab-*"`
For RHEL based systems (CentOS, RHEL, OL, Scientific, OpenSUSE, SLES), paste the output of:
`rpm -qa | grep 'gitlab'`
If you can't provide these details, see https://about.gitlab.com/getting-help/ page on where to ask your question.
-->
</pre>
</details>
### Environment details
* Operating System: `Linux, Ubuntu 20.04.2 LTS`
* Installation Target, remove incorrect values:
* Bare Metal Machine
* Installation Type, remove incorrect values:
* New Installation
* Is there any other software running on the machine: `android studio`
* Is this a single or multiple node installation?
* Resources
* CPU: `7.5 GiB / Intel® Core™ i5-1035G1 CPU @ 1.00GHz × 8 `
* Memory total: `512GB`
### Configuration details
<details>
<summary> Provide the relevant sections of `/etc/gitlab/gitlab.rb` </summary>
<pre>
<!--
Paste the details but omit lines starting with `#`. Relevant lines can be retrieved by
running `grep -v -e '^#' -e '^$' /etc/gitlab/gitlab.rb`. Be certain to sanitize sensitive
configuration such as passwords and secrets.
-->
external_url 'http://gitlab-server.example.com'
</pre>
</details>https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6280external_url changes when stopped and restarted on azure2021-12-30T04:30:09ZBenjamin Brummerexternal_url changes when stopped and restarted on azure### Summary
We are running Gitlab EE on Azure, and followed the official guide. If we stop the VM in the Azure panel and start it again, the external_url in the gitlab.rb is changed to external_url "https://ip.ip.ip.ip" (public ip).
##...### Summary
We are running Gitlab EE on Azure, and followed the official guide. If we stop the VM in the Azure panel and start it again, the external_url in the gitlab.rb is changed to external_url "https://ip.ip.ip.ip" (public ip).
### Steps to reproduce
- Setup VM as descriped in the guide https://docs.gitlab.com/ee/install/azure/
- change the external_url to the custom domain e.g gitlab.yourdomain.com
- stop the vm in the azure virtual machine panel
- start the vm in the azure virtual machine panel
### What is the current *bug* behavior?
external_url has the public ip after stop/restart.
### What is the expected *correct* behavior?
external_url should not change
### Relevant logs
<details>
<summary> Relevant logs </summary>
<pre>
<!--
Paste any relevant logs.
Live log output can be found with `sudo gitlab-ctl tail`.
Log files live in `/var/log/gitlab` by default.
-->
</pre>
</details>
### Details of package version
<details>
<summary>Provide the package version installation details</summary>
<pre>
dpkg-query -l "gitlab-*"
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-============================================================
un gitlab-ce <none> <none> (no description available)
ii gitlab-ee 14.0.6-ee.0 amd64 GitLab Enterprise Edition (including NGINX, Postgres, Redis)
</pre>
</details>
### Environment details
https://azuremarketplace.microsoft.com/en-us/marketplace/apps/gitlabinc1586447921813.gitlabeehttps://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6206Integrations failing to function with SSL decryption2021-06-24T16:53:55ZjeanluclariviereIntegrations failing to function with SSL decryption### Summary
Our security team has installed certificates on our on premise servers and enabled SSL decryption out to the internet. As a result, our MS Teams integration appear to be failing. I've confirmed that the `/opt/gitlab/embedded...### Summary
Our security team has installed certificates on our on premise servers and enabled SSL decryption out to the internet. As a result, our MS Teams integration appear to be failing. I've confirmed that the `/opt/gitlab/embedded/ssl/certs/ca-bundle.crt` was updated correctly (it contains all root ca we need) and also ran gitlab-ctl reconfigure many times.
I've also tried copying the cert to /etc/gitlab/ssl/certs and /etc/gitlab/trusted-certs but neither of these worked either.
### Steps to reproduce
Configure SSL decryption on your network going out to the internet
### What is the current *bug* behavior?
After installing the certificate on the server (RHEL 8 ca-bundle.crt) I can curl our MS Teams webhook https://domain365.webhook.office.com/webhookb2/xxxxxxxxxxxxxxxxxxxxxxxxx without any problems, the certificate is trusted. When I attempt to trigger the webhook in Gitlab (create a wiki page) it fails.
### What is the expected *correct* behavior?
We expect the webhook to be successful and trigger the notificatoin in MS teams
### Relevant logs
```gitlab-rails/application_json.log:{"severity":"INFO","time":"2021-06-14T17:22:19.847Z","correlation_id":"01F85RQ8N9EP7J7NY579F2NWWF","message":"MicrosoftTeams::Notifier: Error while connecting to https://domain365.webhook.office.com/webhookb2/xxxxxxxxxxxxxxxxxxxxxxxxx: SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)"}```
### Details of package version
13.12.3