Service Desk Rejected Emails

Summary

We are self-hosting GitLab 17.3.2 (deployed in a docker). We get one of the following errors when replying to a GitLab notification.

On 09/11/2024 10:44, GitLab wrote:

Unfortunately, your email message to GitLab could not be processed.

We couldn't find the project. Please check if there's any typo.

On 09/06/2024 10:32, GitLab wrote:

Unfortunately, your email message to GitLab could not be processed.

Gitlab::Email::InvalidIssueError

Steps to reproduce

Create a new issue by sending an email to Service Desk.
Receive, via email, the Service Desk notification that an issue was created.
Reply to the notification via email.
Receive one of the two error email messages shown above.

What is the current bug behavior?

One reply to notification will work (no rejection, body of email added to issue, notications sent to participants, etc) and then subsequent replys (to most recent or previous notifications for that same issue) will not work (rejection message).

What is the expected correct behavior?

All replies to gitlab notifications should be processed and connected/applied to the appropriate project and issue.

Relevant logs and/or screenshots

I am unsure of what service to tail for the Service Desk. If there is something specific required please point me in the right direction and I will be happy to provide more information. This issue is easily reproducible in our current deployment.

Results of GitLab environment info

System information
System:
Current User:   git
Using RVM:      no
Ruby Version:   3.1.5p253
Gem Version:    3.5.11
Bundler Version:2.5.11
Rake Version:   13.0.6
Redis Version:  7.0.15
Sidekiq Version:7.1.6
Go Version:     unknown

GitLab information
Version:        17.3.2
Revision:       951fd632abf
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     14.11
URL:            https://gitlab.<redacted>.com
HTTP Clone URL: https://gitlab.<redacted>.com/some-group/some-project.git
SSH Clone URL:  git@gitlab.<redacted>.com:some-group/some-project.git
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers:

GitLab Shell
Version:        14.38.0
Repository storages:
- default:      unix:/var/opt/gitlab/gitaly/gitaly.socket
GitLab Shell path:              /opt/gitlab/embedded/service/gitlab-shell

Gitaly
- default Address:      unix:/var/opt/gitlab/gitaly/gitaly.socket
- default Version:      17.3.2
- default Git Version:  2.45.2

Results of GitLab application Check

System information
System:
Current User:   git
Using RVM:      no
Ruby Version:   3.1.5p253
Gem Version:    3.5.11
Bundler Version:2.5.11
Rake Version:   13.0.6
Redis Version:  7.0.15
Sidekiq Version:7.1.6
Go Version:     unknown

GitLab information
Version:        17.3.2
Revision:       951fd632abf
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     14.11
URL:            https://gitlab.<redacted>.com
HTTP Clone URL: https://gitlab.<redacted>.com/some-group/some-project.git
SSH Clone URL:  git@gitlab.<redacted>.com:some-group/some-project.git
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers:

GitLab Shell
Version:        14.38.0
Repository storages:
- default:      unix:/var/opt/gitlab/gitaly/gitaly.socket
GitLab Shell path:              /opt/gitlab/embedded/service/gitlab-shell

Gitaly
- default Address:      unix:/var/opt/gitlab/gitaly/gitaly.socket
- default Version:      17.3.2
- default Git Version:  2.45.2
root@gitlab:/# gitlab-rake gitlab:check SANITIZE=true
Checking GitLab subtasks ...

Checking GitLab Shell ...

GitLab Shell: ... GitLab Shell version >= 14.38.0 ? ... OK (14.38.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Gitaly ...

Gitaly: ... default ... OK

Checking Gitaly ... Finished

Checking Sidekiq ...

Sidekiq: ... Running? ... yes
Number of Sidekiq processes (cluster/worker) ... 1/1

Checking Sidekiq ... Finished

Checking Incoming Email ...

Incoming Email: ... Checking Reply by email ...

IMAP server credentials are correct? ... Checking servicedesk@<redacted>.com
Checking servicedesk@<redacted>.com
yes
Mailroom enabled? ... skipped
MailRoom running? ... skipped

Checking Reply by email ... Finished


Checking Incoming Email ... Finished

Checking LDAP ...

LDAP: ... LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab App ...

Database config exists? ... yes
Tables are truncated? ... skipped
All migrations up? ...
yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Cable config exists? ... yes
Resque config exists? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Systemd unit files or init script exist? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Systemd unit files or init script up-to-date? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Projects have namespace: ...
2/1 ... yes
9/2 ... yes
10/3 ... yes
11/6 ... yes
16/8 ... yes
16/9 ... yes
16/10 ... yes
11/11 ... yes
10/12 ... yes
10/13 ... yes
11/14 ... yes
11/15 ... yes
16/16 ... yes
16/17 ... yes
16/18 ... yes
16/19 ... yes
16/20 ... yes
9/22 ... yes
49/23 ... yes
10/24 ... yes
16/25 ... yes
8/26 ... yes
16/27 ... yes
16/28 ... yes
16/29 ... yes
11/30 ... yes
10/32 ... yes
10/33 ... yes
10/34 ... yes
16/35 ... yes
10/36 ... yes
10/37 ... yes
10/38 ... yes
11/39 ... yes
16/40 ... yes
10/41 ... yes
10/42 ... yes
9/43 ... yes
11/44 ... yes
10/45 ... yes
9/46 ... yes
16/47 ... yes
16/48 ... yes
11/49 ... yes
9/50 ... yes
10/51 ... yes
10/52 ... yes
9/53 ... yes
10/54 ... yes
16/55 ... yes
10/56 ... yes
26/57 ... yes
11/58 ... yes
9/61 ... yes
9/62 ... yes
62/63 ... yes
9/64 ... yes
Redis version >= 6.2.14? ... yes
Ruby version >= 3.0.6 ? ... yes (3.1.5)
Git user has default SSH configuration? ... yes
Active users: ... 17
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes

Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

Workaround (?) We have found a workaround, while restarting the gitlab services (gitlab-ctl restart) does not do anything to resolve the issue, rebuilding the container does. One reply for an issue will go through and then subsequent replies will not. Below is an example of the docker run command.

docker stop gitlab-ce
docker rm gitlab-ce
docker run --detach \
--name=gitlab-ce \
--hostname=<redacted> \
--env=PATH=/opt/gitlab/embedded/bin:/opt/gitlab/bin:/assets:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
--env=LANG=C.UTF-8 \
--env=EDITOR=/bin/vi \
--env=TERM=xterm \
--volume=/config:/etc/gitlab:Z \
--volume=/logs:/var/log/gitlab:Z \
--volume=/data:/var/opt/gitlab:Z \
--volume=/etc/gitlab \
--volume=/var/log/gitlab \
--volume=/var/opt/gitlab \
-p 22:22 \
-p 443:443 \
-p 80:80 \
--restart=always \
--runtime=runc \
gitlab/gitlab-ce:latest