Create project from instance template while Jira integration is active causes "Error importing repository" (import succeeds)

Summary

Stems from customer issue in ticket 140150 (internal)

Creating a new project using an instance level template, while that instance template has Jira integration active, will create the new project successfully but will generate an error that is emailed to the admin:

["Error importing repository  into [FILTERED] - Validation failed: Url must be a valid URL, Url can't be blank, Username can't be blank, Password can't be blank"]

Testing with 12.5.2-ee (c1b3929bc67)

Steps to reproduce

Create an instance-level template project, configure Jira integration under settings > integrations > project services. You can use dummy data for testing. make sure the integration is set to active.

Create a new project using that instance template. The new project is created successfully, and for all intents and purposes, there are no issues. However, the following is generated in production log

Started GET "/root/from-template-using-jira-service/-/import" for x.x.x.x at 2019-12-03 14:56:05 -0500
Processing by Projects::ImportsController#show as HTML
  Parameters: {"namespace_id"=>"root", "project_id"=>"from-template-using-jira-service"}
Completed 200 OK in 181ms (Views: 120.9ms | ActiveRecord: 22.1ms | Elasticsearch: 0.0ms)
Import/Export - Project public-templates with ID: 32 export error - Error importing repository  into [FILTERED] - Validation failed: Url must be a valid URL, Url can't be blank, Username can't be blank, Password can't be blank
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 7b60315c-d589-40e1-8f79-526a1843480a) to Sidekiq(mailers) with arguments: "Notify", "project_was_not_exported_email", "deliver_now", #<GlobalID:0x00007f8e50d8bf40 @uri=#<URI::GID gid://gitlab/User/1>>, #<GlobalID:0x00007f8e50e16320 @uri=#<URI::GID gid://gitlab/Project/32>>, ["Error importing repository  into [FILTERED] - Validation failed: Url must be a valid URL, Url can't be blank, Username can't be blank, Password can't be blank"]
[ActiveJob] [ActionMailer::DeliveryJob] [7b60315c-d589-40e1-8f79-526a1843480a] Performing ActionMailer::DeliveryJob (Job ID: 7b60315c-d589-40e1-8f79-526a1843480a) from Sidekiq(mailers) with arguments: "Notify", "project_was_not_exported_email", "deliver_now", #<GlobalID:0x00007f8e536ad270 @uri=#<URI::GID gid://gitlab/User/1>>, #<GlobalID:0x00007f8e536be9f8 @uri=#<URI::GID gid://gitlab/Project/32>>, ["Error importing repository  into [FILTERED] - Validation failed: Url must be a valid URL, Url can't be blank, Username can't be blank, Password can't be blank"]
[ActiveJob] [ActionMailer::DeliveryJob] [7b60315c-d589-40e1-8f79-526a1843480a] Sent mail to admin@gitlabserver (29.8ms)
[ActiveJob] [ActionMailer::DeliveryJob] [7b60315c-d589-40e1-8f79-526a1843480a] Performed ActionMailer::DeliveryJob (Job ID: 7b60315c-d589-40e1-8f79-526a1843480a) from Sidekiq(mailers) in 62.73ms

What is the current bug behavior?

import/export controller generates an error due to failed validation of project.json, and ActionMailer sends an email to the admin.

What is the expected correct behavior?

No email is sent to user.

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:		Ubuntu 18.04
Proxy:		no
Current User:	git
Using RVM:	no
Ruby Version:	2.6.3p62
Gem Version:	2.7.9
Bundler Version:1.17.3
Rake Version:	12.3.3
Redis Version:	3.2.12
Git Version:	2.22.0
Sidekiq Version:5.2.7
Go Version:	unknown


GitLab information
Version:	12.5.2-ee
Revision:	c1b3929bc67
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	10.9
Elasticsearch:	yes
Geo:		yes
Geo node:	Primary
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers:


GitLab Shell
Version:	10.2.0
Repository storage paths:
  • default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
Checking GitLab subtasks ...


Checking GitLab Shell ...


GitLab Shell: ... GitLab Shell version >= 10.2.0 ? ... OK (10.2.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Gitaly ...


Gitaly: ... default ... OK


Checking Gitaly ... Finished


Checking Sidekiq ...


Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 3
Try fixing it:
sudo service gitlab stop
sudo pkill -u git -f sidekiq
sleep 10 && sudo pkill -9 -u git -f sidekiq
sudo service gitlab start
Please fix the error above and rerun the checks.


Checking Sidekiq ... Finished


Checking Incoming Email ...


Incoming Email: ... Reply by email is disabled in config/gitlab.yml


Checking Incoming Email ... Finished


Checking LDAP ...


LDAP: ... LDAP is disabled in config/gitlab.yml


Checking LDAP ... Finished


Checking GitLab App ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
1/1 ... yes
1/3 ... yes
1/5 ... yes
1/6 ... yes
1/7 ... yes
1/8 ... yes
1/9 ... yes
1/11 ... yes
11/12 ... yes
11/14 ... yes
1/15 ... yes
1/17 ... yes
1/20 ... yes
1/21 ... yes
1/22 ... yes
1/23 ... yes
10/24 ... yes
11/25 ... yes
6/26 ... yes
1/27 ... yes
2/31 ... yes
11/32 ... yes
1/33 ... yes
1/34 ... yes
1/41 ... yes
1/46 ... yes
1/47 ... yes
1/49 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.3)
Git version >= 2.22.0 ? ... yes (2.22.0)
Git user has default SSH configuration? ... yes
Active users: ... 4
Is authorized keys file accessible? ... skipped (authorized keys not enabled)
Elasticsearch version 5.6 - 6.x? ... yes (6.8.2)


Checking GitLab App ... Finished


Checking Geo ...


GitLab Geo is available ... yes
GitLab Geo is enabled ... yes
HTTP/HTTPS repository cloning is enabled ... yes
Machine clock is synchronized ... yes
Git user has default SSH configuration? ... yes
OpenSSH configured to use AuthorizedKeysCommand ... yes
GitLab configured to disable writing to authorized_keys file ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... no
Try fixing it:
Please migrate all projects to hashed storage
to avoid security issues and ensure data integrity.
For more information see:
doc/administration/repository_storage_types.md


Checking Geo ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

Exporting a project generates project.json which includes a hash of the configured services. This hash is built without including url, username, and password (for obvious reasons). Presumably, during import, this is where the failure begins.

Workaround

Set the Jira integration in the instance template project to inactive and then use a service template instead.