Problems sending email in gitlab

I have gitlab installed via the docker image. I have configured email via:

gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "mail-server.com"
gitlab_rails['smtp_port'] = 25
gitlab_rails['smtp_enable_starttls_auto'] = false
gitlab_rails['smtp_tls'] = false

My mail server is setup to talk STARTTLS on port 25 and SSL on port 465 using a certificate with a non-standard CA.

I am able to send a test email from gitlab-rails console:
irb(main):003:0> Notify.test_email('destination_email@address.com', 'Message Subject', 'Message Body').deliver_now

However when I change an issue I get this in my sidekick log.

2017-06-28_15:49:40.88571 2017-06-28T15:49:40.885Z 1841 TID-osxz2klrk WARN: {"context":"Job raised except
ion","job":{"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"ActionMailer::Deliv
eryJob","queue":"mailers","args":[{"job_class":"ActionMailer::DeliveryJob","job_id":"e6bfeb87-c876-463d-9
8d5-23da3077e561","queue_name":"mailers","arguments":["Notify","note_issue_email","deliver_now",2,18],"lo
cale":"en"}],"retry":3,"jid":"d389f4a5d55204ce1f64be21","created_at":1498664747.2179124,"enqueued_at":149
8664980.716833,"error_message":"SSL_connect returned=1 errno=0 state=error: certificate verify failed","e
rror_class":"OpenSSL::SSL::SSLError","failed_at":1498664747.4612129,"retry_count":3,"retried_at":14986649
80.8823452},"jobstr":"{\"class\":\"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper\",\"wrapped\":\"A
ctionMailer::DeliveryJob\",\"queue\":\"mailers\",\"args\":[{\"job_class\":\"ActionMailer::DeliveryJob\",\
"job_id\":\"e6bfeb87-c876-463d-98d5-23da3077e561\",\"queue_name\":\"mailers\",\"arguments\":[\"Notify\",\
"note_issue_email\",\"deliver_now\",2,18],\"locale\":\"en\"}],\"retry\":3,\"jid\":\"d389f4a5d55204ce1f64b
e21\",\"created_at\":1498664747.2179124,\"enqueued_at\":1498664980.716833,\"error_message\":\"SSL_connect
 returned=1 errno=0 state=error: certificate verify failed\",\"error_class\":\"OpenSSL::SSL::SSLError\",\
"failed_at\":1498664747.4612129,\"retry_count\":2,\"retried_at\":1498664863.512033}"}
2017-06-28_15:49:40.88628 2017-06-28T15:49:40.886Z 1841 TID-osxz2klrk WARN: OpenSSL::SSL::SSLError: SSL_c
onnect returned=1 errno=0 state=error: certificate verify failed
2017-06-28_15:49:40.88675 2017-06-28T15:49:40.886Z 1841 TID-osxz2klrk WARN: /opt/gitlab/embedded/lib/ruby
/2.3.0/net/http.rb:933:in `connect_nonblock'

And no email is received and my mail server doesn't show any entries for a connection. I also don't see network connection when watching tcpdump.

I have tried adding to trusted certificates and modifying smtp_ca_file and still can't get it to work.

I would be fine with the docker container using the docker host through a loopback interface, but I don't see how to configure that either.

This seems similar to #1926 (closed), but that issue isn't seeing the SSL error, however it has the same behavior.

What other information can I gather to debug this issue?