Gitlab tries to send E-Mail notifications via IPv6 when IPv6 is unavailable

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

We have been experiencing sporadic E-mail failures, where notifications were sometimes never sent, seemingly at random.

Looking at the sidekiq/current log, the issue is apparently that Ruby's net-smtp will sometimes try to connect to an SMTP server via IPv6 if it finds an AAAA record. When the attempt fails because IPv6 is disabled via net.ipv6.conf.all.disable_ipv6 = 1, it does not retry over IPv4.

Start of the backtrace:

...
  "exception.class": "Errno::EADDRNOTAVAIL",                                    
  "exception.message": "Cannot assign requested address - connect(2) for [IPv6 address censored]:465",                                                            
  "exception.backtrace": [                                                      
    "/opt/gitlab/embedded/lib/ruby/3.2.0/socket.rb:1217:in `__connect_nonblock'"
,                                                                               
    "/opt/gitlab/embedded/lib/ruby/3.2.0/socket.rb:1217:in `connect_nonblock'", 
    "/opt/gitlab/embedded/lib/ruby/3.2.0/socket.rb:60:in `connect_internal'",   
    "/opt/gitlab/embedded/lib/ruby/3.2.0/socket.rb:141:in `connect'",           
    "/opt/gitlab/embedded/lib/ruby/3.2.0/socket.rb:645:in `block in tcp'",      
    "/opt/gitlab/embedded/lib/ruby/3.2.0/socket.rb:231:in `each'",              
    "/opt/gitlab/embedded/lib/ruby/3.2.0/socket.rb:231:in `foreach'",           
    "/opt/gitlab/embedded/lib/ruby/3.2.0/socket.rb:635:in `tcp'",               
    "net-smtp (0.3.3) lib/net/smtp.rb:643:in `tcp_socket'",                     
    "net-smtp (0.3.3) lib/net/smtp.rb:656:in `do_start'",                       
    "net-smtp (0.3.3) lib/net/smtp.rb:611:in `start'",                          
    "config/initializers/mail_starttls_patch.rb:53:in `start_smtp_session'",
    "mail (2.8.1) lib/mail/network/delivery_methods/smtp.rb:100:in `deliver!'",
    "mail (2.8.1) lib/mail/message.rb:2145:in `do_delivery'",
    "mail (2.8.1) lib/mail/message.rb:253:in `block in deliver'",
    "actionmailer (7.1.5.1) lib/action_mailer/base.rb:600:in `block in deliver_m
ail'",
...
Edited by 🤖 GitLab Bot 🤖