Skip to content

Emails redirected to GitLab are dropped if key is in "Received" header

Summary

Email messages redirected to GitLab may not be parsed properly since the token does not end up in the Delivered-to or Envelope-To fields. The token is in the Received: header though it is mixed with other text making it a bit more difficult to find.

Steps to reproduce

Forward an email from incoming-token@gitlab.server.com to incoming@gitlab.mailbox.com (due to the lack of + notation for some email providers).

What is the current bug behavior?

Email is dropped.

What is the expected correct behavior?

Email is processed.

Sample email headers

Received: from exchhy01.example.com (10.0.12.157) by exchhy02.example.com
 (10.0.12.158) with Microsoft SMTP Server (TLS) id 15.0.1395.4 via Mailbox
 Transport; Wed, 22 Jan 2020 15:56:46 -0500
Received: from exchhy01.example.com (10.0.12.157) by exchhy01.example.com
 (10.0.12.157) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Wed, 22 Jan
 2020 15:56:44 -0500
Received: from emgwy2.example.com (160.91.254.10) by exchhy01.example.com
 (10.0.12.157) with Microsoft SMTP Server (TLS) id 15.0.1395.4 via Frontend
 Transport; Wed, 22 Jan 2020 15:56:44 -0500
Received: from emgwyext1.example.com (emgwyext1.example.com [10.0.177.164])
	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by emgwy2.example.com (Postfix) with ESMTPS id 482yPm32dlz2SmtH
	for <gitlab+test-group-test-project-29-issue-@example.com>; Wed, 22 Jan 2020 15:56:44 -0500 (EST)
Received: from ipss1-snip.is.centurylink.net (ipss1-snip.is.centurylink.net [65.153.203.170])
	by emgwyext1.example.com (Postfix) with ESMTP id 482yPm2QhYzlqymC
	for <gitlab+test-group-test-project-29-issue-@example.com>; Wed, 22 Jan 2020 15:56:44 -0500 (EST)
Received: from fireeye01.example.com (unknown [10.0.177.140])
	by ipss1-snip.is.centurylink.net (Postfix) with ESMTP id 0FA0F121C0002
	for <gitlab+test-group-test-project-29-issue-@example.com>; Wed, 22 Jan 2020 20:56:44 +0000 (GMT)
Received: from localhost.localdomain (localhost [127.0.0.1])
	by fireeye01.example.com (Postfix) with SMTP id 482yPm03z1z1ZwW9
	for <gitlab+test-group-test-project-29-issue-@example.com>; Wed, 22 Jan 2020 15:56:44 -0500 (EST)
Received: from mta02.example.com (mta02.example.com [10.0.177.136])
	by fireeye01.example.com (Postfix) with ESMTPS id 482yPh2JYTz1ZwZC
	for <gitlab+test-group-test-project-29-issue-@example.com>; Wed, 22 Jan 2020 15:56:40 -0500 (EST)
Received-SPF: Pass (mta02.example.com: domain of username@source.from
  designates 10.2.40.134 as permitted sender)
  identity=mailfrom; client-ip=10.2.40.134;
  receiver=mta02.example.com; envelope-from="username@source.from";
  x-sender="username@source.from"; x-conformance=spf_only;
  x-record-type="v=spf1"; x-record-text="v=spf1
  ip4:10.1.106.36 ip4:10.1.106.40 ip4:10.2.40.0/24 ~all"
Received-SPF: Pass (mta02.example.com: domain of
  postmaster@mail-40134.source.from designates 10.2.40.134
  as permitted sender) identity=helo; client-ip=10.2.40.134;
  receiver=mta02.example.com; envelope-from="username@source.from";
  x-sender="postmaster@mail-40134.source.from";
  x-conformance=spf_only; x-record-type="v=spf1";
  x-record-text="v=spf1 ip4:10.2.40.134 ~all"
Authentication-Results: mta02.example.com; spf=Pass smtp.mailfrom=username@source.from; spf=Pass smtp.helo=postmaster@mail-40134.source.from; dkim=pass (signature verified) header.i=@source.from; dmarc=pass (p=reject dis=none) d=source.from
IronPort-SDR: [SNIP]
Subject: [EXTERNAL] Test 123
IronPort-PHdr: [SNIP]
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: [SNIP]
X-IPAS-Result: [SNIP]
X-IronPort-AV: E=Sophos;i="5.60,350,1274139600"; 
   d="scan'208";a="84585154"
X-IronPort-Outbreak-Status: No, level 0, Unknown - Unknown
Received: from mail-4.source.from ([10.2.40.134])
  by mta02.example.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jan 2020 15:56:38 -0500
Date: Wed, 22 Jan 2020 20:56:30 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=source.from;
	s=sourcefrom; t=1579726596;
	bh=k13fyF4mZv46lYZajzIh32Lx2ZqWZxXea8NBq6qTJnk=;
	h=Date:To:From:Reply-To:Subject:Feedback-ID:From;
	b=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa=
To: servicedesktest-pe <servicedesktest-pe@example.com>
X-FireEye: Clean
From: username Lname <username@source.from>
Reply-To: username Lname <username@source.from>
Message-ID: <[SNIP]@source.from>
Feedback-ID: [SNIP]:Ext:SourceFrom
Content-Type: multipart/alternative;
	boundary="b1_e41f9d3df8d3f1e188b18ad09e279a77"
X-Spam-Status: No, score=0.8 required=7.0 tests=ALL_TRUSTED,BAYES_40,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,HTML_MESSAGE,
	PDS_TONAME_EQ_TOLOCAL_SHORT shortcircuit=no autolearn=no
	autolearn_force=no version=3.4.2
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on sourcefrom
X-Qwest-Proxy-Host: snip.centurylink.net
X-Qwest-Process-Uuid: [SNIP]
X-Qwest-Status: aaaaaaaaaaaaaaaa
Return-Path: username@source.from
X-MS-Exchange-Organization-Network-Message-Id: [SNIP]
X-MS-Exchange-Organization-AVStamp-Enterprise: 1.0
X-MS-Exchange-Organization-AuthSource: exchhy01.example.com
X-MS-Exchange-Organization-AuthAs: Anonymous
MIME-Version: 1.0

Possible fixes

For test case, add above snippet to gitlab / spec / fixtures / emails / received_for_header.eml

     def key_from_additional_headers(mail)
        find_key_from_references(mail) ||
          find_key_from_delivered_to_header(mail) ||
          find_key_from_envelope_to_header(mail) || 
          find_key_from_received_header(mail)
      end

and

      def find_key_from_received_header(mail)
        Array(mail[:received]).find do |header|
          key = Gitlab::IncomingEmail.key_from_address(header.value)
          break key if key
        end
      end

I don't know how the regex matching works, but if Envelope-To has more characters than just the email address it appears to fail the match. Received header has a lot of characters in it and there are many received headers to go through. This loop structure appears to find the first and then break as soon as it extracts a key from any of them.

Since the received format in the example contains the word for and then a semicolon, string breaking can be used to isolate that segment. It would reduce the usefulness of the parsing however.