Use minimal inline email bodies in specs instead of complex external files
Problem
When working on email ingestion I found it quite challenging to change/extend the specs because:
- Due to external fixture files they require context switching. It's not clear what's tested directly.
- External fixture files include a lot of bloat. It's hard to find the important bits for a specific spec.
- External fixture files are used in multiple places and test different scenarios/outputs.
So instead of getting a fixture file
let(:email) { fixture_file('emails/service_desk_custom_address_cc.eml') }
we could use a minimal inline example that contains exactly whats necessary in a given spec
let(:email) do
<<~EMAIL
From: from@example.com
To: to@example.com
Cc: support+project_slug-project_key@example.com
Subject: Issue title
Issue description
EMAIL
end
(Made up examples. Fixture file doesn't exist)
See this example fixture file spec/fixtures/emails/x_envelope_to_header.eml
Return-Path: <jake@example.com>
Received: from myserver.example.com ([unix socket]) by myserver (Cyrus v2.2.13-Debian-2.2.13-19+squeeze3) with LMTPA; Thu, 13 Jun 2013 17:03:50 -0400
Received: from mail.example.com (mail.example.com [IPv6:2607:f8b0:4001:c03::234]) by myserver.example.com (8.14.3/8.14.3/Debian-9.4) with ESMTP id r5DL3nFJ016967 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for <incoming+gitlabhq/gitlabhq@example.com>; Thu, 13 Jun 2013 17:03:50 -0400
Received: by myserver.example.com with SMTP id f4so21977375iea.25 for <incoming+gitlabhq/gitlabhq@appmail.example.com>; Thu, 13 Jun 2013 14:03:48 -0700
Received: by 10.0.0.1 with HTTP; Thu, 13 Jun 2013 14:03:48 -0700
From: "jake@example.com" <jake@example.com>
To: "support@example.com" <support@example.com>
Subject: Insert hilarious subject line here
Date: Tue, 26 Nov 2019 14:22:41 +0000
Message-ID: <7e2296f83dbf4de388cbf5f56f52c11f@EXDAG29-1.EXCHANGE.INT>
Accept-Language: de-DE, en-US
Content-Language: de-DE
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [62.96.54.178]
Content-Type: multipart/alternative;
boundary="_000_7e2296f83dbf4de388cbf5f56f52c11fEXDAG291EXCHANGEINT_"
MIME-Version: 1.0
X-Envelope-To: incoming+gitlabhq/gitlabhq+auth_token@appmail.example.com
--_000_7e2296f83dbf4de388cbf5f56f52c11fEXDAG291EXCHANGEINT_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
--_000_7e2296f83dbf4de388cbf5f56f52c11fEXDAG291EXCHANGEINT_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Look, a message with some alternate headers! We should really support them.
Why?
- No context switching.
- Minimal setup. Still sufficient to test the use case. Email clients just add bloat. If you test headers, you don't need multipart HTML and plain emails. You only need headers.
- Easy to validate locally. Just copy, and replace emails with the current setup and you're ready to go.
- Adds less code to the codebase, because we use a minimal setup.
- Email ingestion specs are quite old and contain lots of user-defined email addresses which could be replaced with
@example.com
emails etc.
Why not?
-
Makes spec files longer. Yes, but you know what the spec is about at first glance
👀 . If you add the fixture files then it's probably the same length or even less. - Some setups are similar or nearly identical. Yes, I think we can be creative with that and use variables, etc.
Examples
My MR Reads mail-key from CC in general email receiver (!129700 - merged) (diff) adds support for the CC
header and uses this format for the specs.
Proposal
Let's use minimal inline email bodies for new email ingestion specs. If you feel like it, refactor existing specs to that more explicit approach.
Affected files
Click to expand
70 results - 16 files
ee/spec/lib/gitlab/email/handler/create_note_handler_spec.rb:
14
15: let(:email_raw) { fixture_file('emails/valid_reply.eml') }
16 let(:group) { create(:group) }
36 context 'when the note contains quick actions' do
37: let!(:email_raw) { fixture_file("emails/commands_in_reply.eml") }
38
60 context "when the reply is blank" do
61: let!(:email_raw) { fixture_file("emails/no_content_reply.eml") }
62
118 context 'mail key is in the References header' do
119: let(:email_raw) { fixture_file('emails/reply_without_subaddressing_and_key_inside_references.eml') }
120
124 context 'mail key is in the References header with a comma' do
125: let(:email_raw) { fixture_file('emails/reply_without_subaddressing_and_key_inside_references_with_a_comma.eml') }
126
spec/initializers/mail_encoding_patch_spec.rb:
224 context 'frozen email boy content with unsual body encoding' do
225: let(:content) { fixture_file("emails/ios_default.eml") }
226
spec/lib/gitlab/email/attachment_uploader_spec.rb:
7 let(:project) { create(:project) }
8: let(:message_raw) { fixture_file("emails/attachment.eml") }
9 let(:message) { Mail::Message.new(message_raw) }
40 context 'with a signed message' do
41: let(:message_raw) { fixture_file("emails/valid_reply_signed_smime.eml") }
42
57 context 'with a signed message with mixed protocol prefix' do
58: let(:message_raw) { fixture_file("emails/valid_reply_signed_smime_mixed_protocol_prefix.eml") }
59
74 context 'with a message with no content type' do
75: let(:message_raw) { fixture_file("emails/no_content_type.eml") }
76
spec/lib/gitlab/email/failure_handler_spec.rb:
5 RSpec.describe Gitlab::Email::FailureHandler do
6: let(:raw_message) { fixture_file('emails/valid_reply.eml') }
7 let(:receiver) { Gitlab::Email::Receiver.new(raw_message) }
spec/lib/gitlab/email/receiver_spec.rb:
64 context 'when in a Delivered-To header' do
65: let(:email_raw) { fixture_file('emails/forwarded_new_issue.eml') }
66 let(:meta_key) { :delivered_to }
72 context 'when in an Envelope-To header' do
73: let(:email_raw) { fixture_file('emails/envelope_to_header.eml') }
74 let(:meta_key) { :envelope_to }
80 context 'when in an X-Envelope-To header' do
81: let(:email_raw) { fixture_file('emails/x_envelope_to_header.eml') }
82 let(:meta_key) { :x_envelope_to }
88 context 'when enclosed with angle brackets in an Envelope-To header' do
89: let(:email_raw) { fixture_file('emails/envelope_to_header_with_angle_brackets.eml') }
90 let(:meta_key) { :envelope_to }
96 context 'when mail key is in the references header with a comma' do
97: let(:email_raw) { fixture_file('emails/valid_reply_with_references_in_comma.eml') }
98 let(:meta_key) { :references }
106 context 'when all other headers are missing' do
107: let(:email_raw) { fixture_file('emails/missing_delivered_to_header.eml') }
108 let(:meta_key) { :received_recipients }
192 context 'when we cannot find a capable handler' do
193: let(:email_raw) { fixture_file('emails/valid_reply.eml').gsub(mail_key, '!!!') }
194 let(:expected_error) { Gitlab::Email::UnknownIncomingEmail }
206 context 'when the email was auto generated with Auto-Submitted header' do
207: let(:email_raw) { fixture_file('emails/auto_submitted.eml') }
208 let(:expected_error) { Gitlab::Email::AutoGeneratedEmailError }
256 context 'when the email was auto generated with X-Autoreply header' do
257: let(:email_raw) { fixture_file('emails/auto_reply.eml') }
258 let(:expected_error) { Gitlab::Email::AutoGeneratedEmailError }
spec/lib/gitlab/email/reply_parser_spec.rb:
20 it "returns an empty string if there is no reply content" do
21: expect(test_parse_body(fixture_file("emails/no_content_reply.eml"))).to eq("")
22 end
25 it "returns quoted text from email" do
26: text = test_parse_body(fixture_file("emails/no_content_reply.eml"), allow_only_quotes: true)
27
48 it "properly renders plaintext-only email" do
49: expect(test_parse_body(fixture_file("emails/plaintext_only.eml")))
50 .to eq(
66 it "properly renders html-only email with table and blockquote" do
67: expect(test_parse_body(fixture_file("emails/html_table_and_blockquote.eml")))
68 .to eq(
78 it "supports a Dutch reply" do
79: expect(test_parse_body(fixture_file("emails/dutch.eml"))).to eq("Dit is een antwoord in het Nederlands.")
80 end
82 it "removes an 'on date wrote' quoting line" do
83: expect(test_parse_body(fixture_file("emails/on_wrote.eml"))).to eq("Sure, all you need to do is frobnicate the foobar and you'll be all set!")
84 end
86 it "handles multiple paragraphs" do
87: expect(test_parse_body(fixture_file("emails/paragraphs.eml")))
88 .to eq(
101 it "handles multiple paragraphs when parsing html" do
102: expect(test_parse_body(fixture_file("emails/html_paragraphs.eml")))
103 .to eq(
114 it "handles newlines" do
115: expect(test_parse_body(fixture_file("emails/newlines.eml")))
116 .to eq(
125 it "handles inline reply" do
126: expect(test_parse_body(fixture_file("emails/inline_reply.eml")))
127 .to eq(
172 it "properly renders email reply from gmail web client" do
173: expect(test_parse_body(fixture_file("emails/gmail_web.eml")))
174 .to eq(
192 it do
193: expect(test_parse_body(fixture_file("emails/html_only.eml")))
194 .to eq(
224 it "properly renders email reply from iOS default mail client" do
225: expect(test_parse_body(fixture_file("emails/ios_default.eml")))
226 .to eq(
239 it "properly renders email reply from Android 5 gmail client" do
240: expect(test_parse_body(fixture_file("emails/android_gmail.eml")))
241 .to eq(
257 it "properly renders email reply from Windows 8.1 Metro default mail client" do
258: expect(test_parse_body(fixture_file("emails/windows_8_metro.eml")))
259 .to eq(
275 it "properly renders email reply from MS Outlook client" do
276: expect(test_parse_body(fixture_file("emails/outlook.eml"))).to eq("Microsoft Outlook 2010")
277 end
279 it "properly renders html-only email from MS Outlook" do
280: expect(test_parse_body(fixture_file("emails/outlook_html.eml"))).to eq("Microsoft Outlook 2010")
281 end
283 it "does not wrap links with no href in unnecessary brackets" do
284: expect(test_parse_body(fixture_file("emails/html_empty_link.eml"))).to eq("no brackets!")
285 end
287 it "does not trim reply if trim_reply option is false" do
288: expect(test_parse_body(fixture_file("emails/valid_new_issue_with_quote.eml"), { trim_reply: false }))
289 .to eq(
312
313: expect(test_parse_body(fixture_file("emails/valid_new_issue_with_quote.eml"), { append_reply: true }))
314 .to contain_exactly(body, reply)
spec/lib/gitlab/email/service_desk_receiver_spec.rb:
5 RSpec.describe Gitlab::Email::ServiceDeskReceiver do
6: let(:email) { fixture_file('emails/service_desk_custom_address.eml') }
7 let(:receiver) { described_class.new(email) }
34 context 'when in a Delivered-To header' do
35: let(:email) { fixture_file('emails/service_desk_custom_address_reply.eml') }
36
40 context 'when in a Envelope-To header' do
41: let(:email) { fixture_file('emails/service_desk_custom_address_envelope_to.eml') }
42
46 context 'when in a X-Envelope-To header' do
47: let(:email) { fixture_file('emails/service_desk_custom_address_x_envelope_to.eml') }
48
69 context 'when the email contains no key in the To header and contains reference header with no key' do
70: let(:email) { fixture_file('emails/service_desk_reference_headers.eml') }
71
spec/lib/gitlab/email/handler/create_issue_handler_spec.rb:
70 context "creates a new issue with legacy email address" do
71: let(:email_raw) { fixture_file('emails/valid_new_issue_legacy.eml') }
72
spec/lib/gitlab/email/handler/create_merge_request_handler_spec.rb:
88 context "creates a new merge request with legacy email address" do
89: let(:email_raw) { fixture_file('emails/valid_new_merge_request_legacy.eml') }
90
spec/lib/gitlab/email/handler/create_note_handler_spec.rb:
11 let(:note) { create(:diff_note_on_merge_request, project: project) }
12 let(:note) { create(:diff_note_on_merge_request, project: project) }
13: let(:email_raw) { fixture_file('emails/valid_reply.eml') }
13 let!(:sent_notification) do
14 let!(:sent_notification) do
26
27
28: let(:update_commands_only) { fixture_file('emails/update_commands_only_reply.eml') }
29: let(:no_content) { fixture_file('emails/no_content_reply.eml') }
30: let(:commands_in_reply) { fixture_file('emails/commands_in_reply.eml') }
31: let(:with_quick_actions) { fixture_file('emails/valid_reply_with_quick_actions.eml') }
31 end
32 end
33 context 'when the recipient address does not include a mail key' do
34 context 'when the recipient address does not include a mail key' do
35: let(:email_raw) { fixture_file('emails/valid_reply.eml').gsub(mail_key, '') }
35
36
56 let(:verified_email) { 'alan@adventuretime.ooo' }
57 let(:verified_email) { 'alan@adventuretime.ooo' }
58: let(:email_raw) { fixture_file('emails/valid_reply.eml').gsub('jake@adventuretime.ooo', verified_email) }
58
59
78 context 'when no sent notification for the mail key could be found' do
79 context 'when no sent notification for the mail key could be found' do
80: let(:email_raw) { fixture_file('emails/wrong_mail_key.eml') }
80
81
135 context 'when mail key is in the References header' do
136 context 'when mail key is in the References header' do
137: let(:email_raw) { fixture_file('emails/reply_without_subaddressing_and_key_inside_references.eml') }
137
138
142 let(:email_raw) do
143 let(:email_raw) do
144: fixture_file('emails/reply_without_subaddressing_and_key_inside_references_with_a_comma.eml')
144 end
145 end
153 context 'when email contains quoted text only' do
154 context 'when email contains quoted text only' do
155: let(:email_raw) { fixture_file('emails/no_content_with_quote.eml') }
155
156
161 context 'when email contains quoted text and quick commands only' do
162 context 'when email contains quoted text and quick commands only' do
163: let(:email_raw) { fixture_file('emails/commands_only_reply.eml') }
163
164
175 context 'when email contains text, quoted text and quick commands' do
176 context 'when email contains text, quoted text and quick commands' do
177: let(:email_raw) { fixture_file('emails/commands_in_reply.eml') }
177
178
192 context 'when email contains text, quoted text and quick commands' do
193 context 'when email contains text, quoted text and quick commands' do
194: let(:email_raw) { fixture_file('emails/commands_in_reply.eml') }
194
195
216 context 'when email contains text, quoted text and quick commands' do
217 context 'when email contains text, quoted text and quick commands' do
218: let(:email_raw) { fixture_file('emails/commands_in_reply.eml') }
218
219
spec/lib/gitlab/email/handler/create_note_on_issuable_handler_spec.rb:
32 let(:mail_key) { 'gitlabhq-gitlabhq-project_id-auth_token-issue-issue_iid' }
33: let(:email_raw) { fixture_file('emails/valid_note_on_issuable.eml').gsub(mail_key, '') }
34
spec/lib/gitlab/email/handler/service_desk_handler_spec.rb:
168 context 'with legacy incoming email address' do
169: let(:email_raw) { fixture_file('emails/service_desk_legacy.eml') }
170
spec/lib/gitlab/email/handler/unsubscribe_handler_spec.rb:
12
13: let(:email_raw) { fixture_file('emails/valid_reply.eml').gsub(mail_key, "#{mail_key}#{Gitlab::Email::Common::UNSUBSCRIBE_SUFFIX}") }
14 let(:project) { create(:project, :public) }
66 context 'when using old style unsubscribe link' do
67: let(:email_raw) { fixture_file('emails/valid_reply.eml').gsub(mail_key, "#{mail_key}#{Gitlab::Email::Common::UNSUBSCRIBE_SUFFIX_LEGACY}") }
68
85 context 'when no sent notification for the mail key could be found' do
86: let(:email_raw) { fixture_file('emails/wrong_mail_key.eml') }
87
spec/requests/api/internal/mail_room_spec.rb:
35
36: let(:email_content) { fixture_file("emails/service_desk_reply.eml") }
37
spec/workers/email_receiver_worker_spec.rb:
5 RSpec.describe EmailReceiverWorker, :mailer, feature_category: :team_planning do
6: let(:raw_message) { fixture_file('emails/valid_reply.eml') }
7
spec/workers/service_desk_email_receiver_worker_spec.rb:
7 let(:worker) { described_class.new }
8: let(:email) { fixture_file('emails/service_desk_custom_address.eml') }
9
Edited by Marc Saleiko