Skip to content

[SE-3470] Adjust periodic build failure email content

Boros Gábor requested to merge gabor/adjust-email-context into master

Fix the most relevant log collection and attach all the Ansible produced logs to the email. Also, make sure that the configuration is sent as an attachment instead of a plain text within the email.

Structure of the email


Subject: Deployment failed at instance: {instance_name}

Body:
The periodic deployment of {edx_platform_release} failed. Please see the details below.
 
Ansible task name: {ansible_task_name}
Relevant log lines:
{relevant_log_entry}
 
AppServer ID:                     {appserver_id}
Latest successful provision date: {latest_successful_provision_date}
Configuration source repo:        {configuration_source_repo_url}
Configuration version:            {configuration_version}
OpenEdX platform source repo:     {edx_platform_repository_url}
OpenEdX platform release:         {edx_platform_release}
OpenEdX platform commit:          {edx_platform_commit}
 
Attachments:
- build_log.txt -> This contains all the Ansible build logs
- configuration.json -> This contains the configuration we created the appserver with

Screenshots: Always include screenshots if there is any change to the UI. Can be useful for people that are not reviewer but want to know what's going on.

Sandbox URL: N/A

Testing instructions:

  1. Checkout this code on stage environment
  2. Create an instance
  3. Enable periodic builds on the instance
  4. Add your email address to the Periodic build notification emails text field
  5. Set 00:05:00 as build interval (5 minutes)
  6. Misconfigure it to no be able to provision (for example change a database password or similar to have build logs and not fail immediately)
  7. Wait for the email (check your spam folder as well) - It should arrive in 5-10 minutes based on the delay in the queue
  8. Check the email has no sensitive data and has the desired (listed above) content.
  9. Check the email has a build_log.txt attachment and it has content (with no secrets)
  10. Check the email has a configuration.json attachment and it has content (with no secrets)
  11. Fix the configuration
  12. Wait for the ack email - It should arrive in 5-10 minutes based on the delay in the queue

Author notes and concerns:

  1. Testing the flow could take from 5 minutes to 1 hour, depending on how fast the periodic task picked up by the workers.
  2. Checking the whole build log is near impossible since it may have several thousand log records

Reviewers

Merge request reports