Skip to content

OC-1541: Keep the old instance running while deploying a new one

Boros Gábor requested to merge provision-while-running into master

Created by: bradenmacdonald

Changes:

  • Fix occasional pylint warning about undocumented Meta classes
  • Combine LogEntry classes
    • ^ These two commits have been copied to #67 so they can be reviewed more easily and merged sooner
  • Separated [OpenEdX]Instance into Instance (long-lived, mutable, creates AppServers and marks one as active), and AppServer (immutable, owns one VM)
  • Factored Open edX specific code into OpenEdXInstance and OpenEdXAppServer respectively
  • Factored Open edX specific code out of the ansible mixin
  • Change the ansible mixin so it can optionally run multiple playbooks
  • Removed instance.last_provisioning_started since you can compute that from the most recent AppServer.created field
  • Combined forum_version, notifier_version, xqueue_version, and certs_version into a single openedx_release setting, since these were always set to the same value, and if one needs to be set differently it can be done using ansible_extra_settings anyways.
  • Separated PR-watching code to a separate app (OpenEdXInstance should not need to know about pull requests)
  • Updated API
  • Allows provisioning a new AppServer at any time
  • Updated UI accordingly
    • Lovely new home screen ("empty state"): screen shot 2016-05-08 at 7 54 01 pm
    • Instance list indicates the state of the active app server for each instance, as well as the state of any newer app server waiting to be provisioned / reviewed / made active. screen shot 2016-05-08 at 7 54 57 pm
    • New "Pull Request" tab: screen shot 2016-05-08 at 7 56 40 pm
    • When you click "Update Instance Settings from PR", it displays an animation while fetching updates from GitHub, then switches to the "Settings" tab to show you the new settings / commit hash.
    • New Settings tab for instance settings (should be made editable in the future) screen shot 2016-05-08 at 7 58 03 pm
    • New App Servers tab to review app servers, launch new ones, or change which one is active: screen shot 2016-05-08 at 7 59 37 pm
    • Deep linking to any instance or app server :)
  • Behavior: The first time a PR is seen, an instance will be auto-provisioned; on failure, the provisioning will be retried (on a new AppServer) up to 1 additional time; on success, the AppServer will automatically be marked active. On the other hand, whenever the user manually provisions a new instance or AppServer, there is no automatic retry or activation. (To give users time to inspect the errors or the new appserver.)

TODO:

  • Update Jasmine tests

Deployment notes:

  • Run this SQL before deploying to create a backup of the instance_singlevmopenedxinstance data.

Suggestions for future work:

  • Add tests for # of queries used by logging, and by each API view. Add in select_related() calls where needed to optimize
  • Allow stopping an AppServer build
  • Implement the WatchedPullRequest TODO items listed in the code

Merge request reports