[Feature flag] Enables Zuora Billing Account based architecture

Summary

This issue is to rollout Zuora Billing Account as a Customer Account on CustomersDot production, that is currently behind the account_data_structure feature flag.

Owners

Stakeholders

The Rollout Plan

Percentage Rollout on CustomersDot

Expectations

What are we expecting to happen?

For all self served orders

First purchase

  • The creation of an Account object in CustomersDot as a counterpart to the Zuora Billing Account that will be created in Zuora.
  • The creation of an AccountMembership record in CustomersDot that will associate the Customer that made the purchase to the created Account.
  • The SFDC associated account id information will be stored at the created Account level.

Subsequent orders for the same Account

Will reuse the Account created in the first step. Future orders might be initiated by the same or another Customer. A Customer initiating an order will be already associated to the account via an AccountMembership record to be able to place the order so the creation of this membership is not mentioned here.

For sales assisted orders will result in:
  • The creation of a Customer based on the provided information that will be reused as the Bill To / Sold To contact at the Zuora Account Level.
  • The creation of an Account object in CustomersDot as a counterpart to the Zuora Billing Account.
  • The creation of an AccountMembership based on the before mentioned Account and Customer.

Subsequent orders for the same Account

Will reuse the Account created in the first step. To be reviewed: If the new order is associated to a different contact (different email) then we might create a new Customer + Account Membership in CustomersDot.

What might happen if this goes wrong?

  • Purchase flow will not work within CustomersDot.
  • Provision for sales assisted orders will not work.

What can we monitor to detect problems with this?

  • Keep an eye on Sentry.
  • Check #s_fulfillment_daily for errors.
  • Verify provisions are being fulfilled within the Provision Tracking System (Admin > Provisions)

Rollout Steps

Rollout on non-production environments

  • Ensure that the feature MRs have been deployed to non-production environments, i.e. customers-staging.
  • Ensure that the rake task to backfill Account and AccountMembership (introduced in #4798 (closed)) has been run and verified. This data must be populated before we enable the feature flag in the environment.
    • RAILS_ENV=stg ~/.rbenv/shims/bundle exec rake data_maintenance:billing_accounts:backfill_with_customers[false]
    • RAILS_ENV=stg ~/.rbenv/shims/bundle exec rake data_maintenance:billing_accounts:backfill_cloud_activations[false]
    • RAILS_ENV=stg ~/.rbenv/shims/bundle exec rake data_maintenance:billing_accounts:backfill_licenses[false]
    • RAILS_ENV=stg ~/.rbenv/shims/bundle exec rake data_maintenance:billing_accounts:backfill_orders[false]
  • Consider announcing that the flag will be enabled to potentially impacted groups, e.g. #s_fulfillment_engineering
  • Enable the feature globally on non-production environments by applying the appropriate environment, e.g. customers-staging and customers-development.
  • Notify the Software Engineers in Test (SETs) for Fulfillment via Slack and in a comment in this issue when enabling a feature flag in customers-staging as it can impact the QA pipeline.
  • Verify that the feature works as expected. Posting the QA result in this issue is preferable.

Preparation before production rollout

  • Ensure that the feature MRs have been deployed to production
  • Ensure test plan had been executed successfully
  • Ensure that the rake task to backfill Account and AccountMembership (introduced in #4798 (closed)) has been run and verified. This data must be populated before we enable the feature flag in the production environment.
    • RAILS_ENV=production ~/.rbenv/shims/bundle exec rake data_maintenance:billing_accounts:backfill_with_customers[false]
    • RAILS_ENV=production ~/.rbenv/shims/bundle exec rake data_maintenance:billing_accounts:backfill_cloud_activations[false]
    • RAILS_ENV=production ~/.rbenv/shims/bundle exec rake data_maintenance:billing_accounts:backfill_licenses[false]
    • RAILS_ENV=production ~/.rbenv/shims/bundle exec rake data_maintenance:billing_accounts:backfill_orders[false]
  • Check if the feature flag change needs to be accompanied with a change management issue. Cross link the issue here if it does.
  • Ensure that you or a representative in Fulfillment can be available for at least 2 hours after feature flag updates in production. If a different developer will be covering, or an exception is needed, please inform the Fulfillment team in #s_fulfillment_engineering
  • Ensure any documentation has been updated
  • Announce on the feature epic an estimated time this will be enabled on production if applicable
  • If the feature might impact the user experience, notify #s_fulfillment and potentially #support_gitlab-com, as well as your team channel (more guidance when this is necessary in the dev docs).

Global rollout on CustomersDot production

  • Confirm the feature flag is enabled on staging without incident
  • Roll out the feature to targeted testing users first (if applicable)
  • If applicable, incrementally roll out the feature.
  • Verify the change has the desired outcome with the limited rollout before enabling the feature globally on production.
  • Enable the feature globally on production environment
  • Announce on the feature epic that the feature has been globally enabled.
  • Wait for at least one day for the verification term.

Release the feature

After the feature has been deemed stable, the clean up should be done as soon as possible to permanently enable the feature and reduce complexity in the codebase.

Rollback Steps

  • This feature can be disabled by performing the following actions:
Edited by Tyler Amos