Apdex and error rate - list of possible improvements

Problem to solve

The main CustomersDot Grafana page now shows enough information so that we can list which endpoint needs to be taken care of in terms of performance, so that all Fulfillment groups error budgets can be improved across the board.

See also this issue that outlines how a group could possibly improve their Apdex: #4725 (closed)

List of action items

This issue will try to identify and list the different classes which performance needs to improve, based on what comes out of Grafana.

For groupprovision:

• Api::V1::SeatLinksController#create: see #4884 (closed) for more details

For ~"group::billing and subscription management":

• Reconciliations::SendSeatOverageNotificationJob: this job alone seems to degrade significantly the Apdex score for this group.
  • undefined method 'zuora_account_id' for nil:NilClass
• SendUpcomingReconciliationEmailJob: the error rate for this job is higher that any other job classes.
  • undefined method 'zuora_account_id' for nil:NilClass
  • MaxAdditionalSeatsError

For ~"group::purchase"

• Investigate PaymentFormsController#show speed.

For ~"group::commerce integrations":

• Salesforce::CreateOpportunityWorker