Skip to content

Create API to update upcoming reconciliations

What does this MR do?

This is part of #215187

When we implement Quarterly GitLab.com Seat Reconciliation & Billing for SaaS, group owners need to be aware if they have a upcoming charge.

The reconciliations happen in CustomersDot. To show alert to group owners in Gitlab application, we need to get upcoming reconciliations data from CustomersDot. We will save the upcoming reconciliations in the table upcoming_reconciliations, so that Gitlab application will check local table upcoming_reconciliations:

  • it allows better page loading performance
  • it makes Gitlab.com more resilient, in the case of CustomersDot is slow or down

There will be an CustomersDot cron job(MR https://gitlab.com/gitlab-org/customers-gitlab-com/-/merge_requests/3314) to collect the upcoming reconciliations data. It is estimated to have hundreds of upcoming reconciliations everyday. The CustomersDot cron job will pass all these data together to Gitlab in a single API(which we are creating in this MR) call.

This MR creates an internal Gitlab API to update upcoming reconciliations. As mentioned above, this API is called by the CustomersDot daily cron job. This API accepts an array of upcoming reconciliations. Each array item(one upcoming reconciliation) contains: namespace_id, next_reconciliation_date, display_alert_from.

  • If one namespace_id exists in upcoming_reconciliations table, the existing record is updated. Othewise it creates a new record.
  • It validate the records, and then used bulk_upsert! to upsert all records together.
    • If all records are valid and bulk_upsert! successfully, it returns 200.
    • If some records are invalid, it will bulk_upsert! the valid records only, and returns 400 error with the invalid { namespace_id => errors}.
    • bulk_upsert! throw unexpected error, it returns 400 error with both bulk_upsert exception message and the invalid { namespace_id => errors}.
  • Both invalid records found during validation and buld_upsert! error are unexpected. In other words, the API should always succeed with 200 status. The 400 error code means, something really exceptional happens(such as, some data integrity issue between CustomersDot and Gitlab, which is rare but could happen), in this case we should open new issues to analyse and fix.

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Related to #215187

Edited by Qingyu Zhao

Merge request reports