Reduce the number of vulnerability statistics records adjusted in single SQL query

Why are we doing this work

We are adjusting the vulnerability_statistics records every day by a cronjob to fix the discrepancies if there are any. In that cronjob, we schedule multiple Sidekiq jobs to adjust the records in batches. Those batches are processed by the Vulnerabilities::Statistics::AdjustmentService. In that service class, we try to adjust the vulnerability_statistics for the whole list of 500 projects which can cause timeouts and prevent adjusting the records.

To address this potential issue, we either need to reduce the batch size and schedule Sidekiq jobs for a smaller number of project IDs or implement an extra level of batching in Vulnerabilities::Statistics::AdjustmentService to adjust the smaller amount of records in a single UPSERT query.

Reducing the batch size to schedule Sidekiq jobs for a smaller number of project IDs can cause scheduling too many jobs and delay in running the logic. Therefore, my initial idea is to implement an extra level of batching in the service class instead.

Implementation plan

TBD

Verification steps

TBD