Performance test of advisory scans

Problem to solve

The benchmark

Proposal

On one of the reference architectures, ingest N advisories that match M projects, and check the following:

• The advisory scans are processed in a reasonable amount of time.
• Scans don't cause congestions or deadlocks.

Implementation plan

• Describe scenario to be tested.
  • 5k advisories
  • 1k projects
  • 100 components per projects
  • half of the components are affected
• Specify expectations.
  • Less than 1 minute per advisory being "scanned".
• Prepare test bucket of advisories export.
  • Copy advisories from production bucket.
  • Set the publication date so that only N advisories are going to be ingested. See #421294 (closed)
• Identify how to monitor scanning jobs.
• Implement seeder.
• Prepare GitLab instance.
  • Select one of the referenced architectures.
  • Deploy GitLab on this architecture.
  • Configure the instance to use test bucket (offline setup).
  • Execute seeder.
• Check performance and data consistency on first ingestion.
  • Sync the instance with the test bucket.
  • Monitor advisory scan worker.
  • Monitor database.
  • Check statistics on vulnerabilities.
  • Check vulnerabilities in a project.
• Check performance and data consistency on second ingestion (idempotent)
  • Reset sync checkpoint, and sync again.
  • Monitor advisory scan worker.
  • Monitor database.
  • Check statistics on vulnerabilities.
  • Check vulnerabilities in a project.

Outcome

• Duration of advisory scan: min, max, median
• SQL time: min, max, median
• CPU time: min, max, median
• Move Ingest vulnerabilities from multiple projects a... (#420768 - closed) to Continuous Vulnerability Scans for DS Post MVC ... (&10133) if performance is sufficient.

/cc @hacks4oats @ifrenkel @willmeek

Auto-Summary 🤖

Discoto Usage

---

Points

Discussion points are declared by headings, list items, and single lines that start with the text (case-insensitive) point:. For example, the following are all valid points:

• #### POINT: This is a point
• * point: This is a point
• + Point: This is a point
• - pOINT: This is a point
• point: This is a **point**

Note that any markdown used in the point text will also be propagated into the topic summaries.

Topics

Topics can be stand-alone and contained within an issuable (epic, issue, MR), or can be inline.

Inline topics are defined by creating a new thread (discussion) where the first line of the first comment is a heading that starts with (case-insensitive) topic:. For example, the following are all valid topics:

• # Topic: Inline discussion topic 1
• ## TOPIC: **{+A Green, bolded topic+}**
• ### tOpIc: Another topic

Quick Actions

Action	Description
/discuss sub-topic TITLE	Create an issue for a sub-topic. Does not work in epics
/discuss link ISSUABLE-LINK	Link an issuable as a child of this discussion

---

Last updated by this job

• TOPIC Test instance #423578 (comment 1535108035)
• TOPIC Expectations #423578 (comment 1535114619)
• TOPIC Seeder #423578 (comment 1535132251)
• TOPIC Test scenario #423578 (comment 1535153943)
• TOPIC Test second ingestion #423578 (comment 1535177609)
• TOPIC Monitoring #423578 (comment 1535190418)
• TOPIC Performance of DS scans when OS data ingested #423578 (comment 1547794260)
• TOPIC Test scenario 1 on test instance #423578 (comment 1604926161)
• TOPIC Test scenario 2 on Gitpod GDK #423578 (comment 1611014403)
• TOPIC Comparison with StoreScansWorker #423578 (comment 1612173093)
• TOPIC Conclusion #423578 (comment 1615208412)

Discoto Settings

---
summary:
  max_items: -1
  sort_by: created
  sort_direction: ascending

See the settings schema for details.