Vulnerabilities index and schema for ES

What does this MR do and why?

Creates the Vulnerabilities ES index along with the document schema and default index settings.

Database

Preloading logic query plans:

• Vulnerabilities::Scanner Load - https://console.postgres.ai/gitlab/gitlab-production-main/sessions/37864/commands/115763
• Vulnerability Load - https://console.postgres.ai/gitlab/gitlab-production-main/sessions/37864/commands/115764
• Project Load - https://console.postgres.ai/gitlab/gitlab-production-main/sessions/37864/commands/115765
• Vulnerabilities::Finding Load - https://console.postgres.ai/gitlab/gitlab-production-main/sessions/37864/commands/115766
• Namespace Load - https://console.postgres.ai/gitlab/gitlab-production-main/sessions/37864/commands/115767
• Vulnerabilities::FindingIdentifier Load - https://console.postgres.ai/gitlab/gitlab-production-main/sessions/37864/commands/115768
• Route Load - https://console.postgres.ai/gitlab/gitlab-production-main/sessions/37864/commands/115769
• Vulnerabilities::Identifier Load - https://console.postgres.ai/gitlab/gitlab-production-main/sessions/37864/commands/115770

How to set up and validate locally

Setup

Seed vulnerabilities in local:

1. Import the project from here into local using the import by url option.
2. In the imported project, run pipeline on the master branch and allow the pipeline to complete. This will seed the vulnerabilities data.
3. To populate the pm_cve_enrichment table with data for epss_scores field, follow the instructions on the readme.md file on the imported project.

Run the ES migration:

1. Run the migration in Rails console Elastic::DataMigrationService[20250226195021].migrate.

Backfill ES index with documents manually:

1. In Rails console run the below commands

Vulnerabilities::Read.all.each { |v| ::Elastic::ProcessBookkeepingService.track!(Search::Elastic::References::Vulnerability.new(v.vulnerability_id, "group_#{v.project.namespace.root_ancestor.id}")) }

2. Run the bookkeeping command.

Elastic::ProcessBookkeepingService.new.execute

Validation steps:

1. GET gitlab-development-vulnerabilities/_settings in Kibana Dev console or curl "http://localhost:9200/gitlab-development-vulnerabilities/_settings" should list the new index being created after running the migration command above.
2. Find the name of the full index from the above request's response, lets say the index name from the response is gitlab-development-vulnerabilities-20250319-2109. Verify that the mappings are created successfully by the request GET gitlab-development-vulnerabilities-20250319-2109/_mapping in Kibana or `curl "http://localhost:9200/gitlab-development-vulnerabilities-20250319-2109/_mapping". It should look like the below response

{
  "gitlab-development-vulnerabilities-20250402-1515": {
    "mappings": {
      "dynamic": "strict",
      "_meta": {
        "created_by": "17.11.0-pre"
      },
      "properties": {
        "archived": {
          "type": "boolean"
        },
        "auto_resolved": {
          "type": "boolean"
        },
        "casted_cluster_agent_id": {
          "type": "long"
        },
        "cluster_agent_id": {
          "type": "text"
        },
        "created_at": {
          "type": "date"
        },
        "dismissal_reason": {
          "type": "short"
        },
        "epss_scores": {
          "type": "float"
        },
        "has_issues": {
          "type": "boolean"
        },
        "has_merge_request": {
          "type": "boolean"
        },
        "has_remediations": {
          "type": "boolean"
        },
        "has_vulnerability_resolution": {
          "type": "boolean"
        },
        "id": {
          "type": "long"
        },
        "identifier_names": {
          "type": "keyword"
        },
        "location_image": {
          "type": "text"
        },
        "project_id": {
          "type": "long"
        },
        "report_type": {
          "type": "short"
        },
        "resolved_on_default_branch": {
          "type": "boolean"
        },
        "scanner_external_id": {
          "type": "text"
        },
        "scanner_id": {
          "type": "long"
        },
        "schema_version": {
          "type": "short"
        },
        "severity": {
          "type": "short"
        },
        "state": {
          "type": "short"
        },
        "traversal_ids": {
          "type": "keyword"
        },
        "type": {
          "type": "keyword"
        },
        "updated_at": {
          "type": "date"
        },
        "uuid": {
          "type": "binary"
        },
        "vulnerability_id": {
          "type": "long"
        }
      }
    }
  }
}

Related to #515553 (closed)