DAST On-Demand: Move DAST site validation to runner

Problem to Solve

When a user saves a url, the built-in url validation prohibits the saving of any url that resolves to a non-routable address as documented by users, #270751 (closed). This is done as a security measure so that the monolith does not access a non-routable url.

Solution

1. Move the domain validation work to a runner. By moving the validation to a runner, the runner can validate a domain regardless of whether that domain resolves to an internal ip or an external ip.

2. Once the domain validation is done exclusively on the runner, and monolith/sidekiq no longer visit domains from their trusted servers, the domain validation should be removed from the model.

The validation error exposed in #270751 (closed), will auto-resolve.

Implementation

• Add feature flag (:dast_runner_site_validation) !61649 (merged)
• Add internal API !61649 (merged)
• Add container for validation gitlab-org/security-products/dast-runner-validation!1 (merged)
• Release container https://gitlab.com/security-products/dast-runner-validation/container_registry
• Add validation strategy to the monolith behind feature flag !65794 (merged)
• Add new ci_pipelines.source for validation (e.g. ondemand_dast_validation) !66991 (merged)
• Add documentation
• Add support for offline environments by updating Secure-Binaries.gitlab-ci.yml (documentation on how this works https://docs.gitlab.com/ee/user/application_security/offline_deployments/) !66501 (merged)