Extend DastSiteProfile with more config options

Related Issue(s)

• #225406 (closed)
• #277353 (closed)

What does this MR do?

adds additional some additional configuration for DastSiteProfiles and stubs out some of GraphQL arguments and fields until the backend for storing them securely is in place.

Why?

we want to offer users more control over their dast on-demand scans.

GraphQL

Query

query project($fullPath: ID!) {
  project(fullPath: $fullPath) {
    dastSiteProfiles(first: 1) {
      nodes {
        id
        profileName
        targetUrl
        editPath
        excludedUrls
        requestHeaders
        validationStatus
        normalizedTargetUrl
        auth { enabled url usernameField passwordField username password }
        referencedInSecurityPolicies
      }
    }
  }
}

Mutations

dastSiteProfileCreate

mutation($dastSiteProfileCreateInput: DastSiteProfileCreateInput!) {
  dastSiteProfileCreate(input: $dastSiteProfileCreateInput) {
    clientMutationId
    errors
    id
  }
}

{
   "dastSiteProfileCreateInput":{
      "fullPath":"namespace10/project10",
      "profileName":"Open-source intermediate methodology",
      "targetUrl":"http://example18.test",
      "excludedUrls":"http://example18.test/signout",
      "requestHeaders":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0",
      "auth":{
         "enabled":false
      }
   }
}

dastSiteProfileUpdate

mutation($dastSiteProfileUpdateInput: DastSiteProfileUpdateInput!) {
  dastSiteProfileUpdate(input: $dastSiteProfileUpdateInput) {
    clientMutationId
    errors
    id
  }
}

{
   "dastSiteProfileUpdateInput":{
      "fullPath":"namespace25/project25",
      "id":"gid://gitlab/DastSiteProfile/24",
      "profileName":"c7ad43f42039f195dc8654f6389f57b9",
      "targetUrl":"http://example36.test",
      "excludedUrls":"http://example36.test/signout",
      "requestHeaders":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0",
      "auth":{
         "enabled":false
      }
   }
}

Database

Migrations

% rails db:migrate:up VERSION=20210302025305                                                                                                                                                                                                                                                                                                                                    
== 20210302025305 AddExcludedUrlsAndRequestHeadersToDastSiteProfiles: migrating
-- add_column(:dast_site_profiles, :excluded_urls, :text)
   -> 0.0016s
-- add_column(:dast_site_profiles, :auth_enabled, :boolean, {:default=>false, :null=>false})
   -> 0.0028s
-- add_column(:dast_site_profiles, :auth_url, :text)
   -> 0.0012s
-- add_column(:dast_site_profiles, :auth_username_field, :text)
   -> 0.0009s
-- add_column(:dast_site_profiles, :auth_password_field, :text)
   -> 0.0009s
-- add_column(:dast_site_profiles, :auth_username, :text)
   -> 0.0009s
-- transaction_open?()
   -> 0.0000s
-- current_schema()
   -> 0.0002s
-- execute("ALTER TABLE dast_site_profiles\nADD CONSTRAINT check_07ae59ac33\nCHECK ( char_length(excluded_urls) <= 2048 )\nNOT VALID;\n")
   -> 0.0012s
-- current_schema()
   -> 0.0002s
-- execute("SET statement_timeout TO 0")
   -> 0.0006s
-- execute("ALTER TABLE dast_site_profiles VALIDATE CONSTRAINT check_07ae59ac33;")
   -> 0.0012s
-- execute("RESET ALL")
   -> 0.0006s
-- transaction_open?()
   -> 0.0000s
-- current_schema()
   -> 0.0002s
-- execute("ALTER TABLE dast_site_profiles\nADD CONSTRAINT check_d446f7047b\nCHECK ( char_length(auth_url) <= 1024 )\nNOT VALID;\n")
   -> 0.0010s
-- current_schema()
   -> 0.0002s
-- execute("ALTER TABLE dast_site_profiles VALIDATE CONSTRAINT check_d446f7047b;")
   -> 0.0011s
-- transaction_open?()
   -> 0.0000s
-- current_schema()
   -> 0.0002s
-- execute("ALTER TABLE dast_site_profiles\nADD CONSTRAINT check_5203110fee\nCHECK ( char_length(auth_username_field) <= 255 )\nNOT VALID;\n")
   -> 0.0012s
-- current_schema()
   -> 0.0002s
-- execute("ALTER TABLE dast_site_profiles VALIDATE CONSTRAINT check_5203110fee;")
   -> 0.0014s
-- transaction_open?()
   -> 0.0000s
-- current_schema()
   -> 0.0004s
-- execute("ALTER TABLE dast_site_profiles\nADD CONSTRAINT check_c329dffdba\nCHECK ( char_length(auth_password_field) <= 255 )\nNOT VALID;\n")
   -> 0.0011s
-- current_schema()
   -> 0.0002s
-- execute("ALTER TABLE dast_site_profiles VALIDATE CONSTRAINT check_c329dffdba;")
   -> 0.0010s
-- transaction_open?()
   -> 0.0000s
-- current_schema()
   -> 0.0002s
-- execute("ALTER TABLE dast_site_profiles\nADD CONSTRAINT check_f22f18002a\nCHECK ( char_length(auth_username) <= 255 )\nNOT VALID;\n")
   -> 0.0016s
-- current_schema()
   -> 0.0002s
-- execute("ALTER TABLE dast_site_profiles VALIDATE CONSTRAINT check_f22f18002a;")
   -> 0.0011s
== 20210302025305 AddExcludedUrlsAndRequestHeadersToDastSiteProfiles: migrated (0.1245s)

% rails db:migrate:down VERSION=20210302025305                                                                                                                                                                                                                                                                                                                                    
== 20210302025305 AddExcludedUrlsAndRequestHeadersToDastSiteProfiles: reverting
-- remove_column(:dast_site_profiles, :auth_username)
   -> 0.0018s
-- remove_column(:dast_site_profiles, :auth_password_field)
   -> 0.0010s
-- remove_column(:dast_site_profiles, :auth_username_field)
   -> 0.0009s
-- remove_column(:dast_site_profiles, :auth_url)
   -> 0.0008s
-- remove_column(:dast_site_profiles, :auth_enabled)
   -> 0.0008s
-- remove_column(:dast_site_profiles, :excluded_urls)
   -> 0.0008s
== 20210302025305 AddExcludedUrlsAndRequestHeadersToDastSiteProfiles: reverted (0.0133s)

Does this MR meet the acceptance criteria?

Conformity

• Does this MR need a changelog?
  • I have included a changelog entry.
  • I have not included a changelog entry because _____.
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team

.gitlab/ci/rules.gitlab-ci.yml

@@ -193,6 +193,7 @@
   - "config.ru"
   - "{,ee/}{app,bin,config,db,haml_lint,lib,locale,public,scripts,symbol,vendor}/**/*"
   - "doc/api/graphql/reference/*"  # Files in this folder are auto-generated
+  - "data/whats_new/*.yml"
 
 .qa-patterns: &qa-patterns
   - ".dockerignore"
@@ -215,6 +216,7 @@
   - "config.ru"
   - "{,ee/}{app,bin,config,db,haml_lint,lib,locale,public,scripts,symbol,vendor}/**/*"
   - "doc/api/graphql/reference/*"  # Files in this folder are auto-generated
+  - "data/whats_new/*.yml"
   # Backstage changes
   - "Dangerfile"
   - "danger/**/*"
@@ -240,6 +242,7 @@
   - "config.ru"
   - "{,ee/}{app,bin,config,db,haml_lint,lib,locale,public,scripts,symbol,vendor}/**/*"
   - "doc/api/graphql/reference/*"  # Files in this folder are auto-generated
+  - "data/whats_new/*.yml"
   # QA changes
   - ".dockerignore"
   - "qa/**/*"
@@ -261,6 +264,7 @@
   - "config.ru"
   - "{,ee/}{app,bin,config,db,haml_lint,lib,locale,public,scripts,symbol,vendor}/**/*"
   - "doc/api/graphql/reference/*"  # Files in this folder are auto-generated
+  - "data/whats_new/*.yml"
   # Backstage changes
   - "Dangerfile"
   - "danger/**/*"