Document detected secrets
-
Please check this box if this contribution uses AI-generated content as outlined in the GitLab DCO & CLA.
What does this MR do?
Creates 2 new docs page which list the types of secrets detected by:
- Pipeline secret detection
- Secret push protection
Related issues
Author's checklist
-
Optional. Consider taking the GitLab Technical Writing Fundamentals course. -
Follow the: -
If you're adding a new page, add the product availability details under the H1 topic title. -
If you are a GitLab team member, request a review based on: - The documentation page's metadata.
- The associated Technical Writer.
If you are a GitLab team member and only adding documentation, do not add any of the following labels:
~"frontend"
~"backend"
~"type::bug"
~"database"
These labels cause the MR to be added to code verification QA issues.
Reviewer's checklist
Documentation-related MRs should be reviewed by a Technical Writer for a non-blocking review, based on Documentation Guidelines and the Style Guide.
If you aren't sure which tech writer to ask, use roulette or ask in the #docs Slack channel.
-
If the content requires it, ensure the information is reviewed by a subject matter expert. - Technical writer review items:
-
Ensure docs metadata is present and up-to-date. -
Ensure the appropriate labels are added to this MR. -
Ensure a release milestone is set. - If relevant to this MR, ensure content topic type principles are in use, including:
-
The headings should be something you'd do a Google search for. Instead of Default behavior
, say something likeDefault behavior when you close an issue
. -
The headings (other than the page title) should be active. Instead of Configuring GDK
, say something likeConfigure GDK
. -
Any task steps should be written as a numbered list. - If the content still needs to be edited for topic types, you can create a follow-up issue with the docs-technical-debt label.
-
-
-
Review by assigned maintainer, who can always request/require the reviews above. Maintainer's review can occur before or after a technical writer review.
Merge request reports
Activity
changed milestone to %17.1
added docsimprovement documentation maintenancerefactor typemaintenance labels
assigned to @rdickenson
added devopssecure groupsecret detection labels
added sectionsec label
CC @phillipwells I needed something "simple" to work on on Friday afternoon, so I tackled this one. When it's been reviewed and approved by an SME I'll assign you as the final reviewer.
1 Message This merge request adds or changes documentation files. A review from the Technical Writing team before you merge is recommended. Reviews can happen after you merge. Documentation review
The following files require a review from a technical writer:
-
doc/user/application_security/secret_detection/pipeline/detected_secrets.md
(Link to current live version) -
doc/user/application_security/secret_detection/secret_push_protection/detected_secrets.md
(Link to current live version)
The review does not need to block merging this merge request. See the:
-
Metadata for the
*.md
files that you've changed. The first few lines of each*.md
file identify the stage and group most closely associated with your docs change. - The Technical Writer assigned for that stage and group.
- Documentation workflows for information on when to assign a merge request for review.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Danger-
added docs-only label
- Resolved by Russell Dickenson
@smeadzinger @amarpatel I put this MR together from CSV content provided by @ahmed.hemdan [1]. I've marked it as a draft because I think it needs some improvement:
- Sort in alphabetical order.
- Resort the columns into: description, ID, keywords.
Perhaps in the preparation for GA, I think we could improve on this MR, which I consider to be an "MVP":
- Combine the tables, with columns that indicate support for pipeline secret detection, secret push protection, or both.
- Add a column to indicate secrets for which we provide an automatic response.
Edited by Russell Dickenson
- Resolved by Russell Dickenson
@ahmed.hemdan I tried to recreate the CSV output by using the commands you listed in #454905 (comment 1864440235).
When I tried the command for the pipeline SD
gitleaks.toml
file, I got the following error:Error: cannot join with !!null, can only join arrays of scalars
When I tried the command for the secret push protection
gitleaks.toml
file I found that the description was being repeated.How can I change the commands to have the fields listed in order by: description, ID, keywords?
added TW-DRIRussell label
- Resolved by Russell Dickenson
Sequence of commands to convert content in the
gitlab.toml
files into a publishable Markdown file:For pipeline secrets detection:
-
In a local clone of the
gitlab
project, go to thepipeline
secret detection directory:https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc/user/application_security/secret_detection/pipeline
-
Copy the
gitleaks.toml
file into the current directory. -
Extract the desired fields from
gitleaks.toml
file used by pipeline secrets detection:yq eval '.rules[] | [.description, .id, (.keywords | join(";"))] | @csv' gitleaks.toml > detected_secrets.md
-
Move the
detected_secrets.md
file to thepipeline
docs directory.
For secret push protection:
-
In a local clone of the
gitlab
project, go to thesecret_push_protection
secret detection directory:https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc/user/application_security/secret_detection/secret_push_protection
-
Copy the
gitleaks.toml
file into the current directory. -
Extract the desired fields from the
gitleaks.toml
file used by secret push protection:yq eval '.rules[] | [.description, .id, .keywords | select(.) | join(";")] | @csv' gitleaks.toml > detected_secrets.md
-
Move the
detected_secrets.md
file to thepipeline
docs directory.
For each of the above files, convert from CSV formatting to Markdown formatting:
-
Sort entries into alphabetical order:
sort --field-separator=, --key=1n detected_secrets.md --output=detected_secrets.md
-
Prefix each line with a
|
(pipe character, followed by a space character):sed -i -e 's/^/| /' detected_secrets.md
-
Append each line with a
|
(space character, followed by a pipe character):sed -i -e 's/$/ |/' detected_secrets.md
-
Replace the comma separators with the Markdown table cell separator, surrounded by single space characters -
|
:sed -i.bak 's/,/ \| /g' detected_secrets.md
-
Replace each of the semicolons with a commas followed by a space (
,
):sed -i.bak 's/;/, | /g' detected_secrets.md
-
Delete any backups of the
detected_secrets.md
created by the previous commands. -
Delete the local copy of the
gitleaks.toml
file. -
Using a VS Code Markdown table formatter extension, reformat the table so that each column is of equal width.
Edited by Russell Dickenson -
added Technical Writing label
- Resolved by Russell Dickenson
@ahmed.hemdan For the "systemd machine ID" and "Password in URL" I marked their keyword column as "N/A" because they didn't have a 3rd field. Is this correct, or should I replace "N/A" with something else?
mentioned in merge request gitlab-docs!4841 (closed)
- Resolved by Phillip Wells
@phillipwells Could you please review this MR, and merge it if you approve?
I've prepared an MR to have these new pages added to the docs left navigation:
requested review from @phillipwells
enabled automatic add to merge train when the pipeline for 0bfa4a3f succeeds
mentioned in commit 21e848e8
removed review request for @phillipwells
mentioned in merge request gitlab-docs!4842 (merged)
mentioned in merge request !155944 (merged)
mentioned in issue #454905 (closed)
added workflowstaging-canary label
added workflowcanary label and removed workflowstaging-canary label
added workflowstaging label and removed workflowcanary label
added workflowproduction label and removed workflowstaging label
mentioned in issue #467110 (closed)
I created a follow-up issue to determine if there is a more efficient method of creating the Markdown tables:
added releasedcandidate label
added releasedpublished label and removed releasedcandidate label
mentioned in issue #471390 (closed)
added tw-leadsecure label
added Category:Secret Detection label