Add Cobra for managing Go scripts
What does this MR do and why?
Adds the Cobra library for managing our Go scripts.
We'll be using this for two kinds of scripts:
- Build and maintenance scripts. Things we used rake tasks for in the Nanoc project (e.g, clone site, update redirects, etc)
- Migration scripts, used to manage the Nanoc -> Hugo transition
Why:
- The Cobra project is very active, well documented and supported
- Established patterns for creating commands
- A single binary can can be produced and distributed which means Go will only be required for development (if desired)
- Adding new commands should require less code
- Common logic can be more easily shared using Go packages
- Help can be easily accessed for the tool and/or each command
Copied this list from Ash's comment in a previous MR
Also h/t to this great presentation on Cobra: https://spf13.com/presentation/building-an-awesome-cli-app-in-go-oscon/
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
-
Fetch this branch and update dependencies: git fetch origin cobra-cli && git checkout cobra-cli && make setup
-
Verify the clone projects script still works: go run cmd/gldocs/main.go clone
REMOVE_BEFORE_CLONE=true go run cmd/gldocs/main.go clone
- The script should clone source projects into the same dir as
gitlab-docs-hugo
, skipping them if they already exist, unlessREMOVE_BEFORE_CLONE
is set to true, same as before.
-
Verify the frontmatter migrate script still works: - Test against a single project:
go run cmd/gldocs/main.go migrate ../gitlab-runner/docs/**/*.md
- Check the diff:
git -C ../gitlab-runner diff
. You should see that the front matter script has migrated h1s to title front matter, same as before.
- Test against a single project:
-
Verify these run as expected end-to-end - Run a full new build:
REMOVE_BEFORE_CLONE=true make clone-docs-projects && make view
- Run a full new build:
Expected output for clone
~/dev/gdh/gitlab-docs-hugo cobra-cli *7 REMOVE_BEFORE_CLONE=true go run main.go clone
2024/03/26 14:31:26 [Info] ../gitlab-operator already exists, removing it
2024/03/26 14:31:26 [Info] Fetching operator on branch master at commit heads/master
2024/03/26 14:31:28 [Info] Last commit: ed89e65 Merge branch 'add-tls-generator' into 'master'
2024/03/26 14:31:33 [Info] ../gitlab already exists, removing it
2024/03/26 14:31:33 [Info] Fetching gitlab on branch master at commit heads/master
2024/03/26 14:31:49 [Info] Last commit: 897f7dea3 Merge branch 'more-actions-dropdown-tooltip' into 'master'
2024/03/26 14:31:50 [Info] ../omnibus-gitlab already exists, removing it
2024/03/26 14:31:50 [Info] Fetching omnibus on branch master at commit heads/master
2024/03/26 14:31:51 [Info] Last commit: 454718e Merge branch 'deps/e771939-a995f18' into 'master'
2024/03/26 14:31:51 [Info] ../gitlab-runner already exists, removing it
2024/03/26 14:31:51 [Info] Fetching runner on branch main at commit heads/main
2024/03/26 14:31:52 [Info] Last commit: f898c1e Merge branch 'dependencies-fix-moby-spdystream-race-condition' into 'main'
2024/03/26 14:31:52 [Info] ../charts-gitlab already exists, removing it
2024/03/26 14:31:52 [Info] Fetching charts on branch master at commit heads/master
2024/03/26 14:31:53 [Info] Last commit: 607752a Merge branch 'docs-aqualls-fix-trailing-spaces' into 'master'
~/dev/gdh/gitlab-docs-hugo cobra-cli *7 go run main.go clone
2024/03/26 14:38:07 [Info] gitlab directory already exists, skipping
2024/03/26 14:38:07 [Info] omnibus directory already exists, skipping
2024/03/26 14:38:07 [Info] runner directory already exists, skipping
2024/03/26 14:38:07 [Info] charts directory already exists, skipping
2024/03/26 14:38:07 [Info] operator directory already exists, skipping
~/dev/gdh/gitlab-docs-hugo cobra-cli *7 ls -lah ../
total 0
drwxr-xr-x 8 sarahg staff 256B Mar 26 14:40 .
drwxr-xr-x 40 sarahg staff 1.3K Feb 29 16:36 ..
drwxr-xr-x 35 sarahg staff 1.1K Mar 26 14:31 charts-gitlab
drwxr-xr-x 118 sarahg staff 3.7K Mar 26 14:31 gitlab
drwxr-xr-x 43 sarahg staff 1.3K Mar 26 14:40 gitlab-docs-hugo
drwxr-xr-x 37 sarahg staff 1.2K Mar 26 14:31 gitlab-operator
drwxr-xr-x 52 sarahg staff 1.6K Mar 26 14:31 gitlab-runner
drwxr-xr-x 50 sarahg staff 1.6K Mar 26 14:31 omnibus-gitlab
Expected output for migrate
~/dev/gdh/gitlab-docs-hugo cobra-cli *7 find ../gitlab-runner/docs -type f -name "*.md" -print0 | xargs -0 go run main.go migrate
2024/03/26 16:21:04 Error processing file ../gitlab-runner/docs/internal/ci/packages_iteration.md: [WARN] No valid YAML front matter found
2024/03/26 16:21:04 Error processing file ../gitlab-runner/docs/internal/engineering/executor-interface/index.md: [WARN] No valid YAML front matter found
Expected output for full build
~/dev/gdh/gitlab-docs-hugo cobra-cli *7 REMOVE_BEFORE_CLONE=true go run main.go clone
2024/03/26 16:22:45 [Info] ../gitlab already exists, removing it
2024/03/26 16:22:45 [Info] Fetching gitlab on branch master at commit heads/master
2024/03/26 16:23:04 [Info] Last commit: edd1c5be9 Merge branch 'fix-commit-status-api-overwriting-merge-request-head-pipeline' into 'master'
2024/03/26 16:23:04 [Info] ../omnibus-gitlab already exists, removing it
2024/03/26 16:23:04 [Info] Fetching omnibus on branch master at commit heads/master
2024/03/26 16:23:05 [Info] Last commit: 454718e Merge branch 'deps/e771939-a995f18' into 'master'
2024/03/26 16:23:05 [Info] ../gitlab-runner already exists, removing it
2024/03/26 16:23:05 [Info] Fetching runner on branch main at commit heads/main
2024/03/26 16:23:06 [Info] Last commit: f898c1e Merge branch 'dependencies-fix-moby-spdystream-race-condition' into 'main'
2024/03/26 16:23:06 [Info] ../charts-gitlab already exists, removing it
2024/03/26 16:23:06 [Info] Fetching charts on branch master at commit heads/master
2024/03/26 16:23:07 [Info] Last commit: 607752a Merge branch 'docs-aqualls-fix-trailing-spaces' into 'master'
2024/03/26 16:23:08 [Info] ../gitlab-operator already exists, removing it
2024/03/26 16:23:08 [Info] Fetching operator on branch master at commit heads/master
2024/03/26 16:23:08 [Info] Last commit: ed89e65 Merge branch 'add-tls-generator' into 'master'
~/dev/gdh/gitlab-docs-hugo cobra-cli *7 REMOVE_BEFORE_CLONE=true make clone-docs-projects && make view ✔ 31s 16:23:09
INFO: Fetching docs content sources...
2024/03/26 16:23:51 [Info] ../gitlab already exists, removing it
2024/03/26 16:23:51 [Info] Fetching gitlab on branch master at commit heads/master
2024/03/26 16:24:07 [Info] Last commit: edd1c5be9 Merge branch 'fix-commit-status-api-overwriting-merge-request-head-pipeline' into 'master'
2024/03/26 16:24:07 [Info] ../omnibus-gitlab already exists, removing it
2024/03/26 16:24:07 [Info] Fetching omnibus on branch master at commit heads/master
2024/03/26 16:24:09 [Info] Last commit: 454718e Merge branch 'deps/e771939-a995f18' into 'master'
2024/03/26 16:24:09 [Info] ../gitlab-runner already exists, removing it
2024/03/26 16:24:09 [Info] Fetching runner on branch main at commit heads/main
2024/03/26 16:24:10 [Info] Last commit: f898c1e Merge branch 'dependencies-fix-moby-spdystream-race-condition' into 'main'
2024/03/26 16:24:10 [Info] ../charts-gitlab already exists, removing it
2024/03/26 16:24:10 [Info] Fetching charts on branch master at commit heads/master
2024/03/26 16:24:11 [Info] Last commit: 607752a Merge branch 'docs-aqualls-fix-trailing-spaces' into 'master'
2024/03/26 16:24:11 [Info] ../gitlab-operator already exists, removing it
2024/03/26 16:24:11 [Info] Fetching operator on branch master at commit heads/master
2024/03/26 16:24:12 [Info] Last commit: ed89e65 Merge branch 'add-tls-generator' into 'master'
INFO: Running content update scripts...
Renaming index files in ../gitlab...
Updating content in ../gitlab...
Renaming index files in ../omnibus-gitlab...
Updating content in ../omnibus-gitlab...
Renaming index files in ../gitlab-runner...
Updating content in ../gitlab-runner...
2024/03/26 16:24:22 Error processing file ../gitlab-runner/docs/internal/ci/packages_iteration.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:22 Error processing file ../gitlab-runner/docs/internal/engineering/executor-interface/_index.md: [WARN] No valid YAML front matter found
Renaming index files in ../charts-gitlab...
Updating content in ../charts-gitlab...
Renaming index files in ../gitlab-operator...
Updating content in ../gitlab-operator...
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0002-use-rh-codebase.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0001-record-architecture-decisions.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0012-makefile-modifications.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0006-gitlab-application-versions-supported.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0016-replace-makefile-with-taskfile.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0011-operator-release-process.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0004-integration-of-the-gitlab-chart.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0013-stable-branch-policy.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0003-use-operator-sdk.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0005-versioning-of-the-operator.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0007-dependency-on-other-external-components.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0009-version-tagging.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0010-ci-environments.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0008-operator-scoped-to-namespace.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0018-obsolete-gitlab-cr-v1beta1.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0017-structured-spec-for-gitlab-cr.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0014-supported-openshift-versions.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0015-ci-pipelines-in-dev.md: [WARN] No valid YAML front matter found
INFO: Content updates complete!
vite v5.1.5 building for production...
✓ 1000 modules transformed.
themes/gitlab-docs/static/vite/icons.svg 193.69 kB │ gzip: 45.99 kB
themes/gitlab-docs/static/vite/main.css 7.09 kB │ gzip: 2.15 kB
themes/gitlab-docs/static/vite/main.js 132.86 kB │ gzip: 45.56 kB
✓ built in 1.80s
Watching for changes in /Users/sarahg/dev/gdh/{charts-gitlab,gitlab-docs-hugo,gitlab-operator,gitlab-runner,gitlab,omnibus-gitlab}
Watching for config changes in /Users/sarahg/dev/gdh/gitlab-docs-hugo/config/_default, /Users/sarahg/dev/gdh/gitlab-docs-hugo/go.mod
Start building sites …
hugo v0.123.7-312735366b20d64bd61bff8627f593749f86c964+extended darwin/arm64 BuildDate=2024-03-01T16:16:06Z VendorInfo=gohugoio
| EN
-------------------+-------
Pages | 3094
Paginator pages | 0
Non-page files | 841
Static files | 427
Processed images | 0
Aliases | 0
Cleaned | 0
Built in 2287 ms
Environment: "development"
Serving pages from disk
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop
Merge request acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this merge request.
Edited by Sarah German