Skip to content

Add Cobra for managing Go scripts

Sarah German requested to merge cobra-cli into main

What does this MR do and why?

Adds the Cobra library for managing our Go scripts.

We'll be using this for two kinds of scripts:

  1. Build and maintenance scripts. Things we used rake tasks for in the Nanoc project (e.g, clone site, update redirects, etc)
  2. Migration scripts, used to manage the Nanoc -> Hugo transition

Why:

  • The Cobra project is very active, well documented and supported
  • Established patterns for creating commands
  • A single binary can can be produced and distributed which means Go will only be required for development (if desired)
  • Adding new commands should require less code
  • Common logic can be more easily shared using Go packages
  • Help can be easily accessed for the tool and/or each command

Copied this list from Ash's comment in a previous MR 😄

Also h/t to this great presentation on Cobra: https://spf13.com/presentation/building-an-awesome-cli-app-in-go-oscon/

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Fetch this branch and update dependencies: git fetch origin cobra-cli && git checkout cobra-cli && make setup
  2. Verify the clone projects script still works:
    • go run cmd/gldocs/main.go clone
    • REMOVE_BEFORE_CLONE=true go run cmd/gldocs/main.go clone
    • The script should clone source projects into the same dir as gitlab-docs-hugo, skipping them if they already exist, unless REMOVE_BEFORE_CLONE is set to true, same as before.
  3. Verify the frontmatter migrate script still works:
    • Test against a single project: go run cmd/gldocs/main.go migrate ../gitlab-runner/docs/**/*.md
    • Check the diff: git -C ../gitlab-runner diff. You should see that the front matter script has migrated h1s to title front matter, same as before.
  4. Verify these run as expected end-to-end
    • Run a full new build: REMOVE_BEFORE_CLONE=true make clone-docs-projects && make view
Expected output for clone 
~/dev/gdh/gitlab-docs-hugo  cobra-cli *7  REMOVE_BEFORE_CLONE=true go run main.go clone

2024/03/26 14:31:26 [Info] ../gitlab-operator already exists, removing it
2024/03/26 14:31:26 [Info] Fetching operator on branch master at commit heads/master
2024/03/26 14:31:28 [Info] Last commit: ed89e65 Merge branch 'add-tls-generator' into 'master'
2024/03/26 14:31:33 [Info] ../gitlab already exists, removing it
2024/03/26 14:31:33 [Info] Fetching gitlab on branch master at commit heads/master
2024/03/26 14:31:49 [Info] Last commit: 897f7dea3 Merge branch 'more-actions-dropdown-tooltip' into 'master'
2024/03/26 14:31:50 [Info] ../omnibus-gitlab already exists, removing it
2024/03/26 14:31:50 [Info] Fetching omnibus on branch master at commit heads/master
2024/03/26 14:31:51 [Info] Last commit: 454718e Merge branch 'deps/e771939-a995f18' into 'master'
2024/03/26 14:31:51 [Info] ../gitlab-runner already exists, removing it
2024/03/26 14:31:51 [Info] Fetching runner on branch main at commit heads/main
2024/03/26 14:31:52 [Info] Last commit: f898c1e Merge branch 'dependencies-fix-moby-spdystream-race-condition' into 'main'
2024/03/26 14:31:52 [Info] ../charts-gitlab already exists, removing it
2024/03/26 14:31:52 [Info] Fetching charts on branch master at commit heads/master
2024/03/26 14:31:53 [Info] Last commit: 607752a Merge branch 'docs-aqualls-fix-trailing-spaces' into 'master'
 
~/dev/gdh/gitlab-docs-hugo  cobra-cli *7  go run main.go clone
2024/03/26 14:38:07 [Info] gitlab directory already exists, skipping
2024/03/26 14:38:07 [Info] omnibus directory already exists, skipping
2024/03/26 14:38:07 [Info] runner directory already exists, skipping
2024/03/26 14:38:07 [Info] charts directory already exists, skipping
2024/03/26 14:38:07 [Info] operator directory already exists, skipping

~/dev/gdh/gitlab-docs-hugo  cobra-cli *7  ls -lah ../                                                                                                        
total 0
drwxr-xr-x    8 sarahg  staff   256B Mar 26 14:40 .
drwxr-xr-x   40 sarahg  staff   1.3K Feb 29 16:36 ..
drwxr-xr-x   35 sarahg  staff   1.1K Mar 26 14:31 charts-gitlab
drwxr-xr-x  118 sarahg  staff   3.7K Mar 26 14:31 gitlab
drwxr-xr-x   43 sarahg  staff   1.3K Mar 26 14:40 gitlab-docs-hugo
drwxr-xr-x   37 sarahg  staff   1.2K Mar 26 14:31 gitlab-operator
drwxr-xr-x   52 sarahg  staff   1.6K Mar 26 14:31 gitlab-runner
drwxr-xr-x   50 sarahg  staff   1.6K Mar 26 14:31 omnibus-gitlab
Expected output for migrate 
~/dev/gdh/gitlab-docs-hugo  cobra-cli *7  find ../gitlab-runner/docs -type f -name "*.md" -print0 | xargs -0 go run main.go migrate                       
2024/03/26 16:21:04 Error processing file ../gitlab-runner/docs/internal/ci/packages_iteration.md: [WARN] No valid YAML front matter found
2024/03/26 16:21:04 Error processing file ../gitlab-runner/docs/internal/engineering/executor-interface/index.md: [WARN] No valid YAML front matter found
Expected output for full build 
~/dev/gdh/gitlab-docs-hugo  cobra-cli *7  REMOVE_BEFORE_CLONE=true go run main.go clone                                                               
2024/03/26 16:22:45 [Info] ../gitlab already exists, removing it
2024/03/26 16:22:45 [Info] Fetching gitlab on branch master at commit heads/master
2024/03/26 16:23:04 [Info] Last commit: edd1c5be9 Merge branch 'fix-commit-status-api-overwriting-merge-request-head-pipeline' into 'master'
2024/03/26 16:23:04 [Info] ../omnibus-gitlab already exists, removing it
2024/03/26 16:23:04 [Info] Fetching omnibus on branch master at commit heads/master
2024/03/26 16:23:05 [Info] Last commit: 454718e Merge branch 'deps/e771939-a995f18' into 'master'
2024/03/26 16:23:05 [Info] ../gitlab-runner already exists, removing it
2024/03/26 16:23:05 [Info] Fetching runner on branch main at commit heads/main
2024/03/26 16:23:06 [Info] Last commit: f898c1e Merge branch 'dependencies-fix-moby-spdystream-race-condition' into 'main'
2024/03/26 16:23:06 [Info] ../charts-gitlab already exists, removing it
2024/03/26 16:23:06 [Info] Fetching charts on branch master at commit heads/master
2024/03/26 16:23:07 [Info] Last commit: 607752a Merge branch 'docs-aqualls-fix-trailing-spaces' into 'master'
2024/03/26 16:23:08 [Info] ../gitlab-operator already exists, removing it
2024/03/26 16:23:08 [Info] Fetching operator on branch master at commit heads/master
2024/03/26 16:23:08 [Info] Last commit: ed89e65 Merge branch 'add-tls-generator' into 'master'
 ~/dev/gdh/gitlab-docs-hugo  cobra-cli *7  REMOVE_BEFORE_CLONE=true make clone-docs-projects && make view                                                ✔  31s  16:23:09

INFO: Fetching docs content sources...
2024/03/26 16:23:51 [Info] ../gitlab already exists, removing it
2024/03/26 16:23:51 [Info] Fetching gitlab on branch master at commit heads/master
2024/03/26 16:24:07 [Info] Last commit: edd1c5be9 Merge branch 'fix-commit-status-api-overwriting-merge-request-head-pipeline' into 'master'
2024/03/26 16:24:07 [Info] ../omnibus-gitlab already exists, removing it
2024/03/26 16:24:07 [Info] Fetching omnibus on branch master at commit heads/master
2024/03/26 16:24:09 [Info] Last commit: 454718e Merge branch 'deps/e771939-a995f18' into 'master'
2024/03/26 16:24:09 [Info] ../gitlab-runner already exists, removing it
2024/03/26 16:24:09 [Info] Fetching runner on branch main at commit heads/main
2024/03/26 16:24:10 [Info] Last commit: f898c1e Merge branch 'dependencies-fix-moby-spdystream-race-condition' into 'main'
2024/03/26 16:24:10 [Info] ../charts-gitlab already exists, removing it
2024/03/26 16:24:10 [Info] Fetching charts on branch master at commit heads/master
2024/03/26 16:24:11 [Info] Last commit: 607752a Merge branch 'docs-aqualls-fix-trailing-spaces' into 'master'
2024/03/26 16:24:11 [Info] ../gitlab-operator already exists, removing it
2024/03/26 16:24:11 [Info] Fetching operator on branch master at commit heads/master
2024/03/26 16:24:12 [Info] Last commit: ed89e65 Merge branch 'add-tls-generator' into 'master'

INFO: Running content update scripts...
Renaming index files in ../gitlab...
Updating content in ../gitlab...
Renaming index files in ../omnibus-gitlab...
Updating content in ../omnibus-gitlab...
Renaming index files in ../gitlab-runner...
Updating content in ../gitlab-runner...
2024/03/26 16:24:22 Error processing file ../gitlab-runner/docs/internal/ci/packages_iteration.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:22 Error processing file ../gitlab-runner/docs/internal/engineering/executor-interface/_index.md: [WARN] No valid YAML front matter found
Renaming index files in ../charts-gitlab...
Updating content in ../charts-gitlab...
Renaming index files in ../gitlab-operator...
Updating content in ../gitlab-operator...
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0002-use-rh-codebase.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0001-record-architecture-decisions.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0012-makefile-modifications.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0006-gitlab-application-versions-supported.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0016-replace-makefile-with-taskfile.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0011-operator-release-process.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0004-integration-of-the-gitlab-chart.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0013-stable-branch-policy.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0003-use-operator-sdk.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0005-versioning-of-the-operator.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0007-dependency-on-other-external-components.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0009-version-tagging.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0010-ci-environments.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0008-operator-scoped-to-namespace.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0018-obsolete-gitlab-cr-v1beta1.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0017-structured-spec-for-gitlab-cr.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0014-supported-openshift-versions.md: [WARN] No valid YAML front matter found
2024/03/26 16:24:24 Error processing file ../gitlab-operator/doc/adr/0015-ci-pipelines-in-dev.md: [WARN] No valid YAML front matter found
INFO: Content updates complete!
vite v5.1.5 building for production...
✓ 1000 modules transformed.
themes/gitlab-docs/static/vite/icons.svg  193.69 kB │ gzip: 45.99 kB
themes/gitlab-docs/static/vite/main.css     7.09 kB │ gzip:  2.15 kB
themes/gitlab-docs/static/vite/main.js    132.86 kB │ gzip: 45.56 kB
✓ built in 1.80s
Watching for changes in /Users/sarahg/dev/gdh/{charts-gitlab,gitlab-docs-hugo,gitlab-operator,gitlab-runner,gitlab,omnibus-gitlab}
Watching for config changes in /Users/sarahg/dev/gdh/gitlab-docs-hugo/config/_default, /Users/sarahg/dev/gdh/gitlab-docs-hugo/go.mod
Start building sites …
hugo v0.123.7-312735366b20d64bd61bff8627f593749f86c964+extended darwin/arm64 BuildDate=2024-03-01T16:16:06Z VendorInfo=gohugoio


                   |  EN
-------------------+-------
  Pages            | 3094
  Paginator pages  |    0
  Non-page files   |  841
  Static files     |  427
  Processed images |    0
  Aliases          |    0
  Cleaned          |    0

Built in 2287 ms
Environment: "development"
Serving pages from disk
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop

Merge request acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Sarah German

Merge request reports