Report on pages not in nav

added Technical Writing typemaintenance labels

changed the description

Hi @sarahgerman I opened this issue after a discussion with @dianalogan. I think we might have a "pages not in nav" report somewhere already, but I'm not quite sure. It would be nice to periodically bring it to the team's attention so we can keep it cleaned up.

Copying @claytoncornell because I know you worked on this, @axil because you might have script ideas, and @susantacker for visibility.

Thanks.

I think slide 23 of Craig's deck on the content audit might help.

@sselhorn I remember a related issue that we talked about this again

I think I had proposed something like, make a list of the Markdown files and then compare them with what we have in navigation.yaml.

@sselhorn @axil I'd assumed that we ran a script or something to generate this report, and that we could turn that logic into a script like:

A script like that is easy to call locally and easy to call in CI/CD job.

I'd like to get to a place where we script things like this, but automatically create issues for the community to help address them.

Or maybe we get to a point where we have the exception list synced up with the nav and we could fail the pipeline or produce a warning...

@sselhorn @sarahgerman @axil

Just to make a bit of a start on scripting, I've come up with some rough CLI commands to generate lists we can compare against:

To get a list of all the pages in the global nav, you can run the following command the root of gitlab-docs:
```
sed -n -E "s/.*[section|category|doc|]_url: '(.*)'/\1/p" content/_data/navigation.yaml | sort
```
That hopefully works on both macOS and Linux.
To get a list of all the HTML files we publish in the public directory, run the following in the the public subdirectory of a fully compiled documentation site:
```
find * -name "*.html" | sort
```

The biggest problem with lists these generate is that index.html file names are absent in the global nav (per: https://gitlab.com/gitlab-org/gitlab-docs/-/blob/21b197f9ceb69d3507065185106523a9c714e3e6/content/_data/navigation.yaml#L8), so they would have to be handled differently at the moment. That also affects sorting, FYI.

Another alternative for processing navigation.yaml is a tool like yq: https://mikefarah.gitbook.io/yq/. I thought it would be better to use a YAML-specific tool for this, but when I had a very quick look at it, I couldn't see any specific advantage for our purposes.

@eread Before running your first sed, just do an extra find and replace -> Find: /', replace with /index.html', then you should get a full list with the next step. Output those two sorted files, then compare them to each other. If they don't match, we've found what's not in the nav automatically. Then compare those to an exception list?

This definitely seems doable with just a few script steps. We could add a job to the pipeline that is allowed to fail. We wouldn't break master or block deployments, but we'd notice pretty quick and be able to check the change?

Before running your first sed, just do an extra find and replace -> Find: /', replace with /index.html', then you should get a full list with the next step.

@marcel.amirault I should've said the biggest problem with my commands

I posted them here in their raw format so the whole team could help refine, making it more likely will build a solution without it consuming too much of one person's time.

This definitely seems doable with just a few script steps.

Yeah, I like the idea of using a Bash script, which we can hook up into Makefile (set a dependency on make compile) and also CI/CD.

The script can have the exception list.

I'm very intrigued into turning this into a rake task

Your sed command is a good first step

@axil I went ahead and raised: !2993 (merged). I went with make instead of rake.

I'll probably extend it later to include this check: https://gitlab.com/gitlab-org/gitlab-docs/-/blob/b9a174871c524f7fe696a8c9a8ffb5e721609709/.gitlab/ci/test.gitlab-ci.yml#L37. I'd like to see all these checks:

Manually runnable with make test.
Run on git push when folks have turned on Lefthook.
Runnable in the pipeline.

Nice idea! Sad lost pages are no good. This doesn't seem too tough to report on. I'll work with Susan to prioritize.

added Category:Docs Site label

added workflowready for development label

@sarahgerman This was the spereadsheet that Craig used to track orphaned pages: https://docs.google.com/spreadsheets/d/1Ar87vjcliYlCBER99DcX_9-YjxGUu7xmFCvAzD01Vrk/edit#gid=722849967 I'm not usre what he did to maintain it... it was a manual process

WHOA! That spreadsheet is a beast! Thanks for posting this, it's a very helpful reference. I like the idea of outputting results of a report like this in CSV format. I'll see what I can do. This will be fun!

@sarahgerman When Craig left, this spreadsheet was assigned to me to maintain. I haven't had time in the past few months, but I tried a few days ago and realized a lot needs to be done to bring it up-to-date. I'll likely create a snapshot of it and start fresh with April data. Craig gave us the steps in the presentation Diana linked above, and I have a video where he explained it and some notes in a Google doc...

I'm just telling you all of this because it sounds like you might want/be able to automate it? If that's the case, let me know and I'm happy to brain dump what I know onto you...

(This is a separate request from comparing the list of files in the navigation.yml to the list of markdown files in the four docs projects.)

@sselhorn I would love to automate whatever we can in the content audit. I have a lot of experience writing scripts to wrangle data for reporting/analysis. Maybe when you get into updating the content audit we can write up issues for the pieces that would be impactful to have a script or automation for.

For now I'll work on the issue at hand with the navigation items.

An idea: adding a line to the metadata for pages we KNOW shouldn't be in the left nav:

navigation: excluded

It feels like we'd need two checks to be complete:

Inclusion: are all files eligible to be in the left nav, present in the left nav? (Has anyone forgotten to add a page to the nav?)
Exclusion: do the number of files currently excluded pages match the previously known number of exclusions? (Has anyone sneaked in another exclusion?)