Tracking Issue: Implement GLFM scripts per the specification guide
Overview
This is the single Tracking Issue for the following tasks. Each individual task will only have MRs to represent the work, not separate issues. See the following thread for more discussion on how we arrived at this approach: &7719 (comment 933806489)
See the parent epic for other Tracking Issues: GitLab Flavored Markdown (GLFM) Specification D... (&7719)
See the GLFM Specification Guide development documentation for much more context on all of this work: https://docs.gitlab.com/ee/development/gitlab_flavored_markdown/specification_guide
Tasks
Main Implementation Tasks
-
Implement GLFM update-specification.rb
script -
Implement GLFM update-snapshots.rb
script (see old issue, closed in favor of this one): -
Add proper support for all new scripts and files to https://gitlab.com/gitlab-org/gitlab/blob/master/tooling/danger/project_helper.rb
. See thread with details: !84347 (comment 936095519): -
Add UPDATE_GHFM_SPEC_TXT
env var, and only download latest GHFMspec.txt
when it is set. Otherwise, use the current version committed to the repo. This will be a guard against potential injection attacks if the canonical GFM spec.txt hosted by GitHub is temporarily compromised. See related threads here and here: -
Finish support for using glfm_example_status.yml
to prevent overwriting manually-edited entries in example snapshot YAML files. -
Improve GLFM static HTML snapshots -
Add Markdown Snapshot spec for GLFM backend/static HTML examples -
Add support for regex-ignoring variable values in static HTML in spec/fixtures/glfm/example_snapshots/html.yml
. This is a port of the "substitutions" logic from Golden Master specs. -
Add Markdown Snapshot spec for GLFM frontend/WYSIWYG HTML examples - MR: Create spec/frontend/content_editor/markdown_sn... (!89953 - merged) (branch
caw-glfm-frontend-example-snapshot-specs
)
- MR: Create spec/frontend/content_editor/markdown_sn... (!89953 - merged) (branch
-
Move everything from /spec/fixtures/glfm
to under/glfm_specification
- Enrique and Himanshi are in favor.- MR: Move GLFM example_snapshots under glfm_specific... (!90554 - merged) (branch
caw-glfm-move-example-snapshots
)
- MR: Move GLFM example_snapshots under glfm_specific... (!90554 - merged) (branch
-
Implement scripts/glfm/run-snapshot-tests.sh
- MR: Add scripts/glfm/run-snapshot-tests.sh (!90555 - merged) (branch
caw-glfm-create-run-snapshot-tests-script
)
- MR: Add scripts/glfm/run-snapshot-tests.sh (!90555 - merged) (branch
-
Make fixes to reduce variability in example snapshots, in order to reduce the need to manually curate normalizations: -
Introduce setting to override random behavior of footnote IDs ( GITLAB_TEST_FOOTNOTE_ID
ENV var)- MR: Allow override of random footnote ID (!90959 - merged) (branch
caw-glfm-allow-override-of-random-footnote-id
) - MR: Set test footnote ID for GLFM snapshot examples (!90992 - merged) (branch
caw-glfm-set-test-footnote-id
)
- MR: Allow override of random footnote ID (!90959 - merged) (branch
-
-
Support GLFM example snapshots in third-level headings - MR: Add support for nesting GLFM examples in H3s (!95302 - merged) (branch
caw-add-glfm-h3-example-support
)
- MR: Add support for nesting GLFM examples in H3s (!95302 - merged) (branch
-
Convert render_static_html.rb
to use rspec, and add deterministic fixtures.- MR: Convert render_static_html.rb to rspec (!95437 - merged) (branch
caw-convert-render-static-html-script-to-rspec
)
- MR: Convert render_static_html.rb to rspec (!95437 - merged) (branch
-
Refactor YAML access and symbols in GLFM scripts and tests - MR: A01. Refactor YAML access and symbols in GLFM s... (!96318 - merged) (branch
caw-glfm-yaml-symbol-refactors
)
- MR: A01. Refactor YAML access and symbols in GLFM s... (!96318 - merged) (branch
-
DRY up fixture creation between spec/support/shared_contexts/markdown_snapshot_shared_examples.rb
andscripts/lib/glfm/render_static_html.rb
. This is necessary to prevent failures when runningspec/support/shared_contexts/markdown_snapshot_shared_examples.rb
due to non-normalized values. This should also include documentation updates to the normalization section to indicate other means we use to avoid the need for normalization (i.e. fixtures, andGITLAB_TEST_FOOTNOTE_ID
)- MR: A02. DRY up and clean up GLFM fixtures (!96325 - merged) (branch:
caw-glfm-cleanup-fixtures
)
- MR: A02. DRY up and clean up GLFM fixtures (!96325 - merged) (branch:
-
Add API request overrides support to GLFM example snapshot specs. This will allow examples which use different preview_markdown
contexts, and also drive support for EE examples such as group wikis.- MR: A03. Add metadata support to GLFM example snaps... (!92507 - merged) (
caw-glfm-add-examples-metadata
)
- MR: A03. Add metadata support to GLFM example snaps... (!92507 - merged) (
-
Raise an exception if any of the manually-curated config files ( glfm_specification/input/gitlab_flavored_markdown/glfm_example_*.yml
) contain references to example names which do not exist inglfm_specification/example_snapshots/examples_index.yml
. This will catch errors when inserting new example header sections that are not at the last/end position, but not updating the manual files with the new renumbered example names.- MR: Add error checking for GLFM config files (!98735 - merged) (branch
caw-glfm-error-checking-example-names
)
- MR: Add error checking for GLFM config files (!98735 - merged) (branch
-
Rename intro.txt
tointro.md
andghfm_spec_v_0.29.txt
toghfm_spec_v_0.29.md
- MR: Rename GLFM input files to *.md (!100227 - merged) (
caw-glfm-rename-md-files
)
- MR: Rename GLFM input files to *.md (!100227 - merged) (
-
Implement split between official specification and internal extensions as documented by Update GLFM docs for official specification vs ... (!98493 - merged) - MR: !100472 (merged) (
caw-glfm-impl-split-official-vs-extensions
)
- MR: !100472 (merged) (
-
Add support for automatically generating spec.html
when runningupdate-specification.rb
- MR: !100489 (merged) (
caw-glfm-generate-spec-html
)
- MR: !100489 (merged) (
-
Setup new CI job which runs the update-specification.rb
andupdate-example-snapshots.rb
scripts, and fails if this results in any diffs to the generated and committed output specification files or example snapshot files. This job will ensure that all changes to the input specification files are reflected in those output files.- MR: !101118 (merged) (branch
caw-glfm-ci
) -
Revisit CI job which was reverted due to problems with docs-only and FOSS-only pipelines: Remove .gitlab-ci.yml for GLFM verification script (!101453 - merged) - MR: !101589 (merged) (branch
caw-glfm-ci-2
)
- MR: !101589 (merged) (branch
- MR: !101118 (merged) (branch
-
Refactor/rename input files as discussed and agreed in this thread: #375300 (comment 1131831270). - MR: Reorganize GLFM input and output file contents (!101413 - merged) (branch
caw-glfm-reorg-input-files
)
- MR: Reorganize GLFM input and output file contents (!101413 - merged) (branch
-
DRY up duplication with import statements and createTestEditor
extensions list betweenspec/frontend/content_editor/render_html_and_json_for_all_examples.js
andspec/frontend/content_editor/services/markdown_serializer_spec.js
- MR: DRY up GLFM tiptapEditor helper (!101868 - merged) (branch
caw-glfm-dry-up-js-extensions
)
- MR: DRY up GLFM tiptapEditor helper (!101868 - merged) (branch
-
TODO: Look into possibility of adding CI rules to force spec/frontend/content_editor/markdown_snapshot_spec.js
frontend spec to run when GLFM specification files change. See !102244 (comment 1150457127) for context. See also ENOENT: no such file or directory, open '/build... (#379080 - closed)- Use imports to solve this: !102244 (comment 1150751210) and !102244 (comment 1150753245)
-
MR: Run markdown_snapshot_spec.js when needed (!102688 - merged) (branch caw-glfm-ci-triggers
)
-
- Also fix for backend:
-
MR: Run markdown_snapshot_spec.rb when needed (!102693 - merged) (branch caw-glfm-ci-triggers-2
)
-
- Use imports to solve this: !102244 (comment 1150751210) and !102244 (comment 1150753245)
-
Perform static analysis to enforce that every existing backend and frontend example has at least one corresponding example in the spec. See related comment-
Need to decide exactly what this means, and if it is even possible. -
glfm_specification/output/glfm_snapshot_spec.md
already contains all defined markdown examples. Do we want to try to somehow look at the existing code, specs, or user-facing markdown docs to try to determine if any examples are missing? How would we even do that? - UPDATE: This doesn't seem feasible/practical. Marking this task as closed without implementation.
-
-
Backfill all missing GLFM examples from existing Golden Master examples, with appropriate(This is being tracked in a separate issue: Tracking Issue: Migrate existing specs Golden M... (#361242 - closed))glfm_example_status.yml
entries for unimplemented/unsupported functionality -
Update spec.html
andsnapshot_spec.html
to use the similar standard formatting, e.g. example numbering, as the official CommonMark/GHFMspec.html
. This will require simulating the formatting of the Lua script and template from the CommonMark project: https://github.com/commonmark/commonmark-spec/blob/master/Makefile#L11. See also this related comment: !98381 (comment 1158434632)- Part 1 - split examples - MR: GLFM HTML formatting Part 1: Split GLFM example... (!102822 - merged) (branch
caw-glfm-format-html-part-1
) - Part 2 - styling, numbering, and table of contents - MR: GLFM HTML formatting Part 2: Styling, numbering... (!103094 - merged) (branch
caw-glfm-format-html-part-2
) - See follow-on tasks below in "Other cleanup tasks" section below
- Part 1 - split examples - MR: GLFM HTML formatting Part 1: Split GLFM example... (!102822 - merged) (branch
-
Review brainstorming document from Himanshu and Chad, and update tasks on this and other issues to reflect and flesh out that new planned direction around the parallel support for the "remark" vs "hybrid" deserializers: https://docs.google.com/document/d/1HUQKQFXYD5r4Rs7bPx96b3z5MpnfNhQNhuVhEBnUe-A/edit -
Implement support for running Markdown Conformance Testing for GLFM implementation against GHFM/CommonMark specification and GLFM spec.txt
.- MR: TODO: (branch
TODO
)
- MR: TODO: (branch
-
By default, any errors from rendering during scripts/glfm/update-example-snapshots.rb
should fail the script. Require a--allow-unskipped-errors
option to be passed, which will have existing behavior of includingError - check implementation
as rendered output. The failure output should include the error message, as well as a pointer to the documentation on how to add an entry to skip the example toglfm_specification/input/gitlab_flavored_markdown/glfm_example_status.yml
-
Changes to example snapshot files should run relevant frontend and backend specs. See https://gitlab.com/gitlab-org/gitlab/-/pipelines/707254208 pipeline in Convert GLFM examples from old GM approach (!104933 - merged) for an example of them not getting run when they should. -
Make update-example-snapshots.rb
check for any git diffs to snapshot files after running, and if there are any, print out a big red warning saying to verify all the changes are exactly as expected, and if not, to fix the problem or add appropriate entries to `glfm_example_status.yml. See related discussion in !106733 (comment 1209495621) -
Add a placeholder script for run-spec-tests.sh
which just prints out a "NOT YET IMPLEMENTED" message with some explanatory text. -
Decide where and how we will host spec.html
andsnapshot_spec.html
. Followup to this thread: !84220 (comment 909137776)- MR: TODO: (branch
TODO
)
- MR: TODO: (branch
Other Documentation Tasks
-
Update GLFM docs to fix flowchart (!86528 - merged) -
Update GLFM docs with new headers (!86604 - merged) -
Add links in GLFM dev guide docs to source for ... (!90562 - merged) -
Make a diagram documenting hierarchy and terminology of various markdown specifications. -
Add a "Summary" section: Add summary to GLFM guide (!98273 - merged) -
Update documentation to distinguish between the GLFM official specification and GLFM internal extensions. This is a follow-up to plans outlined in these threads: -
Fix broken link. See !85982 (comment 1031524242) -
Update https://docs.gitlab.com/ee/user/markdown.html to add a mention of and link to https://docs.gitlab.com/ee/development/gitlab_flavored_markdown/specification_guide/ -
Add/update docs around input specification config files, to reflect changes made in Add error checking for GLFM config files (!98735 - merged) - Add a new
H4
heading after https://docs.gitlab.com/ee/development/gitlab_flavored_markdown/specification_guide/#input-specification-files namedInput Specification Config Files
which containsglfm_example_*.yml
files. - MR: Reorganize GLFM input specification files section (!99078 - merged)
- Add a new
-
Clearly indicate non-implemented parts (e.g. conformance testing). See !85982 (comment 1031528958) -
Add docs for Add error checking for GLFM config files (!98735 - merged), including around special case for Input Specification Config Files where examples starting with 00_
are skipped for validation, and can be used for YAML anchors.- MR: Add docs on error handling for GLFM input speci... (!99083 - merged), Branch:
caw-glfm-add-note-on-00-example-naming
- MR: Add docs on error handling for GLFM input speci... (!99083 - merged), Branch:
-
Fix snapshot_spec filename in GLFM docs (!102825 - merged) -
Add note about run-spec-tests.sh
not yet being implemented, and only showing a placeholder message. -
Remove obsolete note about spec.html: -
TODO: Switch to shortcut reference links as suggested in this comment -
TODO: Document usage and details of snapshot test spec files: - Update of section on
scripts/glfm/run-snapshot-tests.sh
- Add details on CE and EE versions of
spec/requests/api/markdown_snapshot_spec.rb
- Add details on CE and EE versions of
spec/frontend/content_editor/markdown_snapshot_spec.js
- document usage of
FOCUSED_MARKDOWN_EXAMPLES
- Update of section on
-
TODO: Add documentation around details and requirements of spec.txt
header and disabled example parsing that are currently only documented in code/tests inscripts/lib/glfm/parse_examples.rb
andscripts/lib/glfm/update_example_snapshots.rb#add_example_names
. See discussion in these threads: !90561 (comment 999901748) and !90561 (comment 1004779132) -
TODO: Add some text to https://docs.gitlab.com/ee/development/gitlab_flavored_markdown/specification_guide/#markdown-snapshot-testing which explains how our snapshot testing is acting as end-to-end "top of pyramid" integration testing. -
TODO: Make a doc note that only the pre-deserialization, non-dynamic frontend behavior is being tested. -
TODO: Update workflow sections with more details (based on feedback from Enrique) - Add initial instructions on how to add an example
- Make clear that conformance testing is not yet implemented.
- Change split of two workflows. Instead have one workflow for adding examples for already-implemented functionality, and another for adding examples and status entries for TDD-ing functionality that does not yet exist
- Document meaning of tokens and extensions, and special
gitlab
token in backtick-delimited example format - Explain how to make a canonical example. It should be the minimal representation, which is a common subset of the static and wysiwyg HTML that can be derived via normalization. Also document that there are some cases where this may not be possible, such as PlantUML diagrams.
-
TODO: Add a mermaid diagram to docs section for verify-all-generated-files-are-up-to-date.rb
(flowchart showing success/failure cases). -
TODO: Add note to docs section on various specifications about GHFM HTML version being outdated, with link to issue: https://github.com/github/cmark-gfm/issues/288 -
TODO: Revisit terminology of dialect
vs.flavor
. Follow-up from this thread: !84220 (comment 903091005) -
TODO: Add some notes on how and where to update example numberings if you reorder examples in the input markdown files. Use this comment's explanation as a starting point: !106733 (comment 1206573879) -
TODO: Update/add workflow docs with these details: !106733 (comment 1206725109) -
TODO: Update/add some docs (maybe workflow section, maybe other sections) with these explanations of deserialization and related topics: !106733 (comment 1209495621) -
TODO: Update text Goals
section of the Specification Guide and the output specification files section to reflect the fact that it's not justspec.txt
anymore, since that no longer contains CommonMark+GFM examples. -
Update diagram and text in https://docs.gitlab.com/ee/development/gitlab_flavored_markdown/specification_guide/#update-specificationrb-script to reflect the fact that spec.txt
only contains the GLFM official specification, and no longer contains CommonMark+GFM examples or internal extensions. -
Update https://docs.gitlab.com/ee/development/gitlab_flavored_markdown/specification_guide to fix outdated references and charts for scripts and filenames.
Other Cleanup Tasks
-
Support full diffs in fast_spec_helper (!89205 - merged) -
Fix roulette for GLFM scripts (!89222 - merged) -
Fix GLFM typos and lint/inspection warnings (!90560 - merged) -
Fix bug in GLFM spec handling (!90561 - merged) -
Simplify headers logic in parse_examples (!90993 - merged) -
Uncomment jest in markdown_snapshot_spec.rb (!92504 - merged) -
Add some clarification to GLFM fixture (!94905 - merged) -
Remove unnecessary footnote normalization from ... (!95537 - merged) -
Remove unused GLFM example extension annotations (!95761 - merged) -
Minor cleanups to GLFM update_example_snapshots... (!95765 - merged) -
Fix typo in scripts/lib/glfm/render_static_html.rb (!100474 - merged) -
Switch all manual ENV
stubs to usestub_env
helper: Switch to stub_env in update_specification_spec.rb (!101870 - merged) -
Improve output and usefulness of glfm-verify job (!103130 - merged) -
TODO: Extract javascript out of scripts/lib/glfm/specification_html_template.erb
and add unit tests for it. -
TODO: Add formatting to spaces within examples. See TODO comment in CSS in scripts/lib/glfm/specification_html_template.erb
. -
TODO: Refactor parse_examples.rb
to be cleaner and more idiomatic Ru by than original straight port of Python code. Goal: Remove need forrubocop:disable Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity, Metrics/AbcSize
-
TODO: Add "This file was generated, do not edit!" to the top of all generated files -
TODO: DRY up duplication in pattern for raising error if files are empty. See review comment here: !100472 (comment 1135764904). Completed as part of Reorganize GLFM input and output file contents (!101413 - merged) -
TODO: Clean up warnings in JetBrains IDEs and add docs. See start in branch caw-glfm-jetbrains-ide-warnings-cleaup
-
Remove unnecessary CE
reference in spec/frontend/content_editor/markdown_snapshot_spec.js (fixed in Run markdown_snapshot_spec.js when needed (!102688 - merged)) -
Inline spec/frontend/content_editor/markdown_snapshot_spec_helper.js
intospec/frontend/content_editor/markdown_snapshot_spec.js
, because there no longer needs to be a CE vs. EE split on the frontend. -
TODO: Revisit usage of scripts/lib
directory, because it sometimes isn't handled properly by Spring. Possibly needs to be added to Spring config. Follow up to this thread: !84220 (comment 912060119)
Third Party Tasks
-
GitHub Flavored Markdown HTML-rendered spec is out of sync with their latest spec.txt
. See issue: https://github.com/github/cmark-gfm/issues/288. See also slack thread.
Current MR Rebase Branch Queue
placeholder-upstream-1
git co placeholder-upstream-2 && git rebase head~1 --onto placeholder-upstream-1
Edited by Chad Woolley