Support running GitLab Markdown Golden Master tests against standard GitHub+CommonMark spec.txt
What does this MR do and why?
Related: #345726 (closed)
Provides a process to run the existing backend and frontend Golden Master Markdown testing framework
against the standard GitHub-Flavored Markdown spec.txt
as specified in
https://github.com/github/cmark-gfm/blob/master/test/spec.txt
(which is a strict superset of the CommonMark spec).
NOTE: Currently only supported for backend, frontend tests in
Overview of process:
- Run
scripts/markdown/convert_spec_txt_to_golden_master_examples_yml.rb
to generate the YML file (see file comments for description and usage notes). NOTE: You don't have to do this step, the generated file is already checked in. - Run backend spec against the generated file:
spec/requests/api/markdown_github_golden_master_spec.rb
- Run frontend spec against the generated file:
spec/frontend/content_editor/markdown_github_processing_spec.js
Both specs currently fail on most of the examples due to differences in GitLab's markdown rendering from the original
spec.txt
spec.
TODO
-
Question: Why are there more tests (674 backend, 673 frontend) being run than there are examples in the spec.txt
(649)🤔 ???
Initial commit run spec compliance
- For frontend, 73 out of 673 examples pass. NOTE: These failures are due to HTML-to-Markdown rendering differences, e.g. rendering headers with leading
#
instead of a line of---
underneath. - For backend, 287 of 674 examples pass. NOTE: Some of these may still be false negatives because we need to strip out GitLab-specific metadata attributes such as
id
,class
,href
,aria-hidden
- I've already got a method in place to strip these out, I just need to identify what they all are. However, others seem legit, like self-closing tag differences, etc.).
Latest commit run spec compliance
(may not be up-to-date)
- For backend, 382 of 674 examples now passing
Failures remaining to address
We should look at failures caused by these, and either fix them in the markdown rendering, or else make the spec ignore them:
-
Self-closing tags -
base64-encoded <img>
data -
<img>
tag wrapped in<a>
tag instead of just bare<img>
tag -
Wrapping HTML in <div>\n
- NOTE: Some are already ignored as metadata -
Other extra <div>
tags in HTML (which may have only had ignored metadata attributes) -
Unnecessary extra <span>
tags - NOTE: Some are already ignored as metadata -
???
Screenshots or screen recordings
Example backend results:
Example frontend results:
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.