Support wiki processors

Problem to solve

As a user, I want to read and share structured content inlined in issue text and comments more easily. The content such as CSV, diffs, code sections from VCS history, sanitized HTML, rst rendering, etc. As an admin, I also want to extend with custom processors for such content.

Some examples of what is possible.

• https://trac.edgewall.org/wiki/WikiProcessors
• https://trac-hacks.org/wiki/FoldMacroProcessorMacro
• https://trac-hacks.org/wiki/GanttChartPlugin
• https://trac-hacks.org/wiki/PerlPodMacro
• https://trac-hacks.org/wiki/ScrippetMacro
• https://trac-hacks.org/wiki/TracCodeProcessorMacro
• https://trac-hacks.org/wiki/TracBlockDiagPlugin
• https://trac-hacks.org/wiki/TracMathJaxPlugin
• https://trac-hacks.org/wiki/TracYetAnotherChartMacro

As a developer, I also want to reuse Trac plugins to move issues to GitLab.

Proposal

Use either Trac {{{ syntax or extend ``` from GFM.

Further details

Permissions and Security

Links / references

• #14174 (closed)

changed the description

mentioned in issue gitlab-org/quality/triage-reports#189 (closed)

mentioned in issue #22156

mentioned in issue gitlab-org/quality/triage-reports#190 (closed)

/cc @cdybenko

added Category:Wiki devopscreate + 1 deleted label

changed the description

changed milestone to %Backlog

added typefeature label

@abitrolly can you speak more about these specific use cases in a wiki?

Do you know if this can run as a layer on top of any markdown format or does it need to be its own markdown type?

cc @toupeira

added [deprecated] Accepting merge requests label

@cdybenko this can be run on top of any markdown format, but custom code is needed. The most important take away is that people can add their own extensions.

For the specific use case I need, to render CSV with csv processor I would add the following block to issue text.

{{{
!#csv
id,funding_session_id,state,user_id
266,22,uninitialized,
262,20,not_charged,
276,23,uninitialized,
}}}

And it will be rendered as.

id	funding_session_id	state	user_id
266	22	uninitialized
262	20	not_charged
276	23	uninitialized

From which I can easily spot that user_id is empty.

Other use cases are well described at https://trac.edgewall.org/wiki/WikiProcessors page. Notable examples are rendering code sections.

And diffs.

@abitrolly thanks for opening this issue! I remember Trac fondly

There's a few considerations around this:

• Any new syntax we introduce should be supported in all Markdown fields, not just wikis.
• In wikis we also need to support non-Markdown formats like AsciiDoc, ReST, org-mode.
• User-extensible syntax would be very awesome, but also brings up a lot of questions around security and maintainability.

For some of this it's probably best to reuse code block syntax, e.g. for CSV files you'd use (in Markdown)

```csv

The advantage is that all markup formats we support already have a construct for this, so we can have a generic processor that operates on the rendered HTML (e.g. our Mermaid diagrams currently work this way).

Diffs can be rendered with a patch code block, but we could definitely extend this to automatically embed a snippet when linking to code lines (looks like that's covered by #22156).

As a developer, I also want to reuse Trac plugins to move issues to GitLab.

Did you mean importing this kind of markup from Trac? Not sure how feasible that is as some of it will have other dependencies (e.g. the diagram plugins).

marked this issue as related to #217614 (closed)

I thought that all GitLab specific markdown rendering already goes through the same pipeline, and there is no need to care about every field. In fact, I didn't think about GitLab Wiki at all - my primary usage for this is within the issues. It is just that Trac names these markup processors as Wiki Processors, and Wiki is the central part of each Trac project.

Given the above, I am not sure it worth to add this functionality to GitLab Wiki because that will make the content GitLab specific, and people will not be able to convert it into static website, or migrate to different platform without losing the functionality. I would keep those processors limited to issues and repo data structure. An aid for looking at the data, but not the primary tool for authoring content.

Security and maintainability is a question of support. Official extensions are no less secure that GitLab code. And unofficial extensions can be tested by security professionals of the company what uses them.

Not sure how feasible that is as some of it will have other dependencies (e.g. the diagram plugins).

Process containers (docker, podman) are already there, so I see no problems with that. The only problem is the extension interface that allows GitLab to start and use these services with dependencies. Common interface for a call, and for a lifecycle of this single function container.

mentioned in issue #228538 (closed)

mentioned in merge request !39055 (closed)

added sectiondev label

added groupeditor [DEPRECATED] label and removed 1 deleted label

Related #335992

mentioned in issue #335992

added groupknowledge label and removed groupeditor [DEPRECATED] label

added devopsplan label and removed devopscreate label