Feature request: Embedding rich media in GFM
It would be nice to have the ability to directly embed things like tweets, (publicly accessible) slack messages, youtube videos, etc. with GitLab flavored markdown.
We could use the image embed syntax and automatically recognize certain URLs:
![twitter](https://twitter.com/gitlab/status/867954134191718400)
could show up as:
I think this would be a good MVP, but we could expand on this further, but auto-recognizing normal link URLs and automatically providing previews for them, like slack and twitter do themselves:
# here's a header
here's a link:
https://youtu.be/UFhHetf7kHM
^ I think you should watch this video
could render as
Attachments approach above may require extra work. We can rely on oEmbed to control how things are rendered inside GitLab, without having to code each one of them.
Here is a proposal using some pre-defined oEmbed services:
We could initially support these services (they all have oEmbed support) and expand/test more in the future:
- Youtube
- Slideshare
- Speakerdeck
oEmbed works by visiting the resource URL, parsing the HTML looking for either a <link rel="alternate" type="application/json+oembed" href=...>
or a <link rel="alternate" type="text/xml+oembed" href=...>
tag that will provide discovery to the API endpoint responsible for returning embeddable HTML snippet.
Because this will generate HTML, we have to trust the other peer, as we are potentially opening the door for a CSRF attack. By hard-coding a whitelist of services, we can still use generic oEmbed libraries that will handle the oEmbed protocol/steps.
There are pros and cons on using either the embed syntax: ![]()
or just the text/link alone.
Implementations like the oEmbed in WordPress, either rely on special tags like: [embed]url[embed]
, or [youtube]url[youtube]
, or they rely on just the URL being inserted in a single line.
The idea here is that you may have valid reasons to place the URL as part of the text, and don't want that to be turned into a rich component. So we should respect users intention here.
In that way we can decide on two simple rules for when to trigger oEmbed:
- Link for a whitelisted service in its own paragraph alone
- using Embedding syntax.
In this example:
![youtube](https://www.youtube.com/watch?v=zR7kMUejooU)
is rendered in HTML as:
<a class="no-attachment-icon" href="https://www.youtube.com/watch?v=zR7kMUejooU" target="_blank" rel="nofollow noreferrer noopener"><img src="https://www.youtube.com/watch?v=zR7kMUejooU" alt="youtube" class="js-lazy-loaded qa-js-lazy-loaded"></a>
That means we will have to parse in the HTML pipelines for <a href=...><img src=...></a>
and check the URLs to see if they are one of the embeddable ones and then trigger/rewrite that piece when rendering the final version.
Because oEmbed requires external requests to the services, we should cache/store that on the database, so we do that only once and refresh when needed.
As the embedding can happen in any place where we have markdown, we can potentially have a table with the original URL and the intended snippet to be used.
here is a draft:
Table: oembeddables
Fields:
-
uuid
(to be used with the oEmbed tag, not using sequential IDs to reduce the possibility of enumeration attacks) -
normalized_url
(keep only the identifying part, remove any extra anchors or extra marketing query strings) -
oembed_html
(this is the result returned from the service) -
updated_at
(we can use this to trigger refreshes whenever needed... we can use a default of 30 days before refreshing the oembed snippet)
Whenever our parser finds services that can be "oembeddable", it should first query the table to look for existing items, if it finds and it's not "expired", just reuse them. If it's expired, triggers a refresh. If its a new url, add it to the table and trigger a refresh (oembed_html
will be null until oembed fetchs it)
The rendered HTML should ideally contain only a reference to this table, in order to allow the embeddable content to be refreshed later on. We could rely here on custom HTML tag that will fetch from a special endpoint in GitLab.
Here is an example of a markdown content with before and after the transformation:
Please take a look at this **YouTube** video about GitLab Commit event: ![](https://www.youtube.com/watch?v=VBxnxJqWo3o)
Please take a look at this **YouTube** video about GitLab Commit event: <gitlab-oembed>40e6215d-b5c6-4896-987c-f30f3678f608<gitlab-oembed>
in a oembeddable table:
uuid="40e6215d-b5c6-4896-987c-f30f3678f608"
normalized_url="www.youtube.com/watch?v=VBxnxJqWo3o"
oembed_html="<iframe width="560" height="315" src="https://www.youtube.com/embed/VBxnxJqWo3o" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>"
<gitlab-oembed>
tag could cache the HTML on users browser as well (Local Storage), and only when it didn't find it there it would query GitLab for the content. We can enable a graphql endpoint that would respond with oembed_html
for each given UUIDs.