Marcelo Rivera requested to merge feat/implement-yt-sdk into epic/youtube-migrator-130 Mar 26, 2020

Summary

API for importing videos from YouTube

Closes #1415 (closed) #1417 (closed)

Endpoints

api/v3/media/youtube-importer/account (GET): used when a user tries to connect to his YouTube channel
api/v3/media/youtube-importer/account (DELETE): used when a user tries to disconnect from his YouTube channel
api/v3/media/youtube-importer/account/redirect (GET): automatically called once you log in to YouTube and authorize Minds to use this.
api/v3/media/youtube-importer/videos: retrieves a list of videos (more on this endpoint later)
api/v3/media/youtube-importer/videos/import (POST): imports a video
api/v3/media/youtube-importer/subscribe (POST): subscribes to push notifications for a given YouTube channel. If a new video is uploaded to that channel, we should automatically download it.
api/v3/media/youtube-importer/subscribe (DELETE): unsubscribes from push notifications for a given YouTube channel.
api/v3/media/youtube-importer/hook (GET): gets called by the push notification when something in a subscribed channel gets updated

Setting it up

Create a new credential in the Google Developers Console
Make sure to verify it, otherwise the user will get prompted that it's not verified.
You need to request an access token with an authorization code and set these up: helm-charts!38 (merged)

Registering your channel

Call api/v3/media/youtube-importer/oauth. This endpoint will return a url field which you must redirect to
Go through Google's sign in and it will automatically redirect you to Minds. After this, that youtube channel ID will be associated with your user

Disconnecting from your YouTube channel

Endpoint: api/v3/media/youtube-importer/account (DELETE) Params:

channelId

Getting a list of videos

Endpoint: api/v3/media/youtube-importer/videos (GET) Params:

channelId
status (optional). Possible values: queued, created, transcoding, failed, completed (we will probably only care about queued, transcoding and completed)

When calling it with a null status, it will query YouTube videos using their SDK. For every video returned, we will query Cassandra and see if we have an instance of Entities\Video for it, and return it inside the response. If we don't have such an instance, we will return raw YouTube data. On the other hand, if you do specify a status, it will only query Cassandra for videos.

Importing a video

Endpoint: api/v3/media/youtube-importer/videos/import (POST) Params:

channelId (youtube channel ID)
videoId (youtube video ID)

channelId is verified against the registered channels in the User entity, as we don't allow Users to import from channels that aren't theirs. There's a limit of 10 daily videos that you can import from YouTube. The Video entity will first get saved with a transcoding_status set to queued and, if you haven't yet reached the threshold, it will automatically queue the video for transcoding. In the case you've already surpassed the threshold, there's a YouTubeImporter CLI that I set to run twice a day, that retrieves all queued videos and sends them to the Transcoding queue if the User is now below that daily limit.

Webhook (⚠ I still need to test this one on the review site)

The webhook's located in api/v3/media/youtube-importer/hook.
We have to subscribe to push notifications for each channel we want updates from, and we only take action if there's no associated Minds video to a given YouTube video ID.
The user can only toggle the subscription to that channel if he has already associated it

Subscribing

Endpoint:

api/v3/media/youtube-importer/subscribe (POST) Param:
channelId (youtube's channel ID)

Unsubscribing