Feat/metascraper stage 2 front#5815
Ticket(s) / Related Merge Requests
- Issue card front#5815 (closed)
- minds branch (add to start command) - minds!209 (merged)
Summary of Changes
Use Metascraper for parsing Twitter and Youtube importers.
Testing Considerations
This is all going to be quite tricky to test on sandboxes, however:
Youtube
source /vault/secrets/env && php cli.php YouTubeImporter import --video_id='4drttTSsWjM' --channel_id='UCGksFXDnH3swdUfRV6Dhs3w'`
Check /nemofin
to see the video. To repeat, you will need to delete Redis to it reboots due to rate limiting.
Alternatively do it yourself, connect a Youtube channel and then swap the params in the above command to post it to your channel.
Note the thumbnail isn't appearing - I believe this to be a CDN issue because it also doesn't work for newsfeed preview which isn't touched by these changes when on sandbox BUT the thumbnail works fine on newsfeed post preview on production and locally.
To get TwitterSync to work I've had to do quite a bit of hacking around bits of code in the pods engine branch to make the request work, manually inserting the user GUID into the URL etc, and added various var_dumps.
For some reason my Twitter account is linked to https://feat-metascraper-stage-2-5815.minds.io/tanyatest4/
I've set it so the timestamp is ALWAYS the start of this month, so you can run the below command repeatedly and it should sync the same tweets to that channel.
source /vault/secrets/env && php cli.php TwitterSync sync
From above:
Note the thumbnail isn't appearing - I believe this to be a CDN issue because it also doesn't work for newsfeed preview which isn't touched by these changes when on sandbox BUT the thumbnail works fine on newsfeed post preview on production and locally.
Similar here, it works on prods newsfeed and locally but not on sandboxes. HOWEVER there are KNOWN issues with Metascraper and Twitter so it would not surprise me if some do not work. See: https://github.com/microlinkhq/metascraper/issues/260
Deployment Considerations
Needs deploying to runners. Gated by feature flag front-5815-metascraper-stage-2 which will need flipping.
Be aware stage 1 (direct feed posting / preview endpoint) is controlled by front-5392-metascraper-previews
Regression Scope
Youtube Importer and Twitter Sync
Platform Affected (web, mobile, etc)
Runners
Developer Testing Completed
Manual, unit.
Screenshots / Screen Recording
N/A
Does this impact
-
Localization -
Dark/light mode -
Guest mode
Definition of Done Checklist
-
The Acceptance Criteria has been met -
Code is tested: Testing includes unit/spec, E2E/automated and manual testing -
Merge requests description has been filled out