Master

Newer version of MR !282 (closed) Mark's latest code for Deamos to review.

• Much faster page loads. Mostly by reducing the number of database calls used (DarkCow did 60 database calls to load the main page for example whereas my fork of Darkcow now does 6 database calls to display main ,V2 did 26)
• Onetimestartup service which cleans up things like active streams on a restart if you want that.
• trimUnused python which recursively searches your /vaw/www/videos dir for things which are not known to the database and deletes them.
• Faster string handling used in places.
• More fault tolerant video deletion.
• Ability for admins to delete any video via admin controls.
• New xDrDarkDoc deployment which is a fork of Moo's lovely Darkcow. DarkDoc works well on mobile as channelplayer.html has been written to use viewport scaling so it works in all resolutions and screen sizes. This deployment runs the fastest and is the one I tested the most, but my performance changes have mostly been applied to all deployments.
• Scrolling front page carousel which shows who is currently live.
• Optimisation of database calls.
• Added sysSettings to redis. This greatly reduces the number of database calls needed.
• Use "pottery Redlock" to protect critical section.
• Optimised notifications
• Fixed top bar on all pages and sidebar on xDrDarkDoc channel player. Sidebar shows who is live along with their live gif and also who the user is following.
• "Verified" badge can be given to streamers and shows up in xDrDarkDoc channelplayer.html. This allows them more restreams and to appear in "recent" on the front page.
• Stripe credit card payments can be taken as xDrDarkDoc has the ability to link to Stripe, We took real payments using Stripe using this.
• Limit number of notifications / webhooks emails which can be sent out in time to stop spam.
• Channels which don't have publish set now don't send notifications & emails to allow them to be used for direct streaming which does not cause spam.
• Faster handling of 404 and 500 errors to make it harder to DDoS us.
• "420 dicks" file used in app.py. Code is commented out but it's now easy to add back and see what gets into flask which should not have. At Prohibited this caught lots of stuff...
• Clickable links in chat (channel owner can turn them off in channel settings).
• More easy to find / stream keys / rtmp key in my channels
• 5 recent and 10 random videos shown on front page of xDrDarkDoc.
• RTMP settings stored locally to save getting them from the database all the time (this will not work with edge and proxy and will need reworking for that).
• Fixed bug which caused vanity links to not work correctly.
• Limit number of restreams a user can have to 2 for "not verified" and 5 for verified. Also stopped looping back of restreams to channels on the same server which previously could be used to maliciously crash the server.
• Removed "user left" message from chat.
• Various bug fixes
• Various optimisations
• Use the "getgit.sh" bash script (found in /opt/osp) to update from an existing 8.7 or 8.8 OSP to my code. This script will get the code and (if you are running a local DB) make the required database changes to your database to make it compatible by calling "aftergetgit.sh". If you try this obviously back stuff up first as it's not well tested..
• Real-time live and following sidebar on every page.
• Real-time live scrolling carousel on the main page
• "Live now" is now real-time on the the main page.
• The front page live now, carousel and live part of the sidebar are the same for all users, so when a user loads a page the code renders it in the back end and stores the rendered template in redis. Anyone else who then loads the sidebar / main page live feed / slider within 6 seconds does not need to call the database to do it... They don't even need to render the template again. It's already stored in the servers ram and it just feeds it out to everyone already rendered... Then when after 6 seconds the redis dies and the next person who tries to get the data will actually hit the database to get fresh data, render the templates again and store them in redis for everyone else to access later...
• Reduced the number of database calls done while on channel player. We were polling doing several rather expensive database calls including one in particular to channel player which was really slow and I have managed to reduce those down to one socket IO call. The results of this are stored in redis so that everyone else who is viewing that channel shares the same redis store and so there is much less thrashing of the database.
• Added the ability for admins to set donation url links in admin settings.
• Various other optimisations
• Currently I'm working on the actual playback system. Previously we had a 10 second timer which governed things and I'm working on trying to make the lag lower and connections more consistent and faster. I think the playback needs to check the state of the player before we start to play cos it fails sometimes and so that's what I'm working on now...