Skip to content

spike: Fetch pipeline graph stages and then job per stage

Frédéric Caplette requested to merge fc-spike-fetch-jobs-per-stage into master

What does this MR do and why?

Exploration for performance improvements to fetch only the stages first, and then allow the user to expand/collapse each column to see all jobs. If you take a look at the videos below, you can notive that we are going down from 16 seconds to 6 for the initial data fetching

Another very convenient improvement would be that we would then be able to paginate jobs per stage, meaning we could easily implement the "load more" button for jobs per column.

The biggest con is polling. Is a user opens all stage columns, and if you have a lot (in this example, there are 40!) that would mean polling for 40 queries 😅 This is a space where switching from polling to subscription probably be necessary to avoid that much overhead OR we would need to build some kind of mecanism where the client polls every 10 seconds on stages and gets an array of values (one per stage) to know if any value has changed since the last poll. Then we could send the update query.

Screenshots or screen recordings

Before After
Screen_Recording_2023-08-17_at_1.05.27_PM Screen_Recording_2023-08-17_at_12.53.20_PM

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports