From Q4 OKR: Complete integration of Optimize experiences into the dashboard framework
The Optimize team has a high profile dashboard, AI Impact Analysis, that has some elements in the framework, and others that have deviated. The goal here is to ensure that this team can completely resolve any deviations, have their dashboards fully integrated with the framework, and that we use that as a marker for understanding the needs of future teams as they also adopt and have special circumstances.
I would like to use this issue to start a discussion with the Optimise team and understand their requirements better, how they deviated from the framework, and what it would mean for team to fully integrate with the framework.
Daniele Rossettichanged title from Discussion: Gather requirements for Customizable Dashboard framework from Optimise to Discussion requirements for Customizable Dashboard framework from Optimise
changed title from Discussion: Gather requirements for Customizable Dashboard framework from Optimise to Discussion requirements for Customizable Dashboard framework from Optimise
Daniele Rossettichanged the descriptionCompare with previous version
This issue was automatically tagged with the label ~"group::certify" by TanukiStan, a machine learning classification model, with a probability of 0.9.
If this label is incorrect, please tag this issue with the correct group label as well as automation:ml wrong to help TanukiStan learn from its mistakes.
Authors who do not have permission to update labels can use the @gitlab-bot label ~"group::<correct group name>" command, or leave the issue to be triaged by group leaders initially assigned by TanukiStan.
This message was generated automatically.
You're welcome to improve it.
@ekigbo@apennells I hear you guys might be the best people to talk to about this? As mentioned in the issue description, I'm trying to understand and scope the work needed to fulfil the "Complete integration of Optimize experiences into the dashboard framework" goal.
I'm relatively new to the dashboarding world within GitLab, and spent the last few weeks playing around with it and building some PoCs by integrating (1,2) it with o11y data. From that, I became aware of some of the pain points that might come out of working with dashboards in the current state. For instance:
Data explorer is tied to PA and it's not straightforward to add other data sources (#500475)
Default filters might not work well for all datasources ( which I see you also bumped into #452228 (closed) )
But I'd like to understand from your side if you have any immediate blockers or high priority requirements that you'd need from the framework in order to fully integrate with it. e.g, having the dashboard and viz designer work with Optimise.
Here's also the summary I've put together of the framework #500821 (comment 2208097749) - there might be some points that you also could find useful.
@drosse in the immediate, Optimize is focused primarily on AI Impact Analytics, with the next goal(s) of splitting the dashboard into two distinct dashboards:
Usage (some design explorations, but not finalized -- #455860 (closed))
Impact (next iteration of the UI/UX - not finalized -- #480770)
I'd say in terms of immediate blockers, I think they would be:
Connecting Optimize data to the dashboard framework
Ensuring support for all the proposed visualizations (e.g. time-series bar chart with x (date), y (metric value), y2 (secondary metric value)
In terms of thinking a bit more holistically about what we would ideally have ASAP:
In dashboard CRUD for tiles
Date range selector, date aggregation options (hourly, daily, weekly, monthly)
Ability to create different "sections" on a dashboard and organize visualizations into each section
Filtering by event meta-data (project, group, attributes, ...)
Open questions that would be good to get answers to sooner rather than later:
When it comes to filtering and date range selection, I can see value in having both dashboard-wide options as well as specific configuration options per tile/visualizaton. I'm not sure if we've decided how we want to support this
Probably more but I'll let @lvanc and @blabuschagne weigh in with more context
When it comes to filtering and date range selection, I can see value in having both dashboard-wide options as well as specific configuration options per tile/visualization. I'm not sure if we've decided how we want to support this
Connecting Optimize data to the dashboard framework
@gweaver as part of this, would you also expect the team to be able to use the Dashboard and Visualisation ( soon to be renamed Data explorer) designer ? i.e. create dashboard/viz through the UI rather than code/yaml?
It would also be useful to understand what were the reasons to deviate from the framework in the first place, for instance when integrating a new datasource, and resolve those issues first.
But that's probably more of a question for @gitlab-org/plan-stage/optimize-group/engineers/frontend I guess.
as part of this, would you also expect the team to be able to use the Dashboard and Visualisation ( soon to be renamed Data explorer) designer ? i.e. create dashboard/viz through the UI rather than code/yaml?
@drosse eventually, but I don't believe the AI Impact Analytics dashboard currently supports any configuration via YML files. THis would ideally be supported sooner rather than later ;)
It would also be useful to understand what were the reasons to deviate from the framework in the first place, for instance when integrating a new datasource, and resolve those issues first.
It would also be useful to understand what were the reasons to deviate from the framework in the first place, for instance when integrating a new datasource, and resolve those issues first.
@drosse Hopefully I can give some insight here. The main reason for the differences here is due to the Value Streams Dashboard having been built as a separate application first, prior to it being integrated into the shared dashboard framework. It was originally built as a standalone analytics dashboard with a customizable YAML configuration. Obviously this carried a fair amount of overlap with the product analytics dashboard framework, so it made sense to move it there.
The migration process wasn't exactly straightforward though, nor could it even really be considered complete. There wasn't a decision made to deviate from the framework, it just happened naturally during the migration. Some things needed to be added to support the VSD (Group-scope analytics dashboards) and some other framework features weren't really compatible (Dashboard-wide date filtering).
I'll try to list out examples of things that still need to be adopted here. These are just off the top of my head, so they may or may not have issues created already:
Replace custom YAML configs with the dashboard designer: The YAML config has additional options not currently supported by the dashboard designer, so additions to the framework would likely be necessary to have feature parity.
Moving data loading out of the component and to a data source: The VSD sends a lot of requests on page load, so we lazy load the results and render them in the table as they resolve. The data-source/visualization pattern used by the dashboard framework loads all data before rendering the components, so again we'd have to make some changes there
Dashboard date filters: The VSD has fixed date-ranges which line up with the table structure. In order to adapt a dashboard-wide date filter like the other analytics dashboards, we'll need to redesign the table structure for different time ranges.
Thanks for jumping in @apennells ! That's super useful.
Replace custom YAML configs with the dashboard designer: The YAML config has additional options not currently supported by the dashboard designer, so additions to the framework would likely be necessary to have feature parity.
Two questions here:
Is the goal here to fully replace YAML-based dashboards with dashboards created through the designer? And when you say dashboard designer, I guess you mean both the viz(currently tied to PA) and dashboard designer?
Moving data loading out of the component and to a data source: The VSD sends a lot of requests on page load, so we lazy load the results and render them in the table as they resolve. The data-source/visualization pattern used by the dashboard framework loads all data before rendering the components, so again we'd have to make some changes there
I'll have to look into this a bit more though. Maybe @rob.hunt has a better context here.
Dashboard date filters: The VSD has fixed date-ranges which line up with the table structure. In order to adapt a dashboard-wide date filter like the other analytics dashboards, we'll need to redesign the table structure for different time ranges.
We discussed adding support to panels where the date is fixed and outside the scope of the dashboard-wide date filter. This is being looked at by @lvanc and @lfarina8 I think.
Thanks @apennells that's a great rundown. I know @lvanc is working on the design side of things for how we handle some of the items that deviated.
I think we have two goals:
Move existing prebuilt dashboards completely to the framework, aka close all gaps, locate and document any issues we encounter along this path that other teams would experience. This will include documenting in Pajamas and moving through to implementation in GitLab UI (@rob.hunt explains this a lot better). Since there are a number of static dashboards that are composed of basic panels, we likely can handle this migration for various teams to help document that workflow for simple dashboards that exist under the Analyze header.
Add new data sources for querying and custom dashboard creation. This means using the Data Explorer (formerly known as viz designer) to unlock the ability for customers to create their own dashboards by mixing and matching data from any supported sources. By the end of this quarter, we would hope that a customer could create a basic dashboard using Optimize data. If there are specific requirements that might mean that we need to keep this hidden, no problem, but ultimately, we will only be looking at Optimize data for now. Product analytics and o11y will be internal only features, and won't be prioritized as data sources in the custom dashboard creation for now. Hopefully, that keeps things focused.
Thanks @lfarina8! The goals make much sense. Let's focus on the first goal then!
Move existing prebuilt dashboards completely to the framework,
By "existing prebuilt dashboards" I guess we are referring to the value stream dashboard and ai impact dashboard. Are there more?
And who owns the migration? Is it going to be a shared effort between PI and Optimise?
@apennells are there other blockers ( in addition to the ones you listed above ) that come to mind? If there are game stoppers that you are already aware of that's good, otherwise I think we can just pick up one of the dashboard and get started with the migration. I'm sure we'll find those blockers quite quickly.
aka close all gaps, locate and document any issues we encounter along this path that other teams would experience.
This has partially been done by my spike integration of o11y with dashboards (#500821 (comment 2208097749)). But I'm sure we will identify more points when migrating Optimise dashboards.
This will include documenting in Pajamas and moving through to implementation in GitLab UI (@rob.hunt explains this a lot better).
I've created an &15999 for this work. I also think this could happen in parallel ideally and it's not necessarily a blocker to start the porting.
By "existing prebuilt dashboards" I guess we are referring to the value stream dashboard and ai impact dashboard. Are there more?
I think this is referenced to all static dashboards currently implemented across the product, i.e. basically everything under the Analyze menu item.
Shared ownership makes sense and this is already part of groupoptimize's technical roadmap.
One additional blocker for migrating some dashboards is that the framework is currently supported only on GitLab Ultimate. I've opened Dashboarding framework for premium features (#503761) to discuss this topic further. But this isn't an immediate need, not until we start focusing on dashboards other than VSD and AI Impact.
are there other blockers ( in addition to the ones you listed above ) that come to mind?
@drosseThe link that Ezekiel posted sums up his investigation on the matter. One item that's worth highlighting is that the dashboard designer only currently works at the project level. groupoptimize is responsible for group level dashboards, but a side-effect is that the designer was never implemented there.
Daniele Rossettichanged title from Discussion requirements for Customizable Dashboard framework from Optimise to Discussion requirements for Customizable Dashboard framework with Optimise team
changed title from Discussion requirements for Customizable Dashboard framework from Optimise to Discussion requirements for Customizable Dashboard framework with Optimise team
Is the goal here to fully replace YAML-based dashboards with dashboards created through the designer? And when you say dashboard designer, I guess you mean both the viz(currently tied to PA) and dashboard designer?
@drosse I'd imagine that we'd like to fully replace it so we don't have multiple solutions to the same use-case, but I'll defer to @gweaver or @blabuschagne for that. Dashboard designer is our primary goal. Viz designer is on our radar, but isn't our focus currently.
I think we should rework all this naming to be less confusing, cause even I'm confused . I consider all modifications dashboard designing (moving things around, filtering, adding new tiles from library, querying data and adding a new viz, etc.)
Dashboard designer-- Minimal edits allowed for GitLab prebuilt dashboards, engaged by user hitting Edit on an existing dashboard. I'd really love to revisit the entire workflow around the "Add new viz" option. Are you envisioning having a list of prebuilt charts here? My vision was that this step below would actually take you to the data explorer instead of a long scroll list of possible random chart names.
Data Explorer (Viz designer)- Ability to explore data, write queries, and add new visualizations to a dashboard. This can happen from a Create New on dashboard list, directly from the Add new viz on the edit dashboard flow from prebuilt or cloned dashboards. We need to do a lot of clean up to this workflow.
I think we should rework all this naming to be less confusing, cause even I'm confused
Yes please
And when we talk about the framework I think we should start making the distinction between the Customizable Dashboard framework, which is purely made of data-agnostic UI components (eventually to be moved to GitLabUI) and the Analytics Dashboard framework: opinionated framework built using the Customizable Dashboard UI framework, which handles fetching data from different sources, filtering, includes the Data explorer and the dashboard designer etc. I've created some issues to update the docs and clarify this (#505304 (closed))
@lfarina8 We'll align behind whatever naming convention ya'll want to move forward with. I, too, think the distinctions are a bit...confusing. IMHO, there are a few basic workflows for dashboards:
Create/edit chart - Title + query (events + filters + aggregation/formulas) + select visualization (line, table, bar, ...). Something similar to Snowflake (less DB query and more user friendly / non-developer approachable)
Organize charts in a dashboard - resize, drag and drop placement, group into different sections.
Share/embed - get embed code or workflows for incorporating existing charts into different areas of the product (issue descriptions, wiki, ...)
Minimal edits allowed for GitLab prebuilt dashboards, engaged by user hitting Edit on an existing dashboard.
Is there a difference between a "pre-built" (GitLab-built) dashboard and dashboards we ship by default that we want to allow users full control over editing? I think so...
GitLab-built: These dashboards/charts are integrated into different parts of the product and we want to limit the ability for users to permanently mess them up. In this case, I would expect basic controls over the chart data (time range, hide/show series, ....) but be able to edit the underlying events being shown, permanently over-write default view settings, ...
Dashboards we ship by default - Things that we recommend as "best practices" so customers don't have to set anything up, but we give full control to the customer to edit charts, remove charts, delete the dashboard, ...
Just realized I accidentally started a new thread when I meant to reply to the previous one
Are you envisioning having a list of prebuilt charts here?
@lfarina8 This is our immediate goal. The panels shown in the VSD should be accessible when a user creates a custom dashboard. Seems like a good first step in the right direction
Agreed @gweaver, really we should think about this all as dashboard designing and the list of actions at each level. I don't think we really can say that a user can truly build custom dashboards until they can query the data however they'd like.
From dashboard listing page
Create new
View existing (user can toggle date/time or filters if applicable on that dashboard, they can also hit the Edit button)
Clone existing (create a new dashboard by making a copy of an existing dashboard)
Delete dashboard
Actions on Edit existing:
Change name/description
Add new visualization (current flow allows user to pick from list of panels (aka queries) available, but they cannot query the data themselves)
Change layout (drag/drop, resize charts)
In the future, we will unlock the data explorer capabilities and that will change the flow of the add new visualization, but I think we need to sort out how we want this to work, and will need designs.
The GitLab built label is on dashboard that are "locked" in the Dashboards listing view. You'll see that none of those have the option to edit, the user can however, clone them as a starting point to creating a new dashboard that they can modify.