Improve Analytics dashboards schema errors

What does this MR do and why?

Replaces the vague GraphQL error messages returned when an Analytics dashboard configuration is invalid with friendlier error messages from the new errors field in the CustomizableDashboard GraphQL object type to display in the UI when they are present. In order to bypass the default GraphQL error handling to display these schema errors, this MR allows any fields coming from the YAML config – specifically from ProductAnalytics::Dashboard and ProductAnalytics::PanelType – to be nullable.

This MR also does the following:

• Adds a Contains errors badge to any invalid dashboards listed in the Analytics dashboards page. Previously, no errors were displayed.
• Updates all built-in dashboard configurations – Value Streams Dashboard, Behavior, Audience, AI Impact – to ensure that they match the dashboard JSON schema defined in ee/app/validators/json_schemas/analytics_dashboard.json.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

View	Before	After
Dashboards List
Dashboard

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

1. Please make sure that you have a GitLab Ultimate license.

2. Create a group or select an existing one

3. Follow the instructions to add a YAML configuration for the group and use the example below for your schema:

   title: 'Custom Dashboard - Test VSD'
   description: 'Custom description'
   panels: 
     - visualization: dora_chart # broken panel - missing `title` and `gridAttributes`
       queryOverrides: {}

4. Navigate to Analyze > Analytics Dashboards in the left hand menu

5. Verify that a Contains errors badge should appear next to Custom Dashboard - Test VSD

6. Select Custom Dashboard - Test VSD

7. Verify that an alert appears listing all of the dashboard schema's errors with a link to the troubleshooting section in the Analytics dashboards doc

Related to #444915 (closed)

changed milestone to %17.1

added devopsplan featureenhancement frontend groupoptimize priority2 sectiondev typefeature workflowin dev labels

assigned to @rcrespo3

added backend documentation labels

	1 Warning
	This merge request contains lines with testid selectors. Please ensure e2e:package-and-test job is run.

	1 Message
	This merge request adds or changes documentation files. A review from the Technical Writing team before you merge is recommended. Reviews can happen after you merge.

Documentation review

The following files require a review from a technical writer:

• doc/api/graphql/reference/index.md (Link to current live version)
• doc/user/analytics/analytics_dashboards.md (Link to current live version)
• doc/user/analytics/value_streams_dashboard.md (Link to current live version)

The review does not need to block merging this merge request. See the:

• Metadata for the *.md files that you've changed. The first few lines of each *.md file identify the stage and group most closely associated with your docs change.
• The Technical Writer assigned for that stage and group.
• Documentation workflows for information on when to assign a merge request for review.

Multiversion compatibility

This merge request updates GraphQL backend and frontend code.

To prevent an incident, ensure the updated frontend code is backwards compatible.

For more information, see the multiversion compatibility documentation.

testid selectors

The following changed lines in this MR contain testid selectors:

ee/app/assets/javascripts/analytics/analytics_dashboards/components/list/dashboard_list_item.vue

+            data-testid="dashboard-errors-badge"

ee/app/assets/javascripts/analytics/analytics_dashboards/components/analytics_dashboard.vue

+        data-testid="analytics-dashboard-invalid-config-errors"

If the e2e:package-and-test job in the qa stage has run automatically, please ensure the tests are passing. If the job has not run, please start the manual:e2e-test-pipeline-generate job in the prepare stage and ensure the tests in follow-up:e2e:package-and-test-ee pipeline are passing.

For the list of known failures please refer to the latest pipeline triage issue.

If your changes are under a feature flag, please check our Testing with feature flags documentation for instructions.

Reviewer roulette

Category	Reviewer	Maintainer
backend	@sam.figueroa (UTC+0, 4 hours ahead of author)	@engwan (UTC+8, 12 hours ahead of author)
frontend	@minahilnichols (UTC-4, same timezone as author)	@justin_ho (UTC+7, 11 hours ahead of author)
UX	@lvanc (UTC-10, 6 hours behind author)	Maintainer review is optional for UX

Please check reviewer's status!

• Reviewer is available!
• Reviewer is unavailable!

Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

added pipelinetier-1 label

Bundle size analysis [beta]

This compares changes in bundle size for entry points between the commits 97740bd8 and 199f8d14

Special assets

Entrypoint / Name	Size before	Size after	Diff	Diff in percent
average	4.3 MB	4.3 MB	-	0.0 %
mainChunk	3.3 MB	3.3 MB	-	0.0 %

---

Note: We do not have exact data for 97740bd8. So we have used data from: 5857a69c.
The target commit was too new, so we used the latest commit from master we have info on.
It might help to rerun the bundle-size-review job
This might mean that you have a few false positives in this report. If something unrelated to your code changes is reported, you can check this comparison in order to see if they caused this change.

Please look at the full report for more details

---

Read more about how this report works.

Generated by Danger

changed the description

changed the description

added Technical Writing label

changed the description

added UX label

marked this merge request as ready

changed the description

Hey @apennells, when you get the chance, could you please do the initial frontend review? Once done, could you please pass it over to @ekigbo? Thanks!

requested review from @apennells

Hey @lvanc, could you please do the UX review here? Thanks!

requested review from @lvanc

Hey @lciutacu, when you get the chance, could you please do the Technical Writing review? Thanks!

requested review from @lciutacu

Hey @tyleramos, hope all is well! When you get the chance, could you please do the backend review? Once done, could you please pass it over to @felipe_artur for the backend maintainer review? Thanks!

requested review from @tyleramos

added workflowin review label and removed workflowin dev label

changed the description

approved this merge request

removed review request for @lciutacu

added pipeline:mr-approved label

added pipelinetier-2 label and removed pipelinetier-1 label

@lciutacu, thanks for approving this merge request.

This is the first time the merge request has been approved. To ensure we don't only run predictive pipelines, and we don't break master, a new pipeline will be started shortly.

Please wait for the pipeline to start before resolving this discussion and set auto-merge for the new pipeline. See merging a merge request for more details.

E2E Test Result Summary

allure-report-publisher generated test report!

e2e-test-on-gdk: test report for 199f8d14

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Create      | 110    | 0      | 10      | 93    | 120   | ✅     |
| Verify      | 31     | 0      | 1       | 30    | 32    | ✅     |
| Govern      | 64     | 0      | 1       | 41    | 65    | ✅     |
| Plan        | 54     | 0      | 2       | 47    | 56    | ✅     |
| Data Stores | 31     | 0      | 0       | 22    | 31    | ✅     |
| Analytics   | 1      | 0      | 1       | 0     | 2     | ✅     |
| Package     | 19     | 0      | 12      | 19    | 31    | ✅     |
| Monitor     | 8      | 0      | 0       | 7     | 8     | ✅     |
| Release     | 5      | 0      | 0       | 5     | 5     | ✅     |
| Manage      | 0      | 0      | 1       | 0     | 1     | ➖     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 323    | 0      | 28      | 264   | 351   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

e2e-package-and-test: test report for 199f8d14

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Plan        | 127    | 0      | 11      | 0     | 138   | ✅     |
| Create      | 318    | 0      | 36      | 6     | 354   | ✅     |
| Data Stores | 22     | 0      | 0       | 0     | 22    | ✅     |
| Monitor     | 8      | 0      | 0       | 0     | 8     | ✅     |
| Verify      | 10     | 0      | 0       | 0     | 10    | ✅     |
| Govern      | 26     | 0      | 0       | 0     | 26    | ✅     |
| Release     | 2      | 0      | 0       | 0     | 2     | ✅     |
| Package     | 6      | 0      | 8       | 0     | 14    | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 519    | 0      | 55      | 6     | 574   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

Generated by gitlab_quality-test_tooling.

---

Slow tests detected in this merge request. These slow tests might be related to this merge request's changes.

Click to expand

Job	File	Name	Duration	Expected duration
#6865913768	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	69.98 s	< 27.12 s
#6872133036	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	81.61 s	< 27.12 s
#6873512899	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	70.9 s	< 27.12 s
#6893947509	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	73.39 s	< 27.12 s
#6896678584	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	68.11 s	< 27.12 s
#6902732828	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	77.83 s	< 27.12 s
#6904432824	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	79.0 s	< 27.12 s
#6904438476	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	86.77 s	< 27.12 s
#6904433008	spec/features/projects/badges/list_spec.rb#L44	list of badges user changes current ref of build status badge	65.91 s	< 50.13 s
#6904438664	spec/features/projects/badges/list_spec.rb#L44	list of badges user changes current ref of build status badge	66.01 s	< 50.13 s
#6905880278	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	71.27 s	< 27.12 s
#6905880382	spec/features/projects/badges/list_spec.rb#L44	list of badges user changes current ref of build status badge	65.86 s	< 50.13 s
#6907145683	spec/lib/release_highlights/validator_spec.rb#L82	ReleaseHighlights::Validator when validating all files they should have no errors	77.77 s	< 27.12 s

added rspec:slow test detected label

requested review from @jiaan

approved this merge request

removed review request for @tyleramos

added 282 commits

• a0adf813...28866a4d - 280 commits from branch master
• f88e609f - Improve Analytics dashboards schema errors
• e8744cb2 - Fixed failing rspec tests

Compare with previous version

reset approvals from @lciutacu by pushing to the branch

added 43 commits

• e8744cb2...0222cce6 - 38 commits from branch master
• 736f641b - Improve Analytics dashboards schema errors
• 17176c60 - Fixed failing rspec tests
• f3d67b7e - Updated schema to align with VSD config
• cc2dacac - Bumped up schema version
• 82c00b9b - Remove hardcoded raise for visualization field

Compare with previous version

removed review request for @jiaan

added 106 commits

• 82c00b9b...6e0ae9b7 - 95 commits from branch master
• 0fe4c79c - 1 earlier commit
• 2325bdd8 - Fixed failing rspec tests
• 3b552665 - Updated schema to align with VSD config
• e88fa2bb - Bumped up schema version
• 46b5f415 - Remove hardcoded raise for visualization field
• 8c36f146 - Added a little top margin to alert
• e4821ebc - Bumped up schema version and removed made nullable fields optional
• 45b1b74a - Removed required queryOverrides field
• 02caf737 - Removed unnecessary visualization error specs
• 121b6252 - Refactored dashboard error state
• 820928d8 - Added version to YAML config section of VSD docs

Compare with previous version

requested review from @lciutacu

Thanks for the thorough review (and for verifying that all is well with the Product Analytics dashboards) @elwyn-gitlab! I've addressed your comments, mind taking another look?

requested review from @elwyn-gitlab

requested review from @felipe_artur

requested review from @apennells

approved this merge request

removed review request for @apennells

requested changes

approved this merge request

added pipelinetier-3 label and removed pipelinetier-2 label

added 3 commits

• 9582eb2f - Addressed visualization resolver spec nitpick
• 2d117ed9 - Made newly nullable fields required again in schema
• 96ff8fbc - Removed optional tags from required fields in VSD docs

Compare with previous version

reset approvals from @felipe_artur by pushing to the branch

mentioned in issue #462677

requested review from @elwyn-gitlab

approved this merge request

resolved all threads

approved this merge request

enabled an automatic merge when the pipeline for 238b0614 succeeds

mentioned in incident gitlab-org/quality/engineering-productivity/approved-mr-pipeline-incidents#274 (closed)

approved this merge request

removed review request for @lvanc

approved this merge request

removed review request for @lciutacu

aborted the automatic merge because source branch was updated

added 407 commits

• 96ff8fbc...44cc7ab4 - 393 commits from branch master
• 44cc7ab4...8ea58818 - 4 earlier commits
• 0b5f8759 - Remove hardcoded raise for visualization field
• b33369f0 - Added a little top margin to alert
• b4ffc3ae - Bumped up schema version and removed made nullable fields optional
• 7c2d4784 - Removed required queryOverrides field
• 11235753 - Removed unnecessary visualization error specs
• 97faf66d - Refactored dashboard error state
• 55b8a677 - Added version to YAML config section of VSD docs
• e0852c20 - Addressed visualization resolver spec nitpick
• 991c339f - Made newly nullable fields required again in schema
• 199f8d14 - Removed optional tags from required fields in VSD docs

Compare with previous version

Hey @felipe_artur, when you get the chance, could you please set to MWPS I had to rebase due to a merge conflict

resolved all threads

enabled an automatic merge when the pipeline for 199f8d14 succeeds

merged

mentioned in commit aeb4ccf4

mentioned in incident gitlab-org/quality/engineering-productivity/master-broken-incidents#6452 (closed)

@rcrespo3 was this confirmed to work with the dashboard editor? On my GDK with the latest master branch I'm getting errors on all custom dashboard created with the dashboard editor.

I think we should revert for now and rethink how to solve these errors for custom dashboards. I've created an MR Revert improve analytics dashboards schema errors (!153832 - merged).

/cc @rcrespo3 @felipe_artur @elwyn-gitlab

was this confirmed to work with the dashboard editor?

@jiaan Not sure that it was, although I did somewhat anticipate this occurring, thinking that the updated documentation + friendly errors would be enough to rectify them (without thinking of the impact on custom dashboards created with the editor) Nevertheless, I do think a revert would be best here

No worries @rcrespo3 we managed to catch this before it hit production. I think this MR was on the right track and we just need to solve for custom dashboards. Also I think this shows there's room for improvement so I'd be in favour of doing a blameless retro to see if there's any ways we can catch these issues earlier in the future.

mentioned in commit 4c5af150

mentioned in merge request !153832 (merged)

added workflowstaging-canary label and removed workflowin review label

added workflowcanary label and removed workflowstaging-canary label

added workflowstaging label and removed workflowcanary label

added workflowproduction label and removed workflowstaging label

mentioned in merge request !154012 (merged)

mentioned in issue #444915 (closed)

mentioned in merge request !155896 (merged)

mentioned in issue #467670

added releasedcandidate label

added releasedpublished label and removed releasedcandidate label

mentioned in merge request kubitus-project/kubitus-installer!3098 (merged)