Implement custom dashboards storage solution
## Problem to solve
> Goal: Empower users to tailor dashboards to their unique workflows.
Dashboard customization is limited to Analytics dashboards using repository file storage. This structure does not allow for dashboards storage outside of the group → project hierarchy and it is not easy to search or manage dashboards across multiple project repositories.
### Proposal
Store dashboard configurations in a new PostgreSQL table. Users should be able to:
- CRUD dashboards on the UI via a API (preferably GraphQL)
- Search for dashboards in an organization or group
- Mark a dashboard as a favorite / stared
- Can see by who and when a dashboards was last edited
- See when a dashboard was `Created by GitLab` or is custom / user created
See the research issue with prototype MRs https://gitlab.com/gitlab-org/gitlab/-/issues/555058+.
See the UX designs for dashboards CRUD flow https://gitlab.com/gitlab-org/gitlab/-/issues/528016+
### Functional Requirements
* **Platform Compatibility:** Must work on both SaaS and customer instances (self-hosted, dedicated).
* **Access Control:**
* Prevent unauthorized access to dashboards.
* Support dashboard sharing and fine-grained permission management across different organizational levels (e.g., group, project, user, external sharing).
* **Data Hierarchy:** Support dashboard configurations that exist outside the traditional group \> project hierarchy.
* **Auditing and Versioning:**
* Provide an auditable and traceable log of all changes (persistent change events).
* Support the ability to roll back a dashboard to a previous version.
* **Concurrency:** Safely handle concurrent write operations from multiple users to prevent conflicts and data corruption.
* **User Experience:** Provide useful and clear error messages to the end-user.
### Non-Functional Requirements
* **Conflicts**: Optimistic locking to prevent users from saving a change over the change of another user similar to when editing issues.
* **Performance:** API responses for dashboard configurations must be within a 200-500ms range to ensure a responsive user experience.
* **Scalability:** The solution must be able to scale to handle multiple dashboard configurations, with consideration for the maximum number of dashboards; as well as the maximum size of each configuration file.
* **Availability and Recovery:**
* Must have a clear and automated plan for backup and disaster recovery (DR).
* The system should be designed for high availability (HA), with zero-downtime updates and locally saving state in case of outages.
* **Security and Compliance:**
* Dashboard configurations should be treated as sensitive data, potentially containing private information.
* The solution should support data governance needs, such as data residency and the ability to produce audit trails for compliance with regulations like GDPR or HIPAA.
* **Monitoring:** The system must be monitorable to provide insights into cost, usage, and performance.
### Future and Architectural Requirements
* **Advanced Search:** The solution must eventually support complex searching and filtering operations on dashboard configurations (e.g., searching for dashboards containing specific panel types or metrics).
* **Zero-Downtime Schema Evolution:** Must support the ability to make changes to the _dashboard configuration schema_ itself (e.g., adding or renaming fields within the JSON/YAML structure) without causing application downtime.
* **Real-Time Capabilities (Optional but considered):** The architecture should ideally not prevent the future implementation of real-time features, such as live collaboration on dashboards.
### Definition of done
1. Updated the architectural design document with the PostgreSQL storage solution: https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/dashboard_customization_framework/
1. Users can CRUD dashboard configuration via the dashboard listing page and the customization UI https://gitlab.com/groups/gitlab-org/-/epics/18158+.
## References
- [Dashboard foundations roadmap epic](https://gitlab.com/groups/gitlab-org/-/epics/18072)
- [Architectural design document](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/dashboard_customization_framework/)
- [Implement dashboard customization UI framework epic](https://gitlab.com/groups/gitlab-org/-/epics/18158)
epic