## Overview Our existing GitLab.com <> Customers, License, Zuora integration has several data integrity issues. Examples of recent issues we've dealt with include: - [Only 85-90% of Paid SaaS can be stitched to Usage Data](https://docs.google.com/presentation/d/1DBOqLO8cFzTsxzZUgzYFq4TFJB3Odm3yFToDOnKLHnY/edit#slide=id.g88ab15452f_0_0) - [SaaS trials are not expiring](https://gitlab.slack.com/archives/CNLHTE457/p1592585200341500) - [Trials may be dropped going to Salesforce](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/1537#note_364665626) - [Duplicates are being submitted to SFDC for Gold Trials](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/1357) - [SaaS trial fields not being sent to SFDC](https://gitlab.com/gitlab-org/gitlab/-/issues/214849) - [Customers <> Salesforce Missing SaaS trials](https://gitlab.com/gitlab-org/growth/team-tasks/-/issues/156) - [Ensure SaaS Gold trials are receiving 50,000 CI minutes](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/1525) - [Data cannot be retriggered from Customers to downstream systems](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/1537#note_362566159) - [Manually generated license data is not recorded in SFDC](https://gitlab.com/gitlab-org/license-gitlab-com/-/issues/164) ## Problem The core data problem lies in our integration across Customers, License, Zuora, and GitLab.com. * We need one **single view of a customer account that can be stitched together end to end** from it’s subscription to usage data. For GitLab.com, this means stitching together GitLab.com Postgres <> Customers. For self-managed, this means stitching together Usage Ping <> Customers. This single view can then be used downstream in Data / Sales / Marketing. Currently we have varying definitions of the data model and how to join the data across applications. We need to align each team and update the data models in each application so they all share the same definition. Here's an example of what a proper data model might look like: * Customer (Account) has many Namespaces. * Namespaces can have many Subscriptions. * Subscriptions can have many Licenses. * Licenses can have many Usage Pings. * Salesforce is then layered on top and can have many Customers (Accounts) * We need the **Customer's app to be the single source of truth** for all payment and license data. Currently Customers <> License <> Zuora is not matching 1:1:1 making it impossible to stitch together. This is due to the multiple entry points without a well defined source of truth. For example: Sales Assisted purchases go through Salesforce to Zuora and may not end up in Customers. Another example: TAMs/SALs/CS can manually generate licenses which may not end up in Customers. All entry points to purchase and generate a license need to go through Customers app. We need to make it so: * Every license must exist in Customers. * Every Zuora subscription must exist in Customers. * Every GitLab.com subscription/trial must exist in Customers. * Every self-managed account must exist in Customers. * We need the **Customer's app exposed to GitLab.com via a well documented API**. All interactions between the applications must pass through this API and the API should be our data contract between the applications. * We need **all user flows documented** to understand how data is flowing across each application. Currently we have minimal documentation and each team has a different understanding of how each flow works. * We need **GitLab.com to stay in sync with Customers**. Once we agree Customers is our single source of truth of customer data, GitLab.com should replicate this data and stay in sync. Currently there is a gap where some data lives in GitLab.com and other data lives in Customers. Fixing this core integration and it's data quality issues are fundamental to the Growth team's ability to build experiments - we're currently building on top of an unstable integration that constantly breaks. This core integration creates the foundation of our product data. We need 100% data integrity here before shipping it downstream to Salesforce / Marketo. The goal of this epic is to document and fix the integration. The outcome will be no more gaps in the data so we can reliably build on top of this integration. ## Why Fix This Now? 1. The number of issues reported by other teams is beginning to balloon. Customer Support, Sales, Marketing, Data are reporting more issues every day. 2. The organization wants this data. Sales wants a single view of a Customer in SFDC. Marketing wants leads accurately sent to Marketo. Product wants subscription data tied to usage data. 3. It's frustrating to work with an integration that is fragile and constantly breaks. ## List of Flows There are currently eight flows that related to this integration. We will need to document and fix each of these flows. ### Signup Flow * [ ] Starting on about.GitLab.com * [ ] Starting on GitLab.com ### Trial Flow * Trial SAAS Gold * [ ] Starting on about.gitlab.com * [ ] Starting on Customers * [ ] Starting on GitLab.com * [ ] Trial expiring * Trial Self Managed * [ ] Starting on about.gitlab.com * [ ] Starting on Customers * [ ] Starting on GitLab * [ ] Trial expiring ### Purchase Flow * Web Direct/SaaS - First Purchase * [ ] Starting on about.gitlab.com * [ ] Starting on Customers * [ ] Starting on GitLab.com * Self-managed purchase * [ ] Starting on about.gitlab.com * [ ] Starting on Customers * [ ] Starting on GitLab * Sales Assisted - First Purchase *no customer facing UI* * Reseller - First Purchase *no customer facing UI* * EDU/OSS - First Purchase *no customer facing UI* ### Storage Purchase Flow * Web Direct/SaaS - Add Ons (.com) * [ ] Starting on Customers * [ ] Starting on GitLab.com * Self-managed Add Ons * [ ] Starting on Customers * [ ] Starting on GitLab * Sales Assisted - Add Ons *no customer facing UI* * Reseller - Add Ons *no customer facing UI* * EDU/OSS - Add Ons *no customer facing UI* ### CI Purchase Flow * Web Direct/SaaS - Add Ons (.com) * [x] Starting on Customers * [x] Starting on GitLab * Self-managed Add Ons * [ ] Starting on Customers * [ ] Starting on GitLab * Sales Assisted - Add Ons *no customer facing UI* * Reseller - Add Ons *no customer facing UI* * EDU/OSS - Add Ons *no customer facing UI* ### Renewal Flow * Web Direct/SaaS - Renewals / subscription expiring * [ ] Starting on Customers * [ ] Starting on GitLab.com * [ ] Auto-renew * [ ] Without auto-renew * [ ] CI minutes expiring (might be combined with subscription) * [ ] Storage plan expiring (might be combined with subscription) * Self-managed * [ ] Starting on Customers * [ ] Starting on GitLab * [ ] Auto-renew * [ ] Without auto-renew * Sales Assisted - Renewals *no customer facing UI* * Reseller - Renewals *no customer facing UI* * EDU/OSS - Renewals *no customer facing UI* ### Upgrade Flow * Web Direct/SaaS - Upgrades * [x] Starting on Customers * [x] Starting on GitLab.com * Self-managed * [ ] Starting on Customers * [ ] Starting on GitLab ### Cancel / Expire Flow * Web Direct/SaaS - Cancel * [ ] Starting on Customers * [ ] Starting on GitLab.com * Self-managed - Cancel * [ ] Starting on Customers * [ ] Starting on GitLab ## Resourcing Plan | Flow | Primary Team | Supporting Teams | Documentation Milestone | |-|-|-|-| | Signup Flow | Acquisition | Marketing | 13.3 | | Trial Flow | Conversion| Marketing, Fulfillment | 13.3 | | Purchase Flow | Acquisition | Fulfillment | 13.3 | | Storage Purchase Flow | Fulfillment | - | 13.4 | | CI Purchase Flow | Fulfillment | - | 13.4 | | Renewal Flow | Retention | Fulfillment | 13.3 | | Upgrade Flow | Expansion | Conversion, Fulfillment | 13.3 | | Cancel Flow | Retention | Fulfillment | 13.4 | ## Next Steps - [ ] [DATA] Customer Product Usage Data Investigation https://gitlab.com/groups/gitlab-data/-/epics/130 - [x] [PRODUCT] Create a resourcing plan https://gitlab.com/gitlab-org/growth/product/-/issues/1590 - [ ] [UX] Document all flows between GitLab.com <> Customers, License, Zuora https://gitlab.com/groups/gitlab-org/-/epics/3603 - [ ] [ENG] Document all flows between GitLab.com <> Customers, License, Zuora https://gitlab.com/groups/gitlab-org/-/epics/3605 - [ ] [ENTAPPS](https://gitlab.com/groups/gitlab-com/business-ops/bizops-bsa/-/epics/12) - [ ] [ENG] Create plan to fix all flows between GitLab.com <> Customers, License, Zuora https://gitlab.com/groups/gitlab-org/-/epics/3646 - [ ] [QE] Define end to end test strategy and coverage for Customer portal https://gitlab.com/groups/gitlab-org/-/epics/3530 - [ ] [ENG] Setup monitoring for all flows between GitLab.com <> Customers, License, Zuora https://gitlab.com/groups/gitlab-org/-/epics/3648 - [ ] [ENG] Resync all data between GitLab.com <> Customers, License, Zuora https://gitlab.com/groups/gitlab-org/-/epics/3649 ## Deliverables ### Documentation | Flow | UX (SM) | UX (SaaS) | Technical (SM) | Technical (SaaS) | Business Process (SM) | Business Process (SaaS) | |-----------------------|---------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|----------------|----------------------------------------------------------------------------------------|-----------------------|-------------------------| | Signup Flow | https://www.figma.com/file/DCq7K8Srsv79tbH1yRkGbl/Document-user-flows-%5Bgitlab-org%2F-%2Fepics%2F3603%5D | https://www.figma.com/file/DCq7K8Srsv79tbH1yRkGbl/Document-user-flows-%5Bgitlab-org%2F-%2Fepics%2F3603%5D?node-id=1%3A2 | | https://gitlab.com/gitlab-org/gitlab-docs/-/issues/794#note_367430307 | | | | Trial Flow | https://www.figma.com/file/DCq7K8Srsv79tbH1yRkGbl/Document-user-flows-%5Bgitlab-org%2F-%2Fepics%2F3603%5D?node-id=28%3A22 | https://www.figma.com/file/DCq7K8Srsv79tbH1yRkGbl/Document-user-flows-%5Bgitlab-org%2F-%2Fepics%2F3603%5D?node-id=28%3A22 | | https://gitlab.com/gitlab-org/gitlab-docs/-/issues/793#note_404734376 | | | | Purchase Flow | | | | https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/staging/doc/purchase_flow.md | | | | Storage Purcahse Flow | | | | | | | | CI Purchase Flow | | | | https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/staging/doc/ci_minutes.md | | | | Renewal Flow | https://www.figma.com/file/DCq7K8Srsv79tbH1yRkGbl/Document-user-flows-gitlab-org-epics-3603?node-id=72%3A35 | https://www.figma.com/file/DCq7K8Srsv79tbH1yRkGbl/Document-user-flows-gitlab-org-epics-3603?node-id=96%3A59 | | | | | | Upgrade Flow | https://www.figma.com/file/DCq7K8Srsv79tbH1yRkGbl/Document-user-flows-gitlab-org-epics-3603?node-id=66%3A42 | https://www.figma.com/file/DCq7K8Srsv79tbH1yRkGbl/Document-user-flows-gitlab-org-epics-3603?node-id=99%3A0 | | https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/staging/doc/upgrade_flow.md | | | | Cancel / Expire Flow | https://www.figma.com/file/DCq7K8Srsv79tbH1yRkGbl/Document-user-flows-gitlab-org-epics-3603?node-id=96%3A677 | https://www.figma.com/file/DCq7K8Srsv79tbH1yRkGbl/Document-user-flows-gitlab-org-epics-3603?node-id=96%3A614 | | | | | [Editable Link](https://docs.google.com/spreadsheets/d/1wKGBCu0j97bp0Z6crZbE5dp5vt70Vr_sKLeWrCG2uSc/edit?usp=sharing)