# **Overview** [[_TOC_]] ### What is the problem? CustomersDot and integrated QTC systems don't have capabilities to process necessary data changes. Some of data changes are necessary to resolve subscription related issues Data changes are required during the following stages of customers' subscription lifecycle <table> <tr> <th>Stage</th> <th>Example data changes</th> </tr> <tr> <td>Initial discovery to paid subscription</td> <td>Editing of trial end dates, trial type</td> </tr> <tr> <td>Migration (SaaS to SM or Vice versa)</td> <td>Editing trial end dates</td> </tr> <tr> <td>Business continuity needs during subscription term</td> <td> Editing linked namespace or linked subscription, Add ons modifications </td> </tr> <tr> <td>Renewals</td> <td>Extended trial end date</td> </tr> </table> _\*Trial license is being repurposed for various reasons \[_[_Detailed description below_](https://gitlab.com/groups/gitlab-org/-/epics/14169#what-do-these-data-changes-enable-for-the-business)_\]_ These changes cannot built as self serve customer functionality due to * **Potential revenue implications :money_with_wings:** * **Complexity of changes :spider_web:️** * **Need for controls for better risk management :safety_vest:** * **Potential abuse management (customer) :octagonal_sign:** * **Need for auditing and reporting clarity :bar_chart:** * **Need for data integrity and consistency of CustomersDot Data :clipboard:** ### **How are we solving this now? And what are the current limitations?** **Support team is using a built in tool called Mechanizer to make these data updates** * **Mechanizer was originally built in place by the support team to enable the above mentioned functionalities** by the support team because the only alternative to making the above changes were directly from the console rails rather than Customersdot admin view.  * **Mechanizer is directly integrated with Zendesk UI** where Support engineer can add values that will in turn create a pipeline in backend and execute the changes in production with record indicating which support engineer initiated the change.  While Mechanizer **does have a few benefits (Ex- Ease of use for support engineers, Faster turnaround time)** it falls short in the following aspects <table> <tr> <th>Problem/Limitation</th> <th>Details</th> </tr> <tr> <td> **Low/No maintenance of Mechanizer creates significant gaps in the system** _(Between CustomersDot and Mechanizer - Former being the system that is being changed consistently)_ </td> <td> * Ex- **_Duo pro Add ons, Enterprise Add on plan or trial data etc are not editable_** via mechanizer - therefore support will be unable to help the customers without significant overhaul of mechanizer * Mechanizer based updates to **CDot data has no consistent overview from Fulfillment engineering**. This has caused issues in the past where we had to invest effort to rectify the issues * Requires us to review periodically to ensure compliance & security _\[We currently don’t have any pending items but changing mechanizer to address new functionality might require additional effort\]_ * Any tech debt improvements will have to account for Mechanizer </td> </tr> <tr> <td> **New cells architecture introduces a new authentication system that will not work with Mechanizer without significant overhaul** </td> <td> * Mechanizer **_will not work with_** **_the new authentication system that is being introduced as part of the Cells architecture._** * An alternative without significant effort to change Mechanizer is to use multiple personal access tokens with mechanizer which opens up security concerns and **adds a lot of inefficiency for the support team defeating its original purpose \[** https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/9900#note_1945847652\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\] </td> </tr> <tr> <td> **Changes to sensitive and revenue impacting product data is high risk situation with minimal to no oversight or approvals** </td> <td> * Does it sync back with Zuora? - Assuming it should but we don't monitor this exclusively - we fix issues when it comes up * **\[Solved\]** Auditing CDot data changes requires additional effort if changes were made by mechanizer \[ https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/3417+\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\] * External BPO support team will have access to make sensitive changes in production - There is traceability but no easy way to report or monitor proactively </td> </tr> </table> ### Existing/Past efforts across fulfillment group <details> <summary>Here is everything that has been done in the past to reduce the need for manual intervention of support team</summary> * **\[Solved with more potential improvements\]** Expired trials are linked and not de-provisioned properly \[ https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4390 & https://gitlab.com/gitlab-org/fulfillment/meta/-/issues/1308 \] * **\[Partially solved\]** purchasing issues blocked Add-on purchases (Minutes & storage) ( https://gitlab.com/groups/gitlab-org/-/epics/9486+) & ( https://gitlab.com/groups/gitlab-org/-/epics/7714+) * **\[Solved\]** Partner-reseller who don’t have access to customer portal require manual intervention to associate their namespace to subscription \[ https://gitlab.com/groups/gitlab-org/-/epics/8941+\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\] * \[Partially solved\] </details> ### What do these data changes enable for the business? Overall we have Mechanizer reports that showcase volume over the past few months [here](https://gitlab.com/gitlab-com/support/internal-requests/-/issues/?sort=updated_desc&state=opened&label_name%5B%5D=MMR&first_page_size=20). The most commonly used Mechanizer functions are listed in the order of usage (approx requests per month) 1. Update Gitlab plan - 100-150 2. Force re-association - 40-45 3. Extra Minutes - 25-30 4. Unlink GL account - 15-25 5. Change Max seats - 10-12 6. Clear subscription - 8-10 7. Emergency license generation - Spikes during certain months (likely holidays) 8. Storage - 2-3 <details> <summary> Here are some of **the business scenarios in detail** that are currently being processed via Mechanizer </summary> 1. **Reformulating Trial licenses (Update gitlab plan)** 1. **Initial stages of customer lifecycle are truly exploratory** where sometimes it was an unintentional trial start and the sales team wants to drive a conversation requesting support team to change the trial (ex- Trial plan upgrade or downgrade & extensions) 2. **Downgrading/Upgrading trial licenses:** When customer signs up for Ultimate trial but eventually wants to test features that they intend to purchase (Premium) & Vice versa 3. **Repurposed trial licenses:** Customers are given the option to use a trial license for proof of concept validation or migration with longer validity (Saas to SM & Vice versa) 4. **Creating a Not for resale (NFR) subscription** on an adhoc basis for partners by updating an existing trial subscription to modify to order to extend it for 1 year 5. **Emergency license generations** On call support engineers who don’t have access to Cdot will use this (weekends, holidays etc) 2. **Namespace linking/unlinking (Force re-associate ):**  1. Customer reached out to correct an Incorrectly linked namespace to a subscription  2. Existing (manually changed) trial needs to be removed and new subscription needs to be linked if provisioning does not happen on time 3. **\[Solved\]** Partner-reseller who don’t have access to customer portal require manual intervention to associate their namespace to subscription \[ https://gitlab.com/groups/gitlab-org/-/epics/8941+\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\] 3. **Change max billable seats to enable dispute management:** Customer runs into billing issues (ex- QSR disputes) that requires a change in core subscription information like Max seat  (Ex- if they were refunded by Sales team, they will get those additional seats waived off so that the subscription will reflect correct Max seats with subscription) 4. **Clear subscription with no pending charges** Customer needs to have their subscription cleared for following scenarios 1. **\[Solved with more potential improvements\]** Expired trials are linked and not de-provisioned properly \[ https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/4390 & https://gitlab.com/gitlab-org/fulfillment/meta/-/issues/1308 \] 2. **\[Requires investigation\]** Extended trial now requires a manual intervention so that customer can purchase a subscription \[if we can resolve any purchasing limitation that customers may have\] 3. Subscription expired and customer is trying to purchase a new plan subscription  4. Force re-association indicates there is one of the above cases that requires a clear subscription before the namespace can be linked to another subscription 5. **Add ons modifications for business continuity** 1. CI minutes modifications (self serve & sales assisted) 2. **\[Partially solved\]** Used for purchasing issue and wanted to buy minutes and it failed - we assign manually as an interim’ solution till they can buy it ( https://gitlab.com/groups/gitlab-org/-/epics/9486+) & likely will be reduced once we roll out 3DSecure 3. Sometimes its used to give courtesy credit for bugs faced 4. No changes to monthly quota - because we can do it on CDot Admin 5. Additional storage for emergencies 6. **Unlink gitlab.com account:** Subscription contact management during personnel change and also during SSO login issues are driving the need for this functionality on mechanizer </details> --- ### Long term: How do we solve this problem? * Our primary goal is to **enable only necessary Admin functionality on CustomersDot** to make acceptable changes to CustomersDot data. * We will create proper access controls in CustomersDot * We will group the functionality and create new permissions based on Business reason * Emergency Lite - Only for emergency situations to help customers * Emergency Billing - For changing revenue impacting data * Other permissions (Based on how Phases 1-3 progress) * We will continue to identify issues and improve our overall fulfillment systems to ensure we solve the problem at the source _\[Some of problems are solved with past efforts-Read here\]_ * We will enable self serve admin functionality on other systems (where possible) once we have completed phases 1-3 # Proposed Phases \[Updated Feb 2025\] #### :white_check_mark: \[Launched as of Jan 31st 2025\] **Phase 1 & 2: Introduce 'Support' permission - Users will be able to perform changes on CDot for customer business continuity with certain guardrails in place** * Phase 1 : Update gitlab plan - Edit trial end date & plan (ultimate or premium) for Saas (For SM the team will just leverage creating a new license) - only limitation is inactive trials * Phase 1 : Create emergency license - ~~X days max cap~~ * Phase 2 : Clear subscription & force re-associate * Phase 2: Emergency Add on consumables (Storage or Minutes) - Max capped at certain values * Phase 2: Unlink GL account #### :x: **~~Phase 3: Introduce 'Billing' - Users will be able to perform changes on CDot that may have some revenue impact or is sensitive billable information~~ \[Updated on Feb 2025\]** * ~~Phase 3 : Change Max billable users \[For dispute management\] - Ideally we will see reduced use of this function based on how overages is rolled out and we can evaluate the real need for this~~ * :notepad_spiral: :exclamation: **This has been removed and replaced with a new Phase2b** \[[Review additional context here](https://gitlab.com/gitlab-org/fulfillment/meta/-/issues/2084#note_2332684815)\] #### :white_check_mark: Phase 2b (Completed April 2025): Enhance support group functionality based on feedback from Phase 1,2 and 2b and address edge cases to eliminate the use of mechanizer * Enhanced logging * Include edge cases from Mechanizer (primarily for update gitlab functionality) * Reset max seats for support group #### :new: Phase 2c (New April 2025): Enhance support group functionality by adding SaaS extensions functionality to eliminate the use of mechanizer * Introduce SaaS extensions for support group to eliminate the last pending case of mechanizer #### **Phase 3 (updated on Feb 2025): Expand scope for non support members to allow self serve via Salesforce (directional approach defined)** * OPT1 - Use Salesforce integration to enable this functionality use an approvals system inside salesforce for exceptions (likely the preferred pathway) * Enable APIs to allow for self serve pathways for Sales requests as supported by Sales policies * **\[FY26 Q1 priority\]** Renewal extension with approvals (multiple extensions per opportunity will be enabled and likely have an impact on support operations queue) * ~~OPT2- Use Admin view to create a request approval list for assigned users to approve directly on CustomersDot~~ _~~once the necessary functionality has been created (phase 1-3) on CustomersDot, we could potentially just extract it and create pathways from external tools (Ex- Salesforce or Zendesk etc) to enable this functionality for a wider group~~_ <table> <tr> <th>Phase</th> <th>What will be enabled</th> <th>Notes</th> </tr> <tr> <td>Phase 1</td> <td> * Create new permissions "Support" * Enable only for support team on CDot * Capture reason for change for monitoring and future improvements * Deprecation of Mechanizer "Update Gitlab plan" </td> <td>Guardrails will be added if needed</td> </tr> <tr> <td>Phase 2, 2a, 2b</td> <td> * Add new functionality for this set of permission * Test and Deprecation of Mechanizer "Clear subscription & Force re-associate , unlink GL account & reset max seats" </td> <td></td> </tr> </table> All phases defined highlight which user group will have access to the functionality in addition to the support team. Ideally each individual team will leverage these functionalities in a way that is helpful for them without relying on support to process the requests unless there is a grave error that requires complex debugging and resolution <table> <tr> <td> **Functionality** </td> <td> **Who**  </td> <td> **Can this be self-serve for the end user/customer?** </td> <td> **Additional description** </td> </tr> <tr> <td>Update GitLab Plan</td> <td> Phase 1-2 : Support team Phase 4+ : Sales team Phase 4+ : \[If needed\] Proper user management in CDot for future access </td> <td>No</td> <td>In the past we have launched self serve extend option for customers and decided to roll it back due to sales team concerns and also potential for abuse</td> </tr> <tr> <td>Force Reassociation</td> <td> Support team  (eventually customer once we implement abuse prevention mechanism) </td> <td>Partially</td> <td> We now allow one namespace change  for customers as a self serve action. Support can change the namespace as needed In the future we could have more rules to prevent namespace abuse \[ Ex - Additional verification before manual intervention etc\] </td> </tr> <tr> <td>Unlink GL account</td> <td>Support team to unblock customer</td> <td>Yes</td> <td>We could leverage this opportunity to improve SSO experience</td> </tr> <tr> <td>Clear Subscription</td> <td>Support team to unblock customer</td> <td>No</td> <td></td> </tr> <tr> <td>Extra minutes</td> <td>Support team to unblock customer or emergency situation</td> <td> This is mostly for emergencies - In an ideal world the customer would be able to purchase and use the Add on  This could also be voluntary effort from Gitlab when there are bugs or other issues in the system </td> <td> Multiple active orders has resolved most occurrences 3DS could probably bring down the need for this \[if the customer faced this issue during purchasing\] </td> </tr> <tr> <td>Add Storage </td> <td>Support team to unblock customer or emergency situation</td> <td> This is mostly for emergencies - In an ideal world the customer would be able to purchase and use the Add on  This could also be voluntary effort from Gitlab when there are bugs or other issues in the system </td> <td> 3DS could probably bring down the need for this \[if the customer faced this issue during purchasing\] </td> </tr> <tr> <td>Emergency License Generation</td> <td>Support team for emergencies only</td> <td>No</td> <td></td> </tr> <tr> <td>Sales Assisted CI Minutes</td> <td>Support team for emergencies only</td> <td>No</td> <td></td> </tr> </table> --- **Support Priority Score:** 19 <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> > [!important] > This page may contain information related to upcoming products, features and functionality. > It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. > Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc. <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->