Opportunity Canvas Lite - Allow web users to easily enable/disable Dependency Scanning - Problem & Solution Validation
Instructions
-
Create a ♻ Retrospective Thread to capture future improvements to this issue template -
Complete the sections in the issue template below -
Link any relevant issues or research -
Assign to Manager -
Assign to VP and EVP of Product for approval @sfwgitlab
@adawar
-
Include VP of UX for visibility @clenneville
-
Once issue is approved, add the label ~oppty-lite::approved
close issue and share in relevant slack channels -
Follow up on the ♻ Retrospective Thread items and propose changes to this issue's template
Note: If issue is not approved, the issue can be closed without adding any labels.
Why Canvas Lite?
I decided to utilize a Opportunity Canvas Lite because this is for an existing category and feature for a understood problem and the solution proposed is leveraging existing patterns.
Customer Pain and Problem
Depending on the size of the company, the persona who needs to do the following JTBD will vary. It could be Cameron (Compliance Manager), Delaney (Development Team Lead), Sasha (Software Developer), or Sam (Security Analyst) depending on the size of the organization and who is driving the trial.
"When I use the GitLab security feature for the first time, I want to configure all necessary features, so that the security team can start using them for GitLab projects." JTBD
We want to be easy to use with or without deep git or GitLab knowledge / having to read documentation.
Problem Details
Today, to enable or disable Dependency Scanning, users must either enable Auto DevOps and get all Secure feature or users must enable, or disable, dependency scanning by creating an MR on the pipeline config YML and copy-paste information from the user documentation.
Today many users end up needing the help of support, sales, or customer success, this delays their evaluation of the product and increases the cost of the sale. As a company we are striving to improve our Self-Service trials and purchasing, and this confusion inhibits that, or in some cases may stop a user from proceeding forward with the evaluation.
Users are paying for, and expect, a product and features that just works, when they need to do reading and work just to turn it on they could get frustrated or end up with a negative first impression. In many cases users start with an initial test as a proof of value test. If users start with a negative experience starting with setup they may view the feature less favorably.
Proposed Solution
We will add add a button to the setup page that automatically generates the MR with the correct content for basic setups. After the MVC we can then iterate to allow for configuration of the most popular variables.
Mockup
Configuration Page
When button is clicked, in either case it goes to the Create MR Page
Title: Set .gitlab-ci.yml to enable or configure Dependency Scanning
Description: Set .gitlab-ci.yml to enable or configure Dependency Scanning security scanning using the GitLab managed template. You can add variable overrides to customize the settings.
File content:
include:
- template: Dependency-Scanning.gitlab-ci.yml
Business case
RICE score
2 = (3.0x2x1)/3
Reach
What count of users are you expecting to reach with this feature?
~25% of ultimate users use Dependency Scanning - and this will improve those users disable experience and then increase that number.
Rationale
if it takes the planned amount of time, 3 months, for ~1 person per month (UX, FE, BE) thats 37,500 for the feature, which is 378 ultimate seats to pay for the feature.
This will impact new project setups, such as new projects and trials. Placing this directly in the UI will make it easier to discover if you are looking at the secure configuration page.
I think that dependency scanning Month-Over-Month average MAU growth in non-gold tiers would stay steady at 5% in gold, and increase to 6% from 4% in paid and overall.
I would expect our overall percent of users in gold using DS to hold steady at 25%, but paid percent of users to increase from 2% to 3% and % of total users to increase from 5% to 6%.
I looked into the release of SAST to Core which was combined with the improved setup UI. Although difficult to tell what percent increase was due to bringing SAST to Core and what percent was improving the setup experience.
I firmly believe not as many users would have tried SAST in non-gold versions if they had to configure SAST in the CI-Pipleine-YML as when working with sales and customer success setup is the most frustrating part of the process for ultimate users, in addition our UX score in this area is failing.
Dependency scanning has been growing at an average of 5% with a range between -1 and 11% in 2020 in gold as compared to SAST (AVG: 9% range 5-22%) and Dependency scanning only has an 4% average growth overall (range between -4-11%) in 2020 for all users as compared to SAST (AVG:16% Range:-1-70%). Note the difference in averages from before and after the release of SAST 13.3 is -1% for gold and +9% overall.
Impact
Describe the impact this feature will have for those users.
High
Rationale
- Increased revenue - We are potentially exposing users to a new stage, and statistically our paid conversion increases as the number of stages increases. SPU
- I believe it will increase NPS by setting a good first impression, and making proof of value easier, and self service.
- Increased revenue - leaving more time for proof of value exercises and less getting the proof of value enabled will allow customers to better evaluate and then hopefully select our product. This only occurs once per pov so not a large impact.
- Decreased cost - Currently support, sales, and customer success have to walk users through setting up and troubleshooting dependency scanning jobs. I believe this will become less frequent, and faster for those occurrences that happen, decreasing support costs. This only occurs once per project so not a large impact.
Confidence
How confident are you in the problem and solution?
High = 100%
Rationale
How well do we understand the customer problem? Very well, we frequently need to help troubleshoot, and frequently get the feedback.
How well do we understand the solution and implementation details? Very well, SAST has already implemented a pattern we can utilize.
Effort
Approximately how many developers and how many months will be required to deliver this feature?
3 person months
Rationale
How many person months do we estimate this will take to build? 3 releases - with one person working on it each month (where it is part time UX, part time backend, part time frontend)
Note I have limited UX (.5) which influences how many releases this will take in calendar months as opposed to effort potentially
RICE Questions
- Roughly what percent of those users would you expect to be Starter? Premium? Ultimate?
- I do not expect my target 2% MoM-MAU increase to change, however I expect this to make proof of values decrease setup and support time, and make self-trials easier.
- The cost will be ~$40k or ~32 annual user licenses.
- What other companies offer a feature like this?
- GitHub allows easy webpage enabled and configured add-ons in their marketplace.
- When do you intend do deliver this feature?
- MVC Q1/Q2
- What would happen if we waited to deliver this feature later than you have planned?
- We will likely continue to score poorly in usability, which is one of the items preventing us from moving depending scanning to complete.
Qualitative Evidence
One customer I had feedback from was searching for documentation and ended up with very old gemnasium documentation and was disappointed and frustrated with setup and that was one key point they brought up was how they expected a better experience from us. They did not end up choosing Ultimate/Gold.
Analysts have asked If our setup is available though API, Web, and command-line as part of their evaluation of our enterprise support. Today we only offer configuration via command-line using git to update YML and this will allow us to add Web as a supported configuration option.
Feature Maturity Plan
By offering the feature additions outlined in this Canvas Lite, we would expect to make the feature maturity:
-
Viable - Encourage adoption for up to 50% of our users - Feature will be fully available for Dogfooding -
Complete - Encourage adoption for up to 75% of our users - Feature will be fully available for external users -
Lovable - Encourage adoption for up to 100% of our users - Feature will be fully usable for external users, be a switching reason, or a key differentiator for our product category.
One element of complete is usability - which this will help improve. To Complete epic;
Feature Development Plan
- What is the minimal change to make progress?
- Enable and Disable
- What are the next steps needed to address the problem?
- Configure and Customize
Related Epics or Issues
Epic which I will bulk out based on the results of this canvas.