Please take notice of the new data source request [handbook page](https://about.gitlab.com/handbook/business-technology/data-team/how-we-work/new-data-source/) before submitting this new data source request. If you have any questions, please ask in the #data slack channel. ## Business Use Case (Please explain what this data will be used for): In analytics~24933738, we [want](https://gitlab.com/gitlab-org/frontend/pajamas-adoption-scanner/-/issues/85) to start making [Pajamas](https://design.gitlab.com/) Adoption in the GitLab codebase a KPI. > For now, we'll want the following: > > 1. Overall adoption % (e.g. `88%`) > 1. Overall adoption ratio (e.g. `2256/16728`) > 1. Adoption % by group > 1. Adoption ratio by group ## Request checklist To support the new data source triage process, try to complete the below checklist. If there is anything not sure or unknown, you can skip that point. * [x] Prefix the issue name with 'New Data Source: ', e.g. 'New Data Source: NetSuite AP data' * [ ] Review the [current data available in the EDW](https://about.gitlab.com/handbook/business-technology/data-team/platform/#extract-and-load) and confirm it concerns a new pipeline, or a change/extension of an existing pipeline: - [ ] Complete new pipeline - [ ] Change/extension of an existing pipeline: ` ` * [ ] Do any objects in this data source need to be snapshotted? If yes, please open separate issues to have the objects snapshotted. * [ ] Does it contain MNPI data? - No * [ ] Does it contain Personal Data? - No * [ ] Severity in case of an incident - [ ] Critical - S1 - [ ] High - S2 - [ ] Medium - S3 - [ ] Low - S4 * [ ] Who will be using this data, and where (dashboards, snowflake UI, etc.)? - analytics~24933738 in Tableau * [ ] Please list and describe any data from this source that is sensitive (Classified as Red or Orange in our [Data Classification Policy](https://about.gitlab.com/handbook/engineering/security/data-classification-standard.html#data-classification-levels#data-classification-levels))? - None * [ ] Does this data have any agreed [SLO](https://about.gitlab.com/handbook/business-technology/data-team/platform/#slos-service-level-objectives-by-data-source) attached to it? If not: * [x] How often does the data need to be refreshed? * Daily at most. Weekly might also be acceptable. ## People matrix | Role | Name | Gitlab Handle | | ---- | ---- | ------------- | | System owner | `Please provide` | Mark Florian | @markrian | | Technical contact for data related questions | Mark Florian | @markrian | | Technical contact for infrastructural related questions | `Please provide` | `Please provide` | | Data access approval* | `Please provide` | `Please provide` | | Business users who need to be informed in case of data outage | Mark Florian | @markrian | * Data access approval will be involved in the Access Request process and need to give approval if a GitLab team member applies for raw data access. ## Integration Preparation <!-- Sufficient access needs to be granted and verified before we can begin working on an automated extraction ---> **Will there need to be access granted in order for a Data Engineer to extract this data? (example: New permissions or credentials to Salesforce in order to access the data)** - [ ] Yes - [x] No - [ ] I don't know **If Yes:** - Prioritize giving access to a service account rather than any individual Data Engineer, if uncertain on which account to use contact the Data Engineer assigned below for confirmation. - Where will access be required? - Link to Access Request: <!-- This can be blank to start, will need to be added for prioritization --> ## Data Engineer tasks **Triage** * [ ] Determine the extraction solution via decision [diagram](https://about.gitlab.com/handbook/business-technology/data-team/how-we-work/new-data-source/#extraction-solution): * [ ] Estimate the [issue points](https://about.gitlab.com/handbook/business-technology/data-team/how-we-work/#issue-pointing), based on the current information* * [ ] This new data source is lised in the [tech_stack.yml](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/tech_stack.yml) and does not have `data_classification: Red`. If the data source does have `data_classification: Red`, the following approvals are required: - [ ] BT/Data VP-level - [ ] Security (Security Risk Management Officer) - [ ] Privacy (Lead Legal Counsel - Privacy) - [ ] Not applicable - this data source is already extracted and it concerns extending the current pipeline **Admin** * [ ] Create issue for creation of extract process (not needed if using Stitch/FiveTran) * [ ] Create issue for dbt models * [ ] Create and link merge requests for updating relevant documentation **Who should be responsible for making the data accessible and usable in the data warehouse?** - [ ] Analyst: <!-- please tag them --> - [ ] Analytics Engineer <!-- please tag them --> - [ ] Data Engineer: <!-- please tag them --> **Who can we contact in the case of external errors or source freshness problems** - [ ] Ensure the [data source contact sheet](https://docs.google.com/spreadsheets/d/1VKvqyn7wy6HqpWS9T3MdPnE6qbfH2kGPQDFg2qPcp6U/edit#gid=0) is updated. <!-- Do not edit below this line -->