## Overview This epic covers the implementation of the PyPI virtual registry feature in GitLab. The PyPI virtual registry will leverage our existing virtual registry architecture to provide a complete solution with multiple upstream sources, UI management, and cache clearing capabilities. The beta phase will focus on validating performance and reliability with real-world Python package usage patterns. ## Goals - Beta implementation of a PyPI virtual registry with full feature parity to our Maven and npm virtual registries - Provide comprehensive UI and API interfaces for creating, configuring, and managing PyPI virtual registries - Ensure seamless package resolution and proxying from multiple upstream sources - Validate performance and reliability with real-world usage patterns ## Scope ### In Scope - Create a PyPI virtual registry with multiple upstream sources - Configure and manage shareable upstream configurations - Full UI integration for registry management - List and search virtual registries - Deleting a virtual registry - Package resolution and proxying from configured upstreams - Cache clearing capabilities - Comprehensive error handling and logging - Performance monitoring and optimization ### Out of Scope - Package publishing to virtual registries (planned for post-beta) - Advanced caching policies beyond those already implemented for Maven and npm ## User Stories 1. As a DevOps engineer, I want to create a PyPI virtual registry via UI and API so that I can start aggregating packages from multiple upstream sources. 2. As a developer, I want to configure multiple upstream sources for my virtual registry so that I can specify the order and priority of package resolution. 3. As a project maintainer, I want to share upstream configurations across multiple registries so that I can ensure consistency and reduce configuration overhead. 4. As a system administrator, I want to clear the cache for specific packages or entire registries so that I can ensure my teams have access to the latest versions. 5. As a developer, I want to resolve and download Python packages through the virtual registry so that I can use it as a single source for my project dependencies. 6. As an administrator, I want to monitor the performance and usage of my PyPI virtual registries so that I can optimize them for my organization's needs. ## Acceptance Criteria - UI and API interfaces are fully implemented and functional - Virtual registries correctly proxy packages from multiple configured upstreams - Shareable upstream configurations work correctly across registries - Cache clearing functions operate as expected - Error scenarios are properly handled and logged - Documentation clearly explains all features and configuration options - Performance meets established benchmarks for high-volume Python package usage - Test coverage meets GitLab standards ## Metrics for Success - Successful creation and configuration of registries via UI and API - Correct package resolution and download through virtual registries - Response times within acceptable thresholds under load - Error rates below defined thresholds - Cache hit ratios meeting or exceeding targets - Positive feedback from beta customers regarding usability and reliability ## Adoption Tracking and Analytics - Implement instrumentation to track the adoption and usage of virtual registries, including: * Total number of groups with the feature enabled * Number of packages pulled from virtual registries * Cache hit ratio * UI usage data (e.g., number of virtual registries per group, number of upstream sources per virtual registry) ## Beta Customer Engagement Plan - **Recruitment**: Target 5-10 key customers with high Python package usage for beta participation - **Onboarding**: Provide detailed documentation and dedicated support for initial setup - **Feedback Collection**: * Bi-weekly check-ins with beta participants * Structured feedback form for feature usability and performance * Bug reporting channel for immediate technical issues - **Success Tracking**: * Monitor usage metrics across beta participants * Track package resolution success rates and performance * Document customer use cases for future marketing materials - **Graduation Criteria**: * Zero P1 bugs reported for two consecutive weeks * Performance metrics meeting or exceeding targets * 80%+ of beta participants actively using the feature