Draft: PoC: Enhanced IAM Role Support for S3 Audit Event Streaming

Overview

This PoC implements a robust solution for using AWS IAM roles instead of access keys for S3 audit event streaming, addressing critical security concerns raised by enterprise customers like Amazon.

Problem Statement

Current S3 audit streaming requires long-lived AWS access keys, which:

• Pose security risks and don't align with AWS best practices
• Are problematic for customers with strict security policies
• Cannot leverage AWS's credential provider chain

Solution Highlights

✅ Database-First Approach: Proper migration before code changes ✅ Smart Model Validation: Conditional validation based on authentication method ✅ Enhanced Error Handling: Custom exceptions and comprehensive logging ✅ Simplified JSON Schema: Let model handle complex validation logic ✅ Comprehensive Testing: Unit, integration, and error scenario coverage ✅ Full Backward Compatibility: Zero disruption to existing configurations

Key Improvements Over Previous PoC

1. Proper Implementation Order: Database migration → Model → Logic → API
2. Better Validation Strategy: Model-based instead of complex JSON schema
3. Enhanced Error Handling: Custom exceptions with proper error tracking
4. Comprehensive Testing: All authentication scenarios and edge cases covered
5. Clear Documentation: Complete setup and usage instructions

Implementation Details

Database Changes

• Add use_iam_profile boolean column with proper defaults
• Maintain backward compatibility with existing configurations

Model Enhancements

• Conditional validations for access keys (only required when not using IAM)
• Mutual exclusion validation (cannot use both authentication methods)
• Helper method use_iam_profile? for cleaner code

AWS S3 Client Updates

• Support both authentication methods with clear separation
• Enhanced error handling for credential issues
• Comprehensive logging for debugging

GraphQL API Updates

• Add optional use_iam_profile argument
• Make access key arguments conditional
• Update types and interfaces appropriately

Usage Examples

IAM Profile Configuration

mutation {
  auditEventsAmazonS3ConfigurationCreate(
    input: {
      groupPath: "my-group"
      bucketName: "audit-logs"
      awsRegion: "us-east-1"
      useIamProfile: true
    }
  ) {
    amazonS3Configuration { id name useIamProfile }
    errors
  }
}

AWS Setup

1. Create IAM role with S3 permissions
2. Attach role to GitLab instance/container
3. Configure streaming with useIamProfile: true

Testing Coverage

• ✅ AWS S3Client with both authentication methods
• ✅ Model validation for all scenarios
• ✅ Streaming destination logic
• ✅ GraphQL mutations and error handling
• ✅ Backward compatibility scenarios
• ✅ Error handling and logging

Risk Assessment: LOW

• Well-understood AWS SDK patterns
• Minimal code changes required
• Strong backward compatibility
• Comprehensive test coverage
• Clear customer demand

Effort Validation: Small-Medium (1 engineer, 1 milestone) ✅

The implementation confirms the original estimate:

• Database: Simple column addition
• Logic: Straightforward conditional authentication
• API: Optional parameter addition
• Testing: Comprehensive but manageable scope

Customer Impact

• Amazon: Addresses security concerns about long-lived credentials
• Enterprise: Enables compliance with strict security policies
• All Customers: Optional enhancement, zero disruption

Next Steps

1. Review implementation approach and architecture
2. Validate test coverage and error handling
3. Consider feature flag strategy for gradual rollout
4. Plan documentation updates for end users

---

Addresses: #454683 Documentation: See POC_IAM_ROLES_README.md for complete details