Skip to content

[Config] refactoring to track set config values and unify config object with metadata

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Problem

The current configuration system has a tight coupling between environment_config and config that creates several problems:

  1. Lost context: Once environment variables are parsed into the config object, we lose the connection to which environment variable was set by the user and what the value was. This makes it difficult to provide helpful error messages that reference specific configuration variables.

  2. Duplication: We maintain two parallel structures - environment_config (with ENV values) and config (with typed values) - leading to redundant definitions and code. This makes a start on implementing deduplication.

  3. Difficult to extend: Creating new, smaller config objects (like a bootstrap config for scan summary observer) requires either copying large amounts of tightly-coupled parsing code or duplicating the existing pattern.

  4. No traceability: When errors occur during validation or at runtime (e.g., timeouts), we cannot easily trace back to the environment variable name to tell users which configuration to adjust.

  5. Complex parsing code: The translation from environment_config to config involves complex intermediate code that's all strongly coupled to these two specific objects.

Current architecture

Environment Variables
        ↓
environment_config (ENV values)
        ↓ (translation)
config (typed values)
        ↓ (context lost)
Usage (no reference to original env variable set)

Proposed Solution

Replace the current split between environment_config and config with a unified configuration approach using a ConfigValue wrapper type that carries metadata alongside values.

Architecture

type ConfigValue[T any] struct {
    Value          T      // The parsed, typed value
    EnvName        string // The environment variable name used (may be alias)
    CanonicalName  string // The documented/canonical variable name
    OriginalValue  string // The raw string value before parsing
}

Single config object approach

Instead of two separate objects, have one config object where:

  • Fields are defined as ConfigValue[Type] (e.g., ConfigValue[time.Duration])
  • Parsing happens inline with the field definitions
  • Environment variable names are captured during parsing
  • No translation phase needed between two separate structures

Implementation Details to follow

Current (two objects):

Proposed (single object):

type Config struct {
    Username ConfigValue[string] `env:"DAST_AUTH_USERNAME"`
}

// Access value: config.Username.Value
// Access metadata: config.Username.EnvName, config.Username.CanonicalName

Benefits

Immediate benefits

  1. Better error messages: Can reference specific environment variable names in all error messages, not just validation errors
   Error: connection timeout exceeded
   To adjust, set DAST_TARGET_CHECK_TIMEOUT (currently: 60s)
  1. Single source of truth: One config object instead of two parallel structures
  2. Easier to create smaller configs: Can create focused config objects for specific components without copying complex parsing machinery
  3. Runtime context: Can reference configuration metadata at any point during execution, not just during parsing

Long-term benefits

  1. Alias handling: Can show canonical name in documentation links even when user used an alias
  2. Better help text: Can programmatically generate help documentation from config struct
  3. Audit trail: Can log which configuration values were explicitly set vs defaulted
  4. Debugging: Can trace configuration decisions back to source

Technical Approach

Initially: Prototype with new bootstrap config on subset of config

Start by implementing this approach for a new, minimal bootstrap configuration needed for scan summary observer and logger initialization:

  1. Create ConfigValue[T] wrapper type (structure already POCed by @DavidNelsonGL in this branch)
  2. Add field to EnvValue to pull in envName
  3. Define bootstrap config struct with ConfigValue fields
  4. Implement parsing logic that captures metadata
  5. Test the approach in limited scope

Example bootstrap config:

type BootstrapConfig struct {
	TargetURL     ConfigValue[string]        `env_name:"DAST_TARGET_URL"`
}

Draft acceptance criteria for MVC:

  • ConfigValue[T] type defined with value and metadata fields (initially just the current name not the alias)
  • EnvValue enhanced to track which env var name was used
  • Bootstrap config uses ConfigValue wrapper for all fields
  • Parsing correctly populates both value and metadata
  • Can access both config.Field.Value() and config.Field.EnvName()
  • Error messages reference specific environment variable names
  • No dependencies on existing environment_config or config objects
  • .Value() accessor provides clean access to typed values

Notes

  • Details may fluctuate as part of the POC, and I will update the comments and description with evolving details.

CC: @gitlab-org/secure/dynamic-analysis

Edited by 🤖 GitLab Bot 🤖