Define Production Readiness Criteria for GLAS PHP Support

Problem

We need to establish clear, measurable criteria for when GLAS PHP support can be considered production-ready and competitive with existing solutions.

Without defined evaluation criteria, we risk:

• Releasing before our solution is sufficiently competitive with existing tools
• Missing coverage of critical vulnerability types or frameworks
• Inconsistent coverage compared to our other supported languages
• Poor first impressions that could be difficult to overcome

Currently, we lack:

• Concrete metrics for evaluating detection efficacy
• Clear benchmarks against competitor capabilities
• Framework coverage requirements
• Performance baselines for PHP codebases

Proposal

Quality Readiness

1. Cross-File Analysis Capability
   • Primary objective: Support the most common dependency management patterns in PHP
   • Namespace imports (with and without alias)
   • Function imports (with and without alias)
   • Static imports (require, include, require_once, include_once)
   • Grouped use statements (with and without alias)
   • Const imports deemed non-critical after evaluation
   • Dynamic imports deferred to future implementation
2. Detection Quality
   • Precision Rate ≥ 80% across supported CWEs
   • Benchmarking using a comprehensive test corpus for each supported CWE
   • Testing against common PHP frameworks (Laravel, Symfony, WordPress)

Coverage Readiness

To be considered production-ready, GLAS PHP support should meet the following criteria:

1. Priority-Based Coverage
   • Must cover all critical PHP injection vulnerabilities:
     • SQL Injection (CWE-89)
     • Cross-site Scripting (CWE-79)
     • Command Injection (CWE-77, CWE-78)
     • Path Traversal (CWE-22)
     • Insecure Deserialization (CWE-502)
   • Must cover common PHP security misconfigurations:
     • Weak Cryptography
   • Must have solid coverage alignment with industry standards:
     • OWASP Top 10 PHP vulnerabilities
     • SANS CWE Top 25 (where applicable to PHP)

Support Readiness

Documentation must be complete in:

1. Supported languages
   • PHP listed and described as supported language
2. GitLab Advanced SAST CWE coverage
   • PHP column added to CWE coverage table
   • All supported CWEs properly marked

Additionally, the dynamic imports limitation should be documented.

Performance Readiness

1. Standard Performance:
   • Medium-sized applications: < 10 minutes
   • Very large-sized applications: < 30 minutes with --multi-core 2
2. Benchmarks:
   • Medium: Laravel (~266K LOC based on slocount metrics)
   • Large: WordPress (~310K LOC based on slocount metrics)
   • Very large: Symfony (~1480346 LOC based on slocount metrics)

Context

Initial discussion took place in this thread.