Split Secure common library into Go modules
Summary
The common library is a collection of Go packages used by all SAST, Dependency Scanning, and Container Scanning analyzers. Currently this library is a single Go module and all the Go packages share the same versions and changelog,. This is not flexible enough, and is expensive to maintain:
- When upgrading the
common
dependency in the analyzer projects, it's not clear why we are updating, and what the benefit is. - We cannot introduce a breaking change in any of the Go packages
common
is made of without upgrading the major number ofcommon
itself, which is expensive (extra maintenance in the analyzer projects using the common library). Actually, the analyzer projects might very well upgradecommon
to the next major version w/o any API change on their side, which is confusing. - We cannot remove any of these Go packages without upgrading the major version of
common
, even if this Go package isn't used by any analyzer - that's the case for theorchestrator
package.
The solution is to split common
into multiple Go modules. Each Go module has its own versions and its own changelog. A module can introduce a breaking change or can be removed without any side-effect in the analyzer projects that don't use it.
Technically each Go module has its own git tags. See documentation on Go modules.
Improvements
Better communication, reduced maintenance, increased flexibility.
Risks
Broken build in the analyzer projects.
Involved components
- common library
- analyzer projects
- possibly the orchestrator projects, that is sast and dependency-scanning, until we deprecate them
Optional: Intended side effects
- The issue package can be renamed, and its versions can match the ones of the common security report schema. See #8398 (closed)
Key Takeaways
- There are 3 types of modules we are dealing with:
-
framework cli modules (
command.go
) -
group-specific library modules (
ruleset.go
) -
shared library modules (
issue.go
,logutil.go
)
-
framework cli modules (
- Any modules we plan to split out should have a single owner.
- Group-specific modules should belong to their own project. The most immediate candidate for this is
ruleset.go
package, which is owned by SAST. - Framework cli modules should be split out and provided by each group individually. The clearest example of this is
command.go
, used by DS and SAST. This is likely to occur anyway as DS deprecates its use, but removal from common removes dead-code. - Shared library modules will be part of an ongoing effort to reduce conflicts in ownership. There are 2 groups, based primarily around number of depending groups
-
Utilities; i.e.
logutil.go
cacert.go
,pathfilter.go
. Most of these are quite stable and do not require updates. If they do, they should be considered for splitting out. An owner should be determined for these and either split the modules into separate projects or simply use CODEOWNERS until changes are needed. -
Interfaces; i.e.
issue.go
. Issue module is the highest touch but the most difficult to split. The main benefit for splitting our common is forced version upgrades sincecommon
is inextricably tied to this module. The best direction here is to refactor the module to reflect our schemas more closely using embedded structs and a "base issue" (struct SastIssue { Issue }
). We can then use CODEOWNERS to segment ownership of the individual structs.
-
Utilities; i.e.
- There's no immediate need for git submodules, but they can be a useful addition if we need to relink any dependencies in the future
Actions
- Clean up easy-wins to reduce surface area
- Split
common/template
into separate project - Remove
common/table
(unused)
- Split
- Split group-owned modules into separate project
-
ruleset.go
(owned by SAST)
-
- Refactor
issue.go
into Category-specific file-separated embedded structs with relevantCODEOWNERS
. This should mirror thesecurity-report-schemas
composition model. This could be done as a separate project to help with versioning - thus requiring opt-in.common/issue.go
is put in maintenance-mode, future updates will require teams to use the new structs. - [Consider] Placing
common
in maintenance mode for stable packages with future updates encouraging separation of modules.
Edited by Lucas Charles