Jamie Tanna authored 1 year ago and Jamie Tanna committed 1 year ago

It's been deprecated officially for a week or so, but hasn't been
actually wired into Advisories, as we've had deps.dev's data which is
superior.

By removing this, we'll also hopefully reduce the time it takes folks to
build their databases, as osv.dev lookups were fairly slow.

As raised in #421, it'd be handy to announce whether the release
introduces breaking changes, until we've got to a point where we're a
stable release.

To do this is temporarily a bit of a hack, as we're waiting for upstream
to support this - via some contributions I've raised upstream - so in
the meantime we can pin to a pre-built binary and overwrite the
downloaded `hooks-goreleaser` with my fork.

This flags whether a major-inducing change is part of the release,
automagically detected by go-semantic-release.

Closes #421.

For instance with `internal/endoflifedate/db/schema.sql`, where it's the
Dependabot-related tables that are deprecated.

As it hasn't actually been working as expected 😅

We can use `blocks` as a slightly nicer way of formatting output. This
will release to the DMD Slack, and be an improved notification compared
to the GitLab/RSS feed options.

Jamie Tanna authored 1 year ago and Jamie Tanna committed 1 year ago

Similar to the `policy_violations` table, this introduces a single view
of all Advisories data that affects the dependencies in the database.

This makes it simpler to query all package advisories by "simply"
querying the `advisories` table, which includes all relevant information
for package advisories.

This allows us to remove a lot of duplication in our SQL queries for
looking up advisories data, making sure that it's a much simpler process
for querying data.

We can still directly query the tables as before - following the example
from `RetrievePackageAdvisoriesFromSeparateTables` - if we need to.

We can also remove the references in `policy_violations` that indicates
that `policy_violations` is better than `advisories`, as it no longer
is!

Because we want to make it easier to surface the `eol_from` and
`supported_until` dates, too, we should add them as nullable fields to
`advisories` - and defaulting it as `NULL` for tables that don't surface
the data.

Closes #414.

Jamie Tanna authored 1 year ago and Jamie Tanna committed 1 year ago

BREAKING CHANGE:

For some time, I've regretted the naming choice, as it can be a little
confusing because the `advisories` table doesn't include all the
advisories present in the dependency trees.

This now makes it more explicit by using the new name
`custom_advisories`.

Jamie Tanna authored 1 year ago and Jamie Tanna committed 1 year ago

It's not necessary, especially as we now have better database schema
documentation, so we can remove it.

This has been accidentally misaligned for some time, and will cause some
confusion when querying the data.

To keep this more consistent, we should make sure we return a known
`advisory_type`.

Closes #413.

Jamie Tanna authored 1 year ago and Jamie Tanna committed 1 year ago

As part of #412, it is noted that GitHub validates the pURL types[0]
that are sent to it, only allowing "known" pURL types to be consumed.

To make `renovate-to-sbom` a little more strict, we can use a new flag,
`only-include-known-purl-types`, which will trim the dependency output
to ensure that

This also adds a log notification to make sure it's visible that they're
being skipped.

We can use a `map` of `struct{}` to:

- make sure we have an indication of the pURL `type`s that are skipped
- are removing duplicates in the list of names
- reduce the allocations with a `struct{}`

Closes #412.

[0]: https://github.com/package-url/purl-spec/issues/286

Closes #402.

To allow us to more appropriately target organisational policies around
i.e. whether a repository is public, used for sensitive purposes, or
otherwise, we can now consume the data available in
`repository_metadata` with our Policies.