New RSS Drupal Security Advisories failing to parse

Background

Historically, security advisories were fetched via the RSS feeds for Core and Contrib advisories respectively. Advisories were issued as forum posts according to a template. Since the data was not validated, this often required hand parsing of the HTML in posts to determine project and version matching. The RSS feeds did not support pagination, so old advisories would become unavailable through that method.

As of October 2017, the Drupal security team has moved to a dedicated "sa" content type for security advisories, and a lot of the pertinent information can now be extracted via the Drupal.org API. After comms with the security team, methods were discovered to retrieve older core and contrib security advisories via "taxonomy_forums" codes (now added to their docs).

The API natively supports pagination on content, though there is currently a bug when using a .json or .xml extension. Accept: application/json headers don't currently work, as the query part of the URL is stripped off.

Current approach is either to wait for upstream bugfixes, or insert a .json into the URLs when following if there isn't already one there.