Make package metadata parsing errors recoverable
Summary
PackageMetadata Sync execution can be halted if imported data is missing attributes.
Steps to reproduce
Run sync processing the advisory json below.
export timestamp=$(date +%s)
mkdir -p "vendor/package_metadata/advisories/v2/conan/$timestamp"
mv 000000000.ndjson vendor/package_metadata/advisories/v2/conan/$timestamp/
bin/rails console
lease = Gitlab::ExclusiveLease.new('sync-advisories', timeout: 1.hour); lease.try_obtain
PackageMetadata::SyncService.execute(data_type: 'advisories', lease: lease)
An ArgumentError is thrown and execution stops for the duration of the lease.
000000000.ndjson
000000000.ndjson
{"advisory":{"id":"950539e3-9c79-455d-8778-d0e343e6f0db","source":"glad","title":"Excessive Iteration","description":"Issue summary: Checking excessively long DH keys or parameters may be very slow.\n\nImpact summary: Applications that use the functions DH_check(), DH_check_ex()\nor EVP_PKEY_param_check() to check a DH key or DH parameters may experience long\ndelays. Where the key or parameters that are being checked have been obtained\nfrom an untrusted source this may lead to a Denial of Service.\n\nThe function DH_check() performs various checks on DH parameters. After fixing\nCVE-2023-3446 it was discovered that a large q parameter value can also trigger\nan overly long computation during some of these checks. A correct q value,\nif present, cannot be larger than the modulus p parameter, thus it is\nunnecessary to perform these checks if q is larger than p.\n\nAn application that calls DH_check() and supplies a key or parameters obtained\nfrom an untrusted source could be vulnerable to a Denial of Service attack.\n\nThe function DH_check() is itself called by a number of other OpenSSL functions.\nAn application calling any of those other functions may similarly be affected.\nThe other functions affected by this are DH_check_ex() and\nEVP_PKEY_param_check().\n\nAlso vulnerable are the OpenSSL dhparam and pkeyparam command line applications\nwhen using the \"-check\" option.\n\nThe OpenSSL SSL/TLS implementation is not affected by this issue.\n\nThe OpenSSL 3.0 and 3.1 FIPS providers are not affected by this issue.","cvss_v3":"CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L","published_date":"2023-07-31","urls":["https://nvd.nist.gov/vuln/detail/CVE-2023-3817","https://www.openssl.org/news/secadv/20230731.txt","https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=6a1eb62c29db6cb5eec707f9338aee00f44e26f5","https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=9002fd07327a91f35ba6c1307e71fa6fd4409b7f","https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=91ddeba0f2269b017dc06c46c993a788974b1aa5","https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=869ad69aadd985c7b8ca6f4e5dd0eb274c9f3644","http://www.openwall.com/lists/oss-security/2023/07/31/1","http://seclists.org/fulldisclosure/2023/Jul/43"],"identifiers":[{"type":"cve","name":"CVE-2023-3817","value":"CVE-2023-3817","url":"https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-3817"},{"type":"cwe","name":"CWE-1035","value":"1035","url":"https://cwe.mitre.org/data/definitions/1035.html"},{"type":"cwe","name":"CWE-834","value":"834","url":"https://cwe.mitre.org/data/definitions/834.html"},{"type":"cwe","name":"CWE-937","value":"937","url":"https://cwe.mitre.org/data/definitions/937.html"}]},"packages":null}
Example Project
n/a
What is the current bug behavior?
Encountering a data object with a missing expected attribute throws an error and stops the sync service.
What is the expected correct behavior?
No error should be thrown.
Possible fixes
Catch the error and track it.
Edited by Igor Frenkel