BUG: Delta boms not able to load from tar.gz bundle
name: Bug Report
about: Create a report to help us improve
Hoppr Framework Version : 1.8.7
Python Version : 3.11.0rc1
Platform:
Distributor ID: Ubuntu
Description: Ubuntu 22.04.2 LTS
Release: 22.04
Delta capability doesn't work with tar'd bundles, as identified in the CLI help options. There are two possibilities here.
- The tar is too big; the bundle I'm testing is 13 gigs, which could be a memory error.
- Hoppr can't load a tar.gz
Example:
$ hopctl bundle --transfer test/integration/IntTest_1/transfer.yml --log hoppr_log.txt --verbose test/integration/IntTest_1/manifest.yml
[2023-06-26 15:48:04,997] - [HopprProcessor--26] - [INFO] - Collecting metadata file manifest.yml
[2023-06-26 15:48:04,998] - [HopprProcessor--26] - [INFO] - Collecting metadata file transfer.yml
[2023-06-26 15:48:04,998] - [HopprProcessor--26] - [INFO] - Collecting metadata file credentials.yml
[2023-06-26 15:48:05,767] - [HopprProcessor--26] - [INFO] - Beginning Hoppr Process execution, max_processes=10
[2023-06-26 15:48:05,768] - [HopprProcessor--26] - [INFO] - ====================================== Beginning Stage Delta =======================================
[2023-06-26 15:48:06,115] - [DeltaSbom--26-1] - [DEBUG] - Starting DeltaSbom.pre_stage_process
[2023-06-26 15:48:06,115] - [DeltaSbom--26-1] - [INFO] - Creating delta/update SBOM, previous SBOM being retrieved from previous-bundle.tar.gz
[2023-06-26 15:48:06,116] - [DeltaSbom--26-1] - [ERROR] - Unexpected exception running DeltaSbom.pre_stage_process:
[2023-06-26 15:48:06,146] - [DeltaSbom--26-1] - [ERROR] - Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/hoppr/base_plugins/hoppr.py", line 82, in wrapper
result = func(self, *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/hoppr/core_plugins/delta_sbom.py", line 52, in pre_stage_process
previous_sbom = self._get_previous_bom(previous_source) # pyright: ignore[reportGeneralTypeIssues]
File "/usr/local/lib/python3.10/site-packages/hoppr/core_plugins/delta_sbom.py", line 90, in _get_previous_bom
Manifest.load(Path(source))
File "/usr/local/lib/python3.10/site-packages/hoppr/models/manifest.py", line 469, in load
manifest_file = cls.parse_file(path)
File "/usr/local/lib/python3.10/site-packages/hoppr/models/manifest.py", line 224, in parse_file
data = hoppr.utils.load_file(path)
File "/usr/local/lib/python3.10/site-packages/hoppr/utils.py", line 113, in load_file
content: str = input_file.read()
MemoryError
Expected outcome:
Hoppr loads manifest from bundled gzip.