Skip to content

Update NdjsonExtractor to support file extraction from object storage in offline transfer

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Update NdjsonExtractor to use a new file download service, BulkImports::OfflineFileDownloadService, for offline transfer. This must be behind a feature flag. The new file download service should use the object storage client to fetch files from object storage. Updating the existing NdjsonExtractor allows existing, simple ndjson pipelines to function in offline and direct transfer since they do the same ETL process, except files are downloaded from object storage.

Edited by 🤖 GitLab Bot 🤖