Support early downloading and preprocessing of forecasts
This is marked as WIP because I'd like to delay merging until we've tested the approach in the 201906 run.
This merge request adds a
prepare_forecasts target to the workflow generator that runs all steps associated with downloading forecasts, bias-correcting them, and writing them to the
forcing folder in the format expected by WSIM. This lets a user begin the monthly run early. Importantly, this also lets the work happen when forecast files are still in the 7-day rolling archive, before they potentially get lost in the transition to the long-term archive.
The merge request also adds a
--forecast-lag-hours argument to the generator. This lets a user omit creating preprocessing steps for forecasts based on initial conditions that are less than
N hours old. This supports the use of the
prepare_forecasts target in a cron job, where we want to avoid reporting failure when we could not download and process forecasts that have not yet been issued.
I set up a job at http://sycamore.corp.isciences.com:9000/job/WSIM%20Download%20CFSv2%20Forecasts/ that will generate and run the
prepare_forecasts every 6 hours. We'll see how it performs in advance of the 201906 run.