Skip to content

Fixes bug which prevented download of yearly file

Adithya Venkateswaran requested to merge bugfix/skip_file_download into master

For years before 2019, only yearly files are present in SWPC. The if condition handling these files failed to add the basename (the one passed to SWPC) to the basename list. This meant that these files were not downloaded. This has been fixed.

A test has been added for fetching large intervals of time as well.

A couple of statements had the logging section wrong (used logging instead of LOGGER).

How do I test?

$ python sample_script.py
2020-08-04 21:43:05,170 - vinvelivaanilai.space_weather.sw_file_fetch - INFO - Attempt number: 1 to fetch index DGD
2020-08-04 21:43:05,170 - vinvelivaanilai.space_weather.sw_file_fetch - DEBUG - Getting files list
2020-08-04 21:43:05,170 - vinvelivaanilai.space_weather.sw_file_fetch - DEBUG - List of files ['/home/adithya/dev/test_vv/2018_DGD.txt', '/home/adithya/dev/test_vv/2019Q1_DGD.txt']
2020-08-04 21:43:05,171 - vinvelivaanilai.space_weather.sw_file_fetch - DEBUG - /home/adithya/dev/test_vv/2018_DGD.txt does not exist
2020-08-04 21:43:05,171 - vinvelivaanilai.space_weather.sw_file_fetch - DEBUG - /home/adithya/dev/test_vv/2019Q1_DGD.txt does not exist
2020-08-04 21:43:06,991 - vinvelivaanilai.space_weather.sw_file_fetch - INFO - 230 Anonymous access granted, restrictions apply
2020-08-04 21:43:07,810 - vinvelivaanilai.space_weather.sw_file_fetch - INFO - Fetching file at /home/adithya/dev/test_vv/2018_DGD.txt
2020-08-04 21:43:10,419 - vinvelivaanilai.space_weather.sw_file_fetch - INFO - Fetching file at /home/adithya/dev/test_vv/2019Q1_DGD.txt
2020-08-04 21:43:13,135 - vinvelivaanilai.space_weather.sw_extractor - INFO - Extracting data from /home/adithya/dev/test_vv/2018_DGD.txt
2020-08-04 21:43:13,180 - vinvelivaanilai.space_weather.sw_extractor - INFO - Extracting data from /home/adithya/dev/test_vv/2019Q1_DGD.txt

sample_script.py

Signed-off-by: Adithya Venkateswaran adithyav1511@gmail.com

Merge request reports