Performance issue when downloading/retrieving data
Wondering if a performance issue has recently been introduced in the transfer activities.
Despite the known issue of the 30sec. timeuout (dotstatsuite-kube-rp#3 (closed)) where for instance downloaded files are truncated because the request took more than 30 seconds, we are experiencing unexpected long-time requests for short queries that should only take few seconds.
Steps to reproduce
- go to http://dlm-staging-oecd.redpelicans.com/
- select space=OECD-staging and type=dataflows
- look for a DF with data, e.g. Labour force status by educational attainment, all countries
- in the hamburger of the DF, select Export data > in CSV format
The request takes more than 30 seconds and the download file is therefore truncated, BUT this query shouldn't take that long.
Structures all OECD.EDU-EDU_NEAC_EAG_LFS_2-1.1-all.xml
