Retry downloads immediately after a curl 56 error
Some sites use a crawler blocking measure where they give an empty response for the first access. Browsers will retry accessing the site right away.
It's not a happy solution but I suppose the crawler ought to try something alike as well. Perhaps with a parser config flag to only enable that behavior for those sites that require it. An example site that does this: http://www.pennyblackfeather.co.uk/blog/?feed=rss2