Be more robust to Mastodon API errors
Thanks for feed2toot! My Mastodon instance (like many) is flaky right now; API requests randomly fail. feed2toot
doesn't seem to handle this well. It exits with an unhandled exception, leaves a lockfile behind, and doesn't seem to record if some toots posted correctly (so they will be reposted on retry). Here's a specific example of a run I just tried with the errors (using --debug
). Some toots were posted but after this ran my feed2toot.db
was 0 bytes and feed2toot.lock
was left behind, even though the process exited. I think the server was sending 500 errors.
$ feed2toot --debug -c feed2toot.ini
configured stdout level 10
lockfile ./feed2toot.lock created.
found feed entry https://www.somebits.com/linkblog/b4bce0ea932d814e4e3c0ecab16fb964
Tooting with visibility "public": Saturn in daytime: Remarkable amateur astrophotography
https://twitter.com/gp_o11/status/1590392544650985472
#photography #astronomoy #stacking #saturn #photos #space #+
Starting new HTTPS connection (1): tech.lgbt:443
Starting new HTTPS connection (2): tech.lgbt:443
Traceback (most recent call last):
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.10/http/client.py", line 1374, in getresponse
response.begin()
File "/usr/lib/python3.10/http/client.py", line 318, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.10/http/client.py", line 287, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
raise value.with_traceback(tb)
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.10/http/client.py", line 1374, in getresponse
response.begin()
File "/usr/lib/python3.10/http/client.py", line 318, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.10/http/client.py", line 287, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/mastodon/Mastodon.py", line 3414, in __api_request
response_object = self.session.request(method, base_url + endpoint, **kwargs)
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/requests/adapters.py", line 547, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/nelson/src/mastodon/linkblog-toot/venv/bin/feed2toot", line 24, in <module>
Main()
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/feed2toot/main.py", line 50, in __init__
self.main()
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/feed2toot/main.py", line 123, in main
send_message(config, clioptions, options, entrytosend, finaltweet, cache, rss)
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/feed2toot/message.py", line 76, in send_message
twp = TootPost(config, options, finaltweet)
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/feed2toot/tootpost.py", line 30, in __init__
self.main()
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/feed2toot/tootpost.py", line 44, in main
mastodon.status_post(self.toot, visibility=toot_visibility)
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/mastodon/Mastodon.py", line 102, in wrapper
return function(self, *args, **kwargs)
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/mastodon/Mastodon.py", line 1837, in status_post
return self.__api_request('POST', '/api/v1/statuses', params, headers = headers, use_json = use_json)
File "/home/nelson/src/mastodon/linkblog-toot/venv/lib/python3.10/site-packages/mastodon/Mastodon.py", line 3416, in __api_request
raise MastodonNetworkError("Could not complete request: %s" % e)
mastodon.Mastodon.MastodonNetworkError: Could not complete request: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))