Verified Commit c7b6f736 authored by Antoine Beaupré's avatar Antoine Beaupré
Browse files

fix crash in feed2exec parse 

For some reason, I was getting this backtrace here when running a test
feed:

    plugin generated exception: expected str, bytes or os.PathLike object, not NoneType, skipping
    Traceback (most recent call last):
      File "/home/anarcat/src/feed2exec/feed2exec/plugins/__init__.py", line 105, in output
        return plugin.output(*args, feed=feed, item=item, lock=lock)
      File "/home/anarcat/src/feed2exec/feed2exec/plugins/maildir.py", line 42, in __init__
        prefix = os.path.expanduser(feed.get('mailbox', '~/Maildir'))
      File "/usr/lib/python3.7/posixpath.py", line 235, in expanduser
        path = os.fspath(path)
    TypeError: expected str, bytes or os.PathLike object, not NoneType

This was caused by the maildir plugin constructor, which was looking
up the `mailbox` field in the field. Because that is unset in my
feed2exec.ini (because I was testing against a new feed), that
`mailbox` key was just not present. The feedparser dict behavior in
this case is to return None, instead of the default, which I find
strange and unusual.

Tests still seem to pass here after the patch, so I guess this should
work in the future, and does un-break the feed test.

I can't quite figure out how to come up with a unit test for that one,
but now I can't help but think that feed configuration should just be
split up from the feedparser dict, because so much of those issues
come from that (strange) coupling.
parent a44497c1

feed2exec/model.py

+17 −0
Original line number Diff line number Diff line
@@ -65,6 +65,23 @@ class Feed(feedparser.FeedParserDict): 
        super().__init__(*args, **kwargs)

        self['name'] = name



    def get(self, key, default=None):

        """override upstream getter



        in my own configuration, the maildir output plugin is default,

        yet it doesn't have a maildir or folder defined. somehow in

        there, the getter here ends up returning None for those,

        because those keys do not exist.



        that's not exactly what we expect here: what we want is to

        return the default in those cases.

        """

        v = super().get(key, default)

        if v is None:

            return default

        else:

            return v



    def normalize(self, item=None):

        """normalize feeds a little more than what feedparser provides.