Bounce runner doesn't validate email addresses
At Wikimedia, running Mailman 3.3.3, in the bounce runner's debug log we're seeing stuff like:
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email /xml is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email xml is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email /p is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email /head is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email br is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email /a is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email /center is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email o:PixelsPerInch is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email ![endif]-- is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email td role="module-content" is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email !--user entered Head Start-- is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email center is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email table is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email /table is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email tbody is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email td width="100%" is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email /tbody is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:57 2021 (29499) Bounce message for a non subscriber: Email !--<![endif]-- is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email title is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email meta http-equiv="X-UA-Compatible" content="IE=Edge" is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email /o:OfficeDocumentSettings is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email head is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email o:OfficeDocumentSettings is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email td is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email /div is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email /body is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email o:AllowPNG/ is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email style type="text/css" is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email style is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email /strong is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email /style is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email !--End Head user entered-- is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email strong is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email notify.wikipedia.org@gurlap.com is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email tr is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email !--[if !mso] is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email /td is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email td width="100%" valign="top" bgcolor="#ffffff" is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email !--[if (gte mso 9)|(IE)] is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email !--[if mso] is not a subcriber of vereinde-l.lists.wikimedia.org
May 09 10:05:58 2021 (29499) Bounce message for a non subscriber: Email /tr is not a subcriber of vereinde-l.lists.wikimedia.org
Occasionally this crashes the bounce runner entirely:
May 9 09:38:37 lists1001 mailman3[19017]: sqlalchemy.exc.DataError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (pymysql.err.DataError) (1406, "Data too long for column 'email' at row 1") [SQL: 'INSERT INTO bounceevent (list_id, email, timestamp, message_id, context, processed) VALUES (%(list_id)s, %(email)s, %(timestamp)s, %(message_id)s, %(context)s, %(processed)s)'] [parameters: {'list_id': 'wikimediaannounce-l.lists.wikimedia.org', 'email': 'p style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: -webkit-standard; font-style: normal; font-variant-caps: normal; font-weight: n ... (67 characters truncated) ... ne; white-space: normal; word-spacing: 0px; -moz-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none; text-align: center; "', 'timestamp': datetime.datetime(2021, 5, 9, 9, 38, 37, 601049), 'message_id': '<E1lf5Ua-0004qv-M2@lists1001.wikimedia.org>', 'context': 1, 'processed': 0}] (Background on this error at: http://sqlalche.me/e/9h9h)
I don't fully understand how it extracts the email from the bounce, but I'd think that it could at least validate the email address before trying to process it and record the event?
Edited by legoktm