accessing plugin config from a plugin's pre_hook or post_hook raises RuntimeError
I have a custom plugin for my mailman installation for which I would like the plugin to access the configuration
attribute from the plugin's section of mailman.cfg so it can load plugin-specific configuration values. The plugin tries to do this by accessing config.plugin_configs[self.name].configuration
(where config
is imported via from mailman.config import config
) because it appears the config is not passed into either pre_hook
or post_hook
. This results, however, in a RuntimeError
being raised by a dictionary under iteration in the plugin setup code in src/mailman/core/initialize.py
.
Here is a minimal plugin that reproduces the issue:
diff --git a/crash-var/etc/crasher.cfg b/crash-var/etc/crasher.cfg
new file mode 100644
index 000000000..22ea0adb7
--- /dev/null
+++ b/crash-var/etc/crasher.cfg
@@ -0,0 +1,2 @@
+[general]
+foo: 1
diff --git a/crash-var/etc/mailman.cfg b/crash-var/etc/mailman.cfg
new file mode 100644
index 000000000..4967906ec
--- /dev/null
+++ b/crash-var/etc/mailman.cfg
@@ -0,0 +1,4 @@
+[plugin.crasher]
+class: crasher.Plugin
+enabled: yes
+configuration: crasher.cfg
diff --git a/src/crasher/__init__.py b/src/crasher/__init__.py
new file mode 100644
index 000000000..1aeb659a9
--- /dev/null
+++ b/src/crasher/__init__.py
@@ -0,0 +1,17 @@
+from mailman.config import config
+from mailman.interfaces.plugin import IPlugin
+from zope.interface import implementer
+
+
+@implementer(IPlugin)
+class Plugin:
+ def pre_hook(self):
+ # NOTE: Accessing config.plugin_configs from the pre_hook or post_hook results in this error:
+ # RuntimeError: dictionary changed size during iteration
+ print(config.plugin_configs[self.name].configuration)
+
+ def post_hook(self):
+ pass
+
+ resource = None
+
Attempting to load this configuration results in the exception (some parts redacted):
$ tox -e py38-nocov --notest -r
py38-nocov recreate: /home/XXX/Projects/Mailman/mailman/.tox/py38-nocov
py38-nocov installdeps: flufl.testing>=0.8, nose2
py38-nocov develop-inst: /home/tdyas/Projects/Mailman/mailman
py38-nocov installed: aiosmtpd==1.2,alembic==1.4.2,atpublic==1.0,authheaders==0.13.0,authres==1.2.0,certifi==2020.4.5.1,chardet==3.0.4,click==7.1.2,coverage==5.1,dkimpy==1.0.4,dnspython==1.16.0,falcon==2.0.0,flufl.bounce==3.0.1,flufl.i18n==2.0.2,flufl.lock==3.2,flufl.testing==0.8,gunicorn==20.0.4,idna==2.9,importlib-resources==1.5.0,lazr.config==2.2.2,lazr.delegates==2.0.4,-e git+git@gitlab.com:mailman/mailman@ae1f48b0b125996a1c779916b669915e1a455ac4#egg=mailman,Mako==1.1.2,MarkupSafe==1.1.1,nose==1.3.7,nose2==0.9.2,passlib==1.7.2,publicsuffix2==2.20191221,python-dateutil==2.8.1,python-editor==1.0.4,requests==2.23.0,six==1.15.0,SQLAlchemy==1.3.17,urllib3==1.25.9,zope.component==4.6.1,zope.configuration==4.4.0,zope.deferredimport==4.3.1,zope.deprecation==4.4.0,zope.event==4.4,zope.hookable==5.0.1,zope.i18nmessageid==5.0.1,zope.interface==5.1.0,zope.proxy==4.3.5,zope.schema==6.0.0
_____________________________________________ summary _____________________________________________
py38-nocov: skipped tests
congratulations :)
$ .tox/py38-nocov/bin/mailman -C crash-var/etc/mailman.cfg
Traceback (most recent call last):
File ".tox/py38-nocov/bin/mailman", line 11, in <module>
load_entry_point('mailman', 'console_scripts', 'mailman')()
File "/home/XXX/Projects/Mailman/mailman/.tox/py38-nocov/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/XXX/Projects/Mailman/mailman/.tox/py38-nocov/lib/python3.8/site-packages/click/core.py", line 781, in main
with self.make_context(prog_name, args, **extra) as ctx:
File "/home/tdyas/Projects/Mailman/mailman/.tox/py38-nocov/lib/python3.8/site-packages/click/core.py", line 700, in make_context
self.parse_args(ctx, args)
File "/home/XXX/Projects/Mailman/mailman/.tox/py38-nocov/lib/python3.8/site-packages/click/core.py", line 1212, in parse_args
rest = Command.parse_args(self, ctx, args)
File "/home/XXX/Projects/Mailman/mailman/.tox/py38-nocov/lib/python3.8/site-packages/click/core.py", line 1048, in parse_args
value, args = param.handle_parse_result(ctx, opts, args)
File "/home/XXX/Projects/Mailman/mailman/.tox/py38-nocov/lib/python3.8/site-packages/click/core.py", line 1630, in handle_parse_result
value = invoke_param_callback(self.callback, ctx, self, value)
File "/home/XXX/Projects/Mailman/mailman/.tox/py38-nocov/lib/python3.8/site-packages/click/core.py", line 123, in invoke_param_callback
return callback(ctx, param, value)
File "/home/XXX/Projects/Mailman/mailman/src/mailman/bin/mailman.py", line 94, in initialize_config
initialize(value)
File "/home/XXX/Projects/Mailman/mailman/src/mailman/core/initialize.py", line 218, in initialize
initialize_2(propagate_logs=propagate_logs)
File "/home/XXX/Projects/Mailman/mailman/src/mailman/core/initialize.py", line 163, in initialize_2
for name in config.plugins: # pragma: nocover
RuntimeError: dictionary changed size during iteration
$
I worked around the issue by giving my plugin a constructor that accesses plugin_configs
with a hard-coded plugin name, but that is ultimately just a hack and not a fix.
It would be great if the plugin setup code in src/mailman/core/initialize.py
did not trigger this exception. Even better would be if the IPlugin interface supported passing in plugin-specific configuration.
I'm willing to submit a Merge Request to fix, but would love feedback on what the preferred solution here is.