GitLab's annual major release is around the corner. Along with a lot of new and exciting features, there will be a few breaking changes. Learn more here.

migration.rst 6.17 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
Migrating from Mailman 2.1 to Mailman 3
=======================================

This guide covers the migration steps required for porting from Mailman 2.1 to
Mailman 3.


Why Upgrade?
------------

Mailman 3 is the new version and actively developed, as compared to 2.1, which
is now in maintenance mode and won't receive any feature updates.

Some of the reasons that might convince you to migrate to Mailman 3 include

- a well tested and rewritten code base,
- support for REST API (if you need to integrate with other services)
- support for multiple domains in same installation (without listname collisions)
- A real database backend for settings and configuration
- Modern Web UI for list administration and subscription management
- Support for social logins (and possibly LDAP/openid integration)
- Much improved and interactive archiver/forum with ability to reply from within
  the Archiver


Other considerations
--------------------

Before you upgrade, you should consider a few things like:

- URLs to archived messages will break, unless you take extra steps to keep them
  around. Upgrade mechanism makes sure to import all your archived messages in
  the new system, but, all the URLs to the new messages are going to be different.

  If you need your URLs for Mailman 2 archives to work, you can keep the HTML
  files generated for the archives around and your web server configuration for
  the archives intact (possibly with a notice to viewers that it is now a
  read-only archive, see `this list
  <https://mail.python.org/pipermail/security-sig/>`_ for example).

- The above mechanism won't work for private archives since the archives are
  gated with password and without a Mailman 2 list, there is no password. You
  can however import them to Mailman 3.

- Some configuration and settings aren't available in Mailman 3's UI yet, so
  even though those settings will be migrated to Mailman 3, you may not be able
  to change them from the Web UI today. All of those settings should be exposed
  in the UI very soon.


51 52 53 54 55
Before you upgrade
------------------

- Make sure that you have no pending subscription requests as those will not be
  ported over to Mailman3.
Mark Sapiro's avatar
Mark Sapiro committed
56

57 58 59 60
- Make sure that you don't have any pending emails in the digest mbox, if there
  are, you can force send the digests before moving to Mailman 3, as those won't
  be upgraded to Mailman3.

61 62 63 64

Upgrade strategy
----------------

65
As of now, there isn't a turn key solution to migrate all your lists from Mailman
66 67 68 69 70 71 72 73 74 75 76 77 78 79
2 to Mailman 3, although, the process is fairly automated. You need shell access
to your installation in order to perform the upgrade.

Mailman 3 is split in two main parts, the Core engine which includes all the
lists and their configuration and the Web UI, which includes the archiver and UI
for information in Core.

Before you start with migration, you need a working Mailman 3 instance, you can
see `here <http://docs.mailman3.org/en/latest/>`_  for recommendations on
installing Mailman.

Some key information that you should know about your Mailman 2 and Mailman 3
installations before you do the migration:

Mark Sapiro's avatar
Mark Sapiro committed
80 81
- Location of list configuration in Mailman 2: This is typically at
  ``$var_prefix/lists/LISTNAME/config.pck`` where ``$var_prefix`` is an
82 83
  installation dependent directory which is typically ``/usr/local/mailman`` or ``/var/lib/mailman``.

Mark Sapiro's avatar
Mark Sapiro committed
84 85
- Location of list archives in Mailman 2: This is typically at Mailman 2: This is at
  ``$var_prefix/archives/private/LISTNAME.mbox/LISTNAME.mbox`` where ``$var_prefix``
86 87
  is as above.

Mark Sapiro's avatar
Mark Sapiro committed
88
- Location of the ``bin/`` commands in Mailman 2: This is at ``$prefix/bin`` where ``$prefix`` is an
89 90 91 92 93 94 95 96 97
  installation dependent directory which is typically ``/usr/local/mailman`` or ``/usr/lib/mailman``


Steps for migration:

- Create the list you are trying to migrate in Mailman 3, for the purposes of
  this guide, we will call it ``foo-list@example.com``

- Migrate the list configuration from Mailman 2 to Mailman 3 by running the
98
  following command [#fn1]_ ::
99 100

	$ mailman import21 foo-list@example.com /path/to/mailman2/foo-list/config.pck
101 102


Mark Sapiro's avatar
Mark Sapiro committed
103 104 105 106 107
.. [#fn1] This, and in general all, ``mailman`` commands should be run as the
  Mailman user and not as ``root``.  Running some of these commands as
  root can create files owned by root that can't be read by the
  Mailman user.
 
108 109

- Migrate the list archives from Mailman 2 to Mailman 3 by running the following
Mark Sapiro's avatar
Mark Sapiro committed
110
  command [#fn2]_::
111

112
	$ python manage.py hyperkitty_import -l foo-list@example.com $var_prefix/archives/private/foo-list.mbox/foo-list.mbox
113

Mark Sapiro's avatar
Mark Sapiro committed
114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129

.. [#fn2] If the Mailman 2 list does not predate Mailman 2.1, its LISTNAME.mbox
  file is probably in good shape, but all mailboxes should be checked
  for defects before importing.  Certain defects such as missing Message-ID: headers or missing
  or unparseable Date: headers will be corrected or ignored by the import process. The one defect
  that will definitely cause problems is lines beginning with From in message bodies.  These
  will be seen as the start of a new message.  There is a Mailman 2 script at $prefix/bin/cleanarch.
  That can identify and fix most such lines, but it is not perfect.  Cases have been observed
  where a post includes in its body a copy of some other message including the From separator.
  This will normally occur only on an old list which includes spam messages or other email problems
  in its subject matter, but is something to be aware of.  Certain other
  message defects can cause the import to abort.  There is a
  ``check_hk_import`` script in ``hyperkitty/contrib`` that can find and
  report messages with these defects.


Mark Sapiro's avatar
Mark Sapiro committed
130
After this, you will need to rebuild the index for this list::
131 132 133

	$ python manage.py update_index_one_list foo-list@example.com

Mark Sapiro's avatar
Mark Sapiro committed
134
After this, you should be able to search your messages in Hyperkitty.
135 136 137 138 139 140 141 142 143 144 145 146 147

- Delete Mailman 2 list::

	$ $prefix/bin/rmlist foo-list


After this, you may need some additional steps based on if you want to keep your
old archives around or not. To add a notice to your list archives, you can edit
``index.html`` available at the root of your mailing list archives.

You may also want to add a redirect at old list page to automatically redirect
your users to Mailman3 list-info page. However, this process is fairly manual
depending on type of webserver you are using.