According to the RFCs List-Post should be left alone and a new one not added, and a new List-ID field should be added when there is already one, but in both cases Mailman 2 replaces an existing field's contents with data for the child list.
RFC 2919 specifies that if the existing List-ID is not a parent list (eg, somebody resent a message from an unrelated list to this one, or reposted from a nested list to a parent), then it should be removed. RFC 2369 doesn't say that explicitly (I didn't check very carefully, though), but I think it should behave the same way: existing List-* fields that aren't from a parent list should be removed.
ISTR there is at least one more RFC for List-* fields, should check.
Designs
Child items
...
Show closed items
Linked items
0
Link issues together to show that they're related or that one is blocking others.
Learn more.
I looked up the RFC's and understand the issue. But I am unable figure out how nested lists can be created(I looked up the docs too). Could you please guide?
In the simplest case two lists are nested if one is a member of the other. E.g., list2 is a member of list1, A post to list1 gets various headers and decorations added and is then delivered to the members which include list2. In this case, list1 is the Parent and list2 is the nested list.
I'm not sure how relevant this is for Mailman 3, where we don't have nested lists. I'm also not sure I really want to add nested lists. The way I'd always thought about supporting something similar was through composable rosters. E.g. if you wanted to send a message to both ListA and ListB, you'd probably create a roster that combines both memberships. Would you create a new mailing list for this? Not sure! Definitely an area for further discussion and thought.
Note that a 'nested list' doesn't have to be a member of an 'umbrella list' per se. I don't think anything in MM 3 prevents subscribing the posting address of list2 to list1. Granted without list1 being an 'umbrella list' there may be difficulty with administrative messages to list2, but it is certainly possible to have a list be a member of another list whether or not we say that there's a better way to accomplish whatever the goal is. In those cases, we should be RFC compliant with our headers, although determining whether or not the upstream list is a 'parent' may be tricky.
I don't see how you support "composable rosters" without creating a list in some sense. In my case I have a seminar for the students I advise, composed of undergraduate students and graduate students. I also keep contacts for "old boys". Finally, although the undergraduate advising is only for senior year, the relationship is multiyear for the grad students, and each year has unique administrativia to deal with. So I want to send messages to current students and old boys, and it's OK and desirable to have two addressees there. But for other messages, I just want to have one, so I have an "all-students" address, the current "ug-year" address, a "grad" address, and one "grad-year" address for each class of grad students. The all-students and grad-year addresses are moderated, to encourage discussion in the other, more appropriate groups for discussion. So each year, I move a couple of laggards to the appropriate list for their new expected graduation, move the classes of this year to old-boys, and add new lists for the incoming students at ug and grad level. Oh, and I unmoderate the grad-year when I move it to old-boys.
In summary, yes, I have rosters that I can compose, but I really do want each roster addressable separately by its own address. If a roster with an address (and configurable access!) isn't a mailing list, what is it? (And what is a mailing list for that matter?) And can you convince users? I think that even if you can convince me that "composable rosters" is a better idea than "nested mailing lists", it will be hell to document and support for typical list owners.
Regard Mark's issue of "tricky to determine parents", I don't want to say too much until Anirudh has presented a design ;-), but I don't think it's a unsolvable problem. You know the existing list posting address and ID, you know the recipient list, so constructing an SQLAlchemy query shouldn't be hard. Or you could automatically update a parents attribute for the list at subscription time.
If a roster with an address (and configurable access!) isn't a mailing list, what is it?
That's pretty much the definition of it, I think!
There's little mechanism, and no ui around this currently, but I do think that creating a lightweight mailing list that doesn't maintain its own roster, but composes them from other existing rosters is doable. It might not be a nested list in @msapiro 's sense, but it's how I imagine supporting @yaseppochi 's use case described above. I like it a lot more than subscribing one list to another.
I have been trying my hands with injecting the mail into a nested list structure from mailman shell and seeing how the headers change, but finally after a lot of time spent on that, I referred to the pipeline handlers and have the following views :
Irrespective of whether we go ahead with composable rosters or not, I feel that we must be RFC compliant in case a list subscribes to another list(but imagine a list being a subscriber to itself, this could maybe crash the system??)
I think that the proposed idea of querying the database is a simple solution, but am afraid that if the mailing lists/recipients increase too much, it could lead to the server slowing down.
The other idea of updating a parents attribute when the mail is received in the sub-list seems better, where we can check sender address being in the parents attribute of the sub-list, and add a flag in meta-data dict accompanying the message through the pipeline. Then handlers would be responsible for checking the flag in the meta-data dict and doing necessary changes to the headers in the message.
@warsaw As Mark says, people will subscribe lists to lists (especially across organizations -- you can't compose a roster you can't read!) How about denoting the composition by "subscribing" another list, but detecting that it is your own list and doing the composition behind the scenes?
However, getting this RFC-ly correct isn't going to be easy. As with many RFC specifications, it's the user's (list admin's) intention (nesting lists) that should determine behavior. So in this case posts should still get two List-IDs (or more, in my example a post to "zemi" goes to "g-zemi" then to each "grad201x" for several values of "x", giving 3 instances). Possibly we can reduce burden on the mail system by doing this processing internally (recursing into the pipelines of the nested lists), but I think conceptually it's easiest to think of subscribing another list, and delivering to that list's mailbox, then redistributing it.
Consider diagnosing "why am I getting these posts, I'm not subscribed to that list?" problems. In that use case we want RFC-ly correct handling of List-ID. (Minor, I admit, and hardly reliable since List-ID is optional.)
Here's a related problem: archiving. If, as in my case, posts are addressed to lists at all levels of nesting, you might want the explicit posting address to archive that message, and have the sublists not. OTOH, you do want direct posts to the sublist to be archived. So you might want a mechanism to set a no-archive flag on redistribution to sublists (actually, if the parent has set a List-Archive or List-Archived-At field, the sublist can decide).
@yaseppochi Could you please elaborate upon what we mean by recursing into pipelines of nested lists here.
Anyways here's what I had in mind. When the sub-list subscribes to a parent list(just the parent here, not the grandparent and so on ) , we add the parent list to the list in parent lists attribute of the sub list. Now when we receive the mail from parent list to sublist,the sublist checks that the mail is from parent list. Now it adds its own headers according to the RFC using a special handler we would need to make for this case(i.e. Sublists SHOULD replace the parent list's List-Help, List-Subscribe,List-Unsubscribe and List-Owner fields).
I have a few doubts about some headers:
I think you mean to say that list-id of the parent list should be left untouched(so as to diagnose the problem of why I am receiving these mails), am I correct here?
About archiving, the RFC states '''If the sublist provides its own archive, it SHOULD replace the List-
Archive with its own. Otherwise, it MUST leave the List-Archive field
untouched.''' So I think that in case of list having its own different archive, we must change the header and send the message to that archive. If not, should we set the no archive flag?
I have to say, I think we should postpone dealing with this level of complexity until after 3.1. This would make for a good whiteboard discussion at Pycon, but there are enough other things that we really have to solve before we can get into nested lists.