Mailinglist functionality

SourceForge used to have a mailinglist. Right now everyone wanting a mailinglist for their project is required to use Google Groups.

It would be nice if this could be done with GitLab. We already have the ability to read email for the reply-to-email feature. And we already have service desk https://docs.gitlab.com/ee/user/project/service_desk.html that implements similar functionality.

We should model it after the git mailing list https://git.wiki.kernel.org/index.php/GitCommunity and the linux kernel mailing list http://vger.kernel.org/vger-lists.html#linux-kernel so that you can subscribe and unsubscribe by sending an email.

1. Active for every project at mailinglist+group/project@example.com (no need to be able to turn off and on, nobody is subscribed by default and people can just unsubscribe
2. Subscribe by sending 'subscribe' as the first word in the body.
3. Unsubscribe by sending 'Unsubscribe' as the first word in the body.

Optional feature: People posting a single issue don't have to subscribe to a potentially high volume list: you will be Cc'd in replies. Please keep the Cc list intact when replying (use "Reply to all"). See https://git-scm.com/community

Optional feature: Store the emails. The first iteration will not store any emails and there will be no archive.

/cc @dzaporozhets @JobV @DouweM

Zendesk issue: https://gitlab.zendesk.com/agent/tickets/14493

Added ~14213 label

Added ~122770 label

Added ~123454 label

How does this align with issues?

If we have email actions for issues (like !create), one could just mention everyone in the project and create an ad-hoc mailing list like that. We could even make it work with !mailinglist, which would do that in one command.

@JobV To make this feature easier to understand I propose we just have people email gitlab-ee/gitlab.org@gitlab.com to email everyone subscribed to the mailinglist and don't work with actions.

Hi everyone, I was the one who tweeted @gitlab about this. Thanks for making the issue @sytses.

My personal experience with gitlab is fairly small. I run it on a server to back up private code (a much older version) and just made my account on gitlab.com this week. I'm not a major stakeholder so perhaps my suggestions don't fit inline with gitlab's values. Given this information, here are some of my suggestions

• Online View

Making sure the email list can be accessed through an online message board. This services 3 purposes. 1. Making it simple for someone to post to the board. 2, Discoverability. If the mailing list is indexable, it'll make it easy search for questions, bugs or feature requests from the past. 3, Linkability, being able to share past threads using a URL is very, very useful.

• Basic Moderation Tools

Spam and harassment are fairly common on several mailing lists I frequent. Most of the issues are regarding promotion or recruitment. It would be nice to block people from emailing the list.

• Subscription Levels

One of the best features of google group is the ability choose your subscription level. You can follow just threads you've posted, get a daily/weekly summary or get nothing at all.

I've had other ideas but I think I'll leave it to Gitlab's PMs.

Thanks @sunnyrjuneja for the feedback, that's great.

We usually work from a minimal implementation and then look at feedback to plan further implementation.

For now, I think we'll leverage GitLab's architecture, so:

• subscription is based on the project you subscribe to the mailinglist to
• moderation is handled by the fact that you need an account on the instance to be able to email it
• not sure about what we should visualise in the first iteration.

I think it makes sense to have a basic visualization https://gitlab.com/gitlab-org/gitlab-ce/mailinglist/ in the first edition. Spam and moderation can wait because of the projects you subscribed to ask @JobV mentioned. Subscription frequency is needed but can also wait.

That sounds very reasonable. I'm very excited for this feature. Thanks again for considering it.

Added Zendesk issue link in description. A customer is interested in this feature.

@JobV Will you please schedule this?

@dblessing I moved this to 'upcoming' in direction, for review.

Milestone changed to 8.8

mentioned in issue #14237 (moved)

Reassigned to @JobV

Added ~182256 label

Milestone changed to 8.9

I think we should postpone this until we figure out reply-by-email for GitLab.com @sytses

Milestone changed to 8.11

Milestone removed

Mentioned in issue #17516 (closed)

https://gitlab.com/gitlab-org/gitlab-ce/issues/4272#note_11814605 that has been addressed by now, right?

removed ~123454 label

This is the only blocker I know about some projects not being able to migrate to gitlab yet.

Last conversation was a year ago, do you see the light at the end of the tunnel? :)

added auto updated potential proposal labels

/cc @victorwu

added Plan [DEPRECATED] label

This is the one thing I miss the most after moving my project. The ability to see a history in a mailing list would be much appreciated

changed the description

@JobV I updated the description to make it simpler. Can we mark this accepting merge requests? And CE?

added [deprecated] Accepting merge requests label

removed assignee

@JobV Before we mark this ~"Accepting Merge Requests", maybe we should get the design (technical and visual) more clear. For instance a few questions:

• Will this be driven by issues?
• If so, we need deeper threading support. We need to determine how Closed/Opened is represented?
• If not, where will we store the emails (database?)? Some visual mock ups would be advised.

These issues might be related:

• Email subscriptions for issues and merge requests gitlab-org/gitlab-ce#39669
• Reply by Email - mbox/maildir support gitlab-org/gitlab-ce#19524

@toon when someone intends to contribute this we can consider the exact details of implementation. Signalling that we're interested in this and that we're willing to accept contributions is a good start, rather than doing work upfront for something that might not end up being implemented.

It makes sense to have it be driven by issues. We don't need to support deeper threading in a first iteration. Nor do we have to consider how closed/open is represented (it simply isn't).

My impetus for wanting this is:

The project I use the most is postgresql jdbc. Historically all issues, discussion, patches came through a mailing list. It was all in one place. Now all the issues are on github. The mailing list sees almost no traffic.

The challenge for me is that it is very difficult to find the history of a conversation about a particular problem because it is now split into commits and issues.

Ideally I'd like to aggregate everything back into a single searchable list. The mailing list just seemed to be the simplest.

It also allows people without git accounts to monitor and or provide feedback.

Hopefully my situation may help drive the requirements.

Dave Cramer

@toon Thanks for asking.

• Will this be driven by issues? => No
• If not, where will we store the emails (database?)? Some visual mock ups would be advised. => We won't for the first iteration, I updated the description to reflect this.

changed the description

Hi all, I'd be interested in working on such a feature, but I'd like to understand it a bit more, to see if I'd be able to implement it.

So, in the first interaction, there will be an address mailinglist+group/project@example.com where people can send "subscribe" and "unsubscribe".

General questions:

• Are we sure about @example.com? I am asking because of How I hacked hundreds of companies through their helpdesk. While I understand it is the easiest thing, maybe we should consider to have a dedicate subdomain?
• Are we sure we want the use the same address to subscribe?
• Do we want a 1:1 relation with the instance users', or any email can subscribe?
• Is this active only for public projects? If not, how we can protect private information in private projects? Do we want to do so?
• There will some email that will be automatically sent by Gitlab to the ML? Like every issue thread? (This can be a question for a second iteration)

Technical questions:

• We have basically a relation between a project and n email addresses. Where do we save them? I think there is a project table in the db? (I think I can figure it out, but I still haven't look at the code)
• How do we manage email confirmation?

• Are we sure we want the use the same address to subscribe?
• Is this active only for public projects? If not, how we can protect private information in private projects? Do we want to do so?

To tackle these two downs, maybe we do subscribe and unsubscribe on another email, something like mailinglist+group/project+subscribe+randomGibberish@example.com?

Also, I think we should also think about how to remove someone from a ML (like, someone removed from a private project).

Or maybe in the first interaction we go only for public projects, and then we add the feature to internal/private project after having implemented a proper authorization / control system?

mentioned in issue #30366 (closed)

@rpadovani Thank you for taking initiative in this proposal!

Let me try to answer a few questions:

• Are we sure about @example.com? I am asking because of How I hacked hundreds of companies through their helpdesk. While I understand it is the easiest thing, maybe we should consider to have a dedicate subdomain?

Agreed, this is a security concern. We have to apply the same security considerations: https://docs.gitlab.com/ee/administration/reply_by_email.html#security-concerns

• Are we sure we want the use the same address to subscribe?

I like your proposal with +subscribe. Although I don't really understand why we need the +randomGibberish?

• Do we want a 1:1 relation with the instance users', or any email can subscribe?

I don't think that's needed in a first iteration, but it would be nice if active GitLab users can easily subscribe with a push of a button. Anyhow, it should also be possible for non-GitLab users to subscribe, because that's the point of having a mailing list and not using issues instead.

• Is this active only for public projects? If not, how we can protect private information in private projects? Do we want to do so?

Good question. But like you proposed, maybe just start off with public projects.

• There will some email that will be automatically sent by Gitlab to the ML? Like every issue thread? (This can be a question for a second iteration)

Let's iterate on that later.

• We have basically a relation between a project and an email addresses. I think there is a project table in the db?

The projects table exists, but try to avoid adding columns to that table, because it's already too big. You can create a new table mailing_lists which has a foreign key project_id. We can store other settings about the mailing list there too.

Where do we save them?

At first, everything for the mailing list would be mail-driven. So subscribe and unsubscribe are done by sending emails (because then anyone can subscribe). We would keep these subscriptions in a new table (e.g. mailing_list_subscriptions). It would contain an email address and a foreign key mailing_list_id.

---

Besides that I have a few other questions/remarks:

• Where would we store the mails themselves? They can become quite large, so storing them in the database won't be a good idea.