In application notifications for code changes
Problem to solve
It is often the case in a big team that you work on some part of the project and you want to be notified when some file/directory is changed. It can be just a button on top of the files tree. One more table (project_id, watch_path, user_id), one more notification type.
It is possible to subscribe to file changes already via atom feed, but it would be nicer to have notifications directly in the application.
Further details
This would be useful in many situations:
- Subscribe to changes to a particular page in the handbook
- Subscribe to changes to certain code when you know that downstream code will break unless it's also updated
- Subscribe to changes to OKRs too keep up with company direction
Proposal
Add a Watch button to the repository interface to subscribe to changes to a file on the default branch
Whenever changes are pushed to the default branch, if the push contains changes to the watched path, the user will receive:
- todo
- email notification (same as https://docs.gitlab.com/ee/user/project/integrations/emails_on_push.html)
| Email Notification | TODO |
|---|---|
 |
 |
A new section will be available in the user Notification Settings (https://gitlab.com/profile/notifications) to see a list of all the notifications and manage their subscription
Edge cases:
- file renames: if a file is renamed the subscription will not be updated automatically
Customers
https://gitlab.lightning.force.com/lightning/r/Account/00161000004yxj9AAA/view (low)
Links / references
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.
Designs
Relates to
- #1285516.75
Activity
Sounds good @vsizov. I think this can be very interesting for large projects in large organisations. I propose we make this an EE feature cc @dzaporozhets
also look at this https://gitlab.com/gitlab-org/gitlab-ce/issues/13326
Edited by Valery Sizov-
@creamzy OK, I'm surprised that mentions do not ping you :)
Edited by Valery Sizov 
@vsizov they do, but as milestone is not set, we just check and push it to the end of the pipeline, and without UX label, the end is dark and cold. Thanks for the label
@vsizov much love!
Milestone changed to 8.7
Milestone changed to 8.8
Any chance to see this in CE too?
We have the same situation that @dzaporozhets mentioned, where some files are very important to some people of the front end, but not so much for the back-end team.
Reassigned to @hazelyang
Just would like to make sure the use case of this functionality I assume below is correct.
- When the user clicks the
watchbutton, it will appear a list includingmain branchandall branches. - Then, the user selects the
one of all branches, if any files in the branch are changed, the user will get the notification through email.
If there is something wrong, please let me know.
And also list of watched paths in the user profile.
Does
list of watched pathsmean all of files and directories in the branch the user selected?- When the user clicks the
@freeman2kbr Thanks for the explanation! It's clearer for me.
My proposed design:
If you have any feedback, please let me know.
Screen 01
Have a
Watchbutton on this screen.
Screen 02
After clicking the button, the state will change to
Watchingand then the file or folder will be added to watch list on profile page. If you want to disable watch, you just click the button again.
Screen 03
Back to the root (gitlab-ce), there will appear a
binoculars iconnext to the name of the folder (or the file) you are watching.
Screen - Profile page
Adding watch list page to show the list of files and folders you are watching.
Is it by going to the file and then clicking the binoculars?
@doug.edey Yes.
-
Looks awesome, @hazelyang!
How would this integrate with the existing notification system? I wrote in https://gitlab.com/gitlab-org/gitlab-ce/issues/13815 a bit about how emails on push are already separated from this, but it also highlights a problem with our custom notification level: we currently assume that it's less than or equal to watch, but there are some notifications we want to make available as custom notifications only (like this). /cc @godfat
@smcgivern I think we certainly need to stop assuming that
watch= all notifications at some point. (or maybe renamewatchto beall:trollface:) So as we extend notifications,watchshould just be a set of notifications we pick and think it's useful in most cases. Notification levels shouldn't really be ordered, too. (don't define which is greater than the other if possible, just assume any of them is a set of values)We also probably need to put this somehow discoverable in notifications settings, not just in watch list.
It could be very challenging to make gitlab-ce
19492 nice regarding to gitlab-ce18332 settings... which is also a beast like permission settings, unfortunately :x Well, I believe it's easier than permission though!Edited by Lin Jen-Shin-
@godfat I agree, I'm going to quote myself from elsewhere:
We might want to consider a (painful) migration to change the
NotificationSettingmodel to have columns for each notification type, and have the different levels fill those in as required. So the watch level would be something like: subscribed + participating + new note + new issue + reopened issue + ..., but any level could be easily converted to a custom level. I've been meaning to comment on this issue, when it was updated last week, but forgot to, until now. Being able to track file changes, has always been popular in Enterprise and it's actually supported by Atlassian's Fisheye product. In all the environments that I've worked at, developers, team leaders, management, etc. are always looking for ways to reduce risk and file change notifications, is usually one of the supported/encouraged solutions.
On paper, notifications based on file changes seems pretty straightforward and it is, but what people never really think about, are the noise related issues. In a lot of ways, you end up creating a "please spam me list", which negates a lot of the intended benefits.
Notifications based on code changes, which includes tracking file changes, i.e. this issue, is something I've been thinking about for years and it drives a lot of the requirements for my search and analytics engine. I personally think notifications around code changes, will play a greater role in the software development life cycle, and I'll briefly explain how I'm tackling this problem, which might help you better understand this issue.
What has been discussed so far, is pretty much how I've seen file tracking being handled when I worked in Enterprise. And how Atlassian handles it with Fisheye, which basically involves:
- You tell the system what files you want to track
- The system will notify you when it detects changes involving said files
What hasn't been discussed, are the challenges with notifying at scale and how to deal with information overload.
Notifying at scale
Creating notifications based on repository pushes, doesn't require a lot resources. Creating notification based on what files were changed, is a lot more resource intensive ... relatively speaking. If GitLab wants to pursue this feature, it would probably make sense to move this to EE or provide two separate solutions. One for CE and one for EE, that can work at scale.
Another thing to consider is, how much value will people get, from only knowing changes to files on the default branch or in all branches?
When I worked in the Enterprise world, we dealt with multiple releases and people wanted to know when changes occurred in the different releases. If branch level file notifications are going to be supported, the complexity will increase by an order of a magnitude, since merge commits will have to be analyzed, to determine what they bring along with them.
Dealing with information overload
GitLab's development repositories are good examples, of how active repositories can be in Enterprise, and if we only take a look at what has changed on gitlab-ce::master, for files in
app/assetsand to theCHANGELOG.mdfile, we can see that, you'll receive a steady stream of notifications.
In the last 3 months, you would have received 1,353 notifications. And on January 9th, you would have received a flurry of notices.
Grokking 151 commits, with 316 changes can present a challenge. As I mentioned to @victorwu, I'm introducing a paradigm shift, with regards to how we can approach search and discovery, and it's to address issues like these and GitLab is free to copy what I've done.
My solution is not built around the idea of viewing one commit at a time, which is how GitLab, GitHub, GitWeb, and others currently work. You can view commits one at a time, but its main purpose, is to collapse multiple commits, into something easier to grokk. The changes tree that I have:
Is designed to let you navigate multiple commits at once. With the changes tree, you can quickly see what files were changed and if you don't care about changes to certain directories/files, you can easily ignore them.
And to view the commits, from a human aspect, you can use the contributors information.
With this, you can see 99% of the changes came from Clement Ho. So if you wanted to know what changed, it's probably quicker to just call him, and ask for a short abstract of what changed.
Basically, GitLab should look at UX solutions, to help mitigate the spamming effect, as this will certainly be an issue for larger companies.
Since this post is much longer than I had expected, I'll basically wrap things up, by saying, you may want to consider building your notification system around your search engine. This is how my notification system is designed to work, and by doing so, I can provide more than just file change notifications.
I can provide notification based on code changes. For example, if somebody modified a function, my diffs search will catch this. If somebody wants to be notified when ever an issue/merge request/etc. is mentioned, my commits search will catch this. Basically, whatever they can search for, they can create rules to be notified with.
In essence, what I'm doing is providing Google alerts, which simplifies a lot of things for me and is probably a route, that GitLab should explore.
@smcgivern : This has been requested in this thread and elsewhere. How realistic is it if we consider actioning on it for 9.2?
I would take @hazelyang 's mockups, and
notnow just implement the binoculars per file view (and maybe even not consider directories if that's additional separate work. And I wouldn't even care about showing a list of notifying files for a user.Does our existing notification system support an expansion such as this?
Edited by Victor Wu-
@victorwu I think it's possible. I don't understand what you mean by not implementing the binoculars per-file / directory. Without either of those, isn't this just emails on push?
Questions:
- If someone pushes (a single push) and updates a file in two commits in that push, should you get emailed once or twice?
- What would the emails look like?
- Could we call this something else other than watching? We already use that for our other notifications, and integrating this with our existing notifications would mean that term was overloaded: https://docs.gitlab.com/ee/workflow/notifications.html
- I'd like to do https://gitlab.com/gitlab-org/gitlab-ce/issues/24892 first
-
Sorry @smcgivern . Sometimes small typos make a huge difference. I corrected it.
-
Ah, right! Yes, that makes much more sense
My other questions still stand, and I think @terrchen makes some great points. Note that for Elasticsearch (which is EE-only, although not enabled on GitLab.com) we currently only index the default branch of a project.
-
For 2. and 3. in https://gitlab.com/gitlab-org/gitlab-ee/issues/1817#note_24928304, let's finalize that together with UX closer to when we want to action on it. If 4. is a blocker, yeah, let's work on that first anyways.
For https://gitlab.com/gitlab-org/gitlab-ce/issues/24892, is that entirely a code refactoring issue? Would there be an customer-facing change in UI or functionality there?
-
@victorwu no user-facing change, but if we want to add anything new notification-wise and don't do that, things will get more painful.
-
@smcgivern: Sounds good. Could we do https://gitlab.com/gitlab-org/gitlab-ce/issues/24892 and this one together then in the same release, if/when we do it.
-
@victorwu I'd personally like to do gitlab-ce#24892 first, because it involves migrating data over several stages, which may need to be done in the .0, .1, and .2 releases (for example) of a major version: https://gitlab.com/gitlab-org/gitlab-ce/issues/26130
-
Thanks @smcgivern . I've indicated that gitlab-ce#24892 is a dependency for this issue here, in the description. Let's consider that one more likely next first. thanks.
If anyone is interessted, we've created a workaround with gitlab-ci & docker. You can watch single or multiple files (no directories) and send diffs to configured email adresses.
https://github.com/pace-car/docker-diff-notification-service
Demo
Hello @tschwaerzl. I am looking at the docker-diff-notification-service repo you shared. I have a question about the installation instructions. I see where you set the secret variables and add a snipped to the .gitlab-ci.yml file, but I don't see how you implement the ruby file. Also do you have to install some executable to run the ruby code? Thank you!
-
@mikebutak It's part of the docker image
image: pacecar/docker-diff-notification-service. You need to be able to run the docker image. In doubt feel free to create your own image based on theDockerfileprovided in the repository.
mentioned in issue #3250 (closed)
An alternative approach to solving the problem of always wanting a certain teams to have eyes on certain code changes: https://github.com/blog/2392-introducing-code-owners
This doesn't solve an observer wanting to watch files like a CHANGELOG, but that specific use case seems better solved by subscribing to an RSS feed of changes or releases.
removed from epic &4 (closed)
added Product Vision FY20 label
added Create [DEPRECATED] and removed ~2278658 labels
marked #3250 (closed) as a duplicate of this issue
removed Plan [DEPRECATED] label
