Access request email notification not sent

Summary

I am running a GitLab Omnibus installation on an Ubuntu 20.04 server and using mailgun as a maildelivery service. When someone requests access to a project there should be an email notification going out to maintainers/owners. However, this is not happening. I can see the access request in GitLab itself but the email is never sent. All other notifications are sent without problems.

Steps to reproduce

Example Project

What is the current bug behavior?

Notification email is not sent. {"severity":"ERROR","time":"2021-01-11T09:37:10.169Z","correlation_id":"d149c1195a3d7a1cab3c7381455ba46d","message":"Cannot obtain an exclusive lease for namespace:namespaces_root_statistics:27. There must be another instance already in execution."}

What is the expected correct behavior?

Notification email is sent to maintainers/owners of the project.

Relevant logs and/or screenshots

This is logged every time an access request is sent: {"severity":"ERROR","time":"2021-01-11T09:37:10.169Z","correlation_id":"d149c1195a3d7a1cab3c7381455ba46d","message":"Cannot obtain an exclusive lease for namespace:namespaces_root_statistics:27. There must be another instance already in execution."}

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:         Ubuntu 20.04
Proxy:          no
Current User:   git
Using RVM:      no
Ruby Version:   2.7.2p137
Gem Version:    3.1.4
Bundler Version:2.1.4
Rake Version:   13.0.3
Redis Version:  5.0.9
Git Version:    2.29.0
Sidekiq Version:5.2.9
Go Version:     unknown

GitLab information
Version:        13.8.0-ee
Revision:       1ae10d09692
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     12.4
URL:            https://gitlab.enclustra.com
HTTP Clone URL: https://gitlab.enclustra.com/some-group/some-project.git
SSH Clone URL:  git@gitlab.enclustra.com:some-group/some-project.git
Elasticsearch:  no
Geo:            no
Using LDAP:     yes
Using Omniauth: yes
Omniauth Providers:


GitLab Shell
Version:        13.15.0
Repository storage paths:
  • default: /mnt/clustravault01/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
Checking GitLab subtasks ...

Checking GitLab Shell ...


GitLab Shell: ... GitLab Shell version >= 13.15.0 ? ... OK (13.15.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Gitaly ...


Gitaly: ... default ... OK


Checking Gitaly ... Finished


Checking Sidekiq ...


Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Checking Incoming Email ...


Incoming Email: ... Reply by email is disabled in config/gitlab.yml


Checking Incoming Email ... Finished


Checking LDAP ...


LDAP: ... Server: ldapmain
not verifying SSL hostname of LDAPS server 'dc.zrh.enclustra.com:636'
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
User output sanitized. Found 58 users of 100 limit.


Checking LDAP ... Finished


Checking GitLab App ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
6/1 ... yes
6/3 ... yes
6/4 ... yes
48/7 ... yes
48/8 ... yes
95/19 ... yes
6/21 ... yes
30/29 ... yes
37/30 ... yes
40/31 ... yes
30/35 ... yes
30/36 ... yes
40/37 ... yes
48/38 ... yes
47/39 ... yes
47/40 ... yes
47/41 ... yes
47/42 ... yes
47/43 ... yes
30/44 ... yes
30/45 ... yes
49/46 ... yes
49/47 ... yes
49/48 ... yes
49/49 ... yes
49/50 ... yes
49/51 ... yes
49/52 ... yes
49/53 ... yes
49/54 ... yes
49/56 ... yes
70/58 ... yes
26/59 ... yes
71/60 ... yes
82/61 ... yes
88/62 ... yes
89/63 ... yes
90/64 ... yes
91/65 ... yes
64/66 ... yes
74/67 ... yes
95/68 ... yes
95/72 ... yes
95/73 ... yes
96/74 ... yes
30/75 ... yes
99/77 ... yes
64/78 ... yes
39/79 ... yes
100/81 ... yes
70/83 ... yes
88/84 ... yes
89/85 ... yes
82/86 ... yes
90/87 ... yes
91/88 ... yes
106/89 ... yes
106/90 ... yes
108/91 ... yes
108/92 ... yes
109/93 ... yes
109/94 ... yes
110/95 ... yes
110/96 ... yes
110/97 ... yes
111/98 ... yes
111/99 ... yes
111/100 ... yes
39/101 ... yes
112/102 ... yes
113/103 ... yes
116/104 ... yes
30/106 ... yes
30/107 ... yes
30/108 ... yes
30/109 ... yes
30/110 ... yes
30/111 ... yes
30/112 ... yes
30/113 ... yes
30/114 ... yes
119/116 ... yes
119/117 ... yes
119/118 ... yes
119/119 ... yes
119/120 ... yes
30/121 ... yes
30/122 ... yes
30/123 ... yes
122/125 ... yes
71/126 ... yes
124/127 ... yes
1785/128 ... yes
1785/129 ... yes
1785/130 ... yes
126/131 ... yes
127/132 ... yes
127/133 ... yes
145/141 ... yes
129/153 ... yes
157/158 ... yes
63/190 ... yes
1786/227 ... yes
133/236 ... yes
49/775 ... yes
506/981 ... yes
959/2053 ... yes
961/2054 ... yes
63/2086 ... yes
48/2087 ... yes
58/2089 ... yes
1116/2515 ... yes
1117/2516 ... yes
1118/2517 ... yes
1119/2518 ... yes
506/2519 ... yes
1120/2520 ... yes
1120/2521 ... yes
506/2522 ... yes
506/2523 ... yes
1120/2524 ... yes
1120/2525 ... yes
1116/2526 ... yes
1116/2527 ... yes
1116/2528 ... yes
1116/2529 ... yes
1116/2530 ... yes
1116/2531 ... yes
1116/2532 ... yes
1116/2533 ... yes
1121/2534 ... yes
1121/2535 ... yes
1121/2536 ... yes
506/2537 ... yes
1122/2538 ... yes
1123/2539 ... yes
1119/2540 ... yes
1116/2541 ... yes
1116/2542 ... yes
1125/2543 ... yes
1125/2544 ... yes
1125/2545 ... yes
1125/2546 ... yes
1125/2547 ... yes
1125/2548 ... yes
1125/2549 ... yes
1125/2550 ... yes
1125/2551 ... yes
1125/2552 ... yes
1125/2553 ... yes
1125/2554 ... yes
1125/2555 ... yes
1125/2556 ... yes
1126/2557 ... yes
1126/2558 ... yes
1126/2559 ... yes
1126/2560 ... yes
1126/2561 ... yes
1126/2562 ... yes
1126/2563 ... yes
1126/2564 ... yes
1126/2565 ... yes
1126/2566 ... yes
1126/2567 ... yes
1126/2568 ... yes
1126/2569 ... yes
1126/2570 ... yes
1126/2571 ... yes
1126/2572 ... yes
1126/2573 ... yes
1126/2574 ... yes
1126/2575 ... yes
1126/2576 ... yes
1126/2577 ... yes
1126/2578 ... yes
1126/2579 ... yes
1126/2580 ... yes
1126/2581 ... yes
1126/2582 ... yes
1126/2583 ... yes
1126/2584 ... yes
1126/2585 ... yes
1126/2586 ... yes
1126/2587 ... yes
1126/2588 ... yes
1126/2589 ... yes
1126/2590 ... yes
1126/2591 ... yes
1126/2592 ... yes
1126/2593 ... yes
1126/2594 ... yes
1126/2595 ... yes
1126/2596 ... yes
1126/2597 ... yes
1126/2598 ... yes
1126/2599 ... yes
1126/2600 ... yes
1126/2601 ... yes
1126/2602 ... yes
1126/2603 ... yes
1126/2604 ... yes
1126/2605 ... yes
1126/2606 ... yes
1126/2607 ... yes
1128/2608 ... yes
1128/2609 ... yes
1130/2610 ... yes
1130/2611 ... yes
1130/2612 ... yes
1130/2613 ... yes
1130/2614 ... yes
1130/2615 ... yes
1130/2616 ... yes
1130/2617 ... yes
1130/2618 ... yes
1130/2619 ... yes
1130/2620 ... yes
1130/2621 ... yes
1130/2622 ... yes
1130/2623 ... yes
1130/2624 ... yes
1130/2625 ... yes
1130/2626 ... yes
1130/2627 ... yes
1130/2628 ... yes
1130/2629 ... yes
1130/2630 ... yes
1131/2631 ... yes
1131/2632 ... yes
1131/2633 ... yes
1131/2634 ... yes
1131/2635 ... yes
1131/2636 ... yes
1132/2637 ... yes
1132/2638 ... yes
1132/2639 ... yes
1132/2640 ... yes
1132/2641 ... yes
1132/2642 ... yes
1133/2643 ... yes
1133/2644 ... yes
1133/2645 ... yes
1133/2646 ... yes
1133/2647 ... yes
1133/2648 ... yes
1133/2649 ... yes
1133/2650 ... yes
1134/2651 ... yes
1136/2652 ... yes
1136/2653 ... yes
1136/2654 ... yes
1137/2655 ... yes
1138/2656 ... yes
1138/2657 ... yes
1138/2658 ... yes
1138/2659 ... yes
1139/2660 ... yes
1139/2661 ... yes
1139/2662 ... yes
1139/2663 ... yes
1139/2664 ... yes
1139/2665 ... yes
1139/2666 ... yes
1139/2667 ... yes
1139/2668 ... yes
1139/2669 ... yes
1139/2670 ... yes
1139/2671 ... yes
1140/2672 ... yes
1140/2673 ... yes
1140/2674 ... yes
1140/2675 ... yes
1140/2676 ... yes
1140/2677 ... yes
1140/2678 ... yes
1140/2679 ... yes
1140/2680 ... yes
1140/2681 ... yes
1140/2682 ... yes
1140/2683 ... yes
1140/2684 ... yes
1140/2685 ... yes
1140/2686 ... yes
1140/2687 ... yes
1140/2688 ... yes
1140/2689 ... yes
1140/2690 ... yes
1141/2691 ... yes
1142/2692 ... yes
1143/2693 ... yes
1144/2694 ... yes
1144/2695 ... yes
1144/2696 ... yes
1144/2697 ... yes
1144/2698 ... yes
1144/2699 ... yes
1144/2700 ... yes
1144/2701 ... yes
1144/2702 ... yes
1144/2703 ... yes
1145/2704 ... yes
1145/2705 ... yes
1145/2706 ... yes
1145/2707 ... yes
1145/2708 ... yes
1145/2709 ... yes
1145/2710 ... yes
1145/2711 ... yes
1145/2712 ... yes
1145/2713 ... yes
1145/2714 ... yes
1145/2715 ... yes
1146/2716 ... yes
1146/2717 ... yes
1146/2718 ... yes
1146/2719 ... yes
1147/2720 ... yes
1148/2721 ... yes
1148/2722 ... yes
1147/2723 ... yes
1147/2724 ... yes
1147/2725 ... yes
1147/2726 ... yes
1147/2727 ... yes
1147/2728 ... yes
1149/2729 ... yes
1150/2730 ... yes
1152/2731 ... yes
1153/2732 ... yes
1151/2733 ... yes
1151/2734 ... yes
1154/2735 ... yes
1156/2736 ... yes
1156/2737 ... yes
1156/2738 ... yes
1156/2739 ... yes
1156/2740 ... yes
1155/2741 ... yes
1157/2742 ... yes
1158/2743 ... yes
1160/2744 ... yes
1161/2745 ... yes
1162/2746 ... yes
1162/2747 ... yes
1163/2748 ... yes
1164/2749 ... yes
1165/2750 ... yes
1167/2751 ... yes
1167/2752 ... yes
1167/2753 ... yes
1169/2754 ... yes
1169/2755 ... yes
1170/2756 ... yes
1170/2757 ... yes
1171/2758 ... yes
1171/2759 ... yes
1172/2760 ... yes
1172/2761 ... yes
1172/2762 ... yes
1173/2763 ... yes
1173/2764 ... yes
1173/2765 ... yes
1174/2766 ... yes
1174/2767 ... yes
1174/2768 ... yes
1175/2769 ... yes
1176/2770 ... yes
1177/2771 ... yes
1178/2772 ... yes
1179/2773 ... yes
1180/2774 ... yes
1181/2775 ... yes
1182/2776 ... yes
1183/2777 ... yes
1184/2778 ... yes
1184/2779 ... yes
1185/2780 ... yes
1189/2781 ... yes
1189/2782 ... yes
1189/2783 ... yes
1189/2784 ... yes
1189/2785 ... yes
1189/2786 ... yes
1189/2787 ... yes
1189/2788 ... yes
1189/2789 ... yes
1189/2790 ... yes
1189/2791 ... yes
1189/2792 ... yes
1189/2793 ... yes
1189/2794 ... yes
1189/2795 ... yes
1189/2796 ... yes
1189/2797 ... yes
1189/2798 ... yes
1189/2799 ... yes
1189/2800 ... yes
1189/2801 ... yes
1189/2802 ... yes
1189/2803 ... yes
1189/2804 ... yes
1189/2805 ... yes
1189/2806 ... yes
1189/2807 ... yes
1190/2808 ... yes
1190/2809 ... yes
1190/2810 ... yes
1190/2811 ... yes
1190/2812 ... yes
1190/2813 ... yes
1190/2814 ... yes
1191/2815 ... yes
1193/2816 ... yes
1193/2817 ... yes
1193/2818 ... yes
1193/2819 ... yes
1193/2820 ... yes
1193/2821 ... yes
1193/2822 ... yes
1193/2823 ... yes
1193/2824 ... yes
1193/2825 ... yes
1193/2826 ... yes
1193/2827 ... yes
1193/2828 ... yes
1193/2829 ... yes
1194/2830 ... yes
1194/2831 ... yes
1194/2832 ... yes
1194/2833 ... yes
1195/2834 ... yes
1196/2835 ... yes
1197/2836 ... yes
1198/2837 ... yes
1199/2838 ... yes
1200/2839 ... yes
1200/2840 ... yes
1200/2841 ... yes
1200/2842 ... yes
1200/2843 ... yes
1202/2844 ... yes
1203/2845 ... yes
1204/2846 ... yes
1204/2847 ... yes
1204/2848 ... yes
1204/2849 ... yes
1204/2850 ... yes
1205/2851 ... yes
70/2852 ... yes
70/2853 ... yes
70/2854 ... yes
70/2855 ... yes
70/2856 ... yes
70/2857 ... yes
1206/2858 ... yes
1207/2859 ... yes
1207/2860 ... yes
1207/2861 ... yes
1207/2862 ... yes
1208/2863 ... yes
1208/2864 ... yes
1208/2865 ... yes
1209/2866 ... yes
1210/2867 ... yes
1211/2868 ... yes
1212/2869 ... yes
1212/2870 ... yes
1212/2871 ... yes
1212/2872 ... yes
1212/2873 ... yes
1212/2874 ... yes
1213/2875 ... yes
1212/2876 ... yes
1212/2877 ... yes
1214/2878 ... yes
1215/2879 ... yes
1216/2880 ... yes
1216/2881 ... yes
959/2882 ... yes
959/2883 ... yes
1217/2884 ... yes
1219/2885 ... yes
1219/2886 ... yes
1219/2887 ... yes
1219/2888 ... yes
1219/2889 ... yes
1220/2890 ... yes
1220/2891 ... yes
1220/2892 ... yes
99/2893 ... yes
1221/2894 ... yes
1222/2895 ... yes
1222/2896 ... yes
1223/2897 ... yes
1224/2898 ... yes
1225/2899 ... yes
1226/2900 ... yes
1226/2901 ... yes
1226/2902 ... yes
1227/2903 ... yes
1227/2904 ... yes
1227/2905 ... yes
1227/2906 ... yes
1227/2907 ... yes
1228/2908 ... yes
1229/2909 ... yes
1230/2910 ... yes
1230/2911 ... yes
1231/2912 ... yes
1231/2913 ... yes
1231/2914 ... yes
1231/2915 ... yes
1232/2916 ... yes
1232/2917 ... yes
1232/2918 ... yes
1233/2919 ... yes
1234/2920 ... yes
1234/2921 ... yes
1234/2922 ... yes
1234/2923 ... yes
1235/2924 ... yes
1235/2925 ... yes
1236/2926 ... yes
126/2927 ... yes
63/2928 ... yes
1251/2929 ... yes
506/2930 ... yes
64/2931 ... yes
49/2932 ... yes
49/2933 ... yes
49/2934 ... yes
49/2935 ... yes
1370/2936 ... yes
1270/2937 ... yes
1270/2938 ... yes
1271/2939 ... yes
1271/2940 ... yes
1271/2941 ... yes
39/3032 ... yes
1368/3084 ... yes
1147/3087 ... yes
1369/3088 ... yes
1782/3089 ... yes
1165/3145 ... yes
49/3146 ... yes
1373/3266 ... yes
1140/3837 ... yes
1769/3848 ... yes
126/3849 ... yes
1782/3850 ... yes
1783/3851 ... yes
1373/3852 ... yes
1116/3859 ... yes
1373/3862 ... yes
1787/3863 ... yes
1787/3864 ... yes
1787/3865 ... yes
1787/3866 ... yes
1374/3867 ... yes
1374/3868 ... yes
1374/3869 ... yes
1374/3870 ... yes
1374/3871 ... yes
1410/3872 ... yes
1411/3873 ... yes
1412/3874 ... yes
1420/3875 ... yes
1421/3876 ... yes
1414/3877 ... yes
1415/3878 ... yes
1413/3879 ... yes
1416/3880 ... yes
1373/3881 ... yes
115/3882 ... yes
115/3883 ... yes
1419/3884 ... yes
1372/3885 ... yes
1372/3886 ... yes
1373/3887 ... yes
104/3888 ... yes
104/3889 ... yes
1271/3890 ... yes
1265/3891 ... yes
1266/3892 ... yes
1266/3893 ... yes
1788/3894 ... yes
1264/3895 ... yes
1263/3896 ... yes
1263/3897 ... yes
1263/3898 ... yes
101/3899 ... yes
101/3900 ... yes
109/3901 ... yes
103/3902 ... yes
103/3903 ... yes
111/3904 ... yes
102/3905 ... yes
102/3906 ... yes
110/3907 ... yes
1257/3908 ... yes
1257/3909 ... yes
1789/3910 ... yes
1260/3911 ... yes
1260/3912 ... yes
1260/3913 ... yes
1790/3914 ... yes
1261/3915 ... yes
1261/3916 ... yes
1791/3917 ... yes
1262/3918 ... yes
1262/3919 ... yes
1270/3920 ... yes
1256/3921 ... yes
1256/3922 ... yes
1258/3923 ... yes
1258/3924 ... yes
1792/3925 ... yes
1259/3926 ... yes
1259/3927 ... yes
1793/3928 ... yes
1255/3929 ... yes
1255/3930 ... yes
1255/3931 ... yes
83/3932 ... yes
83/3933 ... yes
88/3934 ... yes
1418/3935 ... yes
84/3936 ... yes
84/3937 ... yes
89/3938 ... yes
84/3939 ... yes
87/3940 ... yes
81/3941 ... yes
81/3942 ... yes
82/3943 ... yes
81/3944 ... yes
85/3945 ... yes
85/3946 ... yes
90/3947 ... yes
85/3948 ... yes
86/3949 ... yes
86/3950 ... yes
91/3951 ... yes
105/3952 ... yes
105/3953 ... yes
106/3954 ... yes
107/3955 ... yes
107/3956 ... yes
108/3957 ... yes
1374/3958 ... yes
1374/3959 ... yes
506/3960 ... yes
1374/3961 ... yes
1374/3962 ... yes
1374/3963 ... yes
1374/3964 ... yes
1408/3965 ... yes
1794/3966 ... yes
1795/3967 ... yes
1794/3968 ... yes
1373/3969 ... yes
1372/3970 ... yes
1796/3971 ... yes
1416/3972 ... yes
1417/3973 ... yes
1797/3974 ... yes
1797/3975 ... yes
1797/3976 ... yes
1797/3977 ... yes
1797/3978 ... yes
119/3979 ... yes
1835/3980 ... yes
1802/3981 ... yes
1805/3982 ... yes
1806/3983 ... yes
1807/3984 ... yes
1809/3985 ... yes
1812/3987 ... yes
1814/3988 ... yes
1815/3989 ... yes
1810/3990 ... yes
1816/3991 ... yes
1818/3992 ... yes
1821/3993 ... yes
87/3994 ... yes
127/3995 ... yes
71/3998 ... yes
72/4001 ... yes
39/4002 ... yes
95/4003 ... yes
26/4004 ... yes
1832/4005 ... yes
112/4006 ... yes
1373/4007 ... yes
1373/4008 ... yes
92/4009 ... yes
92/4010 ... yes
1850/4011 ... yes
1419/4012 ... yes
1120/4013 ... yes
1852/4014 ... yes
1231/4015 ... yes
39/4016 ... yes
57/4017 ... yes
30/4018 ... yes
64/4019 ... yes
1119/4020 ... yes
1794/4021 ... yes
47/4022 ... yes
1184/4023 ... yes
1853/4025 ... yes
63/4026 ... yes
1419/4027 ... yes
1856/4028 ... yes
124/4029 ... yes
63/4031 ... yes
1419/4032 ... yes
1419/4033 ... yes
1419/4034 ... yes
1419/4035 ... yes
58/4037 ... yes
95/4038 ... yes
1853/4039 ... yes
71/4040 ... yes
72/4041 ... yes
Redis version >= 4.0.0? ... yes
Ruby version >= 2.7.2 ? ... yes (2.7.2)
Git version >= 2.29.0 ? ... yes (2.29.0)
Git user has default SSH configuration? ... yes
Active users: ... 92
Is authorized keys file accessible? ... skipped (authorized keys not enabled)
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Elasticsearch version 7.x (6.4 - 6.x deprecated to be removed in 13.8)? ... skipped (elasticsearch is disabled)


Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes