Notifications not working on new install of 7.10.1

It seems as though notifications are not working 7.10.1 for us. We are hoping to roll this out to production soon but kind of need this capability.

I enabled the "Emails on push" service for a project that I am part of. Pushing to that repo does not generate any emails. However if I go into the "Emails on push" page and click the "Test settings" button then I do get the email. Other emails from gitlab are being sent out for example if someone adds or removes a ssh key. I also tried mentioning someone else in a commit comment and that person did not get an email.

I am not seeing any errors in the logs that would be related to this, I'm not really seeing any errors at all actually.