Don't send notification email to temp-oauth-email

Since gitlab-ce!3477, which was released in a patch release, we can try to send notifications to temp-oauth-emails which does not make sense.