Include username in Gitlab.com access log

Problem to solve

Currently, Gitlab.com disable access from a certain IP address when repeated failed logins are detected. The logs at Gitlab.com, however, do not include the username used in such logins. As a result, the customer is not effectively able to narrow down the cause of the failed logins.

Intended users

Any Gitlab.com customer

Further details

Including the username in the log should save Gitlab Support a large amount of time. For example, support issue #123000 ended up with almost two days of back and forth between Support and the customer. The customer could have been able to narrow down the cause right away if the username were available.

Proposal

Include the username in the logs

Permissions and Security

This feature should only be available to the Gitlab.com internal personnel including Support

Documentation

Testing

What does success look like, and how can we measure that?

Links / references

To do

• Check behavior on production:
  • Auth.log: https://log.gitlab.net/goto/e4b7365fc2be37fc5b61d24c1d94dbce
  • Throttle events: https://log.gitlab.net/app/kibana#/discover?_g=h@dc5c7ed&_a=h@e7d17fd
  • Blacklist events: https://log.gitlab.net/app/kibana#/discover?_g=h@15e5810&_a=h@709edaa
• Confirm IP addreses are no longer being logged on user_id and logs are ingested correctly by elastic search. https://gitlab.com/gitlab-org/gitlab-ce/issues/62756#note_196118941
• Move protected paths throttle from Omnibus to GitLab rails - https://gitlab.com/gitlab-org/gitlab-ce/issues/64278
• Update documentation
• Communicate addition to Support team

Development Log

• backend A MR adding the user information on auth.log was merged https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29821
• While exploring the logs on Kibana, we noticed that for Rack Attack blocking events, the Rack Attack discriminator was not being populated hence showing the user_id as NULL.
  • A MR was created to limit the user information to Rack Attack throttles: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/30467
  • A follow-up issue was created to also include the user information on Rack Attack blocking events https://gitlab.com/gitlab-org/gitlab-ce/issues/64278
• It was noticed that GitLab.com does not have RackAttack options activated by default, so it's not going to log user information (https://gitlab.com/gitlab-org/gitlab-ce/issues/62756#note_189975231)
  • They can't be enabled because they lead to erroneous blocks.
  • Rack Attack throttle it's only active on Omnibus for the protected paths - https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/templates/default/rack_attack.rb.erb
    • Which is what is being currently logged, but that information does not have the current_user
• It appears that some RackAttack events are also logged in the codebase:
  • https://gitlab.com/gitlab-org/gitlab-ee/blob/master/lib/gitlab/auth.rb#L99-118
  • They do not include user information
• It was reported they were wrongly logged, ip was being populated with the wrong information.
  • https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/30467 fixes that problem
• We've decided to move 'protected paths' throttle into GitLab Rails - https://gitlab.com/gitlab-org/gitlab-ce/issues/64278#note_195954136
  • Omnibus configuration is going to be deprecated on %12.3 and removed on %12.4