Review structured logging for standards compliance and security

MR: Pending

NOTE: This issue should be considered blocked until all work in the Robust Error Handling and Logging (&10461 - closed) is completed. Only then can we do a complete review of the final logging statements for compliance and security. For now, a preliminary review has been done and there are no concerns. Therefore, moving this to Workspaces Technical Debt Work (&11041) as it should not be a blocker for GA

Description

As a developer or admin supporting the Remote Development feature, I want adequate logging of backtraces and exception messages so that we are able to debug problems that happen both in SaaS and for on-prem installations, but without leaking sensitve information into logs.

Related MR: Remove manual exception logging (!119559 - closed)

Notes from @cwoolley-gitlab (see internal slack thread):

My main concern is around how the logging of backtraces and exception messages.

On one hand, there’s docs that ask us to avoid this in order to avoid leaking sensitive information into the logs (even though there’s existing places I found in the codebase that do it anyway).

But on the other hand, this is important information in order for us to be able to debug problems that happen both in SaaS and for on-prem installations.

I’m not sure how we are supposed to reconcile these concerns. There’s some helpers around structured logging and such, but I haven’t had time to dig into them.

See original issues for more context:

• Switch all remote_dev branch log statements to ... (#408783 - closed)
• Ensure Rails structured logs are captured in pr... (#409066 - closed)

Acceptance Criteria

Tasks

• Review all the structured logging added as part of Add structured logging and some cleanup (!119146 - merged) for compliance to standards and security (e.g. not leaking sensitive info in exceptions or backtraces). See TODOs added in Remove manual exception logging (!119559 - closed) for context.

Technical Requirements

TODO: Fill out or delete [If applicable, please list out any technical requirements for this feature/enhancement.]

Design Requirements

TODO: Fill out or delete [If applicable, please provide a link to the design specifications for this feature/enhancement.]

Impact Assessment

TODO: Fill out or delete [Please describe the impact this feature/enhancement will have on the user experience and/or the product as a whole.]

User Story

TODO: Fill out or delete [Provide a user story to illustrate the use case for this feature/enhancement. Include examples to help communicate the intended functionality.]