Use OverlayFS to persist changes to workspace HOME directory
MR: Pending ## Description Use OverlayFS to persist changes to workspace HOME directory. 1. Leverage the [`OverlayFS` filesystem](https://docs.kernel.org/filesystems/overlayfs.html) 2. Use OverlayFS to mount the contents of the container's existing `HOME` directory as a "lower layer" (read only), and a directory on the existing persistent volume as the "upper layer" (writeable). 3. Inject commands/script to be run at container start executes the proper `mkdir ...` and `mount -t overlay ...` commands to achieve the above desired OverlayFS mount setup. 4. Execution of these commands/script at container start will achieved be via the support for running specific commands in containers which we are adding as part of https://gitlab.com/groups/gitlab-org/-/epics/15602+ ## Acceptance Criteria - [ ] HOME directory of all containers in a workspaces are persisted across workspace stop/start. - [ ] Relevant documentation updated - [ ] Test with many and/or large files in the HOME directory, and ensure that the overlayfs does not negatively impact performance. ## Technical Requirements - See description for overlayfs details - Ensure that we use the existing persistent volume. ## Design Requirements No UI changes, this should be transparent to the user. ## Impact Assessment Users no longer lose changes in HOME directory when workspace is restarted. <!-- Replace with other type, e.g. bug or maintenance, if appropriate --> <!-- Replace with other subtype if appropriate --> <!-- By default, all issues start in the unprioritized status. See https://handbook.gitlab.com/handbook/engineering/development/dev/create/remote-development/#-planning-process --> <!-- For simplicity and to avoid triage bot warnings about missing workflow labels, we will default to issues starting at the refinement phase -->
issue