Skip to content

mmap: Detect unexpected file size changes

Will Chandler (ex-GitLab) requested to merge wc/detect-file-modification into master

Recently we had an incident where the parent Puma worker and a child worker process were both writing to the same .db file. One process had opened the file when it was 4k, while the other had expanded the file beyond 8k. When the first process resized the file, it write a NUL byte to position 8192, breaking that metric for the other worker.

To prevent a scenario like this from occurring again, when resizing we add a check to confirm that the size of the underlying file matches still matches its size when we first mapped it.

Note that there is the potential for a race between two processes, A and B:

  • A has mapped length of 4096, B has written more and expanded the file to 8192.
  • A hits its size limit and stats the file, sees a length of 8192.
  • B hits its size limit and resizes to 16384.
  • A resizes to 8192, writing a NUL byte to position 8192 and corrupting the metric in that location.

We cannot fully mitigate this risk, but we modify check_expand() to only resize the file a single time, minimizing the critical section, and process A will now throw an exception upon detecting the file size has changed.

Edited by Will Chandler (ex-GitLab)

Merge request reports