Skip to content

Implement lazy popen so LfsChanges doesn't have to wait for rev-list to complete

James Edwards-Jones requested to merge jej/fix-lfs-changes-laziness into master

What

  • Allows Gitlab::Git::Popen#popen to hand stdout to a block as a lazy enumerator
  • Uses this from rev-list
  • Uses the lazy RevList methods from LfsChanges

This is done using blocks so that the IO/process is closed by Open3.popen3 after calling the block. An alternative would have been to return the lazy IO as cmd_output, but this would have left the process running.

Why

We want to be able to take(2000) to get only 2000 lines from rev-list but unlike IO.popen, our implementation was waiting for the long-running process to complete before returning any data. This was partly because the block form of popen waits for the process to terminate.

Are there points in the code the reviewer needs to double check?

  • Is it OK that the this has only changed Gitlab::Git::Popen and not Gitlab::Popen

Acceptance criteria

  • Tests added for this feature/bug
  • Has been reviewed by Backend

Related

Merge request reports