Skip to content
  • Linus Torvalds's avatar
    Make run_diff_index() use unpack_trees(), not read_tree() · d1f2d7e8
    Linus Torvalds authored
    
    
    A plain "git commit" would still run lstat() a lot more than necessary,
    because wt_status_print() would cause the index to be repeatedly flushed
    and re-read by wt_read_cache(), and that would cause the CE_UPTODATE bit
    to be lost, resulting in the files in the index being lstat'ed three
    times each.
    
    The reason why wt-status.c ended up invalidating and re-reading the
    cache multiple times was that it uses "run_diff_index()", which in turn
    uses "read_tree()" to populate the index with *both* the old index and
    the tree we want to compare against.
    
    So this patch re-writes run_diff_index() to not use read_tree(), but
    instead use "unpack_trees()" to diff the index to a tree.  That, in
    turn, means that we don't need to modify the index itself, which then
    means that we don't need to invalidate it and re-read it!
    
    This, together with the lstat() optimizations, means that "git commit"
    on the kernel tree really only needs to lstat() the index entries once.
    That noticeably cuts down on the cached timings.
    
    Best time before:
    
    	[torvalds@woody linux]$ time git commit > /dev/null
    	real    0m0.399s
    	user    0m0.232s
    	sys     0m0.164s
    
    Best time after:
    
    	[torvalds@woody linux]$ time git commit > /dev/null
    	real    0m0.254s
    	user    0m0.140s
    	sys     0m0.112s
    
    so it's a noticeable improvement in addition to being a nice conceptual
    cleanup (it's really not that pretty that "run_diff_index()" dirties the
    index!)
    
    Doing an "strace -c" on it also shows that as it cuts the number of
    lstat() calls by two thirds, it goes from being lstat()-limited to being
    limited by getdents() (which is the readdir system call):
    
    Before:
    	% time     seconds  usecs/call     calls    errors syscall
    	------ ----------- ----------- --------- --------- ----------------
    	 60.69    0.000704           0     69230        31 lstat
    	 23.62    0.000274           0      5522           getdents
    	  8.36    0.000097           0      5508      2638 open
    	  2.59    0.000030           0      2869           close
    	  2.50    0.000029           0       274           write
    	  1.47    0.000017           0      2844           fstat
    
    After:
    	% time     seconds  usecs/call     calls    errors syscall
    	------ ----------- ----------- --------- --------- ----------------
    	 45.17    0.000276           0      5522           getdents
    	 26.51    0.000162           0     23112        31 lstat
    	 19.80    0.000121           0      5503      2638 open
    	  4.91    0.000030           0      2864           close
    	  1.48    0.000020           0       274           write
    	  1.34    0.000018           0      2844           fstat
    	...
    
    It passes the test-suite for me, but this is another of one of those
    really core functions, and certainly pretty subtle, so..
    
    NOTE! The Linux lstat() system call is really quite cheap when everything
    is cached, so the fact that this is quite noticeable on Linux is likely to
    mean that it is *much* more noticeable on other operating systems. I bet
    you'll see a much bigger performance improvement from this on Windows in
    particular.
    
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    d1f2d7e8