• Linus Torvalds's avatar
    Fix big left-shifts of unsigned char · 48fb7deb
    Linus Torvalds authored
    Shifting 'unsigned char' or 'unsigned short' left can result in sign
    extension errors, since the C integer promotion rules means that the
    unsigned char/short will get implicitly promoted to a signed 'int' due to
    the shift (or due to other operations).
    This normally doesn't matter, but if you shift things up sufficiently, it
    will now set the sign bit in 'int', and a subsequent cast to a bigger type
    (eg 'long' or 'unsigned long') will now sign-extend the value despite the
    original expression being unsigned.
    One example of this would be something like
    	unsigned long size;
    	unsigned char c;
    	size += c << 24;
    where despite all the variables being unsigned, 'c << 24' ends up being a
    signed entity, and will get sign-extended when then doing the addition in
    an 'unsigned long' type.
    Since git uses 'unsigned char' pointers extensively, we actually have this
    bug in a couple of places.
    I may have missed some, but this is the result of looking at
    	git grep '[^0-9 	][ 	]*<<[ 	][a-z]' -- '*.c' '*.h'
    	git grep '<<[   ]*24'
    which catches at least the common byte cases (shifting variables by a
    variable amount, and shifting by 24 bits).
    I also grepped for just 'unsigned char' variables in general, and
    converted the ones that most obviously ended up getting implicitly cast
    immediately anyway (eg hash_name(), encode_85()).
    In addition to just avoiding 'unsigned char', this patch also tries to use
    a common idiom for the delta header size thing. We had three different
    variations on it: "& 0x7fUL" in one place (getting the sign extension
    right), and "& ~0x80" and "& 0x7f" in two other places (not getting it
    right). Apart from making them all just avoid using "unsigned char" at
    all, I also unified them to then use a simple "& 0x7f".
    I considered making a sparse extension which warns about doing implicit
    casts from unsigned types to signed types, but it gets rather complex very
    quickly, so this is just a hack.
    Signed-off-by: default avatarLinus Torvalds <[email protected]>
    Signed-off-by: default avatarJunio C Hamano <[email protected]>
delta.h 3.38 KB