too much bytes were processed, causing a heap buffer overrun http://bugzilla.maptools.org/show_bug.cgi?id=2831 the loop counter must be for (col = 0; col < width; col += 8 / bps)
Also the values were not properly calculated. It should be 255-x, 15-x, 3-x for bps 8, 4, 2.
But anyway it is easier to invert all bits as 255-x = ~x, etc. (substracting from a binary number composed of all 1 is like inverting the bits)