handmade_aligned_malloc can write outside its buffer if malloc returns very unaligned memory
Summary
While investigating a mysterious segfault in Eigen, by visual inspection I noticed a buffer overflow in handmade_aligned_malloc. It requires the following:
alignment==sizeof(void*)-
std::malloc()returns a pointer that is not aligned
If this happens original will be written partially outside the buffer returned by malloc. Note that on all common platforms, malloc will not return memory aligned to less than sizeof(void*).
Environment
- Eigen Version : 3.4.0
Minimal Example
Here is the Eigen 3.4.0 version of that function with a small bit of test scaffolding around it to demonstrate the issue.
https://godbolt.org/z/qxxjd4K6W
#define EIGEN_DEVICE_FUNC
#define EIGEN_DEFAULT_ALIGN_BYTES 16
#define EIGEN_USING_STD(func) using fakestd::func;
namespace fakestd {
void* malloc(size_t) { return (void*)12; }
}
#define eigen_assert(x) assert(x)
EIGEN_DEVICE_FUNC inline void* handmade_aligned_malloc(std::size_t size, std::size_t alignment = EIGEN_DEFAULT_ALIGN_BYTES)
{
eigen_assert(alignment >= sizeof(void*) && (alignment & (alignment-1)) == 0 && "Alignment must be at least sizeof(void*) and a power of 2");
EIGEN_USING_STD(malloc)
void *original = malloc(size+alignment);
if (original == 0) return 0;
void *aligned = reinterpret_cast<void*>((reinterpret_cast<std::size_t>(original) & ~(std::size_t(alignment-1))) + alignment);
#if 1
// don't write outside the malloc block
assert((size_t)(reinterpret_cast<void**>(aligned) - 1) >= (size_t)original);
#else
*(reinterpret_cast<void**>(aligned) - 1) = original;
#endif
return aligned;
}
int main(void)
{
auto ret = handmade_aligned_malloc(32, 8);
return 0;
}
Edited by Paul Du Bois