handmade_aligned_malloc can write outside its buffer if malloc returns very unaligned memory
Summary
While investigating a mysterious segfault in Eigen, by visual inspection I noticed a buffer overflow in handmade_aligned_malloc
. It requires the following:
alignment==sizeof(void*)
-
std::malloc()
returns a pointer that is not aligned
If this happens original
will be written partially outside the buffer returned by malloc
. Note that on all common platforms, malloc will not return memory aligned to less than sizeof(void*)
.
Environment
- Eigen Version : 3.4.0
Minimal Example
Here is the Eigen 3.4.0 version of that function with a small bit of test scaffolding around it to demonstrate the issue.
https://godbolt.org/z/qxxjd4K6W
#define EIGEN_DEVICE_FUNC
#define EIGEN_DEFAULT_ALIGN_BYTES 16
#define EIGEN_USING_STD(func) using fakestd::func;
namespace fakestd {
void* malloc(size_t) { return (void*)12; }
}
#define eigen_assert(x) assert(x)
EIGEN_DEVICE_FUNC inline void* handmade_aligned_malloc(std::size_t size, std::size_t alignment = EIGEN_DEFAULT_ALIGN_BYTES)
{
eigen_assert(alignment >= sizeof(void*) && (alignment & (alignment-1)) == 0 && "Alignment must be at least sizeof(void*) and a power of 2");
EIGEN_USING_STD(malloc)
void *original = malloc(size+alignment);
if (original == 0) return 0;
void *aligned = reinterpret_cast<void*>((reinterpret_cast<std::size_t>(original) & ~(std::size_t(alignment-1))) + alignment);
#if 1
// don't write outside the malloc block
assert((size_t)(reinterpret_cast<void**>(aligned) - 1) >= (size_t)original);
#else
*(reinterpret_cast<void**>(aligned) - 1) = original;
#endif
return aligned;
}
int main(void)
{
auto ret = handmade_aligned_malloc(32, 8);
return 0;
}
Edited by Paul Du Bois