Skip to content

handmade_aligned_malloc can write outside its buffer if malloc returns very unaligned memory

Summary

While investigating a mysterious segfault in Eigen, by visual inspection I noticed a buffer overflow in handmade_aligned_malloc. It requires the following:

  • alignment==sizeof(void*)
  • std::malloc() returns a pointer that is not aligned

If this happens original will be written partially outside the buffer returned by malloc. Note that on all common platforms, malloc will not return memory aligned to less than sizeof(void*).

Environment

  • Eigen Version : 3.4.0

Minimal Example

Here is the Eigen 3.4.0 version of that function with a small bit of test scaffolding around it to demonstrate the issue.

https://godbolt.org/z/qxxjd4K6W

#define EIGEN_DEVICE_FUNC
#define EIGEN_DEFAULT_ALIGN_BYTES 16
#define EIGEN_USING_STD(func) using fakestd::func;

namespace fakestd {
void* malloc(size_t) { return (void*)12; }
}

#define eigen_assert(x) assert(x)

EIGEN_DEVICE_FUNC inline void* handmade_aligned_malloc(std::size_t size, std::size_t alignment = EIGEN_DEFAULT_ALIGN_BYTES)
{
  eigen_assert(alignment >= sizeof(void*) && (alignment & (alignment-1)) == 0 && "Alignment must be at least sizeof(void*) and a power of 2");

  EIGEN_USING_STD(malloc)
  void *original = malloc(size+alignment);
  
  if (original == 0) return 0;
  void *aligned = reinterpret_cast<void*>((reinterpret_cast<std::size_t>(original) & ~(std::size_t(alignment-1))) + alignment);

#if 1
  // don't write outside the malloc block
  assert((size_t)(reinterpret_cast<void**>(aligned) - 1) >= (size_t)original);
#else
  *(reinterpret_cast<void**>(aligned) - 1) = original;
#endif
  return aligned;
}

int main(void)
{
    auto ret = handmade_aligned_malloc(32, 8);    
    return 0;
}
Edited by Paul Du Bois