[FR] Reusing rvalues in expressions

Describe the feature you would like to be implemented.

This issue concerns reusing already allocated memory when evaluating expressions and how Eigen could do that. The TL;DR is that I think it's possible but it would be a good bit of work (I'd say 1 GSOC student could at least get a prototype up over a summer).

Eigen currently does not reuse already allocated memory from rvalue types when computing expressions. Looking at the type for the code below we see the type for C is a long named CwiseBinaryOp that has stripped out any information that A was in fact an rvalue.

Eigen::MatrixXd A = Eigen::MatrixXd::Random(2, 2);
Eigen::MatrixXd B = Eigen::MatrixXd::Random(2, 2);
auto C = std::move(A) + B;

https://godbolt.org/z/Y1fzY1689

// C's type
Eigen::CwiseBinaryOp<Eigen::internal::scalar_sum_op<double, double>,
   const Eigen::Matrix<double, -1, -1>, const Eigen::Matrix<double, -1, -1> >

So Eigen allocates memory for C and then packets for A and B are computed into C's memory. What I propose is that Eigen does not drop the rvalue information and instead of allocating memory for C it reuses A's memory. Eigen would then make types like below where the LHS matrix is known as an rvalue.

// C's type
Eigen::CwiseBinaryOp<Eigen::internal::scalar_sum_op<double&&, double>,
   Eigen::Matrix<double, -1, -1>&&, const Eigen::Matrix<double, -1, -1>>

Would such a feature be useful for other users? Why?

For small fixed or dynamic matrices honestly I'm not sure this would give a huge speedup, maybe a few percent if any. But for large matrices like 100x100 etc. yes I do think managing temporary memory becomes a thing worth thinking about.

Any hints on how to implement the requested feature?

I think the steps for this are

  1. Do some very basic benchmarking just to see how much re-using memory with packet math can save
  2. Adding perfect forwarding semantics to the operators so they can know when to dispatch to a version that re-uses memory
  3. Writing versions of the ops and evaluators that know how to re-use memory
  4. ???
  5. Profit

Additional resources

I think for making (2) easier Eigen could use something like Stan's legacy requires. I'm working on making that feature in Stan into its own header only library if Eigen would be interested in using it. Another option would be something like #2319 (closed) where Eigen has its own little type traits header and legacy requires inside of it for Eigen types.