Skip to content

Optimize check_rows_cols_for_overflow

Reference issue

Fixes #2694 (closed)

What does this implement/fix?

check_rows_cols_for_overflow performs a sanity check to ensure that the number of rows and columns does not exceed the maximum permitted by the index type. @imyxh noted that this check could be optimized when the rows are known at compile time. Currently, we always divide by columns, which the compiler is able to optimize for objects such as Matrix<double,Dynamic,1> but not Matrix<double,1,Dynamic>. This fix uses partial template specialization to use compile time information when its available.

Also fixes a nit where matrices such as Matrix<double,Dynamic,0> were tagged with a Dynamic size at compile time. This probably had no effect on runtime performance, but certainly made the template meta programming more annoying for matrices with a zero dimension at compile time.

Additional information

Edited by Charles Schlosser

Merge request reports

Loading