Skip to content

Fix More-Thuente line search

Description

This MR fixes the More-Thuente line search. The fix is three-fold:

  • Explicitly call the line search for a maximization problem.
  • Allow flipping the direction of the step once the optimization problem type is defined to account for the optimizer sometimes asking for a step in a wrong direction.
  • Check that the function values are not equal when finding the minimum of the cubic and quadratic interpolation. Equal values were causing nan values to appear.

Closes #511 (closed)

Notes for Reviewer

There is a couple of things that bother me and for now I have failed to properly understand the cause. It might help when I have a deeper dive into the optimizer.

  • the optimizer sometimes asks for a wrong direction. In my opinion this should never happen and this is the reason for flipping the direction of the step in this MR
  • while this MR makes More-Thuente line search work it still requires some investigation. It feels extremely finicky and the paper does not really help much. This definitely needs more research. For example, currently, it often happens that the termination criteria as defined in the paper are never reached. I believe this is because we constraint the search for the step to some interval and if this interval has no step that satisfies the termination criteria strange things happen.

Checklist

  • Documentation was updated
Edited by Igor Bogoslavskyi

Merge request reports