The source project of this merge request has been removed.
Fix More-Thuente line search
Description
This MR fixes the More-Thuente line search. The fix is three-fold:
- Explicitly call the line search for a maximization problem.
- Allow flipping the direction of the step once the optimization problem type is defined to account for the optimizer sometimes asking for a step in a wrong direction.
- Check that the function values are not equal when finding the minimum of the cubic and quadratic interpolation. Equal values were causing
nan
values to appear.
Closes #511 (closed)
Notes for Reviewer
There is a couple of things that bother me and for now I have failed to properly understand the cause. It might help when I have a deeper dive into the optimizer.
- the optimizer sometimes asks for a wrong direction. In my opinion this should never happen and this is the reason for flipping the direction of the step in this MR
- while this MR makes More-Thuente line search work it still requires some investigation. It feels extremely finicky and the paper does not really help much. This definitely needs more research. For example, currently, it often happens that the termination criteria as defined in the paper are never reached. I believe this is because we constraint the search for the step to some interval and if this interval has no step that satisfies the termination criteria strange things happen.
Checklist
-
Documentation was updated
Edited by Igor Bogoslavskyi