1. 09 Apr, 2019 4 commits
    • Tjerk Vreeken's avatar
      Add mixin to linearize goal order · 6d5cc07d
      Tjerk Vreeken authored
      For some problems it is preferred to not introduce any non-linear terms in
      the objective. For example, when using a MILP solver, one wants the entire
      problem to remain linear. This sometimes clashed with the desire for an
      order 2 goal, which steers the solution in the direction of many small
      exceedences instead of one large exceedence. An order 1 goal does not
      distuinguish between the two.
      This commit adds functionality to linearly approximate the goal order.
      That way, one can keep the problem fully linear, why getting some of the
      benefits of higher order goals back.
      By default, all goals will be linearized when inheriting from
      LinearizedOrderGPMixin. This behavior can also be overridden by setting
      `linearize_goal_order` to False in goal_programming_options(). The
      linearization can also be controlled on the level of a goal by making a
      goal inherit from LinearizedOrderGoal and setting `linearize_order` to
      either True or False.
      Typical use-cases for this fine-grained control are:
      - `linearize_goal_order` is True (default) to avoid a QP problem turning
        into a QCP problem when `keep_soft_constraints` is set. The last
        priority can however be solved as a QP problem (instead of LP), because
        this quadratic objective will _not_ turn into a constraint.
      - For performance reasons, one might want to linearize specific higher
        order goals, but keep others as-is.
      Aside from solvers which only handle LP and not QP, using a linearly
      approximated goal order is also useful when `keep_soft_constraints` is
      set. A quadratic problem in priority 1 would mean a quadratically
      _constrained_ problem in priority 2 with that option, something that can
      be much harder to solve.
      Note that higher order minimization goals do not work cannot be
      linearized. This is because a function range is lacking over which to
      linearize. Instead, users are requested to give these minimization goals
      e.g. a target_min = target_max = 0.0 (and a proper function range), such
      that they can be linearized. Minimization goals with order 1 are
      supported, as they do not have to be linearized.
    • Tjerk Vreeken's avatar
      Support absolute goal order for minimization goals · f17773e2
      Tjerk Vreeken authored
      One would typically use order=2 to minimize the absolute value of the goal
      function, but that makes nominals a bit harder and easily leads to scaling
      issues. A more involved option commonly used by users is to introduce an
      auxiliary variable and additional constraints to minimize the absolute
      value of the goal function.
      It is the latter logic that this commit adds to a new mixin, such that
      the user now only has to instantiate an "MinAbsGoalProgrammingMixin"
      instance. The auxiliary variable and accompanying constraints are
      automatically added behind the scenes.
      Closes #1065
    • Tjerk Vreeken's avatar
      Allow nominals for path and extra variables · 7a7e954b
      Tjerk Vreeken authored
      Up until now, we required the user to properly scale their additional
      path and extra variables. In practice this meant that the user had to
      multiply the variable with its nominal in every constraint where they
      were using it, and also when extracting the results.
      This commit adds support for nominals for path and extra variables, such
      that this process is no longer required. These variables can now be
      used and scaled in the same way that variables from the Modelica model
      Closes #1063
    • Tjerk Vreeken's avatar
      Make tests faster by skipping Modelica libraries · c30319b6
      Tjerk Vreeken authored
      A lot of time was spent in Pymoca's transfer_model to compile the model.
      That is because since d1bc9d42 we have RTC-Tools Channel Flow as a
      dependency, so we would parse its entire library again for every model.
      The RTC-Tools tests do not need any RTC-Tools Channel Flow components
      (or other libraries for that matter), only the examples do. Therefore,
      we can explicitly disable additional library folders using the
      compiler_options() method to make compilation of a model much faster.
  2. 05 Apr, 2019 1 commit
    • Tjerk Vreeken's avatar
      PIMixin: Fix overriding initial state values · 391502da
      Tjerk Vreeken authored
      If a user was using ModelicaMixin and PIMixin, the initial state values
      from ModelicaMixin were skipped. We have to be sure to call the respective
      super() method. Note that CSVMixin's initial_state() also calls super().
      Also note that ModelicaMixin's initial_state() does not call super(), to
      avoid getting stuck in an endless loop due to OptimizationProblem's
      implementation of initial_state() and history(). This is a separate bug,
      which cannot easily be addressed without significantly refactoring the way
      history and initial states are stored.
      Closes #1110
  3. 15 Mar, 2019 2 commits
    • Tjerk Vreeken's avatar
      ModelicaMixin: Fix reading of symbolic initial states · 3213fc10
      Tjerk Vreeken authored
      The initial_state() method has to return a numeric value. For symbolic
      initial state that means that we need to substitute the parameters, just
      like e.g. bounds() and seed() already did.
      Note that, unlike e.g. the seed() method in ModelicaMixin, we raise an
      exception if the symbolic expression cannot be resolved. This is because
      a wrong/missing initial state is worse than a wrong seed value.
    • Tjerk Vreeken's avatar
      ModelicaMixin: Fix reading of symbolic start attributes · 3e4fbba1
      Tjerk Vreeken authored
      Regression in 10da03cb. Only for SX symbols does a symbolic expression
      turn into a NaN. For symbolic MX expressions a RuntimeError is raised.
      We also have to do the Python type conversion on the _substituted_
      value, not the original MX value (which will result in a RuntimeError).
      Lastly, if we cannot resolve the seed value, we should continue to the
      next symbol instead of still setting the seed.
  4. 27 Feb, 2019 1 commit
    • Tjerk Vreeken's avatar
      Timeseries: Fix constructor for ca.DM and support float arguments · 2199c96f
      Tjerk Vreeken authored
      According to the typing hints, a ca.DM should have been supported, but
      it would fail with an Exception because it does not support len().
      Furthermore, the code in the constructor seemed to suggest support for
      floats (and integers), due to the "np.full_like" branch. The typing
      hints did however not cover this, and neither did it work in practice
      because of the len() call.
  5. 25 Feb, 2019 1 commit
    • Tjerk Vreeken's avatar
      GPMixin: Fix Exception when target min or max empty Timeseries · 053baa9a
      Tjerk Vreeken authored
      If target_min and target_max are both empty Timeseries (full of NaNs),
      the "is_empty" attribute is True and the Goal is skipped in its
      entirety. However, in case only one of the two targets is empty, we
      would get an Exception in CollInt due to setting an empty constraint of
      size MX(1x0).
      This bug was introduced in commit a788207e when support for vector goals
      was added. To make sure that results for a vector goal are exactly equal
      to its non-vector counterpart, the slicing of a (vector) soft constraint
      was added in the relevant section of code. This slicing did not account
      for the possibility of no element of the constraint expression being
  6. 16 Feb, 2019 2 commits
    • Tjerk Vreeken's avatar
      GPMixin: Fix scale_by_problem_size with keep_soft_constraints · 6116b456
      Tjerk Vreeken authored
      The combination of these two options would lead to infeasible objective
      constraints, because the size of vector symbols in the objective were
      not taken into account when building the constraint.
      For example, when using `scale_by_problem_size` an objective `e` with
      `e` being a symbol of size 3 would lead to an (expanded) objective of
      `(e[0] + e[1] + e[2]) / 3`. If this minimized to a value of 0.1, we
      would end up with an objective constraint `e[0] + e[1] + e[2] <= 0.1`,
      which is infeasible.
    • Tjerk Vreeken's avatar
      Rename test class and method of vector goals · 86bc4975
      Tjerk Vreeken authored
      The "constraints" name was an unintended remainder of using the vector
      constraints test class and its methods as a starting point.
  7. 15 Feb, 2019 1 commit
    • Tjerk Vreeken's avatar
      CollInt: Fix constraints on initial derivative · b3e07c0e
      Tjerk Vreeken authored
      When history data is available at t-1 but not at t0, we would end up
      using the unscaled symbol of the state, resulting in the wrong value for
      the initial derivative. This was not caught in the relevant test,
      because the nominal on 'w' was 1.0.
      This commit fixes the bug by properly using the scaled version of the
      symbol, and also changes the test such that it would have caught this
      Note that the behavior was/is correct when when history data is
      available at both t0 and t-1, as there is no reference to the initial
      state entry in the state vector in this case (only to numeric values).
  8. 05 Feb, 2019 1 commit
    • Tjerk Vreeken's avatar
      GPMixin: Disallow non-positive weight for goals with target bounds · 091104e6
      Tjerk Vreeken authored
      If a goal has target bounds, a negative weight would mean maximization
      of the violation variable. This makes no sense from the perspective of
      goal programming, which is why it is now explicitly disallowed with a
      If a goal has target bounds and a weight of zero, we would basically
      just be adding useless constraints. This therefore also makes no sense,
      and is now explicitly disallowed.
      If the goal does not have target bounds, a negative goal weight is
      possible if e.g. the user wants to maximize, or flip the sign of the
      goal function.
  9. 02 Jan, 2019 1 commit
  10. 27 Dec, 2018 2 commits
    • Tjerk Vreeken's avatar
      Support vector goals · a788207e
      Tjerk Vreeken authored
      Like the commit for vector constraints, this commit allows one to write
      expressions using vectors. If one would have many goals that are very
      similar, one can easily combine them by vertcatting them. See the tests
      in this commit for some simple examples.
      This commit is primarily aimed as a performance improvement. Every call
      to CasADi incurs a certain overhead due to SWIG, and writing a soft
      constraint means calling CasADi 4 or 5 times _per soft constraint_. If
      one has 1,000+ goals, this is no longer a neglible amount of time. If
      all those goals are similar, one can combine them first by, resulting in
      only a handful calls to CasADi to turn it into a soft constraint.
      The implementation of vector goals is such that it does not matter if
      one is using multiple single goals, or one vector goal (provided the
      order is the same). The resulting problem will be _exactly_ the same,
      and will therefore lead to _exactly_ the same objective value.
    • Tjerk Vreeken's avatar
      Support vector constraints and variables · 916211bc
      Tjerk Vreeken authored
      Sometimes it makes sense to make a vector symbol, and write constraints
      using those vector symbols. This can be particularly interesting if many
      (10,000+) constraints are added. Every call to CasADi induces a certain
      overhead due to SWIG, and combining the 10,000 constraints into one, can
      result in a 10,000x speedup.
      Note that vector symbols are only allowed for additional path variables,
      extra variables and constant inputs (i.e. symbol not part of the DAE),
      and for parameters. The symbols are also not yet allowed to be
      integrated states or control states.
      The implementation of vector constraints is such that it does not matter
      if one is using multiple single constraints, or one vector constraint
      (provided the order is the same). The resulting problem will be
      _exactly_ the same, and will therefore lead to _exactly_ the same
      objective value.
  11. 24 Dec, 2018 1 commit
  12. 06 Dec, 2018 1 commit
    • Jorn Baayen's avatar
      Allow extra variables to appear in path expressions. · 1c0f5f9d
      Jorn Baayen authored
      This is useful for the following type of situation.  Imagine wanting
      to put a uniform upper bound for a state over all time steps.  Then,
      this bound would be an extra variable, and the bound constraint would
      be imposed as a path constraint.
  13. 21 Nov, 2018 1 commit
    • Jesse VanderWees's avatar
      Improve CSVLookupTableMixin and LookupTable tests · e752e0b2
      Jesse VanderWees authored
      Testing of lookuptables was sloppy and not very thorough. Furthermore,
      the tests treated CSVLookupTableMixin as though it is an extension of
      CSVMixin, while the reality is that it is completely independant. This
      commit splits the testing of CSVLookupTableMixin and the LookupTable
      class into its own file, and tests both the symbolic and numerical
      evaluation of LookupTable functions.
  14. 19 Nov, 2018 1 commit
  15. 06 Nov, 2018 1 commit
    • Teresa Piovesan's avatar
      Add alternative goal programming hard constraints · f6eb00d9
      Teresa Piovesan authored
      A new way of turning the soft constraints of goals into hard constraints
      is added in this commit. The user can choose to use this alternative by
      setting setting 'keep_soft_constraints' to True.
      The new implementation keeps the soft constraints and the violation
      (a.k.a. epsilon) variables around, and just constrains the objective
      value that a certain priority evaluated to. Note that the results are
      expected to be the same if all violations will be 0.0. Only when
      violations become nonzero can the new implementation provide more room in
      further priorities.
      The new implementation is expected to help with infeasibilities in lower
      priorities, even if a trivial goal is added. It turned out that the
      current/old way of goal programming could lead to these infeasibilities,
      likely because it generated entirely differently looking hard constraints.
      Note that keeping the violation variables around may or may not lead to a
      performance hit. Because the constraints of the problem remain entirely
      the same except for one additional constraint (and the constraints of the new
      priority), the solver may more readily pick up the seed solution and
      actually be faster in solving the new priority.
      A word of caution when using this option with (MI)QCP solvers. A
      quadratic objective will turn into a quadratic constraint. In other words,
      a MIQP problem can turn into a MIQCPQP problem (if the next priority also
      has a quadratic objective). This is something that solvers like Gurobi and
      CPLEX typically cannot handle.
      Closes #1048
  16. 02 Nov, 2018 1 commit
    • Teresa Piovesan's avatar
      Fix seed derivatives · 80f225f9
      Teresa Piovesan authored
      The value of the derivatives of the initial time step and of the extra
      variables is included the seed.
      Closes issue #1066
  17. 25 Sep, 2018 3 commits
    • Tjerk Vreeken's avatar
      Fix DeprecationWarnings in tests · e82b3069
      Tjerk Vreeken authored
      The assertRaisesRegexp name was deprecated in Python 3.2 in favor of
      assertRaisesRegex. Our minimum version requirement is Python 3.5, so
      this change is fully backwards compatible for our purposes.
      The np.all() check would raise a DeprecationWarning if the lengths of
      the two arrays were not equal. The required additional length check
      means the early termination moves inside the if-else for __iter__ (i.e.
      is "t" an np.ndarray or a float). Note that we can skip the length check
      when "t" is a float, making the early termination a little more
      generic (also working when len(ts) > 1).
    • Tjerk Vreeken's avatar
      Validate goal target min/max variable type · 3069228c
      Tjerk Vreeken authored
    • Tjerk Vreeken's avatar
      Refactor goal programming mixin · 3690cfa4
      Tjerk Vreeken authored
      External API stays the same, but internally things are restructured to:
      1) Share as much code between goals and path goals as possible
      2) Increase readibility of flow in code by moving things into
      3) Prepare for an alternative implementation of the hard constraints
      4) Prepare for vector goals
      As far as the changes are concerned, they can be summarized as:
      1) Merge code paths of goals and path goals as much as possible
      2) Keep track of soft and hard constraints separately. Note that we do not
         need to associate a soft constraint with a function_key, because they
         are guaranteed to be linearly independent. This change is mainly made
         to prepare for a yet-to-be-implemented alternative way of formulating
         hard constraints.
  18. 12 Sep, 2018 3 commits
  19. 24 Aug, 2018 1 commit
    • Tjerk Vreeken's avatar
      Simulation: opt-in partial workaround for delay() · 62b83ef3
      Tjerk Vreeken authored
      A workaround is added to support a delay of zero (or make delay expression
      behave as such). By default, this workaround is disabled. The user has to
      explicitly set "_force_zero_delay" to True to enable it.
  20. 23 Aug, 2018 2 commits
    • Tjerk Vreeken's avatar
      Improve goal function range vs. targets check · 704b34f6
      Tjerk Vreeken authored
      We now check that the target min/max are not equal to the function range
      lower bound/upper bound respectively as well.
      A target minimum equal to the lower bound of the function range does not
      make any sense, as the corresponding epsilon is then free (and will
      therefore always become zero). Effectively, the goal will behave as a
      critical goal, and users should use those instead if such behavior is
    • Tjerk Vreeken's avatar
      Disallow function_range for minimization goals · e9f500d7
      Tjerk Vreeken authored
      Minimization goals only use the function nominal for scaling. Requiring
      the user to specify a function range is misleading.
      Closes #1051
  21. 06 Aug, 2018 1 commit
    • Tjerk Vreeken's avatar
      Rename test models to be able to use pytest · c4219f40
      Tjerk Vreeken authored
      The naming of some classes confused pytest. In general, it looks for
      classes and functions starting with "Test" or "test". In our case this
      meant it would also find (and fail on) the OptimizationProblem classes.
      Renaming all such classes from "TestProblem*" to "Model* fixes this.
      With nose in maintenance mode and pytest getting more popular, it makes
      sense to make this small renaming change to support both.
      Closes #1033
  22. 03 Aug, 2018 1 commit
  23. 26 May, 2018 2 commits
    • Tjerk Vreeken's avatar
      Add support for qpsol and solvers other than IPOPT · b2dc65d6
      Tjerk Vreeken authored
      We change the solver_options API by removing the workaround for IPOPT to
      pass it options. This behavior changed between CasADi 2 and 3. The
      workaround makes using solver other than IPOPT harder however, so instead
      we go back to the old way of just passing through most options directly to
      This commit alo introduces a new option "casadi_solver" for the dictionary
      returned by solver_options(). The default value is "nlpsol" for
      backwards compatibility. Another typical option would be "qpsol".
      It also introduces a new API function "solver_success()" to override
      what solver return status should be considered succesful, and how to
      log said status.
      Closes #1006
    • Tjerk Vreeken's avatar
      Do not cache test models · d81c9113
      Tjerk Vreeken authored
      The optimization test models already had caching disabled in commit
      f0cceab6. The simulation models were still using caching however,
      which could result in odd behavior (e.g. changes not having any effect),
      especially when testing locally.
  24. 25 May, 2018 1 commit
  25. 17 May, 2018 1 commit
  26. 27 Apr, 2018 1 commit
    • Tjerk Vreeken's avatar
      Do not use variables for min/max of path goals · b3f098ed
      Tjerk Vreeken authored
      Using variables for the min/max series of path goals resulted in (at
      least) quadratic constraints. These min/max series variables had equal
      bounds, and most solvers remove these variables in the preprocessor of the
      optimization. So although there was no problem for e.g. IPOPT, it was not
      possible for the user or CasADi to easily detect whether the problem was
      As a concrete example, it was not possible to use CLP (a linear solver),
      because a Hessian check in CasADi would fail. Furthermore, there might be
      solvers that we want to use that do not (by default) support the
      elimination of variables with equal bounds.
      Instead of using variables, we can instead specify the min/max series
      as additional constant inputs.
      Fixes #1005
  27. 14 Feb, 2018 2 commits