Getting a "number of elements exceeds INT_MAX" error when trying to solve a problem involving ~200x200 matrices
If this is the wrong place to post this please let me know where would be more appropriate.
I'm attempting to use picos to solve an SDP problem outlined in these lecture notes on quantum channels, specifically the dual problem given in section 21.2.1. The specific form of the problem I am working with involves a matrix inequality constraint between 2 matrices of size (216 x 216) and when I try to solve the problem using CVXOPT I get an error:
OverflowError: number of elements exceeds INT_MAX
When I remove this constraint involving the large matrices there is no issue. My confusion is that, as far as I'm aware, INT_MAX = 2,147,483,647 and the number of elements in a 216x216 matrix is 46,656, nowhere near that value. Is this behaviour expected? I've heard of people coming across similar issues with CVXOPT when working with much larger matrices but my application seems rather small in the grand scheme of things.
If there is a fix that I'm missing I would greatly appreciate guidance, picos is much more intuitive than the Mathematica SDP solver and I dread the prospect of shoehorning my problem into its restrictive format demands.
Here is some sample code to illustrate my problem:
import numpy as np
import picos
Y = picos.HermitianVariable("Y",27)
#LHS of inner product shown in lecture notes, all zero except for 1 in top left
C = np.zeros((27,27))
C[0,0] = 1
Constant = picos.Constant(C)
#Identity matrix nxn
def Id(n):
return picos.Constant(np.eye(n))
P = picos.Problem()
P.set_objective("min",(Constant | Y))
# The real constraint involves more detailed matrices of the same size but the
# same error occurs even with simple constraints such as this
P.add_constraint(Id(8) @ Y - Id(216) >> 0)
P.solve(solver = "cvxopt")
And the resulting error spiel
---------------------------------------------------------------------------
OverflowError Traceback (most recent call last)
<ipython-input-3-83f5efe38709> in <module>
16 P.add_constraint(Id(8) @ Y - Id(216) >> 0)
17
---> 18 P.solve(solver = "cvxopt")
~/.local/lib/python3.8/site-packages/picos/modeling/problem.py in solve(self, **extra_options)
1647
1648 # Execute the strategy to obtain one or more solutions.
-> 1649 solutions = self._strategy.execute(**extra_options)
1650
1651 # Report how many solutions were obtained, select the first.
~/.local/lib/python3.8/site-packages/picos/modeling/strategy.py in execute(self, **extra_options)
149 # Defer solving to the first reformulation, which is responsible for
150 # applying the extra options (i.e. ExtraOptions).
--> 151 solution = self.nodes[1].execute(**extra_options)
152
153 # Attach the solution to the root problem. Note that reformulations are
~/.local/lib/python3.8/site-packages/picos/reforms/reform_options.py in execute(self, **extra_options)
105
106 # Advance one step and return any solution as-is.
--> 107 return self.successor.execute()
108
109 def forward(self):
~/.local/lib/python3.8/site-packages/picos/reforms/reformulation.py in execute(self)
230
231 # Advance one step.
--> 232 outputSolution = self.successor.execute()
233
234 # Transform the solution of the output problem for the input problem.
~/.local/lib/python3.8/site-packages/picos/solvers/solver.py in execute(self)
601 self._verbose("Starting solution search.")
602
--> 603 solution = self._solve()
604
605 if isinstance(solution, list):
~/.local/lib/python3.8/site-packages/picos/solvers/solver_cvxopt.py in _solve(self)
488 p["hl"], p["A"], p["b"], kktsolver="ldl")
489 else:
--> 490 result = cvxopt.solvers.conelp(
491 p["c"], G, h, dims, A, b, kktsolver="ldl")
492
~/.local/lib/python3.8/site-packages/cvxopt/coneprog.py in conelp(c, G, h, dims, A, b, primalstart, dualstart, kktsolver, xnewcopy, xdot, xaxpy, xscal, ynewcopy, ydot, yaxpy, yscal, **kwargs)
573 raise ValueError("Rank(A) < p or Rank([G; A]) < n")
574 if kktsolver == 'ldl':
--> 575 factor = misc.kkt_ldl(G, dims, A, kktreg = KKTREG)
576 elif kktsolver == 'ldl2':
577 factor = misc.kkt_ldl2(G, dims, A)
~/.local/lib/python3.8/site-packages/cvxopt/misc.py in kkt_ldl(G, dims, A, mnl, kktreg)
1078 ldK = n + p + mnl + dims['l'] + sum(dims['q']) + sum([ int(k*(k+1)/2)
1079 for k in dims['s'] ])
-> 1080 K = matrix(0.0, (ldK, ldK))
1081 ipiv = matrix(0, (ldK, 1))
1082 u = matrix(0.0, (ldK, 1))
OverflowError: number of elements exceeds INT_MAX