QP solvers benchmark

Benchmark for quadratic programming (QP) solvers available in Python. It contains the following test sets:

• Maros-Meszaros: problems designed to be difficult, some of them with non-strictly p.s.d. Hessians.
• Maros-Meszaros dense: subset of the 59 smallest Maros-Meszaros problems, still difficult but accessible to dense solvers.

This benchmark aims to help us compare and select QP solvers. Its methodology is open to discussions. New test sets are also welcome. Feel free to add one that better represents the family of problems you are working on.

Solvers

Results

Check out the full reports for each test set in the results directory.

Maros-Meszaros

The Maros-Meszaros test set contains difficult problems, some of them large, sparse, ill-conditioned or with non-strictly p.s.d. Hessian matrices. For sparse solvers only.

Check out the full report for definitions and details.

Maros-Meszaros dense

Probems in the Maros-Meszaros test set with less than a 1,000 optimization variables and 1,000 constraints. Note that this subset is not representative of the full Maros-Meszaros test set.

Check out the full report for definitions and details.

Limitations

Here are some known areas of improvement for this benchmark:

• Cold start only: we don't evaluate warm-start performance for now.
• Dual feasibility: we don't check the dual multipliers that solvers compute internally, as the API for them is not yet unified.

Running the benchmark

To run the benchmark on your machine, you will first need to install qpsolvers along with the QP solvers you want to test. For instance, to install open source solvers:

pip install qpsolvers[open_source_solvers]

To run e.g. the Maros-Meszaros test set, call:

python benchmark.py run maros_meszaros

Replace maros_meszaros with the name of the test set you want to run. You can also run only a specified solver, problem or set of solver settings, for instance:

python benchmark.py run maros_meszaros_dense --solver proxqp --settings default

Check out python benchmark.py --help for details.

See also

• Optimality conditions and numerical tolerances in QP solvers: note written while figuring out the high_accuracy settings of this benchmark.
• jrl-qp/benchmarks: benchmark of QP solvers available in C++.
• proxqp_benchmark: benchmark examples for the ProxQP solver.