Numerical instability is a generic label often applied to situations where solving an optimization model produces results that are erratic, inconsistent, or unexpected, or when the underlying algorithms exhibit poor performance or are unable to converge. There are many potential causes of this behavior; however, most can be grouped into four categories:
- Rounding coefficients while building the model.
- Limitations of floating-point arithmetic.
- Unrealistic expectations about achievable precision.
- Ill conditioning, or geometry-induced issues.
This section explains these issues and how they affect both performance and solution quality. We also provide some general rules and some advanced techniques to help avoid them. Although we will treat each of these four sources separately, it is important to remember that their effects often feed off of each other. We also provide tips on how to diagnose numerical instability in your models.
Finally, we discuss the Gurobi parameters that can be modified to improve solution accuracy. We should stress now, however, that the best way to improve numerical behavior and performance is to reformulate your model. Parameters can help to manage the effects of numerical issues, but there are limits to what they can do, and they typically come with a substantial performance cost.
- Avoid rounding of input
- Real numbers are not real
- Tolerances and user-scaling
- Models at the edge of infeasibility
- Gurobi tolerances and the limitations of double-precision arithmetic
- Why scaling and geometry is relevant
- Recommended ranges for variables and constraints
- Improving ranges for variables and constraints
- Advanced user scaling
- Avoid hiding large coefficients
- Dealing with big-M constraints
- Does my model have numerical issues?
- Solver parameters to manage numerical issues
- Instability and the geometry of optimization problems
- The case of linear systems:
- The geometry of linear optimization problems
- Multiple optimal solutions
- Dealing with epsilon-optimal solutions
- Thin feasible regions
- Optimizing over the circle:
- Optimizing over thin regions:
- Stability and convergence
- Further reading
- Source code examples: