How Mathematical Optimization Works with Machine Learning

In most real-world applications, ML and MO operate as complementary technologies. 

Machine learning is typically used for prediction—estimating future demand, customer behavior, risk, or price based on historical data. Mathematical optimization then takes those predictions and uses them to prescribe the best possible actions within a set of constraints. 

For example: 

• A retailer might use ML to forecast demand for products. 

• An optimization model then uses those forecasts to decide how much inventory to stock, where to place it, and when to replenish it—while minimizing costs and maximizing service levels. 

This “predict, then optimize” framework allows organizations to combine the predictive power of ML with the decision-making rigor of MO, turning insights into measurable operational improvements. 

Yes—they do. Every ML model is trained by solving an optimization problem that minimizes or maximizes a specific objective function (such as minimizing prediction error or maximizing accuracy). 

However, the type of optimization used in ML training is typically continuous and unconstrained—for instance, gradient descent methods used in neural networks or logistic regression. These algorithms are specialized for smooth optimization problems with millions of parameters and are not what mathematical optimization solvers like Gurobi are designed to handle. 

In contrast, mathematical optimization excels in solving structured, constrained, and often discrete problems—those involving binary or integer decisions, logical rules, and resource trade-offs. While ML training uses optimization in a general sense, MO provides a different, complementary approach for handling complex decision and constraint structures. 

Mathematical optimization becomes valuable when you need greater control over your ML model or when your learning problem involves nonstandard constraints that traditional training algorithms can’t easily address. 

Some examples include: 

• Feature selection: Choosing exactly k features (or limiting the number of features) in a regression or classification model to improve interpretability. 

• Fairness and explainability constraints: Ensuring certain groups are treated equitably or model coefficients follow logical business rules. 

• Custom structural constraints: Requiring relationships among model parameters that reflect domain-specific knowledge. 

These cases often introduce discrete or combinatorial decisions, which are well-suited for mixed-integer programming (MIP) formulations. Using MO, you can impose explicit control over the model’s structure, leading to interpretable and tailored solutions that align with business requirements. 

One clear example is feature selection in linear regression. Suppose you want your regression model to use only 10 features out of hundreds of potential variables. Traditional ML training methods don’t directly enforce that kind of restriction, instead relying on penalty terms in the objective which don’t guarantee your constraint isn’t violated and also make your objective less interpretable 

Using Gurobi, you can model this as a mixed-integer programming problem, where binary decision variables indicate whether each feature is included in the model. The solver then finds the combination of features that minimizes prediction error while satisfying your constraint on feature count. 

This approach produces a model that’s both highly interpretable and aligned with operational needs. We provide a notebook example demonstrating this technique in action. 

Think of ML and MO as two stages in the decision intelligence pipeline: 

• Machine Learning (Prediction): Understands patterns in your data and forecasts what might happen. 

• Mathematical Optimization (Prescription): Decides the best course of action based on those forecasts and your constraints. 

By combining the two, you can move from insight to impact—leveraging machine learning to inform your optimization models, or using mathematical optimization to shape and refine your ML models when additional structure or constraints are required. 

Ultimately, Gurobi empowers data scientists and decision-makers to build solutions that are not only predictive but also prescriptive—bridging the gap between what’s likely to happen and what you should do about it.