First Distributed Optimization in the Cloud
The Gurobi Instant Cloud makes it easy to launch a cluster of machines for distributed optimization. This guide will walk you through the process of completing your first distributed solve in the Cloud.
1. Open the Instant Cloud Manager
Login to wwww.gurobi.com and go to your account page. Then, open the Instant Cloud Manager by clicking on the following link:
2. Create a pool with distributed workers
In the Instant Cloud Manager, go to the 'Pools' section and click on the add new pool button:
Then, open the 'License' tab and set the number of workers to 2.
Finally, create the new pool. Note that a default name is assigned for you such as pool1.
3. Download the pool license file
The list of pools is displayed in the Instant Cloud Manager and your license file is ready to be downloaded with the following button.
The license file contains the default access ID and secret key for the selected pool. You just have to place this file in your home directory which takes precedence, or in one of the following shared locations:
- C:\gurobi\ on Windows
- /opt/gurobi/ on Linux
- /Library/gurobi/ on Mac OS X
In case you previously had a license file installed, please make sure to replace it,
or set the environment variable
GRB_LICENSE_FILE to point to the cloud license file, it
will override the default locations.
You can try to solve a MIP MPS file provided with the Gurobi distribution. Here is an example on Mac OS X:
$ gurobi_cl /Library/gurobi811/mac64/examples/data/misc07.mps Waiting for cloud server to start.......... Capacity available on '999999-pool1' cloud pool - connecting... Established HTTPS encrypted connection Gurobi Optimizer version 8.1.1 Copyright (c) 2019, Gurobi Optimization, LLC Read MPS format model from file /Library/gurobi811/mac64/examples/data/misc07.mps Reading time = 0.47 seconds MISC07: 212 rows, 260 columns, 8619 nonzeros Optimize a model with 212 rows, 260 columns and 8619 nonzeros Coefficient statistics: Matrix range [1e+00, 7e+02] Objective range [1e+00, 1e+00] Bounds range [1e+00, 1e+00] RHS range [1e+00, 3e+02] Starting distributed worker jobs... Using Compute Server as first worker - running now Started distributed worker on ip-52-91-137-123 Started distributed worker on ip-54-159-77-110 Distributed MIP job count: 3 Nodes | Current Node | Objective Bounds | Work Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | ParUtil Time H 0 4155.0000000 - - 3s H 0 3610.0000000 - - 3s H 0 3500.0000000 1415.00000 59.6% 3s H 0 2940.0000000 1415.00000 51.9% 3s H 0 2810.0000000 1415.00000 49.6% 4s 24 22 2810.00000 1544.28571 45.0% 99% 4s 1114 475 2810.00000 1926.66667 31.4% 99% 5s Ramp-up phase complete - continuing with instance 1 (best bd 2175) 7533 931 1492.85714 0 48 2810.00000 2175.00000 22.6% 99% 7s 15311 0 2785.00000 21 13 2810.00000 2810.00000 0.00% 93% 9s Cutting planes: Cover: 2 Clique: 4 MIR: 17 Zero half: 10 Runtime breakdown: Active: 8.09s (88%) Sync: 0.81s (9%) Comm: 0.28s (3%) Explored 15311 nodes (152346 simplex iterations) in 9.17 seconds Distributed MIP job count: 3 Optimal solution found (tolerance 1.00e-04) Best objective 2.810000000000e+03, best bound 2.810000000000e+03, gap 0.0%
Within this log, we have highlighted in bold some important steps. First, the client automatically connects to the Instant Cloud server and checks for the pool status. As the machines are not launched yet, Instant Cloud starts the machines and the client reports that it is waiting until capacity is available.
Then the Gurobi Optimizer detects that the pool is setup with 2 distributed workers. So it automatically starts the solve in distributed mode with 3 workers (the master compute server counts as one worker as well).
5. Terminate the pool (optional)
The machine will auto-terminate once it stays idle for a duration limit called the idle shutdown. The default idle shutdown is 60 minutes, and it can be changed in the settings of the pools and your preferences. Otherwise, you can terminate the pool manually in the Instant Cloud Manager, by selecting the created pool and clicking on the terminate button.