Accessing the Job History
The job history is only supported by the Cluster Manager. While the nodes have a limited recent history of jobs, the Cluster Manager is able to record the jobs and the logs over a longer period of time. The exact lifespan is a configuration parameter that can be set by the system administrator. History information is persistent and can be accessed even if the cluster nodes are not available.
By default, the job history command will display the last jobs in your cluster.
> grbcluster job history JOBID BATCHID ADDRESS STATUS STIME USER OPT API 910878b9 4aba4ad3 server1:61000 COMPLETED 2019-09-22 15:55:24 jones OPTIMAL grbcluster 594d82fc 9bc34333 server1:61000 ABORTED 2019-09-22 15:52:36 admin grbcluster ce7ab3a4 2e05810c server1:61000 COMPLETED 2019-09-22 14:20:14 jones OPTIMAL grbcluster 66d4783b ada0a345 server1:61000 COMPLETED 2019-09-22 14:17:58 admin OPTIMAL grbcluster
In addition, flags are available to query the history by status, username, time period, application name, and more. For example, the following command lists the last two jobs that were ABORTED by the user gurobi:
> grbcluster job history --status=ABORTED --length=2 --username=gurobi JOBID BATCHID ADDRESS STATUS STIME USER OPT API 80334e25 5a4764cc server1:61000 ABORTED 2019-09-20 16:53:23 gurobi Python 0d6d140b f94228b6 server1:61000 ABORTED 2019-09-16 22:04:09 gurobi Python
This history command gives you access to the same views as the
job recent command, but with more filters. For example,
the following command shows the model characteristics of the last two
jobs that were COMPLETED by the application
app1:
> grbcluster job history --status=COMPLETED --length=2 --view=model --app=app1 JOBID STATUS STIME ROWS COLS NONZ ALG OBJ DURATION b9063f12 COMPLETED 2019-09-19 11:22:18 13 9 45 MIP 5 319ms 964a9405 COMPLETED 2019-09-19 11:22:17 13 9 45 MIP 5 126ms
